Opencode file review PDF demo
When I first started working with the County in 2024, we purchased two GPUs with 48 GB each with ~96GB total, as they can be used together. They were relatively cheap compared to todays prices. We've gotten some use out of them, mostly through openwebui: https://docs.openwebui.com/
Not the most intensive use, but the technology for LLMs have expeditiously moved towards more integrated use, and I have been testing out opencode: https://opencode.ai/
And it's quite amazing what you can do without cloud models. The following video demonstrates a Local only (open source) software demo to build up a tool & filing system.
---
Background:
Using openwebui, I created an Outlook macro that saves emails as HTML then converts them to PDF. This macro is run every time I start outlook. It could run whenever an email is received, but Outlook is not friendly to coding as it requires VBA and direct interaction with the GUI. Future work would just put an email client in linux and run a script to do that.
An anaconda coding environment is installed: https://www.anaconda.com/download (any Python coding environment works; in fact, almost any command line/repl coding environment would likely work)
We're on windows which has a different type of toolset; linux is the preferred interactive environment.
We've got ollama (https://ollama.com) running this model: qwen3-coder-next:q4_K_M (https://ollama.com/library/qwen3-coder-next)
The model is already loaded, so the delays at the beginning are purely processing (it's generating tokens and opencode is interacting with those tokens until it starts output)
It's realtime. Total Time is 10 minutes. It picks up quickly once the bulk of the tokens are loaded, skip ahead about 3 minutes; these models excel when they can keep using a Key-Value cache (it loads all previous calculations), but also becomes deficient as the context (token count, see upper right) continues to grow.
---
You can see it makes a few errors in working with opencode; the "coding harness" of opencode, is telling it how to use determinate tools provided by opencode, and the model is providing indeterminate responses. It gets multiple tries, and it still recovers. You are not seeing the full conversation.
At the end, it gits a bit confused at the order one might expect from the command, organizing by type, year then season, but once all the context is loaded, it wasn't much.
---
Just to reiterate: This is a model (open weights) running locally on a County machine, using ~5 year old hardware, with open source software development. It could easily be expanded to do this normally. Just choose some keywords and it'll create pdfs and sort them.
This is what the hype is, but the fact that anyone could have done this stuff without giant data centers is the real gem here.