Copying and pasting sucks. So does buggy software that no one knows how to run unless you speak like three languages, and one of them has to be C++.
We weren’t taught how to vibecode in college.
In my second post, I Built a Local AI Coding Assistant on Consumer Hardware…and It Works. I Think., I left off with a question: what does the workflow look like when the context window doubles and the model isn’t quantized? I think I can answer that now.
The prompt file that gave it away
It started as a single file of prompts I kept copying and pasting between projects. That was the clue. If I’m pasting the same instructions into every session, they’re not instructions anymore, they’re infrastructure that hasn’t been built yet.
So I started breaking them into skills. Small at first, just because it was easier than scrolling through one giant file.
The test that wasn’t a test anymore
At one point I was trying to get the AI to update my tests. It kept editing the program instead. The tests would pass, but the program didn’t work.
This happened repeatedly. I blamed the AI for a while. Took me longer than I’d like to admit to realize it wasn’t Gemma, it was one specific test that had quietly stopped being viable. I deleted it. Gemma and OpenCode can still make that kind of call wrong on their own; that’s still on me to catch.
I was new to Gemma. It took time to learn its limits, and that test was where I learned the first one.
Three sentences that unlocked everything
I took the problem to Gemini and went through an iterative back and forth. What came out the other side was a Skill Instructions section built from three pieces I’d never seen combined before:
**Constraint:** Do not modify any core source files. Only modify files located within the test directories defined in `./Testing Strategy.md`. Use the `todowrite` tool to track all execution items.
⚠️ **STRICT AGENT BOUNDARY & ORCHESTRATION RULES:**
- You are acting SOLELY as an **orchestrator**. You must NOT perform direct work (file edits, code analysis, or terminal execution) in the main interface.
- All technical inspection, file updates, and shell executions MUST be delegated to an `@general` agent via `todowrite`.
**EXECUTION POLICY:** Perform workflow steps strictly in sequence. Do not spawn concurrent agents. Wait for each returned summary before proceeding.
A constraint, a boundary, an execution policy. Separate the orchestrator from the worker and tell it to wait its turn.
I showed this to a guy I talk tech with sometimes. His whole company started using it. He also tried the second trick I’d stumbled into: end every prompt with “Ask me any additional clarifying questions.” That one line stops the AI from guessing and gets it asking instead. One of his employees finished two weeks of planned work in four days.
Once I ran all six of my skills through that structure, things started to unlock.
Open knowledge, not just open source
The skills have their own repository now: sdlc-skills. It started as a folder inside my llama-server-manager repo, so technically it was already open source. But that’s not really the point. Open knowledge, not just open source. If I figured out something useful, I’d rather publish it than sit on it.
The chain works like this: sdlc-plan reconciles Requirements.md against the codebase and writes Plan.md. sdlc-update diffs Plan.md against the actual code and writes Update.md. sdlc-implement works through those gaps and writes the code. sdlc-update-tests keeps the test suite honest. Bugs.md runs alongside all of it, fed by sdlc-bug-create and cleared by sdlc-bug-fix.
sdlc-update-tests was the first one I had to get right, because it’s the one that kept eating my test files. Once it worked, I used it as a template. I fed Gemini three working skills and told it to build a template from them, then apply that template to the rest. It asked me some clarifying questions of its own, and then BOOM, more skills that were worth something.
Where the line actually is
I use Gemini and Claude for the skills themselves and for the two documents that kick everything off, Requirements.md and Testing Strategy.md. That doesn’t break the local-only rule. The experiment is “can I write code locally.” Nothing more, nothing less. Everything outside that stays fair game, and it keeps me moving at a steady pace without paying for a subscription.
What’s actually running
Gemma 4 12B QAT is the winner right now. It runs on a $740 RTX 4070. I checked the price on Newegg while writing this. It was $650 a month ago, so take that as further proof nothing about this holds still.
For Python specifically, I keep falling back to Jackrong’s Qwen3.5-9B-Neo at Q5_K_M. Jackrong fine-tunes a lot of models and not all of them work for me, but this one does. I also tried a 35B MoE model. It doesn’t fit on 12GB of VRAM for anything beyond simple questions. If I had two GPUs and 128GB of system RAM, that MoE model is probably where I’d go next. I don’t, so it isn’t.
The context window went from 120K to the full 262K, and getting there took three things stacking together: the QAT quantization on the model itself, q8_0 cache quantization on both key and value, and the raw size of the model fitting the card. Here’s the config that took me a while to find:
"cache-type-k": "q8_0",
"cache-type-v": "q8_0"
Quantize the cache and you can usually claw back the context window you want, but you’ll lose some accuracy doing it. That’s the actual case for QAT: it buys back what the cache quantization costs you.
The llama.cpp saga, again
Read The Case of the Disappearing Bug for the full story, but the short version is llama-server kept dying when it went to sleep. I mean, I don’t like getting up in the morning either, but quit screwing with me, llama.
It’s mostly fixed. Mostly. Last night I tried updating to the newest llama.cpp release and it wouldn’t even start. Had to roll back to a build I compiled myself. Turns out Meta released Muse Glimmer on Hugging Face yesterday and it needed llama.cpp code changes to run. I think Meta (sorry, [S]uckerberg) — broke it.
Local versus cloud, honestly
I ran Gemma against Laguna S 2.1 (free tier, OpenRouter) on the same task. Laguna probably would’ve produced the higher-quality answer. It also timed out repeatedly, and by the time it finally finished, I’d already moved on with Gemma. Local didn’t win on quality. It won because it actually finished.
What’s still rough
It still gets stuck, mostly on bug fixes from what I can tell, though I don’t have real data behind that, I just run that skill more than the others. My working theory is the llama-server prompt cache, which sits at 8GB. If the same prompt hangs once, it might take a few more tries before the cache cycles it back out. OpenCode recovers and reruns the agent when this happens, which beats the old setup dying outright, but it still costs about 20 minutes when it hits. Keeping the structure documents and skills separate is what keeps the context rot away in the first place; the hangs are a different problem.
The better question
Post two asked whether consumer hardware could run a local coding assistant at all. It can. The workflow’s stable, the skills are public, and I don’t have to babysit it anymore.
So now the question is how far I can push what I already have.
And somewhere behind that one: what happens with a QAT MoE model, once something like that exists in a size I can actually run. I don’t have the hardware to find out yet. But I didn’t have the hardware for this either, until I did.
Links from this post