The thing that clicked today: the model on my own hardware was not failing at the thinking, it was failing at the paperwork, and I had been asking it to do the paperwork.
Built / shipped
An exact plan, declared before the job runs. For work that is fully specifiable in advance, the request now carries the precise sequence of tools the job must call and the exact markers its answer must contain. The system checks that contract before any thinking happens. A malformed contract stops immediately with a distinct failure, rather than burning a few minutes of compute to fail later and less clearly.
The system writes the delivery, not the model. This is the change that actually worked. Instead of asking the model to compose a final structured report, the harness assembles it from the results of the tool calls that genuinely executed, hashes it, and embeds that hash in the response. The model does the work; the system writes down what happened.
No retries. A failed run stays failed and gets diagnosed. Automatic retry would have hidden the very pattern I was trying to see.
A permanent regression suite. Thirty cases covering the good paths, the malformed contracts that must be rejected before any inference, and the awkward middle cases. It runs as a release gate rather than a thing I remember to do.
Problems & fixes
The failure was in the runtime, not my harness. Jobs were dying intermittently with a server error partway through. My leading theory, written down and wrong, was that my own harness was at fault. Reading the actual error body showed the runtime was rejecting the model's own tool-call output as unparseable, before any valid response reached me. My harness was handling that correctly: capturing the error, recording it, and exiting with a specific code.
It reproduced hardest on long, escape-heavy text. The pattern in a hundred-plus archived runs was that the malformed calls were the ones where the model tried to stuff a long report, or a command full of nested quoting, inside a single argument. Some failed on the first call and some after several successful ones, which ruled out a simple "it degrades over time" explanation.
The newer runtime was worse. The obvious move was to upgrade. I tried it, measured, and it performed worse than the version I was on, so I put the old one back and solved the problem in my own code instead. Upgrading is a hypothesis, not a fix.
Compatibility patches got me halfway and were the wrong direction. The first honest run of the new suite scored three out of thirty. Adding a layer that accepted the different shapes the model produced took it to thirteen. That was real improvement and also the clue: I was building an ever-growing tolerance for variation in something the model should never have been authoring. Moving that responsibility into the harness took it to thirty out of thirty and removed the fabrication risk at the same time.
Decisions
The boundary is what the model owns. It reasons and calls tools. It does not author the record of what it did, because a model writing its own proof of work can be both wrong and confident, and no amount of validating the shape of that proof fixes the underlying problem.
Fail closed and fail early. A bad contract stops before inference with its own exit code. Cheap, unambiguous, and it never half-runs.
Keep the verified runtime. Version numbers are not progress. This one is pinned because it is measured, not because it is current.
Learned
Chasing shape variation was a signal I had drawn the boundary in the wrong place. Every patch I added to tolerate another output format was evidence that the thing producing those formats should not have been producing them at all.
A wrong hypothesis written down is still useful. Recording "probably my harness" made it obvious when the evidence disagreed, instead of letting me quietly drift toward whatever I later preferred to believe.
The cheap model is only cheap if it is reliable. Free compute that fails a third of the way through a job is not free, it just moves the cost onto me.
Still open / next
The exact-plan mode only fits work I can fully specify in advance. Anything needing judgment mid-run still goes to a larger model, and I do not have a clean rule for the boundary beyond experience.
I am pinned to a runtime version I know is not the latest. That is a deliberate trade with an expiry date on it, and I will have to re-test rather than assume the next version fixes what the last one broke.