The thing that clicked today: I was choosing which AI model ran each job by guessing which one I had probably used least, which is a scheduling problem I was solving badly in my head several times a day.
Built / shipped
A router that scores the options and picks one. Three services are available to my fleet, each with its own usage allowance and its own reset clock. When I send off a job I no longer name a provider. The router reads current usage for all three, scores them, and takes the best one that is actually safe to use.
Hard gates first, then a score. Anything that cannot do the job at all is eliminated before scoring, rather than scored badly and picked anyway when everything else looks worse. Only the survivors get compared.
Reserves that move with the clock. Holding capacity back matters much more when a reset is five days away than when it is twenty minutes away. So the amount held in reserve shrinks as the reset approaches, and there is a bounded bonus for spending capacity that is about to expire anyway. Unused allowance is not thrift, it is waste.
A shared reservation ledger. This is the part I would defend hardest. Two independent things asking "who is least used right now" get the same answer at the same time and both go there, which is exactly the pile-on the router exists to prevent. So selection and reservation happen as one transaction against a single ledger on the control-plane machine. You cannot win a slot without holding it.
A written receipt for every decision. Each dispatch records which provider was chosen, the score, and the reason, so a later question about why a job ran somewhere has an answer that is not my memory.
Problems & fixes
Missing data is not good news. One provider only reports its weekly window and nothing about the shorter one. The tempting readings are "nothing reported, so nothing used" or "unknown, so unlimited", and both are wrong in the expensive direction. Unknown is now its own state: usable, ranked below anything with complete telemetry, and never treated as empty.
The first live test picked correctly and then produced nothing. The router chose well, wrote its routing record, and released its reservation cleanly. The job itself then stopped on an internal confirmation prompt, exited zero, and left no output at all. Exit zero meant the process ended, not that the work happened. That gap between "the process finished" and "the work finished" is the thing to design against.
The second test answered and still wrote nothing durable. Different cause: the job produced a perfectly good answer to the screen but did not reliably write it to the file that makes it a durable record. The fix was to stop depending on a job to file its own paperwork. If a run exits clean with real output, the system now captures that output as the record itself. It does not invent one when the output is empty, because a fabricated record is worse than a missing one.
Decisions
Never migrate a healthy running job. Once something is running, later readings will eventually suggest a different provider would have been better. Moving it then buys a marginally better score and risks the work. The decision is made once, at dispatch, and then left alone.
Refuse rather than degrade. Providers are graded, and the bottom grade is never launched. I would rather a job wait than run somewhere I have already judged unsafe.
Pins are for capability, not preference. I can still force a specific provider, but that is for a genuine capability requirement, not a hunch. A pin overrides preference and does not override safety, exhaustion, or availability.
Learned
Cost routing changes what you build, not just what you spend. Once a class of work is genuinely free, I stopped rationing it. Broad sweeps and inventories that were never worth paying for now just run, and they surface things I would not have gone looking for.
The scheduling problem was always there. I had been solving it in my head, badly, several times a day, and calling that judgment. Writing down the actual rule took an afternoon and immediately beat the version I was improvising.
A verification that only proves the good case proves very little. Both live failures produced a clean exit code. Anything that treats exit zero as success would have reported both as fine.
Still open / next
The reserve curve is a guess. It is a reasonable shape that has not been tuned against a full cycle of real usage, and I expect the numbers to be wrong in ways I will only see over weeks.
One provider is still ranked on partial information. That stays until it reports the shorter window, or until I find a reliable way to infer it that is not wishful thinking.