Spend tokens only on judgement

When an agent loop is slow and the bill is climbing, the first instinct is to swap in a cheaper model or cut the prompt down. Usually that is treating the wrong thing. Look at what the model is actually doing on each turn and a lot of it was never a judgement call. Counting characters. Checking whether a block of json parses. Comparing two strings. Re-reading a file it read four turns ago. Validating that a note has the right fields at the top.

Inference is the most expensive thing you have in the loop, and you are spending it on work grep does for free.

The test for each step

I go through a loop one step at a time and ask one question. Could a script decide this with certainty? If yes, a script decides it and the model never sees it. If no, the model sees the smallest slice of evidence it needs to decide, not the whole corpus.

That second half matters as much as the first. Asking the model to count is one mistake. Handing it the entire artifact every iteration so it can find the one part it needs is the bigger one. A script can find that part.

Two tools built on this

An image-to-3D pipeline I use turns a reference photo into a procedural model in code, and it goes through several build passes before it is done. Plain Python from the standard library does every check that can be done mechanically, with no packages to install, so there is nothing to debug inside the agent's context. The scripts validate the output, gate each stage, write the spec for the next pass, pull the material properties out of the reference and package a side by side comparison sheet. What the scripts never do is score the visuals. That is the one judgement in the loop. The model's entire job is to look at the comparison sheet and say pass or fail, and it only ever sees one build pass at a time so it never re-reads the whole model per iteration.

A review gate I run before code gets pushed does the same split on its findings. It sorts each one into auto-fix or ask-user. Auto-fix means a script can apply the change without asking, a formatting fix or an unused import. Ask-user means the finding touches intent, and a person has to look. That split is the reason the gate is still switched on. A gate that asks about everything gets ignored within a week, and a gate that fixes everything silently changes code nobody meant to change.

We use it on prose too. Before a proposal goes out, the checks with a correct answer are scripts. Is the body under the character limit. Does it contain any string from the banned list. Is every link in it alive. Does it leak a name from another client. The one check without a correct answer, does it sound like the person sending it, is where the model comes in. Splitting them stops the model from being asked to hold a dozen rules in its head while also writing well, because under load it drops the rules.

Reliability comes free

The bill is the visible reason to do this. The better reason is that a script validating a file's header gives the same answer every run, and a model asked to validate the same header mostly does. Mostly is the word that costs you. Every mechanical step you move out of the model is one fewer place where the loop can be wrong in a different way tomorrow, and the same move that drops the cost is the one that makes the system boring to run.

The limit

The failure mode in the other direction is real too. It is tempting to push judgement into scripts because determinism feels safer. It isn't, when the check needs judgement. A regex cannot tell a genuine aside in parentheses from a dash in disguise, so in our writing checks a hook catches the character and a person, or a model, reads the sentence. Put the sentence-level call in a regex and it produces false positives, and a guard that cries wolf is a guard someone turns off.