Token Count Is Not Engineering Quality
Picture an agent that keeps hitting the same class of ticket. Every time, it reasons from scratch, burns a large token budget, and lands on the same fix it landed on last time. Nothing breaks, so nobody profiles it. That's exactly the moment worth stopping for, because the real cost isn't the tokens. It's a decision the system already learned that has never been encoded anywhere except in the model's runtime reasoning.
That's the distinction worth sitting with. Think about what a CPU profiler actually does: it tells you a function is eating 38% of execution time, and nobody reads that as an instruction to go buy a faster processor. They read it as a pointer to the hot path that needs rewriting. I think reasoning tokens want the same read. Recurrence doesn't mean the problem is hard. Usually it means nobody has checked whether the solution already got encoded somewhere in the system, or whether the model is just re-deriving something it figured out last week.
It's plausible that a platform team without task-level tracking ends up defaulting to the largest available context and reasoning budget, not because anyone chose that, but because nobody was watching the pattern closely enough to choose anything else. Inference spend has gotten large enough that it wouldn't be surprising if finance teams are already asking questions about it, and surveys of cloud and GenAI spending do find most teams describe that spending as unmanageable, which is a different problem from spending too much on any single task. I can't point to a study that measures how common the "default to the biggest budget" pattern is across the industry, and I wouldn't trust one that claimed to. What I can say is that it's a specific, checkable question for whatever agent you're running right now: is it re-deriving a decision it already made, or is it actually facing something new?
Once you're asking that question, the useful move is sorting the recurring work by what kind of uncertainty is actually left in it. A trajectory that produces the same tool calls in the same order every time isn't uncertain anymore. It's a candidate for deterministic code, and paying an LLM to re-derive it is like paying a senior engineer to recompute a lookup table by hand.
Say a support agent keeps getting a ticket about a stuck webhook retry. Every time, it calls the same three tools in the same order: check the delivery log, confirm the endpoint is reachable, requeue the event. That trajectory ran the same way for weeks before anyone noticed the token spend attached to it was almost entirely wasted reasoning. Once someone did notice, the fix was a script that does those three steps directly and only calls the model when the requeue fails twice in a row. That's the moment the decision left the model's head and became code.
A trajectory that varies in its path but keeps landing on the same small set of intermediate states is a different case. The agent isn't wrong to reason there. It's wasteful to reason from zero every time, and that's closer to an argument for a reusable skill or a cached sub-procedure than for a hard rule. Fine-tuning is a third option, but it only pays off once you've confirmed the pattern is real and stable; done earlier than that, it just bakes an unproven habit into the weights.
I don't think any of these buckets beats frontier reasoning in some general sense. Each one beats frontier reasoning only for the specific, already-resolved sliver of the task it covers, which is a narrower claim and a more checkable one.
That sorting isn't free, and it has a real failure mode worth naming honestly. A tool sequence can look stable for months and then break the day an upstream API changes its schema, and a hard-coded version of that sequence fails silently instead of adapting the way the model would have. So the instrumentation you need isn't just counting repetitions. It has to watch for drift, which means it has to keep asking whether this trajectory is still converging on the same answer or whether something upstream shifted underneath it.
Concretely, that means logging each agent run with a task identifier, the ordered tool-call sequence, the final outcome, and a fingerprint of the trajectory shape so near-duplicate runs cluster together automatically. Without those fields you can total up tokens per call, but you can't separate recurrence from novelty, and you can't tell a stable trajectory from one about to break. That's a narrower ask than a full observability platform, and it's the piece that tends to get skipped, because a token counter is easy to bolt on and trajectory fingerprinting is not. It's worth building before reaching for a token cap or a fine-tuning project, since both of those decisions are guesses without it.
Research on chain-of-thought length finds an inverted U-shaped relationship between reasoning length and accuracy on the tasks it tested: performance improves as reasoning gets longer, up to a point, then degrades as errors accumulate in the extra tokens. That held even on harder tasks in the study, which cuts against the assumption that more reasoning is safely monotonic just because a task looks novel. The tasks tested were closed-ended reasoning problems with a checkable answer, not open-ended agent trajectories full of tool calls, so I'd treat the mechanism, errors compounding once reasoning runs long past what the problem needs, as plausible for agents rather than demonstrated for them. It's a claim about tested trajectories, not a verdict on your production workload, but it says token count and answer quality can decouple past some threshold, and that threshold has to be found for the specific task, not assumed from how hard the task looks.
None of this argues for a blanket token cap. Genuinely novel or high-risk tasks can still justify a generous reasoning budget, and the research above says that generosity doesn't pay off indefinitely, but cutting a trajectory off before it's had room to converge is its own mistake, just a quieter one.
The failure I'd actually worry about runs the other way. An agent stumbles into a workable solution through a lot of exploration, someone turns that solution into a rigid heuristic to save on cost, and the heuristic fails silently the first time an edge case shows up that it never saw during training. I don't know of a study that measures how often that happens, so I'd call it a risk to design against rather than a documented outcome. But it's why a blanket cap makes me nervous: it fixes the line item on the spend report and can break the tasks that still needed room to think, and nobody finds out until the failure shows up three steps downstream.
So the question isn't how many tokens to cut. It's simpler than that: has the uncertainty actually left the task, or are you still paying to resolve something the system worked out months ago and never wrote down anywhere but in a model's runtime reasoning. Sort your recurring work by that question honestly, and the token bill mostly takes care of itself.
