A Taxonomy for Sorting Useful AI Inference From Mechanical Token Waste
Every agentic coding session generates a token bill. Calls, input tokens, output tokens, latency, maybe reasoning tokens if you are lucky. Those numbers tell you how much you spent. They do not tell you what you got for it. Two inference calls with similar token counts can play completely different roles in task completion. One might be deciding which source of information matters. The other might be copying a cursor from one response into the next request. Both cost money. Only one resolves meaningful uncertainty.
Jeff Auriemma proposes a way to describe this distinction. Given some task and the sequence of model invocations used to complete it, each invocation can be classified by the functional role that inference plays. He uses four categories: Initialization, Orchestration, Reasoning, and Synthesis. This is a taxonomy of inference calls, not a theory of what happens inside the model. The word "reasoning" is used in a functional sense: an inference call is Reasoning when it resolves uncertainty that matters to task completion. Nothing about the taxonomy requires taking a position on whether a neural network really reasons.
Initialization: The Context You Pay For Before Anything Happens
Before an agent begins making task-specific decisions, the model has to ingest task-independent context. System prompts, tool descriptions, function definitions, policy text. A software development agent might be told it can call read_file, search_code, edit_file, and run_command. A more general agent might receive hundreds of function definitions. This work is primarily an input or prefill cost. The model may produce almost nothing in response, but it still has to process the environment placed into its context window.
As agent environments grow larger, Initialization becomes significant even before any task-specific inference takes place. There is an important boundary here: if the harness parses JSON, increments a retry counter, or dispatches an HTTP request without invoking the model, that does not belong in this taxonomy because it is not inference. If the harness then places a 20,000-token schema into the model context, consuming that context does contribute to inference cost. The distinction is not between agent work and non-agent work. It is specifically about the role played by model inference within the broader agent runtime.
Reasoning: The Judgment That Actually Matters
The most interesting distinction begins once the task is underway. Suppose a user asks an agent why checkout latency increased after a deployment. The agent has access to deployment history, logs, metrics, incident records, and runbooks. If the model decides the first useful step is to inspect recent deployments for the checkout service, that inference is doing more than formatting a tool call. It is deciding what evidence is relevant, which operation should retrieve it, and what parts of the system are connected. That is Reasoning.
The same applies when a coding agent decides which implementation to inspect after reading a stack trace, when a research agent decides that conflicting secondary sources require consulting the original regulation, or when a browser agent discovers the expected navigation path is missing and has to determine where the relevant information lives. In each case, the model is being used because something task-relevant remains unresolved.
Reasoning also includes judgments about whether execution should continue. An agent that has gathered several sources may need to determine whether they are sufficient to support an answer, whether an apparent contradiction matters, or whether another search is warranted. A model invocation that concludes "we have enough evidence" can be Reasoning even if the resulting output is extremely short. The question is whether the invocation had to resolve a meaningful question about how the task should proceed.
Orchestration: When the Model Is Just a Fancy Cursor
Now consider a different situation. The agent has already determined it needs every page of a result set. The first response contains a cursor: {"next_cursor": "cD0y"}. The next request is semantically determined: GET /records?cursor=cD0y. If the model is invoked simply to produce that request, no meaningful task uncertainty has been resolved. The decision to continue pagination has already been made. The cursor has already been supplied by the preceding response. The model is advancing an execution path whose semantics are known. That is Orchestration.
This shows up in many forms. A previous call returns customer_id: "cus_82ca91" after the agent has already decided to fetch that customer's invoices. Another inference call then emits list_invoices(customer_id="cus_82ca91"). An agent could decide the same metric must be collected for every service in a known list and then invoke the model repeatedly to substitute checkout, payments, inventory, and shipping into otherwise identical requests. A retry policy specifies that a 503 should be retried up to three times, yet every 503 is returned to the model so it can decide to retry the same operation.
None of this is unnecessary work. The essential question is why those operations require dispatching calls to a model once the decisions governing them have already been made. Ordinary software is very good at loops, parameter substitution, state transitions, retries, and protocol progression.
A single example exposes the key boundary. Imagine two model calls that both produce: grep -R "refreshToken" packages/auth. In one trace, the preceding state already says "search packages/auth for refreshToken." The model is rendering an established operation in shell syntax. That is Orchestration. In another trace, the model has been shown a bug report, considered several plausible causes, decided refresh-token handling is the most promising place to investigate, and emits the same grep command. That is Reasoning. The output is identical. The function of the inference call is different.
Predetermination is the key criterion, not syntax. You cannot classify an inference call reliably by looking only at what it emits. Any tool invocation may be the result of a task-relevant decision made inside the call. Conversely, Orchestration does not need to produce machine-readable syntax. A model that emits "Now inspect the next page" or "Retry the previous operation" may still be spending inference on execution that was already determined.
Synthesis: Turning State Into an Answer
Synthesis is the terminal inference that turns accumulated task state into the response delivered to the user. An incident investigation has established that a specific version was deployed at 14:03, p95 latency increased four minutes later, logs show repeated inventory RPC deadline failures, the release changed that RPC timeout, and a rollback restored latency to baseline. The inference that converts those facts into a concise explanation is Synthesis.
Synthesis is not necessarily simple. In some tasks it may require more judgment than any individual step that preceded it. It gets its own category because its functional role is different. Reasoning determines how the task should proceed. Synthesis produces the terminal response from the state that process produced.
Reasoning Yield: Measuring What Your Tokens Actually Bought
The economic distinction between Reasoning and Orchestration is the part Auriemma finds most useful. Model inference is expensive machinery. It adds latency and token cost, and because model outputs are probabilistic it can introduce another opportunity for execution to drift from an otherwise valid plan. That cost is justified when the task calls for judgment. It is harder to justify when the next operation is already represented in the system state and another model call exists principally to translate that state into the next mechanical step.
A mature agent architecture should be evaluated in part by how effectively it reserves model inference for work that benefits from model judgment. The goal is not necessarily to minimize orchestration. Every nontrivial task requires it. The optimization target is the amount of inference spent performing orchestration after the relevant choices have already been made.
Auriemma proposes a metric called Reasoning Yield: the share of task-execution inference spent resolving task-relevant uncertainty rather than mechanically advancing already-determined execution. If an agent spends 8,000 tokens on Reasoning and 2,000 on Orchestration, its Reasoning Yield is 80%. If another agent reaches the same result using the same 8,000 Reasoning tokens but another 12,000 tokens of Orchestration, its yield is 40%. All else equal, the former makes more efficient use of model inference.
Reasoning Yield is not a complete quality metric. A higher number does not automatically imply a better system. An agent could achieve a very high yield while reasoning inefficiently, making bad decisions, or failing the task altogether. Initialization and Synthesis should not be ignored either. But within the inference being spent on task execution, the ratio tells you something useful: how much is being used to resolve uncertainty versus advance execution that has already been determined.
There will inevitably be edge cases because predetermination is not always binary. A loop that appears mechanical may encounter one item requiring interpretation. A retry that normally follows a fixed policy may produce a new authentication error that requires a decision about whether to refresh credentials or stop. A browser agent may mechanically click "Next" for five pages and then reach a page whose layout has changed. At that point the character of the next inference changes because task-relevant uncertainty has reappeared. That is not a weakness of the taxonomy. It is the property it is trying to expose: model judgment becomes useful again when the runtime can no longer determine the next meaningful operation on its own.