A log message arrives saying "config reload requested by unknown actor." It carries an INFO label, but repeated occurrences within ten minutes could signal something worth investigating. The decision of whether to wake someone up does not require a language model capable of writing sonnets or solving differential equations. It requires a fast, cheap judgment about severity and routing. That gap between what we ask of large language models and what many tasks actually need is the core argument in a recent post from the team at Expanso.
The mismatch between model size and job scope
The author does not mean parameter count when saying models are too big. The complaint is about scope. Modern LLMs are general-purpose systems that can do nearly anything, and developers have grown comfortable handing them narrow tasks while spending considerable effort keeping them focused. A log router needs to classify events, assign ownership, and route results. It does not need to discuss the philosophy of incident response or generate lengthy explanations before producing a verdict.
When you ask a model to figure out the question, the policy, and the action all at once, you get a system that is harder to debug, more expensive to run, and slower to respond than the task demands. The alternative is a model built from the start for bounded decisions, where the input is context plus a defined question and the output is a typed choice, a score, or a probability.
Jev and the Expanso pipeline
Jev, a model from TypeSafe, is the concrete example. It returns structured judgments without explanations. You provide context and specific questions, and it produces results the next piece of software in the pipeline can consume directly. There is no conversational overhead, no need to parse prose for the actual answer.
In the Expanso log-triage demo, the pipeline works like this. Incoming records arrive and get checked against explicit rules first. Known routine events go straight to archive without involving any model. Events that require judgment get packaged with context, including how often a matching event appeared in the past ten minutes. Jev answers four questions: is the event actionable, how severe is it, which team should own it, and is the recurrence concerning. The pipeline applies its own thresholds and routes the result toward page, notify, review, or archive.
The division of labor is explicit. The model judges the event. The routing policy is code that humans wrote and can inspect. If the model cannot produce a confident answer, the event goes to a review queue while routine traffic continues unaffected. Each component has a clear owner and a clear responsibility.
A small detail that reveals a large problem
The demo includes a normalization step that strips numbers from log messages so similar events group together. This is useful for counting repeated occurrences. But the author points out that normalizing a successful health check and a failing one can make them look identical once you remove the status code and response time. A more capable model does not excuse the pipeline designer from deciding which information to preserve or which shortcuts are safe. That is an engineering judgment, not a model capability.
This matters because it highlights where the actual risk lives in these systems. The model can be fast, cheap, and accurate, and the pipeline can still make wrong decisions if it discards the wrong fields during preprocessing. Tooling and observability around the data flow matter as much as the model choice.
The economics of small decisions at scale
TypeSafe lists Jev at $0.042 per million input tokens, with no charge for output tokens. The company reports subsecond response times in its own benchmarks, though those are vendor numbers and not independent measurements. At that price point, a decision that would be too expensive to make a million times suddenly becomes viable to make a million times. A small decision that is too costly per invocation generally does not get made at all when the volume is high enough.
That changes which problems are worth tackling. Instead of collecting events in a pile, sending them somewhere else, and waiting for someone to interpret a summary, you can evaluate individual events as they move through the system. The cost structure determines the architecture, and cheap typed inference opens up patterns that expensive general-purpose models make impractical.
Before putting this on a critical path, you would still want to measure tail latency under sustained load and understand the failure modes on your actual data. Typed output does not make the judgment correct. A wrong answer in valid JSON is still wrong. You need examples from your own environment, defensible thresholds, and a place for uncertain results to land.
More models like this are coming
The expectation here is that smaller, targeted models will increasingly sit alongside large general-purpose ones. The specialization is not in the model weights. TypeSafe says customers use the same weights for different use cases. The specialization is in what you ask the model to produce. A log router, a support queue triage system, and a data quality checker all want the same kind of bounded judgment, even if none of them needs a chatbot.
The big general models still have their place. Complex reasoning, open-ended generation, and tasks that require broad world knowledge are not going away. But for the large category of infrastructure decisions where the possible outcomes are known in advance and the policy is already written, a model that returns a typed answer quickly and cheaply is a better fit than one that can do everything.