Why silent success is worse than a crash
Vinoth Govindarajan, who leads core data and AI infrastructure at OpenAI and previously built distributed systems at Uber and Apple, opened his talk with a failure shape that does not show up in benchmarks. A user asked an agent to remember a refund. The agent said it would remember. From the user's perspective, everything worked. Behind the scenes, the turn was not recorded in the persistent context the system would need for the next interaction. The delivery edge looked healthy while the durable record had a hole. A crash gives you a boundary: something stopped, you see an error, and you can replay from the last known good point. Silent success is a lie. The channel says success, the user sees a reply, and the operator has no reason to doubt that the system lost part of its memory. The future context inherits that hole and answers confidently over an incomplete record.
This is why model benchmarks alone do not tell you whether a system is production-ready. A benchmark measures whether the model answered correctly. It does not measure whether the persistent edge and the delivery edge agree. Once an agent can send a message, update a database, run a command, or trigger a workflow, the questions change. Which component owns the state? Who committed first when events arrive together? Can you prove what actually happened at the user-visible edge? These are not model questions. They are production questions, and the harness is where they get answered.
Own the state, order the mutation, prove the action
Govindarajan distilled his talk into three principles. Own the state: a fact needs one owner and one replay path. If your system cannot reconstruct the fact later, it does not really own the fact. Order the mutation: concurrency is fine, accidental interleaving is not. Agents can turn out work, read in parallel, and call sub-agents, but shared state needs one mutable commit path. Prove the action: the transcript is not the receipt. The transcript tells you what the agent said or what the model intended. You need to know what was attempted, what was approved, and what was committed at the user-visible edge.
The first principle showed up in an incident where a heartbeat became a state. An agent went quiet for hours. After 64 hours, someone filed a bug report. The heartbeat token, a liveliness mechanism, had crossed the wrong boundary. It was treated as pending delivery, which kept refreshing the session timestamp. The next heartbeat saw pending work already happening and skipped. The bug was not that the model failed. The bug was that the state machine classified an internal state as user-deliverable work. The fix did not make the agent smarter. It made the state machine stricter. Before processing work, decide whether it is actually work.
The second principle appeared in a concurrency bug where two correct writes produced one wrong outcome. A commitment store followed a load-modify-save pattern. One caller loaded, modified one field, and saved. Another caller loaded the same original state, modified a different field, and saved. Neither writer was malformed. Without serialization around the read-modify-write cycle, the last save erased the first modification. The pattern that works: serialize same-process writes with a queue and protect cross-process writes with a lock. The invariant is narrow and important: one ordered commit path per mutable state. Reads can be parallel. Subtasks can be fanned out. But if two operations touch the same session, the same memory record, or the same commitment store, the harness must define the order. Without it, user experience becomes timing-dependent. Sometimes the agent feels successful. Sometimes it feels haunted.
The harness blueprint for production agents
The blueprint Govindarajan described applies to every agent type, from personal assistants like OpenClaw to coding agents like Codex and Claude Code. Events enter from multiple surfaces: webhooks, timers, chat messages, external systems, internal hooks. The control plane maps each event into a session key, which determines the state boundary. A session lane provides a single writer per commit path. A global throttle protects the system. The runtime calls tools and models. The audit trail becomes a run receipt.
The runtime itself is usually stateless between turns. It rebuilds the working state from persistent sources: session transcript, session state, memory or summary, policies, and tools. If any of these are missing, the model can still answer correctly, but it is reasoning over incomplete reality. State ownership matters before model quality enters the picture. A fact that exists only in a successful delivery log is not enough. A fact that exists only in a return value is not enough. A fact that exists only in user memory is definitely not enough. The first review question for any production agent is: for every fact the agent might use in a future turn, who owns it and how can you replay it.
What this means for teams building agents
The failure modes Govindarajan described are not new. Distributed system engineers already know about idempotency, retries, locks, ordering, and state boundaries. What changes with agents is where the failures sit. The model can choose tools dynamically. The context may be rebuilt differently for each turn. Events arrive from users, timers, webhooks, and sub-agents simultaneously. The system acts across more surfaces. The failures are familiar. The agent setting makes them easier to trigger and harder to explain.
The harness turns those familiar reliability concerns into explicit boundaries around non-deterministic behavior. A model proposes, the harness commits, and the receipt proves it. The model can propose a tool call, a message, an edit, a command, or a plan. The harness decides whether the proposal belongs to the write state, whether the mutation is ordered, whether the work is bounded, whether the authority is valid, and whether the outcome can be proven. For teams building production agents, the takeaway is that a powerful engine with no brakes is not autonomy. It is a liability with good acceleration. The harness gives you control, and control is what makes agents safe to operate at scale.