Checkly's Results Daemon handles roughly 92 million messages a day. It processes check outcomes, writes to databases, fires alerts, and pushes WebSocket updates to dashboards. Last year the volume doubled, the Node.js service started paging on-call engineers more often, and the team decided to rewrite it in Go. They also decided to let Claude Code do most of the writing.
The result shipped with zero production incidents. Running pods dropped 70%. Database active sessions fell 60%, and database CPU dropped about 15%. The team freed up 15 vCPU and 45GB of memory across their cluster. The interesting part is not the rewrite itself but how they structured it so an agent could execute the work reliably.
Why Go instead of another JavaScript runtime
Checkly runs synthetic checks, automated scripts that emulate real users, and uptime checks that confirm system components are alive. Results Daemon sits between their runner fleet and everything downstream: PostgreSQL, caches, queues, and the UI. It consumes check results, determines outcomes, schedules retries, and publishes updates.
The Node.js implementation was vanilla JavaScript. At the doubled volume, type safety became a real problem. Every change required careful manual review because the language offered little protection against regressions. Go's static type system changed that calculus. The compiler catches entire categories of bugs before they reach review, and that property turns out to matter more when an agent is generating the code than when a human writes it by hand.
sqlc, the Go code generator that produces type-safe database query code from SQL, also contributed to the performance gains. The generated queries run faster than the hand-written ones in the Node.js version, and Go's concurrency model means database transactions complete faster with less row locking.
The test harness came first
The team built a behavioral test harness before the agent wrote a single line of Go. The harness treats Results Daemon as a black box: it feeds inputs, captures outputs, and compares them byte-for-byte against golden files generated from the running legacy system. The code under test could be Node.js, Go, or anything else. The harness does not care.
Every test case provides a check result and a check configuration. Results carry outcome states like Failed, Degraded, or Success along with metadata. Configurations include retry rules, alert rules, and account-level settings. The combinatorial space of these inputs maps directly to code paths, so coverage of inputs equals coverage of behavior.
To generate realistic inputs, the team extracted all account, group, and check configurations plus every result outcome from the last 24 hours (the longest check interval they run) into a ClickHouse instance. Each unique configuration combination was tagged with its occurrence count. That gave them a distribution-aware test suite that reflects real production traffic, not synthetic scenarios.
Non-deterministic fields like UUIDs and timestamps get placeholder tokens that are still type-checked during assertions. The harness uses Playwright for test orchestration, Docker Compose for spinning up boundary services, and Toxiproxy for simulating network failures like PostgreSQL going down. Each boundary (database, queue, cache) runs as a real container instance, not an emulator.
Oracle classes handle assertions. PostgresOracle.expectResultToMatchSnapshot(testId) fetches output from the real database and compares it against the golden file. Code coverage reports were fed back to the agent to identify gaps and generate additional test cases, targeting the 90-100% range.
What the agent produced
The team gave Claude Code, running through their internal tool Fable, a single instruction: build a Go service that consumes data from input queues, processes messages, and writes outputs to downstream applications. The legacy implementation was available as reference. The main acceptance criterion was that the test harness passed against the new code.
The agent ran overnight and produced approximately 13,000 lines of application code. It stayed within the daily token limits of a $200 Claude subscription. The implementation architecturally mirrored the legacy version. An earlier attempt using Opus did not meet the team's bar and was discarded.
The team reviewed the output and found two things worth fixing by hand. First, the agent had inherited a configuration anti-pattern from the legacy code: runtime derivation of settings from partial environment variables. Under incident pressure, debugging configuration had been a recurring pain point. They replaced it with static environment variables only. Second, the new service needed better end-to-end observability. At 92 million messages per day, low-level metrics like database connection utilization, CPU usage, and individual code execution timings are not optional. They added both improvements with human supervision.
First deployment exposed a harness gap
The initial deployment strategy was simple: feature-flag accounts to route their traffic to the new daemon. The team deployed across all non-production environments, migrated internal accounts, and hit retry failures immediately.
The root cause was a mismatch between the harness's queue topology and production. Locally, retries routed to 3 queues based on check type. In production, routing also depends on priority and hosting type, totaling 18 possible queues per region. The harness modeled the simplified local version. The agent built retry logic around it. The gap propagated into the new application.
The fix required updating the harness to match production, removing code paths that only ran locally, and a human-supervised refactor of the retry module. The team documented three principles from this: the harness environment must match production as closely as possible, every boundary must be specified down to its smallest unit (every queue, every table), and every assumption about surrounding infrastructure must be written down and verified rather than inferred.
Customer migration
After the retry fix, the daemon processed internal account traffic (about 3% of total load) for 24 to 48 hours. The team then migrated customers in stages: free accounts first, then paid monthly subscribers, then enterprise accounts with signed deals. Each cohort flipped via feature flag and ran for 24 to 48 hours before the next group moved.
During the migration window both daemons processed real traffic simultaneously. Any change to one needed to land in the other. The team updated CI to run the harness against both the legacy and new daemon, blocking any pull request that failed for either version. When customers found or reported bugs, the team reproduced the gap in the harness first, then fixed the issue. Most bugs closed in about an hour including CI and deployment.
After a week of staged migration, monitoring, and minor fixes, the legacy workload was decommissioned. The team reported that beyond those edge cases, no customers noticed they had been migrated.
What this means for agentic engineering
The Checkly rewrite is a useful data point because it is not a prototype or a side project. It is a production service processing 92 million messages daily, and it shipped without incidents. The key takeaway is that the harness, not the agent, is the real engineering work. The agent writes code inside constraints defined by humans. When those constraints accurately model production, the agent can execute at scale. When they do not, the agent faithfully reproduces the gap.
Go's type system proved to be a natural fit for agent-generated code. The compiler catches regressions that a human reviewer might miss in 13,000 lines of unfamiliar code. Combined with a behavioral harness that enforces byte-level parity, the team could trust the output without reading every line. That trust showed up as faster shipping, fewer on-call pages, and a system that handles double the load with a fraction of the infrastructure.