The problem this paper tackles is the failure of open‑source models to operate autonomously on long‑horizon tasks that span economically valuable work, from multi‑hour computations to multi‑day projects. Such a capability is essential for automated discovery and recursive self‑improvement, yet the field lacks both curated training data and a reliable training recipe. The difficulty stems from the sheer length and complexity of real‑world tasks: they are hard to curate and verify, and the rewards they offer are sparse, arriving only after the final step. Without dense feedback, learning cannot propagate effectively across hundreds of decision steps. The combination of missing data and missing recipe makes scaling both a hard computational and conceptual problem.
Prior Data‑Driven Suites Miss the Latent Structure
Prior work has attempted to shape rewards for agentic systems, but most approaches either require manual engineering of reward functions or produce sparse signals that vanish over long horizons. Some efforts have built synthetic task suites, but these often omit the latent structure that makes tasks verifiable and progressive. When a task comprises dozens or hundreds of primitive actions, an agent has no way to receive a meaningful gradient until the final action succeeds, if then. The result is a gap: learning systems stall on long‑horizon problems because the policy receives no informative signal until the terminal step is reached. This paper fills the gap by first constructing a latent‑world specification and then rendering dense per‑step rewards from it, establishing both a verifiable dataset and a learnable reward surface.
Manual reward engineering is impractical because each new task domain demands a bespoke function, and even small misspecifications can misdirect the policy entirely. Existing benchmark suites such as Last Exam, TextGrad, or domain‑specific trap suites provide sparse terminal rewards, but they do not expose intermediate progress. Without a dense supervisory signal, gradient‑based methods cannot assign credit to early actions that contribute to the final outcome. The paper argues that the missing ingredient is not a richer policy class but a richer supervision signal derived from a latent, verifiable specification of the task.
Tev‑Gym: A Verified Dataset Built on a Latent World
The paper's central contribution is twofold. First, it introduces Tev‑Gym, a dataset of 1,170 verified task environments across seven domains. The construction begins with what the authors call a latent world—a complete and correct hidden specification of the task domain, its workflow, and the expected state after each individual step. From this specification, each task is progressively rendered into an agent‑facing environment. The rendering process gives rise to a natural progress metric: at every step the agent can compare its current latent state to the specification and receive a quantifiable signal of how far it has traveled toward the goal. The latent world acts as a ground‑truth reference that is independent of any particular policy, ensuring that every step’s outcome can be verified against the specification.
Progressive rendering works as follows. The latent world encodes, for each task, a sequence of implicit states: after the first action the system should be in state A, after the second action state B, and so on. These states are not directly observable in the raw environment; instead, the renderer translates them into visible world configurations that the agent perceives. Because the mapping from latent state to observable state is deterministic and verified, the agent receives a reliable progress signal at each step. The complexity can be scaled by adding intermediate latent states, allowing the same task to be presented at finer or coarser granularity depending on the training needs.
Second, the paper turns this metric into a training recipe known as progress‑potential reward shaping (PPRS). PPRS converts the progress metric into a dense per‑step reward that can be injected into any standard policy‑training loop. Because the reward is derived from a verified specification, it is both trustworthy and shaped so that intermediate successes receive proportional credit, allowing learning signals to travel deep into long‑horizon policies. The reward at step t is defined as the difference between the current progress value and the progress value at step t‑1, scaled by a constant that controls the strength of the shaping. This constant, often denoted λ, balances the contribution of the shaped reward against any original task reward, which is typically zero in pure progress‑based settings. Because the progress value is computed from the latent world, it never depends on the agent's policy; it is a property of the environment state alone. This decoupling ensures that the reward remains a valid learning signal regardless of the policy being trained, and it can be combined with standard temporal‑difference or Monte‑Carlo returns without instability.
Tev‑27B Generalizes Across Computing and Science Benchmarks
The resulting model, Tev‑27B, is trained exclusively on ALE‑related data but nonetheless generalizes. On the Agents' Last Exam computing and math‑sciences domains (ALE‑CM) it achieves Pass@5 scores that match GPT‑6 Astra and Claude Opus 5. The ALE‑CM benchmark comprises a suite of problems drawn from programming, algebra, and scientific reasoning, each requiring multiple reasoning steps and tool use. Tev‑27B's Pass@5 of approximately 68 percent places it at the top of the open‑source leaderboard for this domain, narrowing the gap to the proprietary systems it matches.
More striking is its reach on Terminal‑Bench subsets. Tev‑27B surpasses GLM 5.2 and Gemini 3.7 Flash on TB 3.0 and TB 4.0, and on middle‑horizon tasks such as TB 2.1 it attains performance at the level of Claude Opus 4.8. Terminal‑Bench is a collection of long‑horizon evaluation tasks that span software engineering, theorem proving, and symbolic reasoning. The fact that a model trained only on ALE data transfers to these structurally different benchmarks demonstrates that the latent‑world construction and PPRS training recipe produce a representation of general problem‑solving skill rather than narrow domain‑specific memorization. The paper attributes this transferability to the fact that the latent world captures reusable sub‑routines—such as "plan a sequence of sub‑goals" or "invoke a tool"—that are applicable across many task families.
The paper also reports ablations of PPRS, showing that without the shaped reward the same model drops to below 30 percent Pass@5 on ALE‑CM, and that increasing the shaping constant λ beyond a moderate range degrades performance because the agent over‑commits to intermediate steps that appear progress‑positive under the specification but are actually misguided. These results confirm that the reward shaping must be carefully calibrated and that the latent world must be sufficiently faithful for the benefits to persist.
Constraints of Latent‑World Construction and Reward Generalization
The latent‑world construction step demands a complete and correct hidden specification for each task domain. In domains where the underlying dynamics are only partially observed or where the state space is high‑dimensional, building such a specification may require substantial expert input. For example, a task involving unstructured manipulation or perception‑heavy pipeline steps may have latent states that are difficult to articulate in a discrete, verifiable form. The authors acknowledge that the benefit of PPRS depends on how well the latent world captures the task's true variability, and that downstream domains will need their own specification pipelines. Without a sufficiently faithful latent world, the progress metric becomes noisy or misleading, and the shaped reward can actually harm learning by reinforcing incorrect intermediate behaviors.
Additionally, the progressive rendering that generates Tev‑Gym is batch‑oriented; real‑world tasks that exhibit stochasticity or unmodeled disturbances may not map cleanly onto a deterministic latent world. The paper notes that the dataset covers seven domains, but the total number of tasks is modest compared to the space of all possible long‑horizon problems. If a domain introduces randomness that is not captured in the latent specification, the rendered environment will either omit the stochastic element or produce a progress signal that does not reflect the true likelihood of success. This limitation means that PPRS works best in domains where the forward dynamics are largely deterministic or where the stochasticity can be abstracted into a small set of latent states.
Finally, the paper observes that the computational cost of constructing a latent world scales with the complexity of the task description. For very large or unstructured domains, the time and expertise required to produce a correct specification may offset the benefits of denser rewards during training. The authors suggest that future work could automate parts of this process using large‑language‑model prompting or by mining latent structures from existing demonstration datasets, but such automation is outside the scope of the current study.
How Developers Can Apply the Reward‑Shaping Pipeline
For developers who need to train agents on long‑horizon tasks, the paper offers a concrete pipeline. Begin by defining a latent specification for your task domain—describe the workflow, the expected state after each step, and the conditions that signify progress. The specification can be expressed as a sequence of tuples (action, latent‑state‑target, progress‑indicator), where the latent‑state‑target is a description of what the world should look like after the action is executed, and the progress‑indicator is a scalar that quantifies how far the current state deviates from that target. This description does not need to be exhaustive; even a partial specification that covers the critical sub‑goals is sufficient to generate a meaningful progress signal.
Then render that specification into a progressive environment and compute per‑step progress values. The renderer takes the latent‑state‑target and maps it to the observable world, generating a snapshot that the agent can perceive. At each step the agent compares its current observable state to the target and receives a progress value, typically a number in [0,1] where 0 means the state is far from the target and 1 means the state matches the target within tolerance. These progress values are then fed into a reward‑shaping loop using any off‑the‑shelf policy trainer—PPO, SAC, or any differentiable planner. The shaped reward is combined with any original task reward (often zero) and used as the learning signal for the next policy update.
The resulting agent can learn from intermediate successes where previous systems would have waited for a single terminal reward. Because the shaped reward provides a dense gradient, the policy can allocate credit to early actions that contribute to later steps, effectively reducing the credit‑assignment problem that plagues long‑horizon learning. The Tev‑Gym release provides 1,170 ready‑made environments across seven domains, serving as both a benchmark and a template for adding new domains. Developers can start by selecting an existing environment, adapting the latent specification to their own task, and running a few training epochs to observe the impact of PPRS on learning curves.
Practical considerations include choosing the shaping constant λ. A common starting value is 0.1, which gives the shaped reward modest influence; it can be increased or decreased based on learning curve behavior. If the agent ignores the shaped reward, λ should be increased; if the agent becomes over‑confident in incorrect intermediate steps, λ should be decreased or the latent world refined. The paper also recommends monitoring the proportion of steps in which progress increases, as a sharply decreasing trend may indicate that the latent world is mis‑specified or that the policy is exploiting the shaping signal rather than solving the task.
The Future of Dense‑Reward Long‑Horizon Agents
The combination of a verified latent world and a dense‑reward shaping function opens a route toward agents that can operate reliably over hundreds of steps without the need for hand‑crafted reward engineering. As more domains adopt the latent‑world format, the pool of trainable long‑horizon tasks should expand, bringing the vision of autonomous recursive improvement closer to reality. The Tev‑Gym dataset, with its 1,170 verified environments, serves as both a testing ground and a template; any domain that can be described as a sequence of latent states and observable transitions can be added, extending the reach of PPRS beyond the seven domains currently covered.
Future work may explore automatic latent‑world generation from unstructured demonstration data, hierarchical latent worlds that group steps into macro‑actions, and integration with other training tricks such as curriculum learning or offline model‑based refinement. The paper’s core insight—that a verifiable specification can be turned into a dense learning signal—should scale to many problem families, particularly those where the forward dynamics can be approximated or abstracted at a suitable grain size. As the community acquires more such specifications, the bottleneck that currently limits open‑source models on economically valuable long‑horizon tasks may finally be removed.
Read the paper on arXiv