StateM wraps an unmodified CLI agent in a small state machine: a YAML runbook with phases, checked transitions, and recovery rules. The same model gains 9 points on Terminal-Bench 2.1, the frozen runbook transfers to the next model generation for a raw 95.3%, and an adapted DeepSeek run reaches a comparable score for about $15 instead of $575.
Long-horizon agents fail even when the model can solve each step. The agent loses track of mutable state. It must reconstruct what is done and what is pending from an append-only transcript. It forgets lessons from earlier runs. It skips known procedures. It stops before the work is verifiably complete. Real workflows and benchmarks grade the completed job, not the agent's knowledge of the steps.
The common response is to improve the model: more pretraining, more reasoning, more agents. The authors ask a different question: how much apparent model failure is failure of the harness that maintains state, checks progress, and recovers from errors? They call the systematic improvement of this layer harness scaling.
StateM is a runtime around a general-purpose CLI agent such as Codex or Claude Code. The model weights do not change. The control layer is a human-readable YAML runbook: a state machine over coarse phases of work, for example plan, execute, verify, repair, handoff. The agent keeps its full freedom inside a phase. The runtime controls the transitions between phases.
Each state is a context boundary and a contract boundary. On entry, a hook refreshes the phase instructions and the durable progress record. The agent does not have to infer its current phase from a long transcript. To leave a state, the agent requests a transition. The runtime then verifies that the edge exists, runs the state's exit checks, persists progress, and commits the transition only if every required check passes. A failed check keeps the run in the current state, where the agent can inspect the unmet condition and repair it.
Not every check gives the same strength of evidence, and the paper is explicit about this. A command check runs a real command on the machine and gives reproducible evidence. A checklist only records that the agent confirmed each item itself. An LLM review is a judgment, not verification. The agent's own claim of completion never counts as proof.
The run state lives outside the model context. After a restart, or after compaction shrinks the transcript to fit the context window, the agent asks the runtime for its phase, its history, and its unresolved obligations, and resumes. A stop hook rejects premature exits: if the run is not in a terminal state, the runtime returns the unmet obligations and tells the agent to continue.
The runbook is a normal file in the workspace. The agent operates it through the CLI. The user reads, edits, and versions the same artifact. This makes the harness itself an optimization target: after a run, failures become versioned prompts, checks, and repair paths. The authors call this failure-driven harness optimization. The accumulated rules are procedural memory stored outside the model weights.
Terminal-Bench 2.1 has 89 terminal tasks; the authors run five trials per task. With GPT-5.5 xhigh, StateM raises accuracy from the 83.1% reference to 92.1%. For comparison, the reference harness gains only 1.8 points from a full model-generation upgrade. The harness change is larger than the model change.
The runbook, developed on GPT-5.5 and then frozen, transfers to GPT-5.6 without modification. GPT-5.6 Sol xhigh rises from 84.9% to a raw 95.28%, with at least one success on all 89 tasks. The lower-cost GPT-5.6 Luna rises from 76.7% to 85.4%, above the stronger model's reference score.
The largest gap is in cost. Exact transfer across providers fails: the frozen GPT profile drops DeepSeek-V4 Flash from 82.7% to 82.0%. But adaptation is cheap. For $37 of API spend, the practices are rebuilt for DeepSeek, and the system reaches 88.1% on the full suite and 89.1% on a disclosed 88-task core. The final scoring run costs about $15; the public GPT reference submission cost $574.68.
The limits matter. The 95.28% is a raw public submission, before adjudication: the benchmark maintainers had not yet ruled on flagged runs, and the authors agree that some flagged runs should not count, which gives 94.4% or 93.3%. The measured system is the runtime plus an evolved, benchmark-adapted profile; the numbers do not isolate the state-machine abstraction alone. The reference rows are public scores, not version-matched A/B controls. On BusinessBench, a second benchmark where the runbook stayed frozen, the aggregate gain is small: +0.55 points averaged over task families and +1.34 points averaged over tasks. Two families whose work matches the runbook's phase structure gain 10 points. Two families get worse: the phase gates sit at the wrong points in those workflows, so the runbook adds procedure without protecting the conditions that matter. The authors also show that harness learning can absorb verifier conventions that the task never states: the profile learns to agree with the verifier, not with the specification.
If you run coding agents on long tasks, the transferable ideas are concrete. Keep phase state in a durable store outside the transcript. Gate each phase handoff on machine-checked evidence, for example a request against the live service or a test run, and never on the agent's statement that it finished. Write postmortem lessons as executable checks in a versioned runbook, not as prose in a memory file. Filter before you persist: the BusinessBench results show that accumulated procedure can make an agent worse. And consider the economics: a harness adapted to a low-cost model moved that model near a frontier score for a small fraction of the price. The runtime and several runbooks are open source on GitHub.