Code is the harness: how agents reason, act, and coordinate through executable state
A 42-author survey from UIUC, Meta, and Stanford argues that code is the agent harness — executable, inspectable, stateful — and maps the full stack: a Plan–Execute–Verify control loop, telemetry-driven harness revision, and explicit shared program state as the fix for multi-agent brittleness.
The problem
An agent harness is the software layer around a language model: the tools, sandboxes, memory systems, validators, permission rules, and feedback channels that turn a stateless model into an agent that can work on a task for hours. Teams that build coding agents now spend much of their effort on this layer. But the research literature has treated code mainly as the thing the model produces. Knowledge about the system around the model sits scattered in blog posts, vendor guides, and leaked system prompts, not in one organized reference.
The authors, 42 researchers from UIUC, Meta, and Stanford, collect this material into a single survey. Their claim is stronger than “harnesses matter”. They argue that code itself is the harness: agents reason through programs, act through generated scripts, sense their environment through tests and traces, and coordinate through shared repositories. When an agent behaves unreliably, the cause is often in this loop, not in the model.
The idea
The survey names three properties that make code work as agent infrastructure. Code is executable: the harness can run the model's output and check the result against the goal. Code is inspectable: intermediate computation appears as structured traces that the harness can read, store, and act on. Code is stateful: the growing program records task progress in a durable form that later steps can read and change.
The authors also separate three parts of an agent system. Model-internal capabilities come from the weights. Harness infrastructure — tools, sandboxes, memory, validators, permission boundaries — is built by humans in advance. The third part is the least studied: agent-initiated code artifacts. These are the regression tests, temporary tools, small domain-specific programs, and reusable skills that the agent writes for itself during a task, executes, observes, and sometimes keeps. The survey places these artifacts at the center of its view.
The material is organized in three layers. The interface layer covers code as the medium for reasoning (programs that hand computation to an interpreter, or machine-checked proofs), for acting (generated scripts, reusable skill libraries, GUI automation), and for modeling the environment (repositories, tests, simulators, and execution traces as the world state). The mechanisms layer covers planning, memory and context engineering, tool use, control, and optimization — what keeps an agent reliable past one generation step. The scaling layer covers multi-agent systems that coordinate through shared code artifacts.
The control loop
The most useful reframing is of debugging. The authors describe the harness as a governor: a control layer that observes the effect of each agent action and regulates the next one. They name the loop Plan–Execute–Verify (PEV).
In the Plan phase, the plan is a contract, not a hidden reasoning step: it states which files the agent may edit, which invariants must hold, which validation commands define success, and where the rollback points are. In the Execute phase, actions run inside a sandbox under a permission tier: a read-only tier for browsing and inspection, a sandbox-edit tier for local patches and test runs, and a full-access tier (network, credentials, deployment, history rewrites) that requires a human approval gate. In the Verify phase, deterministic sensors judge the result: parsers, type checkers, linters, unit and integration tests, static analyzers, fuzzers, CI pipelines. These signals cannot hallucinate. Model critique interprets sensor output; it does not replace it. The loop stops when the required checks pass, when progress stops, or when the risk tier changes — not when the model states that it is confident.
Harnesses that edit themselves
A late section names a discipline the authors call Agentic Harness Engineering: measure and revise the harness itself, not the model. The input is deep telemetry — structured traces that record prompts, retrieved context, tool arguments, token cost, permission requests, sandbox snapshots, test results, and human interventions. An Evolution Agent reads these traces, attributes failures to specific harness components (a weak validator, a brittle tool schema, a retrieval stage that spends tokens without improving outcomes), proposes a revision, and evaluates it on held-out tasks or replayed traces. A revision is promoted only when it improves reliability without regressing solved cases. The authors insist that a change to the harness is a change to a safety-critical runtime: it needs a change contract, a regression suite, rollback semantics, and human approval when it moves a permission boundary.
What the synthesis shows
The multi-agent section compares systems such as ChatDev, MetaGPT, MapCoder, and L2MAC. Most of them keep shared state implicit: each agent reconstructs the program state from conversation history at each invocation. The authors identify this as the technical root of multi-agent brittleness — without one explicit store of program state, agents cannot detect when their beliefs diverge from the true state. A second pattern follows: systems that keep an explicit store (L2MAC, with a persistent file store) use simple sequential topologies, while systems without one build elaborate adaptive topologies to compensate. Complex orchestration, in this reading, is often a symptom of missing state, and heavy context management is the cost of it.
One reported result cuts against the view that only real execution gives ground truth: two cited systems report that model-simulated execution predicted real execution outcomes with more than 98% precision and recall, without running the code. The authors conclude that execution feedback is most valuable for the failure modes that language cannot simulate — crashes, resource exhaustion, boundary conditions, performance regressions — and that a mature harness would use model-based checks as the fast path and real execution as the final check.
Caveats: this is a survey and position paper. It runs no new experiments; every number comes from the cited papers under their own conditions, and the 98% simulation figure comes from two systems on their own benchmarks. The taxonomy is the authors' construction, and the explicit shared state they call for is a proposal, not a built system.
What you can do with it
Use the taxonomy as a diagnostic index. When your agent fails, locate the layer: interface (can the agent see and act on real state?), mechanism (planning, memory, tools, verification), or shared state (do two agents hold different beliefs about the repository?). Then use the relevant section as a reading list — the companion GitHub repository indexes the cited papers.
The PEV framing converts directly into engineering practice: make plans artifacts that carry validation commands; put deterministic sensors ahead of model critique; tie loop termination to verification, not model confidence; and record deep telemetry from the start, because it is the input for any later harness improvement. For multi-agent designs, the survey's clearest advice is structural: make shared state explicit and queryable before you add roles or topology.