A model that writes a new agent harness for every task
JIT-Agent is a 27B model that generates a custom agent harness — memory, planner, action loop, and tool policy — for each task at run time. In the authors' tests, an open model inside a generated harness beats the same model inside Claude Code or Codex on most of the tested benchmarks, and it costs less in every test.
The problem
An agent is a model plus a harness. The harness is the code around the model. It decides what history the model sees, how the next step is chosen, which tools the model can call, and when the loop stops. Claude Code, Codex, and OpenCode are harnesses. Engineers design them by hand, and one design must serve every task.
Recent research tries to automate harness design. Most of that work optimizes one durable harness ahead of time: it collects execution traces, edits the harness, and keeps the edits that help. The authors call this the ahead-of-time (AOT) approach. It assumes the next task will look like the past tasks. The authors argue that the best harness depends on the individual task. A wide search task benefits from parallel exploration branches. A command-line task runs best in a lean serial loop. A deep research task needs working memory over collected evidence. One fixed harness cannot match all of these shapes at the same time.
The idea: generate the harness at run time
The authors invert the approach. Instead of optimizing one harness ahead of time, a dedicated model writes a fresh harness for each task, just in time. They name this model JIT-Agent. It has 27B parameters and is trained from Qwen3.6-27B.
To make harness generation tractable, the authors define a fixed protocol with four modules. The memory module compresses the event history into the view the model sees. The planning module turns that view into the next local directive. The capability module selects which tools and skills are exposed at each step. The action module runs the loop: it updates the controller state and emits the next tool call or the final answer. JIT-Agent receives a task description, the available tool registry, and a few reference harnesses. It then emits executable code for all four modules. Any off-the-shelf agentic model then runs inside the generated harness. The paper calls this inner model the backbone.
The four-module protocol is expressive. The authors re-implement 13 published harness designs under it — ReAct, Plan-and-Execute, designs that split a task into recursive subtasks, such as ROMA, and memory-centric designs. This library, named HarnessFactory, supplies the first training data and gives the generator reference material.
Training: generate, repair, evolve
Training has three stages. In stage one, a stronger teacher model writes task-adapted harnesses. A generated harness enters the training set only when it passes protocol validation and executes correctly. JIT-Agent learns to imitate these harnesses. A second objective then trains it to prefer harnesses that score the same or better while they use less time and less money.
In stage two, the failed generations become training data. Each failure carries a diagnostic report: compiler errors, interface mismatches, runtime exceptions. The teacher writes small patches, and JIT-Agent learns to repair a broken harness from its diagnostics in at most two rounds.
In stage three, the authors train evolution itself with an online reinforcement-learning objective they name Evo-GDPO. JIT-Agent proposes candidate harnesses for a task. The candidates run against the best previous harness from an archive. Task score is the primary signal; speed and cost count only when the candidate matches the best previous harness on score. A candidate enters the archive only when it improves the archive's best trade-off of score, speed, and cost. At deployment, the generator's weights stay frozen. In streaming mode, the archive still grows, so experience transfers across a task sequence without any weight update.
Results
The authors evaluate on nine benchmarks in four groups: deep research (for example DeepSearchQA), daily work, planning, and office workspace tasks. Note what is absent: none of these is a coding benchmark.
Three results matter. First, in all 18 tested pairs of backbone and benchmark, a JIT-generated harness improves over the backbone's default harness. With GLM-5.2 as the backbone, the combined system ranks first on seven of the nine benchmarks in the authors' table, ahead of GPT-5.6 and Gemini 3.1 Pro. Second, in a controlled comparison that holds the model fixed and varies only the harness, JIT-Agent achieves the best score in four of six settings against Claude Code, Codex, OpenCode, Hermes, and NanoBot. Third, it is the cheapest harness in all six settings. On DeepSearchQA with DeepSeek-V4-Flash, it scores 85.1 at $0.066 per case; Claude Code scores 79.6 at $0.088; OpenCode scores 75.9 at $0.258. The generated harnesses win with shorter, more selective trajectories, not with more compute.
The limits matter. The benchmarks are search, planning, and office tasks. Claude Code and Codex are built for coding work, and the comparison does not test coding. The two losses in the controlled comparison are real: Claude Code beats JIT-Agent on AgentIF by 3.1 points, and NanoBot beats it on one DeepSearchQA setting. Where the paper states sample sizes, the evaluation subsets are small: 50 to 100 examples per benchmark. The four-module protocol is much simpler than a production runtime; the authors say this simplification is deliberate. All scores are the authors' own measurements.
What the generated harnesses look like
The qualitative section is the most interesting part. For a cross-application office task, the generator produced a harness that turns the request into a dependency graph, runs the nodes that are ready with a limit on parallel work, and stores intermediate results so later nodes do not re-read the transcript. For a multi-hop research question, the same generator produced a different harness: a dynamic decomposer, a generated "delegate" tool that opens a private sub-agent with a five-step budget, and a fact store that extracts key-value facts from the history.
The same generator maps one task to graph execution with an artifact store and another task to recursive delegation with a fact store. The protocol constrains the interfaces, not the behavior.
What you can do with it
The code and the model weights are public (GitHub: bingreeky/JIT). If you run agents on many kinds of tasks, the direct experiment is to put the generator in front of your own backbone and measure it against your fixed harness.
The transferable lesson does not require the model. The paper gives strong evidence that harness choice is task-dependent, and that no fixed harness dominates: each fixed harness in the comparison wins somewhere and loses somewhere. If your agent serves several task shapes, per-task harness configuration — which memory strategy, which planner, which tools are exposed — is a real tuning dimension, and the four-module split is a practical checklist for it. The cost result gives one more lesson: the winning harnesses read less history, not more. Selective context construction beat full transcripts on both score and price.