PILOT pairs a worker agent with a supervisor that watches a live event channel, can steer or abort the worker mid-run, and writes reusable skills during the run. On Terminal-Bench 2.0 it beats single-agent harnesses by about 5 points one-shot, and gains 12 to 15 points over 20 self-improvement sweeps while cutting output tokens by nearly half.
A long-horizon agent task is one where the agent runs a shell or edits a repository for many steps, often for hours. During such a run the agent produces experience: a working procedure, a project convention, a dead end. Most self-improvement methods use this experience only after the run ends. A reflection step, a judge, or a self-evolving harness all work from the completed run. None of them can save the run that produced the lesson, and the lesson is tested only in a later run.
The authors say the cause is an architectural gap. A single-agent loop can correct itself, but one context holds both the tool output and the judgement about that output. Subagent delegation separates the work from the main agent, but the main agent gets only a final summary when the subagent returns. It cannot redirect a subagent that is still running.
PILOT is a supervisor-worker harness. One model fills both roles; its weights never change. The worker executes the task in its own isolated context. The supervisor never touches the task. It holds only the goal, the recent events, and the signs that the run has a problem. Its context stays small; the worker's exploration, dead ends, and verbose tool output never enter it.
The two agents share a live channel with three worker events and two supervisor actions:
Notification. The worker reports progress, an intermediate result, or a risk, then continues. Question. The worker asks for input and pauses until the supervisor replies. Result. The worker's final output, delivered when it finishes. Steer. The supervisor inspects the relevant part of the worker's transcript and queues guidance. The guidance is applied at the worker's next turn. Abort. The supervisor stops a worker whose run is no longer useful.
The runtime also sends execution errors and inactivity alerts to the supervisor. The supervisor can run several workers at once, or start a new one after an earlier one finishes.
The second mechanism is live self-evolution. When the supervisor sees a reusable procedure, a project convention, or a recurring failure in the live transcript, it writes it to a persistent skill library or memory file. Every worker started after that write loads it, in the same task or a later one. Self-improvement here means only the growth of that library and memory.
In infrastructure terms: the worker is a batch job, and the supervisor is a control plane that watches the job's event stream, can signal the job, and updates a shared configuration store that the next job reads at start.
PILOT is an extension to Pi, an open-source coding-agent runtime. The supervisor and each worker are separate in-process sessions. The code is on GitHub.
The authors test two open-weight backbone models, GLM-5.1 and Kimi-K2.6, on Terminal-Bench 2.0 (89 shell tasks), SWE-bench Multilingual (repository repair beyond Python), and SWE-bench Pro (longer, harder issues), against four single-agent harnesses on the same backbone: Pi, OpenCode, Terminus-2, and Hermes, plus Mini-SWE-Agent on the SWE benchmarks. Each configuration runs twice; the paper reports the mean.
In the one-shot setting, where every task starts from a fresh harness, PILOT ranks first in five of six backbone-benchmark pairs. On Terminal-Bench 2.0 it reaches 71.9% with GLM-5.1 and 71.3% with Kimi-K2.6. The best single-agent harness, Pi, averages 66.3% across both backbones, so the gain is about 5 points. On SWE-bench Pro PILOT averages 59.9% against 55.5% for Pi. On SWE-bench Multilingual with Kimi-K2.6 it comes second to Pi.
The caveats matter. The largest gain the authors report, 9.8 points, is the gap to the weakest baseline, averaged over both backbones, not the gap to the strongest. The authors also inspected every successful Terminal-Bench run by hand and counted how many a steer had rescued (turned from failing to passing). The count is small: 2.3% of successful runs with GLM-5.1 and 10.6% with Kimi-K2.6. No easy task was rescued. Hard tasks were rescued most often: 6.1% and 19.7%. So most of the one-shot gain does not come from a visible correction. The paper does not fully explain where the rest comes from. The isolated worker context alone may help.
The self-improvement setting is the stronger result. The authors run all 89 Terminal-Bench 2.0 tasks as one sweep, then repeat the sweep 20 times. Each sweep starts from the same shared skill library and memory. During a run the supervisor may write skills, but it never sees the verifier, the benchmark's pass or fail check. After the sweep, only the updates from runs that passed are merged into the next shared state. The verifier picks which updates survive. It never writes them.
Under this protocol PILOT's best observed pass rate rises from 66.3% to 80.9% with GLM-5.1, and from 68.5% to 80.9% with Kimi-K2.6. These starting rates are below the one-shot rates above, and the paper does not explain the difference. Pi and OpenCode, started from the same skill library and given the same self-improvement instruction, gain 2.3 and 7.9 points. The skill library grows from 62 to 83 skills and from 50 to 81 skills. Mean output tokens per task fall by 42.9% and 47.4%, and successful evaluations per million output tokens more than double. The gains are largest on hard tasks.
Two more caveats. "Best observed" is the maximum over 20 sweeps, not the final value, and the paper does not give the variance across sweeps. Each configuration repeats the benchmark 20 times, so the authors could afford only two backbones and three benchmarks, and no proprietary models. The supervisor and worker always share one backbone. A stronger supervisor with a cheaper worker is untested.
Two case studies show what a steer looks like. In one, the worker spent twenty minutes tuning a Core War program (a game where programs fight in shared memory) against synthetic opponents. The supervisor told it to stop, find a published program, and test against the real opponents. The worker passed. In the other, the worker edited a linear layer split across several GPUs, where an all-reduce step sums each GPU's partial result. It added the bias before the sum, so the bias was counted once per GPU. The supervisor gave the exact fix, and the worker passed all 13 tests.
If you run coding agents with subagents today, the concrete change is the channel. PILOT adds three things you can build yourself: a notification event the worker emits during the run, a question event that pauses the worker, and a steer message that the supervisor queues for the worker's next turn. Inactivity alerts from the runtime are a cheap addition that needs no model call.
The skill-write protocol is also reusable. Let the supervising agent write skills during the run, with no test feedback. Then gate what persists on an outcome check after the run. The check filters the skills but never enters the writing step.
Set expectations from the hand-counted numbers: a steer rescues a small fraction of runs, mostly hard ones. The larger, repeatable win is the skill library that compounds over a stream of similar tasks, and the near-halving of tokens per task that comes with it.