A controlled study switched three coding-harness parts on and off across four models and 176 settings. No part helped everywhere. Context management mattered only when the window was small, planning raised accuracy for the weakest model but only cut cost for the strong ones, and a bare shell beat a typed tool set for the most capable model.
A coding agent is a model inside a harness. The harness is the software loop that sends the model its history, runs the tool calls the model emits, and appends the results. Real products bundle a planning scaffold, a tool set, and a rule for a full context window, and benchmarks score the whole bundle. When one harness beats another, the score does not say which part did the work. The authors ask: is each harness part useful in general, or does its value depend on the model, the task type, and the token budget?
The authors built a small harness from scratch so they could change one part at a time. The loop is fixed. Each turn, the model reasons and emits tool calls, the harness runs them in the task container, and the output goes into the history. Three parts can be switched.
Planning. A system instruction asks the model to write a plan on the first turn and to keep it current through an update_plan tool. The harness injects the current plan into every model input. With planning off, the instruction, the tool, and the injection are all removed.
Action space. Either a typed tool set (file read, write, and edit; list, glob, and grep; web fetch; and bash) or bash alone.
Context management. Five tiers, T0 to T4, built from three mechanisms. Elision replaces the body of an old tool output with a short stub. Recall stores the elided body on disk and gives the model a tool to read it back. Summarization makes a separate call to the same model to fold the oldest messages into a running summary. T0 uses none of them, and the run dies when the window overflows. T1 uses elision. T2 adds recall. T3 uses summarization alone. T4 uses all three in stages: elide when the history passes 60% of the window, summarize the oldest part when it passes 85%. The system prompt, the task, and the last two turns always stay verbatim.
The models are three sizes of Nemotron-3 (30B, 120B, 550B) and Mistral-Medium-3.5-128B. The benchmarks are SWE-Bench Verified (500 real GitHub issues, all Python) and Terminal-Bench 2.1 (89 command-line tasks). The context window is capped at 32k, 64k, 96k, or 128k tokens. That gives 176 settings, each run once per task.
At a 32k window, no management is fatal. On SWE-Bench, T0 success rates sit between 6% and 13% across the four models. Any managed tier lifts them to between 20% and 66%. Nemotron-3 550B goes from 6.4% to between 51% and 58%. At 128k the gap shrinks to a few points: the same model scores 59.8% with T0 and between 65% and 67% with management.
Every managed tier had zero overflow failures at every budget, and T0's loss tracks its overflow rate, which falls as the window grows. At 32k, median SWE-Bench runs under T0 end after 20 to 30 turns, mostly while still locating the bug. Under management they run for 50 to 180 turns and reach verification.
Among the managed tiers, accuracy is similar but cost is not. T4 had the lowest mean cost per task in seven of the eight model-benchmark pairs. Cheap elision runs first, so the harness makes fewer summarization calls and peak context stays lowest. Recall did not help. T1 and T2 differ only in recall, and across 32 paired comparisons T2 won 15, lost 14, and tied 3. In 36 of the 64 settings that offered recall, the model never called it.
With planning on, Nemotron-3 30B goes from 13.6% to 25.2% on SWE-Bench and from 9.0% to 13.5% on Terminal-Bench. Cost goes up, by 75% on Terminal-Bench. Without a plan, 68.6% of its SWE-Bench runs end without editing any file, and the median run lasts 5 turns. With a plan, those numbers are 27.8% and 40 turns. The plan keeps the model working long enough to attempt an edit.
For the strong models, the effect reverses. Nemotron-3 550B drops from 67.8% to 65.8% on SWE-Bench, but its cost per task falls from $3.31 to $2.33. Mistral drops from 69.0% to 68.6%, and its cost falls from $4.65 to $3.14. The removed turns are almost all checks the model makes after its edit is already done. The median 550B run shortens from 108 to 74 turns. A plan tells a capable model when it is done.
Nemotron-3 30B needs the typed tools. With bash only, its SWE-Bench score falls from 25.2% to 10.2%. The model emits tool calls it learned during training, the bash-only harness has no such tools, and the run ends.
Nemotron-3 550B does better with bash alone. Its SWE-Bench score rises from 65.8% to 69.4%, and its cost per task falls from $2.33 to $1.11. The shell lets it bundle several operations into one command. Re-patches of an already edited file fall from 4.6 to 1.5 per task.
Mistral splits by task type. On SWE-Bench the typed tools win, 68.6% against 45.4%, and 32.8% of its bash-only runs end without editing any file. On Terminal-Bench bash wins, 43.8% against 37.1%. Terminal-Bench tasks are mostly shell work, so typed tools compete with the actions the model prefers there.
Each setting ran once per task, and Terminal-Bench has only 89 tasks, so many of its contrasts are not statistically significant. Planning and the action space were switched only at the 128k, T4 setting. The bash-only switch also removes the read-before-write check and the automatic lint pass, so it measures a whole interface, not tool count alone. The models are open-weight mid-tier models, not frontier models. The authors say every crossover point (the level of model capability at which a part stops helping and starts hurting) should be re-measured before it is applied to another model family.
Measure your overflow rate first. If your agent runs rarely hit the window, a compaction scheme adds little accuracy. If they do, the choice between schemes is about cost.
Stage cheap before expensive. Elide stale tool outputs by rule first. Call the model to summarize only when elision is not enough.
Do not build recall until a trace shows the model asking for it. The models in this study did not use it.
Treat planning as a cost control for capable models. Measure its effect on the verification turns after the edit, not on the pass rate.
Test the tool set per model and per task type. Bash alone halved cost for the strongest model here. Keep typed tools for weaker models.