ContextPipe routes every model call through a five-phase pipeline modeled on a database query executor: a catalog of context sources, a deterministic cache-aware optimizer, and an EXPLAIN ANALYZE trace. On three SWE-bench Pro tasks it cut total prompt tokens by 30 percent and model calls by 17 percent, but more than doubled uncached tokens and lowered the cache-hit rate.
An agent runs in a loop. Each turn, the runtime packs the system prompt, tool schemas, conversation history, retrieved memory, and skill instructions into one model call. The call has a hard size limit, the context window. The provider charges per token and caches any prefix that is byte-identical to an earlier request: repeat last turn's opening bytes and the request is cheap; change one byte early and the cache is lost for everything after it.
The authors say production runtimes handle this with scattered code: a prompt builder per session type, ad hoc compaction routines, provider-specific cache markers, and tool-output injection. Each piece makes a local decision, so nobody can explain the combined result or say which change broke the cache. Order matters too: long-context models give less weight to the middle of the prompt, so section order changes quality, not only cost.
A relational database takes a declarative query, reads a catalog of tables and statistics, and produces a plan. The plan respects a hard memory limit, uses a cache of pages already in memory (the buffer pool), and can report what it did and what each step cost (EXPLAIN ANALYZE). Context assembly has the same shape, the authors argue: the user message is the query, the session state is the catalog, the context window is the memory limit, and the prefix cache is the buffer pool. ContextPipe routes every model call, including sub-agent and compaction calls, through one pipeline with five phases.
Plan is a pure function over a snapshot of the catalog and statistics: no I/O, and the same inputs give the same answer. It computes context pressure: current input tokens divided by the model window minus a reserve for the expected response. The reserve is the 75th percentile of past response lengths, grouped by model and query source, or the 95th after a recent overflow. An average would be too optimistic for long-tail coding turns and cause prompt-too-long retry loops. Plan then picks a compaction tier and a token budget per section.
Bind is the only phase that does I/O. It fetches memory, history, tool schemas, skills, and items carried over from the last turn, all concurrently. Each result is a typed artifact, not a string, so later phases can respect provider rules such as keeping each tool result next to its tool call.
Optimize arranges the sections and shrinks them when the tier requires it. It does not search for the best layout, which has no known fast algorithm; it applies a fixed policy. Each section has a cache scope that says how often it changes: Global (same for every session), Session (same for the whole session), or None (changes every turn). Sections are sorted in that order, least volatile first. Two anchors, Identity (who the agent is) and Constraints (its rules), stay at the front and are never compressed. Cache markers go at the scope boundaries, so the marked prefix is byte-identical turn after turn.
The compaction tiers escalate in fixed steps. TrimSchemas removes tool schemas the turn is unlikely to use. CompactHistory clears the oldest tool results and replaces each with a placeholder, up to a cap on tokens cleared per turn. AggressivePrune also drops the oldest whole conversation rounds. Within a turn a tier can only go up, and a recent overflow error pushes it up further. Each transform sits behind a boolean gate; a blocked transform is logged with the reason, so the trace shows what ran and what did not.
Execute serializes the artifacts once and sends the request. Feedback runs after a successful call and records actual tokens, cache reads, and the cause of any cache break, found by comparing prompt, tool, model, and latch hashes with the previous turn. A failed call writes one failure record and no usage sample, so a broken turn cannot corrupt the response-length statistics.
ContextPipe registers every context source with a lifecycle tier, cache scope, compression priority, and estimated token cost. Two tiers are new. A session latch is a value such as a beta header or a provider feature flag, evaluated once and then frozen for the session, because a mid-session change would invalidate the cache prefix. Emergent context is content discovered mid-turn, after Plan ran, such as a tool result that triggers a new skill. It is queued for the next turn with a one-turn time-to-live, a content hash, and a cap per list, so a replay cannot inject a stale attachment twice.
Every turn emits an EXPLAIN ANALYZE record: pressure, tier, planned and actual tokens per section, compaction actions, marker positions, and the real cache reads joined back in. Eight alert rules run over each trace, among them a cache break with its cause, and a compaction cascade (two compactions in three turns, or three in ten). The authors state three properties, with short arguments from the construction rather than formal proofs: every applied and skipped transform is logged, the same inputs produce byte-identical output, and a failed call leaves persistent state unchanged.
Because any assembly change can break the cache, the paper requires a shadow rollout. The new pipeline runs beside the old path on every turn, but only the old path sends the request. A diff checker compares block and schema hashes, message counts, and marker positions. Rollout moves through four stages: shadow-only, shadow-verified, flip, retire.
The evaluation is small: 3 of the 79 instances in the Qutebrowser subset of SWE-bench Pro, a coding benchmark, three repeats each, with DeepSeek-V4-Pro. It compares the full pipeline with the same pipeline with every optimizer gate closed.
The full pipeline sent 731 thousand total prompt tokens against 1,045 thousand, a 30 percent reduction. It made 25.0 model calls per run against 30.2 and finished in 188 seconds against 204. But uncached tokens more than doubled, from 46.6 thousand to 100.0 thousand, and the cache-hit rate fell from 95.3 to 86.2 percent. Every compaction changes the sent prefix, so every tier crossing costs a near-total cache miss on one call. The paper's own trace shows it: 8 percent hit rate on the call where TrimSchemas engages, then 93 to 100 percent for the next ten calls.
Billed cost depends on the provider's discount for cached tokens. At a DeepSeek-like discount, the gate-closed baseline is about 11 percent cheaper even though it sends 43 percent more context; the full pipeline wins on cost only when a cached token costs more than 14.5 percent of a fresh one. The authors frame the gain as context volume, calls, time, and window headroom, not cost. The abstract quotes a 23 percent call reduction and 9 percent time reduction; the results table gives 17.2 and 7.8. Task quality is reported only as a tie, which the authors call exploratory, and four planned tests that remove one mechanism at a time have not been run.
You do not need ContextPipe to use its discipline. Give every prompt section a cache scope and a compression priority, sort by volatility, and put cache markers at the scope boundaries. Size the response reserve from a percentile of past responses and raise it after an overflow. Gate every compaction step and log whether it ran and why not. Alert on compaction cascades and attribute each cache break to the section whose hash changed. Ship changes to prompt assembly in shadow mode with a byte-level diff. And check billed cost under your provider's cache pricing before you call a compaction ladder a win.