Compaction and the Summarization Trap

Compaction frees context space by condensing history, but over-compacting drops details the model still needs.

Context & Reliability·Lesson 3 of 6·8 min

As a conversation or agent session runs long, the context window fills up with history: prior user turns, tool calls, tool results, and model reasoning. Compaction is the practice of condensing that older history, typically by summarizing it, to free up room for the conversation to continue without hitting the context limit. It's a necessary technique for any long-running agent, but it is also one of the most common sources of silent, hard-to-diagnose failures in production agentic systems.

What compaction is for

Without compaction, a long-running agent session simply stops once the context window fills, forcing a hard restart that loses all prior context. Compaction avoids that cliff by periodically replacing a block of older turns with a shorter summary that preserves the gist while shedding the token cost of the full transcript. Done well, this lets an agent run for hours or across many tool calls while staying within budget.

The summarization trap

The failure mode, often called the summarization trap, is over-compacting: condensing history so aggressively that details the model still actually needs get dropped. This doesn't fail loudly. The agent doesn't crash or throw an error. It keeps producing responses, but those responses become progressively vaguer, more generic, or subtly wrong, because the specific facts that used to ground them (a particular file path, a specific error message, an exact numeric threshold a user specified earlier) were summarized away in favor of a high-level gist. This is worse than a hard failure in some ways, because it's easy to miss in a review and easy to attribute to "the model getting worse" rather than to a context management decision.

A concrete pattern: an agent debugging a production issue is told the exact stack trace and the exact line number where an exception originated in turn 3. By turn 40, aggressive compaction has folded that into a summary like "investigated an error in the payment module." The agent now reasons about "the payment module" in the abstract instead of the specific function, and its later suggestions become generic advice rather than the targeted fix the original detail would have supported.

What to preserve when compacting

Good compaction strategy is selective rather than uniform. Details with high reuse value — file paths, exact error text, IDs, numeric parameters, explicit user decisions or constraints — should be preserved verbatim or extracted into a structured note rather than folded into prose summary. Purely conversational scaffolding — pleasantries, exploratory back-and-forth that led nowhere, redundant restatements — is safe to compress hard. The judgment call the exam tests is whether you can distinguish "this history is safe to compress" from "this history contains facts the agent will need to reference precisely later."

Exam trap A question describes an agent session where compaction is applied uniformly to all history older than N turns, and later the agent gives an unhelpful, vague answer to a question that depends on a specific detail from an early turn. The trap answer blames the model's reasoning ability or suggests using a bigger/better model. The correct diagnosis is a compaction failure: the summarization step dropped a load-bearing detail. The fix is changing what gets preserved during compaction (or how aggressively/uniformly it compresses), not swapping models.

Scenario: A coding agent has been working on a refactor for 90 minutes across many tool calls. Early on, the user specified "do not touch anything under the /legacy directory, it's frozen for a migration." At turn 60, a compaction pass summarizes turns 1-40 into a few sentences about "general refactor goals," and the constraint about /legacy is lost. At turn 75, the agent proposes a change that touches a file under /legacy. What went wrong, and how should compaction have been designed? The constraint was a hard user directive with binding force for the rest of the session, not a detail safe to fold into gist-level summary. Compaction should extract explicit constraints like this into a persistent, separately-maintained "active constraints" note that survives every compaction pass untouched, rather than letting them get swept into the same lossy summarization applied to routine exploratory history.

Try it

Take a long transcript from an agent session you have access to (or simulate one). Manually write a compacted summary of the first half as you'd expect an automated system to produce it, then list every fact in the original transcript that your summary dropped. For each dropped fact, decide whether a later turn in the transcript actually depended on it — that exercise is exactly the audit a good compaction system needs to pass.