Context Engineering: The Discipline That Makes AI Agents Actually Reliable

Context engineering is the discipline that explains exactly why most AI agents fail in production. A 2023 study by Liu et al. (arXiv:2307.03172) found GPT-3.5-Turbo's multi-document question-answering accuracy drops by more than 20% when the one relevant passage lands in the middle of the context. Three years later, that finding is still the most underrated reason agents break. Not the model. The context.

The gap has a name now. Context engineering treats everything you feed a model before it acts, and everything you carry forward between steps, as a designed system instead of a pile of text. This guide walks through the layers that break, the one theorem that explains why, and the playbook I run on my own agents.

What context engineering actually means (and what it isn't)

Prompt engineering optimizes the words you send once. Context engineering optimizes the entire state the model sees on every single turn. The system prompt, the facts you retrieved, the tool schemas, the results from the last step, and the history you chose to keep or drop. The prompt is one input. The context is the whole working memory.

The term only formalized in the last year. "Context Engineering 2.0" (arXiv:2510.26493, Oct 2025) and "Monadic Context Engineering" (arXiv:2512.22431, Dec 2025) both treat context as the engineered surface of an agent, not a side effect of the prompt. Anthropic's own agent guidance points the same way: the model is reliable only as long as the state you hand it is.

The context engineering layers most builders skip

Most agent tutorials show you a while True loop, a tool call, and a print statement. They never show you the five things sitting inside the context on every turn, and those are where reliability is won or lost.

1. The system prompt. What the agent is allowed to do, and what it must never do.
2. Retrieved knowledge. The facts pulled in at runtime, with or without a source attached.
3. Tool descriptions. The schema the model reads to decide which tool to call and how.
4. Working memory. The scratchpad and history carried between steps.
5. Selection and compaction. The logic that decides what gets dropped when the window fills.

Skip any one of these and the agent still runs. It just fails in ways you cannot reproduce, because the failure lives in state you never designed.

The Root Theorem of context engineering

There is a clean result that explains every tactic in this post. "The Root Theorem of Context Engineering" (arXiv:2604.20874, Mar 2026) states it plainly: every system that keeps an LLM conversation alive past a single turn faces two inescapable constraints. The context window is finite. And information quality degrades as accumulated volume grows.

That is the whole game. Every retrieval step, every summary, every compaction policy is a tactic against those two walls. You cannot make the window infinite, so you decide what deserves the space. You cannot stop quality from decaying, so you decide what gets refreshed and what gets dropped.

How context rot actually breaks agents

This is not academic. "TRACE: Trajectory Attribution for Automated Context Engineering" (arXiv:2608.09153, Aug 2026) shows production agents fail when their context sources contain errors or gaps. The paper names the culprits directly: system prompts, knowledge bases, tool descriptions, and procedural skills. Maintenance by hand is the bottleneck, because as interaction volume grows, no one reviews the logs in time to catch a stale fact before the agent acts on it.

I see the same shape on my own stack. The bug is rarely "the model was wrong." The bug is "the context we handed the model was wrong, and we never logged what we handed it." If you cannot replay the exact context from a failed run, you are debugging blind. On one agent I run, a single stale tool description caused 90% of the failures for two weeks before anyone thought to diff the context instead of the prompt.

The context engineering playbook I run in production

Here is the operational version. I apply it to the 14 production agents I run on a single Hetzner VPS, and to the self-hosted runtimes I moved to after leaving third-party APIs. The rules are boring on purpose.

• Keep the system prompt under a hard token budget. If it grew past what fits on one screen, it is doing too many jobs and you should split the agent.
• Write tool descriptions like API docs a stranger would read, not like notes to yourself. The model only knows what the schema tells it.
• Cap history with a summary step, never blind truncation. Blind truncation is how you lose the one fact the next step needed.
• Retrieve on demand and cite the source inline so the agent can re-check it instead of trusting a summary.
• Log the full context on every failure. You cannot debug a state you did not record.

One more rule, because it trips almost everyone: compact on a schedule, not when the window is already full.

• Compact aggressively but always on a schedule. Pick a step count, summarize, and replace. If you wait until the model starts dropping the middle, you have already lost the fact the next turn needed.

If tool calls are where your agents break, the fix usually starts in the tool description, not the model. My writeup on tool-call verification covers how I prove a call actually did what it claimed. And if your agent cannot use tools safely, the root cause is often context design, which is exactly what MCP fixes.

How I measure context health

You cannot improve what you do not measure. Two numbers tell me most of what I need.

First, the retrieval hit rate: of the facts an agent cited in a session, how many came from a source it attached? When that rate drops, the agent is trusting summaries it wrote itself instead of the originals. Second, the compaction loss rate: after a summarize-and-replace step, what fraction of carried facts resurface correctly in the next ten turns? A jump in failures usually shows up here a day before it shows up in eval scores.

I keep both as a one-line log per run. It costs almost nothing and turns "the agent got weird" into a specific, fixable sentence.

The honest part

Context engineering is boring. It is not the model. It is the plumbing. On my own stack I would estimate the prompt is maybe 20% of agent reliability, and context design is most of the rest.

None of it is exciting. All of it is necessary. The teams shipping demos treat context as an afterthought. The teams shipping agents people trust treat it as the product.

Where to start this week

Pick one agent. Print the context it actually sends on a hard turn. Count the tokens you cannot justify, and cut them. Then add one retrieval step that cites its source inline, and turn on full-context logging for every failure. That is the whole discipline in an afternoon.

If you want the memory half of this, read my breakdown of agent memory production failures, and if you are still on hosted APIs, my case for self-hosting explains why owning the runtime makes context engineering survivable.

Teams that treat context as an afterthought ship demos. Teams that engineer it ship agents people trust.