Why a finite context window degrades quality long before it hard-fails, the levers that keep a long-running agent inside its budget — curation, compaction, retrieval, caching — and the error handling, observability, and evaluation practices that let a production system degrade gracefully instead of failing silently.
What is a context window, architecturally?
Every call to Claude carries a finite input budget — the context window — measured in tokens. The system prompt, the tool/function definitions, the full conversation history sent so far, and anything you've retrieved or injected (documents, prior tool results) all draw from that same shared budget, not separate pools.
"Just put everything in context" is a reasonable default for a short conversation, but it breaks down as conversations run longer and tool results pile up: each new turn either eventually exceeds the window outright, or — well before that — leaves so little room for anything else that the model's attention is spread thin across a mountain of mostly-irrelevant history. A context window failure mode is rarely a hard crash at the token limit; it's usually degraded answer quality well before you get anywhere near it.
The rest of this domain is about the practices that keep a long-running agent inside its budget without losing what it actually needs: deciding what belongs in context in the first place, compacting what's grown stale, retrieving only what's relevant on demand, and reusing what hasn't changed.
Key terms for this domain
- Context window
- The finite number of tokens a model can process as input on a single call; the system prompt, tool definitions, conversation history, and any retrieved content all draw from this same shared budget.
- Context engineering
- Deliberately deciding what enters the model's context at each turn — including only the history, tool results, and documents that are still relevant — instead of accumulating everything by default.
- Compaction
- Condensing older conversation turns or verbose tool output into a shorter summary that preserves what is still needed, freeing context budget without discarding information the agent still relies on.
- Retrieval
- Pulling in only the relevant slice of a larger corpus at the moment it is needed, for example via search or embeddings, instead of holding the entire corpus in context up front.
- Prompt caching
- Reusing a previously processed prefix of context, such as a stable system prompt or tool definitions, so repeated or incrementally extended requests skip reprocessing that unchanged portion.
- Observability
- Logging and tracing what an agent actually did on a given run, including tool calls, results, and why it stopped, so a failure can be diagnosed after the fact rather than merely noticed.
- Evaluations
- A repeatable set of test cases with defined pass or fail criteria, run against an agent or prompt before shipping a change, to catch regressions that informal spot checks would miss.
The context budget
┌───────────────────────────────────────────┐
│ Finite context window (tokens) │
│ │
│ ┌───────────────┐ ┌─────────────────────┐ │
│ │ System │ │ Tool / function │ │
│ │ prompt │ │ definitions │ │
│ └───────────────┘ └─────────────────────┘ │
│ ┌───────────────────────────────────────┐ │
│ │ Conversation history (all turns so far) │ │
│ └───────────────────────────────────────┘ │
│ ┌───────────────────────────────────────┐ │
│ │ Retrieved / injected context (docs, │ │
│ │ tool results, search hits) │ │
│ └───────────────────────────────────────┘ │
│ │
│ ← all four compete for the same budget → │
└───────────────────────────────────────────┘Nothing here is free: a longer system prompt, a chattier tool, an unsummarized history, or an over-broad retrieval step all shrink the room left for everything else.
Context engineering: curation, not accumulation
Context engineering is the deliberate discipline of deciding, at each turn, what actually belongs in the model's context — which history, which tool results, which documents — instead of defaulting to "include everything, just in case." The default of accumulating everything feels safer, but it's exactly what fills the budget shown above with content that isn't earning its place.
In practice this means concrete choices: does the agent need the customer's full 50-order history, or just the current order? Does it need the raw 4,000-line build log, or the three failing test names? Does it need every prior turn verbatim, or a compacted summary of the ones no longer being actively worked on? None of these are one-time decisions made at design time — they're ongoing, made as the agent runs.
The instinct to reach for a bigger context window instead of doing this curation is the most common mistake in this domain — covered directly in the exam trap below.
A compaction pipeline
Turns 1–24 (verbose) Turns 25–30 (recent)
┌─────────────────────┐ ┌───────────────────┐
│ full tool calls, │ │ kept verbatim — │
│ full tool results, │ │ still relevant, │
│ full assistant replies │ │ no compaction yet │
└──────────┬───────────┘ └─────────┬─────────┘
▼ │
┌───────────────────────┐ │
│ Summarizer step: │ │
│ condense to what's still │ │
│ needed — decisions, │ │
│ constraints, open items │ │
└───────────┬───────────┘ │
▼ ▼
┌───────────────────────────────────────────┐
│ Compacted context sent to the model: │
│ [short summary of turns 1–24] + [turns 25–30] │
└───────────────────────────────────────────┘Compaction reclaims budget from what's grown stale while keeping recent turns — and anything still binding — intact.
Retry with backoff around a Claude API call
typescriptimport Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic();
const MAX_ATTEMPTS = 5;
const BASE_DELAY_MS = 500;
// Retryable: the transient conditions Anthropic's own API docs and SDKs
// treat this way — rate limits, 5xx server errors, and connection errors.
// 429 rate_limit_error, 500 api_error, 504 timeout_error, 529
// overloaded_error. Not retryable: e.g. a 400 (bad request) — the request
// itself is malformed, and retrying it unchanged will just fail the same
// way again.
const RETRYABLE_STATUS_CODES = new Set([429, 500, 504, 529]);
async function callClaudeWithRetry(
params: Anthropic.MessageCreateParams,
): Promise<Anthropic.Message> {
let lastError: unknown;
for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) {
try {
return await anthropic.messages.create(params);
} catch (error) {
lastError = error;
const status = error instanceof Anthropic.APIError ? error.status : undefined;
const isRetryable = status !== undefined && RETRYABLE_STATUS_CODES.has(status);
if (!isRetryable || attempt === MAX_ATTEMPTS - 1) {
throw error; // non-transient, or out of attempts — surface it, don't swallow it
}
// Exponential backoff with jitter: ~500ms, ~1s, ~2s, ~4s, capped attempts.
const delay = BASE_DELAY_MS * 2 ** attempt + Math.random() * 250;
await new Promise((resolve) => setTimeout(resolve, delay));
}
}
throw lastError; // unreachable given the loop above, but keeps TypeScript happy
}Simplified for teaching. The shape that matters: distinguish retryable (transient) failures from ones that won't resolve on their own, back off between attempts, and always cap the number of attempts — an uncapped retry is Domain 5's version of an unbounded agent loop.
What the exam is actually testing here
The exam won't ask you to recite a definition of "context window." It gives you a system that's degrading — slow, expensive, giving worse answers, or occasionally failing outright — and tests whether you can diagnose which lever actually fixes it:
- Is the problem that irrelevant content is being included by default (a context engineering problem), or that necessary content has grown too verbose over time (a compaction problem)?
- Is the system holding an entire corpus in context when it should be retrieving a relevant slice on demand?
- Is a transient failure (rate limit, timeout, malformed output) being handled with a capped, backed-off retry — or with silence, or an infinite loop?
- Is there enough logging to actually diagnose a failure after the fact, and an eval suite to catch a regression before it reaches production?
Each of these has a specific, correct lever. "Use a bigger model" or "add a bigger context window" is almost never that lever.
Scenario: the research agent that outgrew its context
A long-running research agent investigates a single evolving topic across many sessions, picking up from where the last session left off. After a few weeks of continuous use, responses have gotten slower, more expensive, and noticeably less focused — the agent sometimes forgets constraints the user set early on, and occasionally reintroduces conclusions that were already discarded. The team needs one architectural fix. Which is best?
A long-running research agent accumulates full conversation history, full tool results, and every retrieved document across weeks of continuous sessions, with no summarization or retrieval scoping in place. Responses have grown slower, costlier, and less focused, and the agent has started forgetting constraints set early in the engagement.
What's the best architectural fix?
Retrieval and caching: two different levers
Retrieval and caching both sound like "make context cheaper," but they solve different problems. Retrieval decides what goes into context: instead of holding an entire corpus (a policy manual, a codebase, a knowledge base) in every call, a search or embedding step pulls in only the slice relevant to the current query, at the moment it's needed. This is what makes a corpus far larger than any context window usable at all.
Caching, by contrast, doesn't change what's in context — it changes the cost of reprocessing content that's already there and hasn't changed. Prompt caching lets you reuse a previously-processed prefix of context (a stable system prompt, a fixed set of tool definitions, an unchanging block of retrieved reference material) across calls, so a long-running or incrementally-extended conversation isn't paying full latency and cost to reprocess the same prefix every single turn.
It's an optimization on top of good context engineering, not a substitute for it — caching a bloated, poorly curated context still ships a bloated, poorly curated context, just faster.
Retrieval at query time
┌─────────────────────────┐
│ Full corpus (too large │
│ to fit in any context │
│ window) │
└────────────┬─────────────┘
▼
┌───────────────────┐
│ Search / embedding │
│ index │
└─────────┬─────────┘
▼ query-time lookup
┌───────────────────────┐
│ Top-k relevant slice │
│ (only what this query │
│ actually needs) │
└───────────┬───────────┘
▼
┌───────────────────────┐
│ Inserted into context │
│ alongside history + system │
│ prompt, within budget │
└───────────────────────┘The corpus never has to fit in context — only the slice a given query actually needs does, looked up fresh each time so it stays current.
Observability, evaluations, and graceful degradation
Everything above reduces the odds of a context or reliability failure — it doesn't eliminate them. Observability is what makes a failure diagnosable after it happens: logging which tools were called, what they returned, why the loop stopped, and what was actually in context at the time, so a bad answer in production can be traced to a cause instead of shrugged off.
Evaluations are the complementary, proactive half: a fixed, repeatable set of test cases run against the agent or prompt before a change ships, catching a regression that a couple of informal spot checks would miss. Neither substitutes for the other — logging tells you what already happened; evals test what would happen before it does.
Together, these are what let a system degrade gracefully instead of failing silently: a retrieval outage produces an explicit "couldn't retrieve that" instead of a confidently ungrounded answer, a tool failure past its retry cap surfaces as a clear error instead of a hallucinated result, and a context budget crisis triggers compaction instead of a hard failure mid-conversation.
Exam trap: "a bigger context window fixes it"
The most common wrong answer in this domain is reaching for a bigger context window — or a bigger model — as the fix for a system that's actually suffering from poor context engineering, missing compaction, or no retrieval strategy. A larger window raises the ceiling; it doesn't curate what's inside it, and a system that accumulates everything by default will eventually refill any window you give it, just more slowly.
The mirror-image trap shows up in error handling: treating "add retries" as always safe, without a cap. A retry loop with no maximum attempt count or timeout doesn't fix a persistent, non-transient failure — it just hangs indefinitely, burning cost and latency on a request that was never going to succeed. Retrying belongs with backoff and a bound, exactly like a turn cap belongs on an agent loop — the correct answer names both the retry and the limit, never one without the other.
Mini lab: design a context budget
A customer-support agent handles conversations that can run 60+ turns: it looks up orders, checks policies, escalates edge cases, and occasionally needs to search a large internal knowledge base. It's expected to run continuously in production for months.
- List everything that could occupy this agent's context at turn 50: system prompt, tool definitions, conversation history, retrieved knowledge-base content, tool results. For each, decide whether it should be included in full, summarized, retrieved on demand, or dropped.
- Design a compaction trigger: what specific signal (turn count? token count?) causes older turns to be summarized, and what must that summary explicitly preserve so the agent doesn't lose a standing constraint the customer stated early on?
- Design the retrieval step for the knowledge base: what triggers a lookup, and how do you keep it from re-including the same irrelevant slice on every turn?
- Pick one external call this agent makes (the policy API, the knowledge-base search) and write its retry policy: which failures are retryable, what's the backoff, and what's the cap — and what happens to the conversation when the cap is hit?
- Name one thing you'd log on every turn so that if this agent gives a wrong answer three weeks from now, you can reconstruct what happened.