Context Engineering: What Prompt Engineering Becomes

Million-token windows are now routine ^[1], and the vendors who sell them warn against filling them. Anthropic's own documentation says that "more context isn't automatically better" ^[1]. Our thesis is simple: the unit of design is no longer the prompt but the window, and the window is a budget that has to be spent, aged and cleared on purpose. Prompt engineering does not vanish. It becomes one layer in a larger system, which is why the accurate verb is "subsumes", not "replaces".

The evidence for a budget view is uneven but consistent in direction. Two results, from different tasks and setups, show it: accuracy falls when retrieval is not literal ^[2], and when instructions arrive over several turns ^[3]. A vendor-run report adds that a focused prompt can beat a padded one ^[4]; it is discussed below.

How vendors define context engineering, and where they differ

The major definitions converge. Anthropic calls it "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference" ^[5]. Harrison Chase of LangChain describes "building dynamic systems to provide the right information and tools in the right format" ^[6]. Google's agent-framework write-up says "Context is a compiled view over a richer stateful system" ^[7]. OpenAI's cookbook reduces it to "deciding what goes into the model's input on each request" ^[8]. Thoughtworks moved it from Assess (November 2025) to Adopt (April 2026, Vol. 34, still the latest volume when we checked on 13 October 2026) on its Technology Radar, saying it "has evolved from an optimization tactic into a foundational architectural concern" ^[9].

They differ on scope, and we found no formal standard.

Why more context can make agents worse

"Context rot" is a bundle of mechanisms, not a single law. Separating them matters because the fixes differ.

Position. In Lost in the Middle, accuracy was highest when relevant information sat at the start or end of the input and fell when it sat in the middle, even for models built for long contexts ^[10]. In the worst settings GPT-3.5-Turbo did worse than with no documents at all (56.1% closed-book) ^[10]. Those were 2023 models, so take the pattern, not the magnitude. A 2024 follow-up links the effect to a U-shaped attention bias favouring the start and end "regardless of their relevance" ^[11].

Length itself. Across five models on math, QA and coding, performance fell 13.9% to 85% as input grew, even when the models could retrieve every relevant item, and the drop persisted when irrelevant tokens were replaced with whitespace ^[12].

Distractors and non-literal matching. Chroma, in a vendor-run report without peer review, found that even a single distractor lowers performance, and, unexpectedly, that shuffled haystacks beat logically coherent ones; the authors call it surprising and offer only an untested speculation about attention ^[4]. NoLiMa shows the same fragility once literal overlap is removed ^[2].

Compounding errors. In Laban et al., the 39% average loss (the paper's abstract and text say 39%; its Figure 1 caption says 35%) showed up as a 112% rise in unreliability against a roughly 16% loss of aptitude. Giving the same information in one turn scored 95.1% of the fully specified baseline, so the loss was not missing information. The authors attribute it to early assumptions the model then over-relies on ^[3].

Headline numbers measure the easy case. Google reports about 99% on a single-needle query, but its own documentation says accuracy is lower when several pieces of information must be found ^[13]. RULER tested 17 long-context models that all claimed at least 32K tokens; only about half kept satisfactory performance at 32K ^[14].

The vendor-run Databricks study (over 2,000 experiments on 13 models) concluded that "Longer context is not always optimal for RAG" ^[15]. Its failure analysis shows how uneven the effect is: DBRX-instruct's instruction-following failures rose from 5.2% at 8k tokens to 50.4% at 32k, while gpt-4o, claude-3.5-sonnet and gpt-4o-mini showed little to no performance deterioration ^[15]. The source does not state the denominator for the failure rates, and the models are from 2024. Whether position, length, distractors or step count dominates is unsettled; Anthropic's explanation is an attention budget, because every token attends to every other, giving n² pairwise relationships ^[5]. The practical advice is mechanism-agnostic: less, cleaner, better-placed material, and a reset when history goes bad.

Our opinion: treat the claimed window as a ceiling and the tested window as the number you plan around. Run your own tasks at roughly 10%, 25%, 50% and 100% of the window, plot accuracy and cost, and take the point where the curve bends as the working limit for that model and task. That is a method, not a published threshold, and the answer will change with the next model.

Four levers, and what each costs

LangChain's taxonomy is a useful frame: write context outside the window, select what comes in, compress what stays, and isolate work into separate windows ^[16]. Each lever trades tokens for a specific risk.

Write. Anthropic's memory tool is client-side: Claude asks for file operations and the application executes them. The docs recommend pairing compaction, which keeps active context small, with memory, which preserves what must survive summarisation ^[17].

Select. Tool definitions are context. A five-server setup can consume about 55k tokens before any work, and selection accuracy degrades beyond 30 to 50 tools ^[18]. Anthropic reports that on-demand loading cut total context before any work in one setup from about 77K to about 8.7K tokens, about 72K of the 77K being tool definitions ^[22]. Separately, presenting servers as code APIs cut a Google Drive to Salesforce workflow from about 150,000 to about 2,000 tokens ^[23]; that post does not itemise how much of the saving is definitions and how much is intermediate data staying out of the model's context. Both are vendor examples, not benchmarks. The first post calls its reduction 85%, but its own figures work out to about 89%.

Compress. Anthropic's docs describe compaction as keeping the active context small "because response quality degrades as a conversation grows" ^[20]. It is lossy. In one cookbook probe, a summary kept 3 of 3 high-level facts and 0 of 3 obscure specifics, from a single run scored by simple string matching ^[24]. The ACE paper names two related failures, brevity bias and context collapse, where iterative rewriting erodes detail ^[25].

Isolate. Anthropic reports that multi-agent systems use about 15 times the tokens of chat, and that a lead agent with sub-agents scored 90.2% better than a single agent in a June 2025 internal research eval with Claude Opus 4 leading Claude Sonnet 4 sub-agents ^[21]. This does not contradict the small-window advice. Spend more tokens in total, across several clean windows, but keep each window small. That is NKK's reading: budget per window, not per task.

Caching cuts across all four. OpenAI notes that reuse needs an identical prefix, so stable content goes first, and that "Summarization, compaction, or context truncation can change the prefix and reset cache reuse" ^[26]. An archived OpenAI cookbook page, written for models before GPT-5.6 and undated, puts it more bluntly: context engineering and prompt caching "are inherently at odds" ^[8]; treat that as a historical framing, since the current guide ^[26] says only that these operations can reset reuse. Caching also changes price, not occupancy: cached prefixes still fill the window ^[1].

The NKK Context Layer Stack

The stack is our model for deciding what enters the window and in what order. It builds on Google's session, memory and artifact tiers ^[7], Anthropic's staged loading of skills ^[19] and OpenAI's stable-prefix rule ^[26]. Anthropic also advises putting long documents above the query, with query-at-end improving quality by up to 30% in its tests (test details not given) ^[27], and Google gives the same advice ^[13].

How to use it. Give each layer an owner and the admit, place, age and evict rules. Then measure: log the token share of each layer from the usage field of real runs, and set a trigger for overrun, such as deferring tools when definitions dominate or clearing results when the evidence stream does. The cookbook run, where file reads were 96.3% of context, shows why the split must be measured rather than assumed ^[24]. We publish no recommended percentages because none exist in the sources we trust. The stack and the triage matrix below are the two core models; the ledger and the scorecard are aids for applying them.

Where it breaks. The ordering is a heuristic drawn from vendor guidance and benchmarks, not a measured optimum. Layers blur: a tool result can become memory. The stack is a logical ordering, not a byte layout. Layers 1 to 3 can be physically ordered so they cache, but layers 5 and 6 interleave chronologically in an append-only transcript, and, by the same logic, our inference is that a mid-session memory change that edits the prefix would also reset the cache, so make memory changes at session boundaries. The sources name summarisation, compaction and truncation, not memory edits ^[26][8]. Summaries lose exact values. And rules age with the model. Anthropic's harness write-up found that with one model, "context anxiety" made compaction insufficient and resets essential, while a later model largely removed the behaviour ^[28]. A per-window budget also ignores step-count decay across a long run, whose role is contested.

Deciding what to keep: the Context Triage Matrix

The stack says where things live. The matrix sorts any candidate item by two questions: how often later steps will need it, and how costly it is if it is lost or wrong.

The quadrants map back to the stack: Pin to layer 1 (known up front) or layer 5 (decided mid-session, appended after the cached prefix so it does not edit it), Externalise to layers 3 and 4, Summarise to layer 5, and Clear to layer 6. The treatments also map to documented options. OpenAI separates trimming (deterministic, no added latency, forgets long-range context abruptly) from summarisation (keeps long-range decisions, adds latency and cost, and errors can compound) ^[29]. Its guidance on memory adds that compaction continues one run, memory improves future runs, and compaction belongs "at meaningful workflow boundaries, not after every turn" ^[30]. Anthropic's cookbook states the split: "Memory solves the cross-session problem; clearing and compaction solve the in-session one" ^[24].

Where it breaks. Reuse and loss cost are judgements made before the work happens. The lesson from the 0-of-3 probe is that low-salience exact values, such as the account number in the top-left example above, look unimportant and are not. Anything with an exact-value requirement moves to the top row, however minor it appears.

Persistent context is also an attack surface

Anything the model reads can carry instructions. The OWASP Top 10 for LLM Applications 2025 edition says "Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files." It recommends separating untrusted content, limiting privileges and requiring human approval for privileged actions ^[31]. That wording and those recommendations are from the 2025 edition; OWASP also lists a 2026 edition ^[32]. The agentic list includes "Memory & Context Poisoning" as ASI06 ^[33], and a co-lead of the ASI06 entry writes that "A memory file is not just stored text. It can influence future decisions" ^[34].

Long contexts weaken oversight too. In an Anthropic-affiliated preprint, frontier models used as monitors missed subtly dangerous agent actions 2x to 30x more often when the action followed 800K tokens of benign activity ^[35]. And OpenAI's server-side compaction returns an encrypted item that is "opaque and not intended to be human-interpretable" ^[36]. Our inference: if you cannot read what survived, keep a readable note alongside it.

What to do next

1. Measure, then test. Log tokens by layer for a sample of real runs and do not tune anything until you know where tokens go ^[24]. Then rerun your own tasks at several context sizes, because vendor headline windows measure the easy case ^[14].
2. Move rarely used tools behind search, especially at ten or more tools or above 10k tokens of definitions, the point at which Anthropic's docs recommend tool search ^[18].
3. Set reset and compaction rules per model, and re-test on every model change ^[28].
4. Externalise facts that must survive. Use files and progress notes rather than hoping a summary keeps them ^[37].
5. Give memory an expiry, and treat it as untrusted input. Anthropic's memory tool docs cover expiration ^[17]; OWASP treats memory as an attack surface ^[34]. Recording provenance is NKK's view, not a cited requirement.
6. Grade outcomes, and use pass^k for reliability. At 75% per trial, three consecutive passes occur about 42% of the time ^[38]. OpenAI's cookbook agrees: "evals is all you need for context engineering too" ^[29].

Use this scorecard to prioritise the steps above.

Conclusion

Context is a managed budget, and the system prompt is one line item in it. The best public evidence comes from benchmarks and vendor engineering, with little independent evidence on enterprise outcomes, so claim no more than you have measured. Our stance: before changing any prompt, model or window size, spend this week on one measurement. Log the token share of each layer on one real workflow, set a trigger for the layer that dominates, and let outcome evaluations decide what stays.