Context engineering and memory

Large context windows invite you to put everything in. Anthropic's guidance points the other way: as the number of tokens grows, a model's ability to recall information from its context falls, an effect known as context rot, and it shows up across all models. Models have an attention budget, and every token you add spends some of it. Good context engineering means finding the smallest set of high-signal tokens that makes the desired outcome likely.

In an agent the cost compounds. The loop resends the whole conversation on every turn, including every tool result so far. A 20-turn run that grows to 60,000 tokens of history sends hundreds of thousands of input tokens in total, and latency rises with it. A single verbose tool that returns a full order history instead of the three fields the agent needs can double the bill for the whole run.

So budget explicitly. Decide what the system prompt must contain, keep tool outputs short and structured, and design tools that return identifiers and summaries the agent can drill into, rather than everything at once.

Prompt caching lets the API reuse the processed prefix of a prompt across requests. The cached prefix is the whole prompt in a fixed order, tools first, then system, then messages, up to the block you mark with cache_control. Reads from the cache cost a small fraction of the normal input price and are faster; writing the cache costs somewhat more than normal input. The default lifetime is 5 minutes, refreshed each time it is used, and a 1-hour lifetime is available at a higher write price. Check the pricing page for current multipliers.

Agents are ideal for caching because each turn repeats the same tools and system prompt. What kills the hit rate is anything that changes the prefix: a timestamp or the user's name in the system prompt, tools added or reordered between turns, or an MCP server that lists tools in a different order each time. The 2026-07-28 MCP revision asks servers to return tools in a deterministic order for exactly this reason.

Verify instead of assuming. The response usage reports cache_creation_input_tokens and cache_read_input_tokens. If both are zero, nothing was cached, often because the prefix was below the model's minimum cacheable length.

Anthropic describes three techniques for long-horizon work. Compaction summarises a conversation that is nearing the limit and restarts with the summary; the Claude API also offers server-side compaction. Structured note-taking has the agent write progress to a file or store outside the context, such as a NOTES file or a database row, and read it back when needed. Sub-agent architectures hand focused work to subagents with clean context windows that return condensed results.

The lightest form of compaction is clearing old tool results. Once a tool was called many turns ago, the agent rarely needs the raw output again; replacing it with a one-line summary can reclaim most of the context in a tool-heavy run.

Compaction is lossy, so tune it. Start by making the summary capture everything relevant, then trim. The failure to watch for is an agent that forgets a constraint from early in the run, such as the customer's stated budget, because the summary dropped it.

Session memory is the conversation so far. Persistent memory survives across sessions: a chat history table, a facts store, or notes the agent writes. n8n's AI Agent node, for example, can attach memory sub-nodes backed by Postgres or Redis. Before adding persistent memory, ask whether the fact belongs in your system of record instead. A lead's preferred contact time belongs in a GoHighLevel custom field that humans and workflows can see and correct, not buried in an agent's private notes.

Memory creates new failure modes. Stale facts outlive the truth, PII persists longer than your retention policy allows, and anything written from untrusted input, such as an inbound email, can plant instructions the agent will read later. Give memory a write policy: who may write, with what provenance, and when entries expire.

For knowledge, agents increasingly retrieve just in time. Instead of loading documents up front, the agent keeps lightweight references such as file paths, record IDs or saved queries and calls a search tool when it needs something. Return small chunks with source IDs, cap results per call, and let the agent search again with a better query; this is agentic RAG, and it keeps the context focused on what the current step needs.