Memory for agents
Short chat context versus long-term stores—and why agents still forget.
What you'll learn
- Separate working context from durable memory stores.
- Describe write/read patterns: summaries, vector memory, structured DBs.
- Explain why retrieval and permissions matter for agent memory.
In plain English
Agent memory is everything the system keeps between turns: what fits in the model's context window right now, plus anything saved outside—notes, user profiles, embeddings of past chats, spreadsheets.
Without external memory, long projects fall apart when chat history overflows the window. With memory, new risks appear: stale facts, privacy leaks, and retrieving the wrong old note.
How it works
Working memory is the prompt bundle this step: recent messages, plan, tool outputs. Long-term memory writes selected facts after sessions—often summarized—and reads them back via search when a new task starts.
Design choices include what to remember automatically vs ask the user, TTL on memories, and per-user isolation for security.
- Ephemeral: current thread in context window.
- Summarized: compressed history when threads grow.
- Semantic: embedding search over past interactions or docs.
- Structured: CRM rows, tickets, preferences in a database.
Going deeper
Memory interacts with RAG: both retrieve text into context, but agent memory also stores agent-generated conclusions that may be wrong unless verified.
Compliance-sensitive apps need deletion, audit trails, and clear user controls—not unbounded “remember everything.”
Common misconceptions
- Infinite context removes the need for memory design.
- Long windows help but cost more and still benefit from selective retrieval.
- Vector memory is always accurate recall.
- It returns similar text, which may be outdated or incorrect.
- Agents remember only what users explicitly ask.
- Products may auto-summarize; disclose and control what is stored.
Key facts
- Working memory is bounded by the context window per turn.
- Long-term memory lives in external stores injected when relevant.
- Summarization trades detail for space and can drop constraints.
- Retrieval must respect user boundaries and access control.
- Memory quality depends on write policies and verification, not storage alone.
Sources used
These free resources informed this page. ANN writes original explainers; we do not copy course text behind paywalls.
- The Illustrated Transformer — Context limits that shape agent memory design.
- Hugging Face LLM Course — LLM behavior as the decision core in agent systems.
- Anthropic Prompt Engineering Interactive Tutorial — Tool schemas and structured prompts for agent loops.
Also explore AI companies, Live Feed, and Weekly Brief.