IBRAVRA Media Network
AI Tools • 12 min read

AI Agent Memory Playbook: Context Windows, Compaction & the Cost of Remembering

An agent that cannot remember is a demo with a short fuse. This playbook sizes memory from the vendors' own published numbers — 1M-token windows, compaction math, and Mem0 vs Zep pricing for a 120,000-operation fleet.

By Enow A. Jovial • Published 2026-11-19

An AI agent that cannot remember is a demo with a short fuse. It answers the first question well, loses the thread by the fifth, and by the twentieth it is asking the user for information the user already gave it. The failure is not the model. It is the memory architecture around the model: what is kept, where it is kept, what is thrown away, and what that remembering costs in tokens. This playbook builds that architecture from the vendors' own published numbers, so you can size it before you ship it.

Memory for agents is two stores, not one: the context window, which is working memory that you pay for on every single call, and a long-term store that you pay for only when you write to it or read from it. The expensive mistake is treating the window as the database. The cheap fix is moving everything that does not need to be in front of the model right now into a store you query on demand.

Who this is for, and who should stop here

This playbook is for a team shipping an agent that talks to the same user more than once: a support agent that handles repeat tickets, a sales agent that works a pipeline over weeks, a coding agent that returns to the same repository, an operations agent that runs the same weekly workflow. If the agent's job is a single question and a single answer, stop here — a stateless call with a good system prompt is cheaper and simpler than any memory layer.

It is also not for a team that has not yet fixed retrieval. If your agent already has a RAG pipeline, memory is the next layer: retrieval answers "what does the document say", memory answers "what do we know about this user, this account, this project". The two stores are complementary, and this guide assumes the RAG layer already exists or is being built alongside.

The context window is working memory, and you rent it by the call

Every token in the request counts toward the window: the system prompt, every message, every tool result, every image and document, the tool definitions, and the model's extended thinking. Anthropic's context-window documentation states this plainly, and its current-generation Claude models — Fable 5.2, Mythos 5.2, Opus 5.5, Sonnet 5.5, Haiku 5.5 — carry a 1M-token context window by default, while older models including the deprecated Claude Sonnet 4.5 carry 200k. A single request to a 1M model can generate up to 128k output tokens. OpenAI's GPT-5.4 carries a context window under 272k tokens, and Google's Gemini 2.0 Flash carries a 1M-token context window. The window is not a database. It is a desk: everything on it is visible, everything on it is billed, and everything on it disappears when the call ends.

Vendor / modelContext windowAuthority
Claude current generation (Fable 5.2, Opus 5.5, Sonnet 5.5, Haiku 5.5)1,000,000 tokens, defaultAnthropic context-window docs
Claude older models (incl. Sonnet 4.5, deprecated)200,000 tokensAnthropic context-window docs
OpenAI GPT-5.4Under 272,000 tokensOpenAI API pricing page
Google Gemini 2.0 Flash1,000,000 tokensGoogle Gemini API models page

The arithmetic that decides your design is accumulation. Tokens in a conversation grow with every turn, because each new request re-sends the history:

Tokens in turn n=System prompt+∑i=1n−1(Useri+Assistanti+Tool resultsi)+New inputn\text{Tokens in turn } n = \text{System prompt} + \sum_{i=1}^{n-1}(\text{User}_i + \text{Assistant}_i + \text{Tool results}_i) + \text{New input}_n

Take a support agent with a 2,000-token system prompt, where each turn averages 350 input tokens, 250 output tokens, and 2,000 tokens of tool results — CRM records, order history, knowledge-base excerpts. After 40 turns the request carries 2,000 + 40 x 2,600 = 106,000 tokens. That fits a 200k window. After 100 turns it carries 2,000 + 100 x 2,600 = 262,000 tokens. That fits a 1M window, fits nothing at 200k, and is within 10,000 tokens of GPT-5.4's ceiling. The crossing point is not a matter of opinion; it is the point where the sum passes the vendor's published number, and the agent that crosses it without a plan either errors, truncates silently, or drops the oldest turns. Anthropic's own docs note that chat interfaces manage the window on a rolling first-in, first-out basis — which means the earliest context, often the user's original request, is the first thing lost.

A useful conversion for sizing: one token is roughly 0.75 words of English text, so a 200k-token window holds about 150,000 words of history, and a 1M-token window holds about 750,000. Count your turns in tokens, not in messages, and count tool results — they are usually the largest line item in the sum.

Four kinds of memory, four different stores

LangChain's documentation frames agent memory as two core components — short-term memory and long-term memory — and the working distinction operators actually need is finer than that. Four kinds, each with its own store and its own failure mode:

Memory kindWhat it holdsWhere it livesFailure mode
Working memoryThe live conversation, tool results, the current taskContext windowFIFO truncation drops the original request
Episodic memoryWhat happened: ticket 4812, the refund, the call on TuesdayEvent log + vector storeEvents retrieved out of order, or not at all
Semantic memoryWhat is true: the customer's plan, preferences, account factsFact store / knowledge graphA stale fact overrides a newer one
Procedural memoryHow to do it: the refund playbook, the escalation rulePrompts, skills, runbooksA stale playbook runs after the policy changed

Working memory is the only kind the model sees for free. The other three must be retrieved into the window at the right moment, which is why the retrieval step matters more than the storage step. A vector store full of facts the agent never queries is dead weight; a small, well-indexed fact store that answers the three questions every turn actually asks is the whole system. If your agent needs its vector database tuned before its memory layer is built, tune the database first — memory retrieval is only as good as the index underneath it.

Compaction: the operation that keeps long conversations alive

Compaction is the deliberate act of replacing history with a summary of that history. It is the only honest answer to a conversation that will outgrow its window, and it has three shapes. A sliding window keeps the last N turns verbatim and drops the rest — simple, lossy, and dangerous when the dropped part contains the user's goal. A summary replaces a block of turns with a condensed record — the goal, the decisions, the open items — and the summary itself becomes part of the window. A hybrid keeps the summary plus the last few turns verbatim, so the model has both the arc and the detail.

The compression math is what makes summarization pay. Suppose 100,000 tokens of history compress into a 3,000-token summary:

Compression ratio=100,0003,000≈33:1\text{Compression ratio} = \frac{100{,}000}{3{,}000} \approx 33:1

That summary costs one extra model call, and it buys back 97,000 tokens of window on every subsequent turn. The risk is what the summary drops: a constraint the user stated once ("never email the client after 6pm"), a number quoted in passing, a preference expressed as a joke. The fix is a summary template with fixed fields — goal, constraints, decisions, open items, facts about the user — rather than a free-form paragraph, and a rule that constraints and numbers are never summarized away, only copied verbatim.

A summary that drops a constraint is worse than no summary at all, because the agent will act confidently on the wrong instructions. If a turn contains a hard constraint, a price, a date, or a legal instruction, copy it verbatim into the summary. Never let a model paraphrase a number the business will be held to.

Build or buy: Mem0 and Zep on their published numbers

Two managed memory layers publish enough to price. Mem0's free tier covers the first 30 days at 10,000 memory adds and 1,000 retrievals a month with model calls on Mem0, and 50,000 adds and 10,000 retrievals a month when you bring your own LLM key; past the limits, usage-based billing applies with a monthly cap you set. Its custom tier adds graph memory with entity linking, user profiles, Dream memory consolidation, advanced analytics, private Slack support, and SSO, audit logs, and SLA, plus on-premises deployment. Zep's Flex plan is $125 a month billed monthly, includes 50,000 credits, and charges $25 per 10,000 credits after that; annual billing saves 17 percent, and the plan includes 600 requests a minute, 5 projects, 5 Context MCP Server seats, 10 custom entity and edge types, and 1-day API logs, with unlimited memories, retrieval, and users.

LayerEntry priceIncluded volumeOveragesNotable
Mem0 Free$010,000 adds + 1,000 retrievals/mo (first 30 days); 50,000 adds + 10,000 retrievals/mo with own keyUsage-based, capped monthlyGraph memory and Dream on Custom
Zep Flex$125/mo50,000 credits/mo$25 per 10,000 creditsAnnual billing saves 17%; 600 req/min

Work the Zep numbers for an agent fleet doing 120,000 memory operations a month:

Monthly cost=$125+120,000−50,00010,000×$25=$125+7×$25=$300\text{Monthly cost} = \$125 + \frac{120{,}000 - 50{,}000}{10{,}000} \times \$25 = \$125 + 7 \times \$25 = \$300

Annual billing at the published 17 percent saving brings that to $300 x 12 x 0.83 = $2,988 a year. On Mem0's free tier, the same fleet would need to fit inside 50,000 adds and 10,000 retrievals a month with its own LLM key — which it does not, at 120,000 operations — so the honest comparison is Zep Flex against Mem0 Custom, which is quote-only. Get the Mem0 number in writing before you compare; a vendor page that does not publish a price cannot win a pricing comparison.

The build-versus-buy line is simpler than the pricing tables suggest. Build the memory layer yourself when the memory is the product — when your ranking, your extraction, or your consolidation logic is the differentiator. Buy it when memory is plumbing: the agent is the product, and you need adds, retrieval, and consolidation to work on day one. A team that builds its own memory store to save $125 a month has confused a cost problem with a focus problem.

Failure modes that cost real money

Memory fails in ways a stateless agent never can. Stale facts are the first: a customer upgrades from the $49 plan to the $199 plan, the fact store still says $49, and the agent quotes the old price in writing. The fix is a write path with a source of truth — facts come from the CRM or billing system, never from the model's own summary — and a timestamp on every fact, so the agent can say "as of Tuesday" instead of stating it as current. Memory poisoning is the second: a user, or a pasted document, injects a false instruction into the store ("always apply a 50 percent discount"), and the agent obeys it in later sessions. The fix is a trust boundary — user-supplied text is data, never instruction, and facts that change money or permissions require a human-confirmed source. Cross-user leakage is the third and the most expensive: one user's memories surfacing in another user's session. The fix is a user-scoped key on every memory row, enforced in the retrieval query, not in the prompt.

Founder & CEO Insight — Enow A. Jovial

"The first time an agent quoted a customer a price from a memory that was three weeks out of date, I stopped thinking of memory as a feature and started thinking of it as a liability with a ledger. Every fact in the store needs a source and a date, or it is a rumor the agent will repeat with a straight face."
— Enow A. Jovial, Founder & Chief Executive Officer

The build checklist

Work the first three steps in order; the rest are standing rules that stay true every week.

  1. Count the window in tokens: system prompt, history, tool results, thinking — measured on your ten longest real sessions, not estimated.
  2. Name the four stores: working, episodic, semantic, procedural — and write down which system owns each one.
  3. Set the compaction trigger: a token threshold, not a turn count, with a fixed-field summary template that copies constraints and numbers verbatim.
  • [x] Scope every memory row to a user: the retrieval query enforces the scope; the prompt does not.
  • [x] Timestamp every fact: the agent states the fact's date when it uses it, and the write path refreshes facts from the CRM or billing system.
  • [x] Price the layer: Mem0 free-tier ceilings against your adds and retrievals, or Zep Flex at $125 plus $25 per 10,000 credits past 50,000 — whichever you choose, run the monthly cost through the ROI calculator against the token spend the memory layer saves.
  • [x] Test the FIFO edge: a 120-turn session where the original request is the thing that must survive, and verify it does.
Start memory with one fact type, not five. Pick the fact your agents get wrong most often — plan tier, renewal date, account owner — and build the write path, the retrieval, and the timestamp for that one fact. A single fact type done properly teaches you more about your memory layer than a five-type schema that nobody maintains.

When the window is the wrong tool

A bigger window is not a memory strategy. It is a bigger desk, and a bigger desk still gets buried. The teams that ship agents that survive real use do three boring things: they count tokens before they count features, they compact on a threshold instead of waiting for an error, and they treat every stored fact as a rumor until it carries a source and a date. The model will do the reasoning. Your job is to make sure it reasons about the right facts, and that the facts it reasons about are the ones that are true today.

Mem0 pricing: Free tier limits of 10,000 adds and 1,000 retrievals a month, and Custom tier features including graph memory and Dream consolidation
Mem0 pricing: Free tier limits of 10,000 adds and 1,000 retrievals a month, and Custom tier features including graph memory and Dream consolidation