An LLM can solve a surprisingly complex problem in a single conversation. Ask it to debug an API, write some code, or research a topic, and it can reason through the task using everything currently available in its context.

But start a new conversation and, by default, that history is gone.

The model doesn't inherently know what happened yesterday. It doesn't know that a particular approach failed last week. It doesn't know that you prefer one framework over another, or that your project has a convention that isn't documented anywhere in the current prompt.

This is where memory comes in.

But memory in an AI agent is more complicated than storing conversation history.

A useful memory system needs to answer four questions:

1. What should the agent remember?
2. How should that information be represented and stored?
3. When should it be retrieved?
4. When should it be updated, consolidated, or forgotten?

Those questions are what separate a simple chat history from an actual agentic memory system.

First: Context is not memory

Before talking about different kinds of memory, there is an important distinction to make.

Context is what the model can see right now. Memory is what can survive beyond the current context and potentially be brought back later.

If an agent receives:

"The API is returning a 500. Here's the stack trace..."

and that information is included in the current prompt, it's part of the agent's working context.

If the agent later stores:

"The API had previously failed because duplicate requests were being triggered by the frontend."

and retrieves that information during a similar debugging task next week, that's memory.

You can think of it like this:

The context window is the agent's workspace.

Memory is what allows that workspace to contain useful information from the past.

Borrowing the vocabulary from human memory

AI agent memory terminology borrows heavily from cognitive psychology.

Human memory is commonly described using concepts such as working memory, semantic memory, episodic memory, and procedural memory.

Agent architectures use similar terminology because the analogy is useful, although these should be treated as engineering abstractions, not literal implementations of human memory.

A useful mapping is:

This mapping is particularly useful when designing agents.

A practical classification of agent memory

1. Working memory

Working memory is everything the agent needs right now to solve the current task.

That can include:

• recent conversation
• the current plan
• tool results
• files being edited
• intermediate calculations
• errors encountered during the current task

It is fast and immediately accessible, but constrained by the model's context window.

A coding agent, for example, might currently have:

Task:
Fix the failing purchase request API.

Current file:
apps/backend/src/purchase-request.service.ts

Recent error:
Prisma error P2002

Plan:
1. Find the duplicate insert
2. Check transaction handling
3. Add regression test

None of this necessarily needs to become long-term memory.

It is simply the agent's current workspace.

2. Semantic memory

Semantic memory is knowledge the agent should know independently of a particular event.

For an engineering agent, this could be:

The backend uses Prisma.

Database migrations are run through Nx.

API responses follow the project's standard error format.

All new services require unit tests.

This is similar to documentation or project knowledge.

Files such as CLAUDE.md, project documentation, architecture guidelines, and structured knowledge bases can all act as semantic memory.

The knowledge is meant to persist across interactions rather than belong to a single conversation.

It describes how the world or the project works.

3. Episodic memory

Episodic memory is knowledge about things that happened.

For example:

Yesterday, the deployment failed because the migration
was executed after the application containers started.

Or:

The previous implementation of the webhook handler failed
because retries produced duplicate events.

Unlike semantic memory:

"The application uses PostgreSQL."

episodic memory might say:

"An earlier deployment failed because a required PostgreSQL migration had not been applied."

The second contains an experience.

And experiences are valuable because they can eventually produce reusable knowledge.

4. Procedural memory

Procedural memory is knowledge about how to perform a task.

For an agent, this might look like:

To deploy the backend:

1. Run tests.
2. Generate the Prisma client.
3. Run migrations.
4. Build the application.
5. Deploy.

Modern agent systems often represent this as skills.

Instead of expecting the model to figure out how to perform a task from scratch every time, the agent can retrieve an existing procedure when the task requires it.

This leads to an important evolution in agent design.

From reasoning from scratch to reusable skills

Procedural memory has gone through an interesting evolution.

Stage 1: Re-derive everything

Early reasoning agents essentially started from scratch every time.

If they needed to perform a task, they reasoned through the procedure again.

Task → reason → act

There was no reusable procedure.

This works, but it is inefficient and can produce inconsistent results.

Stage 2: Put everything in the system prompt

The next obvious solution was to write the procedures into the prompt:

If you need to deploy:
  do A
  then B
  then C

If you need to create a PR:
  do X
  then Y
  then Z

If you need to debug:
  do P
  then Q
  then R

This makes the procedures explicit.

But it introduces another problem:

context bloat.

If an agent has 100 possible procedures, most of them are irrelevant to the current task.

Why should a deployment procedure occupy context when the agent is currently fixing a CSS bug?

Stage 3: Store executable skills

A different approach is to make procedures executable.

The agent can have a library of reusable skills:

skills/
├── deploy/
├── database-migration/
├── create-pr/
├── debug-api/
└── run-tests/

When the relevant task appears, the agent can use the appropriate skill.

Systems such as Voyager demonstrated an interesting version of this idea: agents could build reusable libraries of executable skills rather than reasoning about every action from scratch.

Stage 4: Progressive disclosure

A more general approach is progressive disclosure.

Instead of putting the entire procedure into context, the agent keeps a lightweight index:

Available skills:

deploy
database migration
API debugging
create pull request
run test suite

If the user asks:

"Deploy the application."

the agent identifies the relevant skill and loads its full instructions.

This gives the agent explicit procedures without forcing every procedure into its context window.

The important shift is:

Procedural knowledge becomes explicit, but loaded only when needed.

But storing everything isn't memory either

At this point, it is tempting to build a system that simply stores every conversation.

That doesn't solve the problem.

Imagine an agent stores this entire debugging session:

User reported form submission failing with a 500 error. Checked API logs, saw a database unique-constraint error on (formId, userId). Traced it to the frontend calling the create endpoint twice due to a double-click on submit with no debounce. Added a loading-state disable on the button, and a unique constraint check with a friendly error message as backup. Took 45 minutes, touched three files.

Technically, the agent now "remembers" the experience.

But most of that information is not useful later.

The timestamps don't matter.

The number of files doesn't matter.

The fact that it took 45 minutes doesn't matter.

What matters is the reusable lesson.

So the memory could instead become:

Root cause: duplicate form submissions caused by missing debounce. Fix pattern: disable submission after the first click and keep a database constraint as a backstop.

That's distillation.

Distillation: turning experiences into useful memory

Distillation is the process of converting a raw experience into a smaller, more reusable piece of knowledge.

Think of the pipeline as:

The goal isn't necessarily to preserve everything.

The goal is to preserve what will still be useful later.

A useful test is:

Would this information still be useful if I removed everything about when and where it happened?

If yes, it is probably a good candidate for persistent memory.

For example:

Raw experience:

On Tuesday, while working on the purchase request form, the submit button was clicked twice and caused a unique constraint error.

Distilled memory:

Forms should prevent duplicate submissions on the client, with a database constraint as a second layer of protection.

The second version is much more reusable.

Memory is more than one storage technique

Once we understand what memory represents, the next question is:

How do we actually implement it?

There isn't one answer.

Different techniques solve different problems.

Sliding window

Keep only the most recent N messages.

Message 1   ✗
Message 2   ✗
Message 3   ✗
...
Message 98  ✓
Message 99  ✓
Message 100 ✓

It's simple and cheap.

But it has an obvious problem:

Something important from message 2 might disappear even though the agent still needs it.

Conversation summarization

Instead of throwing old messages away, periodically summarize them.

For example:

User is building a Rails application. They are currently migrating a purchase-request flow and previously decided to use X approach.

This preserves the gist without keeping the entire conversation.

The downside is that summarization is lossy.

A summary can accidentally remove a small but important detail.

Fact extraction

Rather than summarizing everything, extract individual pieces of useful knowledge.

For example:

User prefers Python.
Project uses PostgreSQL.
Backend uses Prisma.
Deployment requires migrations first.

These can be stored as individual records and retrieved independently.

This is much more precise when the agent needs to answer:

"What do I know about this project?"

rather than:

"What happened in the previous conversation?"

Retrieval-augmented recall

Instead of loading all stored memories, retrieve only the relevant ones.

For example:

The underlying storage could be:

• a relational database
• vector search
• a knowledge graph
• a document store
• or a combination of these

The important part isn't the database itself.

It's the retrieval strategy.

In practice, memory is usually hybrid

Production systems rarely use only one technique.

A practical agent might use:

For example:

• Sliding window → recent conversation
• Summarization → older conversation
• Fact extraction → durable user/project facts
• Episodic storage → important past experiences
• Skill library → reusable procedures
• Retrieval → bring only relevant memories back

The interesting part is that these mechanisms don't compete with each other.

They operate at different levels.

Looking at memory from three dimensions

The categories above help us describe what an agent remembers, such as facts, experiences, procedures, or information needed for the current task.

But there is another question:

How is that memory represented and managed?

A useful way to separate the two is to think about memory along three dimensions:

1. Form: where does the memory live?

Token-level

Explicit information that can be represented as text or structured data.

Examples:

CLAUDE.md
database rows
vector embeddings
knowledge-graph nodes
skill files

This is where most current agent systems operate.

Parametric

Knowledge encoded directly into model parameters through training or fine-tuning.

The model doesn't retrieve a document containing the fact. The information is embedded in its learned weights.

Latent

Information represented in internal model state, hidden representations, or mechanisms such as KV caches.

This is much closer to the inference/runtime layer than traditional application-level memory.

2. Function: why does the memory exist?

A useful classification is:

Working memory: Information needed for the current task.

Factual memory: Facts about users, projects, environments, or the world.

Experiential memory: Lessons learned from previous task execution.

Procedural memory: Reusable ways of performing tasks.

The boundaries aren't always perfect.

For example, an experience can be distilled into a procedure.

This is why memory categories shouldn't be treated as rigid boxes.

3. Dynamics: how does memory change?

This is arguably the most important dimension when building a real system.

Memory has a lifecycle.

Formation

Something happens.

The system decides whether it is worth remembering.

Interaction
    ↓
Candidate memory

Evolution

New information may update an old memory.

For example:

Old:
User prefers framework A.

New:
User switched to framework B.

The system shouldn't blindly store both forever.

It needs some concept of:

• updates
• conflicts
• confidence
• recency
• relevance
• redundancy

Retrieval

When a new task arrives, the system decides:

"Which memories should I bring into context?"

Forgetting

Not everything deserves permanent storage.

Some memories become:

• stale
• irrelevant
• duplicated
• contradicted
• low-value

A mature memory system needs a way to remove or de-prioritize them.

The complete memory lifecycle

Putting everything together:

This is the part that is easy to miss when talking about "agent memory."

Memory isn't just a database sitting next to an LLM.

It is a lifecycle.

So what does a coding agent actually need?

Different agents need different kinds of memory.

A simple agent might need nothing beyond its current context.

A customer-support agent needs product knowledge and procedures.

A personalized chatbot benefits from remembering user preferences and previous interactions.

A coding agent may need all four:

• current files and errors → working memory
• project conventions → factual/semantic memory
• reusable workflows → procedural memory
• previous bugs and architectural decisions → episodic memory

There is no universal "agent memory architecture."

The right design depends on what the agent actually needs to remember.

If you're building an agent, where should you start?

Don't start by choosing a vector database.

Start by asking what the agent needs to remember.

Step 1: Start with working memory

Keep the recent conversation and current task state in context.

If that is enough, you're done.

Step 2: Add summarization

If conversations regularly exceed the context window, summarize older context.

Step 3: Add durable facts

If the agent needs to remember things like:

user preferences
project conventions
environment configuration
domain rules

extract and store those separately.

Step 4: Add episodic memory

If past experiences matter, store important events and lessons.

Don't necessarily store the entire conversation.

Store the useful residue.

Step 5: Add procedural memory

If the agent repeatedly performs the same tasks, turn those procedures into reusable skills.

Step 6: Add retrieval

Once memory becomes large enough, don't dump everything into context.

Retrieve only what is relevant to the current task.

Step 7: Add evolution

Eventually you'll encounter:

Old memory: User uses React.

New memory: User migrated to Vue.

Now you need conflict resolution, updates, and forgetting.

That's when memory becomes a real system rather than a storage layer.

The bigger picture

There is a tendency to describe agent memory as:

"Store previous conversations in a vector database and retrieve them later."

That's one implementation technique, not a definition of memory.

A better mental model is:

Memory is the system that allows an agent to retain useful information across time and reuse it when appropriate.

That system might contain:

• current working state
• facts
• experiences
• procedures
• summaries
• structured records
• vector indexes
• knowledge graphs
• skills
• even information encoded into model parameters

And the difficult part isn't simply storing any of these.

The difficult part is deciding:

what deserves to be remembered, how it should be represented, when it should be recalled, and when it should be forgotten.

That's what makes agentic memory an architectural problem rather than just a database problem.

Further reading

• CoALA: Cognitive Architectures for Language Agents
• Memory in the Age of AI Agents: A Survey
• Mem0
• Letta
• Voyager: An Open-Ended Embodied Agent with Large Language Models