Agent memory: what it is and where it should live

In short

Agent memory is what an agent keeps between steps and between sessions so it can reuse it later.
It divides into working, session and long-term memory, the last of which brings together facts, preferences, interaction history and learned procedures.
Without structure, memory accumulates errors, goes stale, mixes up customers, ignores permissions and can end up outside the company's perimeter.
In the Company Brain, memory is linked to the graph's entities, with inherited permissions, inside the customer's environment and served to any agent via MCP.

An agent that serves the same account every week can start each conversation from scratch or pick up from what has already happened. The second option depends on agent memory: what an agent keeps between steps and between sessions so it can reuse it later. For a company, the practical question is where that memory lives, who can query it and who can delete it. It is an architecture question, and it leads to the Company Brain, the layer that knows the company: what data exists, where it lives, how it relates and who can see what.

Agent memory keeps what will be reused later

A language model keeps nothing between one call and the next. Everything it uses has to arrive in the prompt. Agent memory is the set of mechanisms that selects what is worth keeping, stores it outside the model and retrieves it at the right moment.

The subject took shape in 2023, when researchers from Stanford University and Google described agents that record every experience in natural language, condense those records into reflections and retrieve the most relevant ones before planning the next action.

Memory divides by lifespan and by type of content

Working memory: what is in the context window during the task, such as the instructions and the tool results.
Session memory: the history of a conversation or run, kept while it lasts and discarded or summarised at the end.
Long-term memory: what survives between sessions. It brings together facts, such as an account's finance contact; preferences, such as the report format a manager asks for; interaction history; and learned procedures, such as the sequence that resolved a type of failure.

Types of agent memory by lifespan: working, session and long-term — Agent memory divides by lifespan and by type of content

Implementations combine summaries, vectors and key-value stores

Summarised history: the conversation is condensed into a short text that goes back into the prompt in the next session.
Vector database: each interaction becomes an embedding, and retrieval looks for the records most similar to the current question.
Key-value store: facts and preferences sit in named fields, looked up by identifier.

Each technique solves one part. The summary loses detail with every round. The vector finds what is similar and ignores the link between records. The key-value store keeps the isolated fact, without the entity it belongs to.

Unstructured agent memory accumulates errors and mixes up contexts

The problems appear when agent memory leaves the prototype and serves many customers:

Accumulated error: a summary with one wrong piece of information is read as fact in the next session and starts being repeated.
Stale memory: the agent records as an account's finance contact a person who has already left the company, because the change in the CRM never reached the memory.
Mixed-up customers: a preference recorded while serving one customer is retrieved in the conversation with another, because the two texts were similar.
Ignored permissions: a note from legal about a dispute with the customer shows up in the answer for the sales team, which has no access to that document at the source.
Outside the perimeter: when memory is stored by the model or tool provider, the company's interaction history ends up living on external infrastructure, under that service's terms.

Five problems of unstructured agent memory in production — Five problems appear when agent memory grows without structure

The relationship between records turns storage into memory

In the anatomy of AI, the brain and memory correspond to the Company Brain, and the principle of that part is that storing is not remembering. A repository keeps the record of the support interaction. Useful memory knows that the ticket belongs to the same customer whose contract expires next month and whose last delivery was late.

That link depends on identifying the entity each memory refers to. A memory tied to the text of the conversation is only retrieved by similarity. A memory tied to the customer, the contract and the order is retrieved by relationship, regardless of how the question was written.

Agent memory that links records to the same customer, compared with a repository of loose records — In agent memory, the relationship between records turns storage into memory

The Company Brain links agent memory to the graph's entities

In the Company Brain, the interaction history is recorded in the knowledge graph. Each support interaction, decision or preference is linked to the entities it involves, with a date and source system. The business ontology defines what each record means, and a change in the CRM updates the entity the memory refers to.

Permissions are inherited from the source. A memory derived from a legal document carries that document's restriction and only comes back to people who already have access to it. Memory is served via MCP, so any agent in the company queries the same base, without each tool keeping its own copy of the history.

The sovereignty of agent memory depends on where it lives and who deletes it

The interaction history records what customers asked for, complained about and negotiated. In the Company Brain, that memory stays in the customer's environment, under the retention and backup policies the company already applies to the rest of its data. The model provider receives the slice needed for each call, according to the contracted architecture.

Deleting is also part of sovereignty. Article 18 of Brazil's data protection law (LGPD) gives data subjects the right to request the deletion of personal data processed on the basis of consent. With memory linked to entities, the request finds every memory associated with that person. With memories scattered across summaries and vectors in several tools, nobody knows for sure where they are.

Agent memory depends on up-to-date sources and retention rules

Memory inherits the quality of the source systems. If the CRM is out of date, the memory linked to it will be too. It is also up to the company to define what is kept, for how long and who reviews the memories the agents generate themselves, such as summaries and learned procedures.

A first use of agent memory starts with accounts that keep coming back

The recommended starting point is a process in which the same account comes back many times, such as serving larger customers or following up on critical suppliers. The first step is to list which facts and preferences are worth keeping and which entity each one links to.

Next, the sources for those entities are connected to the Company Brain. Over the following weeks, the team measures how many answers used memory and how many memories needed correcting.