Announcing US$ 3.2M pre-seed round · OneVC · Maya · Norte Ventures Read →
Back to blog

Context engineering:
what it is and how it changes the prompt

In short

An agent reviewing a refund request makes several calls to the model before it finishes the task. On each one, some part of the system decides what the model receives: the instruction, the policy in force, the customer's history, the result of the previous tool, the list of available actions. Context engineering is the discipline that makes that choice on every call. The quality of the answer and the cost of each run depend on it, and it rests on the Company Brain, the layer that knows the company (what data exists, where it lives, how it connects and who may see what).

Context engineering decides what goes into the model's window

A language model works only with what is in the context window at the moment of the call. Whatever stays outside the window plays no part in the answer. Context engineering treats that window as a limited resource and designs, for each step, the set of information the task requires.

The work comes down to four operations. Selecting which part of the policy goes in. Ordering, so that the instruction and the main data sit in a prominent position. Summarising the history of a long task. Discarding the result of a tool that has already done its job.

Prompt engineering is one part of context engineering

Prompt engineering is about how the instruction is written: how to phrase the task, which examples to include, which output format to ask for. It works well when the task fits in one question and one answer and the knowledge needed is already in the text being sent.

Context engineering includes that instruction and takes care of everything that reaches the model along with it: data retrieved from systems and documents, history, tool descriptions, results of earlier calls and memory of past interactions. In an enterprise agent, the instruction usually takes up the smallest part of the window.

Multi-step agents turned context into a bottleneck

The term gained ground when projects moved from isolated questions to agents that run sequences of steps. Each step adds a tool result to the history, and the window grows at every stage. Without a rule, the last call carries the leftovers of all the ones before it.

Larger windows ease the space limit and bring two side effects. The first is cost, because every token sent is billed on every call. The second is quality. In 2023, Liu and colleagues showed that model performance drops when the relevant information sits in the middle of long contexts, including in models designed for long windows. More text in the window can dilute the very passage that decides the answer.

The components of context have different origins and shelf lives

A call from an enterprise agent combines five components:

  • Instructions: the role, the rules and the output format, stable from one call to the next.
  • Retrieved data: system records and document passages fetched for the question at hand.
  • History: the earlier messages and steps of the same task.
  • Tools: the description of each available action and the results of the ones already triggered.
  • Memory: facts from past interactions that are still relevant, such as a commercial agreement on record with the customer.

Each component ages at its own pace. The instruction changes with each version of the agent, the history is valid for the task in progress, and data from a system can change between one call and the next.

The five components of context engineering and the shelf life of each one
In context engineering, each component of the call ages at its own pace

Context engineering failures have known causes

Errors usually start at one of these four points:

  • Excess: the pipeline sends everything that might be relevant, cost goes up and the decisive passage gets diluted in the middle of the window.
  • Conflict: two sources arrive with different values for the same data point, such as the credit limit in the ERP and the one in an old spreadsheet, and the model fills the gap with the most likely continuation.
  • Staleness: the policy indexed last month takes the place of the current one, and the answer comes out consistent with a rule that has already changed.
  • Missing permission: the context includes data the person who asked is not allowed to see, because retrieval used the agent's credentials.
The four most common context engineering failures
Excess, conflict, staleness and missing permission are the most common context engineering failures

Cost per token is set when the context is assembled

In enterprise agents, most input tokens come from the context, because the instruction is short while retrieved data, history and tool descriptions are long. Every unnecessary passage is paid for on every call and at every step of the agent.

A lean context has a second effect. When the information arrives resolved and free of conflict, the task asks less of the model, and many steps can run on smaller, cheaper models.

The Company Brain is the infrastructure for context engineering

Context engineering depends on knowing, for each question, what data exists, where it is, which version is valid and who may see it. When that answer is built inside each agent, every project redoes the same integration.

In the Company Brain, this work lives in a shared layer. The knowledge graph links the refund request to the customer, the original purchase and earlier tickets. The business ontology defines which policy applies to that type of purchase. The semantic layer translates the question into the right query and returns only the slice that is needed, with permissions inherited from the source system and the source of each piece of information.

That slice is served to the agent over MCP, the open protocol that connects models to data sources. The window receives less text, with no conflicting values and in the current version, and the next use case reuses the same context.

The Company Brain as context engineering infrastructure, serving the right slice over MCP
The Company Brain gives context engineering the right slice for each question

Context engineering starts by measuring each call

The first practical step is to record, for an agent in production, what goes into each call: how much comes from the instruction, from retrieved data, from history and from tools, and which passages actually show up in the final answer.

Passages that go in and are never used point to excess. Wrong answers caused by a conflicting or stale source point to where the first Company Brain slice should start. With that slice in place, context engineering for each new agent starts from a layer that already handles sources, versions and permissions.

Get the next articles on context architecture for AI