Announcing US$ 3.2M pre-seed round · OneVC · Maya · Norte Ventures Read →
Back to blog

AI pilot:
three patterns that stall the move to production

In short

An AI demo usually goes well in the meeting room. The team approves it, and the project joins a queue of adjustments that lasts for months. Then another team starts a new AI pilot, with another dataset and another integration. The cycle repeats because the test was designed to demonstrate, without the conditions that production demands. The starting point changes when the pilot is built on the Company Brain, the layer that knows the company: what data exists, where it lives, how it relates and who may see what.

An AI pilot tests a real use case with limited scope and time

The pilot puts AI into a business process, with real users and a set deadline. The proof of concept comes first and checks whether the technique works. The pilot decides whether the use case goes to production, with everyday volume and data.

In July 2024, Gartner predicted that 30% of generative AI projects would be abandoned after proof of concept by the end of 2025. The reasons it gave were poor data quality, inadequate risk controls, escalating costs and unclear business value. The three patterns below show how those reasons enter the design of an AI pilot.

Three patterns hold the AI pilot back before production

Pattern 1: the lab pilot runs on hand-prepared data

To save time, the team exports a sample of tickets or orders and fixes the fields by hand. The demo goes well because people chose the data.

In production, data comes from the live systems. The same supplier has two records and the cancellation reason is blank in part of the records. Quality drops, and the integration the test avoided has to be built from scratch.

The way out is to connect the real source from day one, with a smaller scope if needed: one ticket type, one product line, one region. Data problems show up during the test, while they are still cheap.

AI pilot on lab data compared with the real production data
An AI pilot built on hand-prepared data does not find the same data in production

Pattern 2: a pilot with no owner ends with no decision

The pilot starts in innovation or in IT, and the team that will run the process follows from a distance. The evaluation uses technical criteria or the impression the demo left.

When time runs out, nobody has the authority to say the result is good enough. Without a baseline measured beforehand, there is no way to show what changed in the process, and the pilot stays in permanent extension.

Before starting, the team that inherits the process names an owner, picks a business metric (closing time, rework on invoices, customer response time) and measures the current value. The criterion for moving to production is written down at the start.

Pattern 3: a pilot outside the perimeter stalls at approval

To move fast, the pilot uses a service account with broad access or a copy of the data in another environment, without security, legal and privacy taking part.

When it is time to go to production, those teams ask the questions that were put off: who can see each answer, where the data stays, how Brazil's data protection law (LGPD) applies and how to audit what the system did. Answering them means redesigning access, and the redesign is the size of a new project.

Security enters the design from the start, and each person accesses what they can already see in the source systems. Each answer points to its source, which gives the review something concrete to assess.

The cost of an AI pilot that never ships repeats in the next pilot

The lost investment is the visible part. The larger part shows up in the next project, which redoes the data extraction, the integration with the same CRM or ERP, the security review and the context text assembled for the model. The company pays several times for the same foundation and never finishes it.

There is also the cost per call. With disorganised context, the test compensates with larger prompts and more expensive models, and the cost per answer ends up above what the operation accepts.

With every test that fails to reach production, it gets harder to secure time and an owner from the business for the next one.

Cost of an AI pilot that redoes extraction, integration and security review with every new test
Each new AI pilot pays again for the same foundation

The Company Brain changes the starting point of every AI pilot

The Company Brain connects CRM, ERP, helpdesk and contracts once, inside the company's environment. The knowledge graph links records of the same entity, the ontology defines each term and the semantic layer translates the question into the right query. The pilot starts on the real source.

Permission is inherited from the source system, and each answer points to the record it came from. The security review assesses a single architecture, which also holds for the use cases that follow.

Context is served over MCP to any model or agent. The second pilot reuses the sources, definitions and permissions of the first and adds only what is missing. Because the model receives only the slice the question needs, the cost per answer falls and the same task can run on a cheaper model.

AI pilot built on the Company Brain, with real sources, inherited permission and reused context
On the Company Brain, each AI pilot starts from the sources and permissions already connected

Owner, metric and reachable sources remain prerequisites

The context layer addresses the first and third patterns in the architecture. The second is organisational. Without a team that takes on the process and a metric measured beforehand, an AI pilot on the Company Brain also ends with no decision.

Sources need to be reachable through an API or a database, and the ontology's definitions depend on the teams agreeing. Quality problems show up earlier and call for a fix at the source.

The next AI pilot starts with three written answers

Before approving the next test, three questions need written answers: which system the real data comes from, who answers for the process and with which metric measured today, and which permission each user inherits. A blank answer shows which pattern the project is going to repeat.

For pilots that have already stalled, the same exercise works as a diagnosis. The pattern that stalled the project shows whether the way forward is to connect the real source, name an owner or bring security into the design, and where to start the first slice of the Company Brain.

Get the next articles on AI implementation