Why AI pilots stall on data, and how to tell if yours is ready

Many AI pilots become "phase one" projects that never reach phase two. The model may work, and the team may be capable. In our team's experience, the real cause is often the same, and it rarely makes the post-mortem.

The data underneath isn't ready. The pilot proved the idea could work; going live proved the data couldn't carry it. By then, much of the budget has gone on tuning a system that was always going to give wrong answers.

How a promising pilot stalls

First, it looks promising. A team picks a use case, such as tender drafting, stock visibility or board packs, and runs a four-week pilot. The agent does something credible, leaders see a demo, and budget is committed.

Then comes connecting it to real systems. The team finds the source data is in twelve places, and one branch's ERP is two versions behind. Nobody on staff can explain how the overnight reconciliation works.

Timelines slip. The pilot now relies on a patchwork of manual exports, held together by goodwill.

Finally, the quiet rename. The phase-two budget moves to "data modernisation", and the agent never quite goes live. Six months later, another firm pitches the same use case, and the cycle restarts.

That matches what our team has seen on delivered projects: the model is rarely the problem. More often, it's the data.

An agent built on bad data gives you fast, confident, wrong answers. That's one of the worst failure modes, and hard to spot until it costs you money.

Four data problems that lead an agent to wrong answers

Here is what our team typically finds when it checks a company's data.

Numbers that don't reconcile. The P&L in the ERP doesn't match the reporting tool, and neither matches the CFO's spreadsheet. An agent querying any one of them returns a number that looks right and isn't.

The same thing under different names. One customer has four spellings in the CRM; one product has three SKUs, depending on the branch. An agent comparing systems, such as quotes against invoices, can match the wrong records.

Data that is out of date. A "current margin" comes from a cost file refreshed each quarter. Ask the agent a live question, and it answers from three-month-old data without knowing it.

Numbers without meaning. A "cancelled" order might mean the customer cancelled, the stock ran out or a rep made an error. A forecasting agent that can't tell them apart produces something that looks like a forecast but isn't one.

Projects that work start with the data, and share four habits

In our team's experience, reliable agents start with data work that often takes longer than the agent build. That's an awkward sentence for a sales page, but it's what we see.

The data work means connecting source systems, cleaning up product codes and agreeing what each number means. That plain work decides whether everything built on top of it works.

They start with an honest review, not a scoping session built to confirm the brief. Some reviews end with "your data is fine; here's the agent plan". Others end with "fix the data first", and both answers are useful.

They separate who builds the data from who uses it. The people who build the data platform own its data flows and rules; others build agents on it. Mixing the roles gives you an agent that works until its builder goes on leave.

They record every change to the data platform. Changes are reviewed and announced, like changes to software code. That sounds obvious until you meet a "data platform" that is a pile of Power Query scripts nobody dares touch.

They agree one definition per metric, in writing, before building an agent. "Revenue" means one thing, from one system, calculated one way. Disagreements get settled before the agent goes live, not after.

Six questions that show whether your data is ready

Ask these six questions before you scope an agent:

• Can you trace each key figure from its source system to the report, with every step written down?
• Is your data labelled in words any manager would recognise, such as customer, order and branch?
• Do customer, product and branch codes match across systems, or is there a written map between them?
• Does access to the data follow your existing staff permissions, not a separate list?
• Do agents and people draw on the same trusted data, so they get the same answers?
• Do you know how often each data feed refreshes, and is that fresh enough for the job?

If any answer is "no" or "not sure", fix the data before you build the agent. We won't build AI on data that isn't ready.

Fix the data first; the model matters less

In our team's experience, many companies need some weeks of data work before their first agent goes live. You can dislike that timeline, but you can't skip it.

The state of your data shapes the outcome more than the model you choose. An average model on clean, trusted, connected data will usually beat the newest model on messy data.

Our advice: worry less about picking the right model, and more about whether your data is ready for it.