Before an AI pilot starts, you must decide the following: which process to start with, who owns the result, what success means and how you will measure it, which data and systems the pilot will use, what production will need, and when you will make the call to scale or stop. Settled in writing, these decisions make the pilot the first phase of a production system. Left open, they get made after the demo, under pressure, by whoever is in the room.
The cost of leaving them open shows up in the numbers. In Dun & Bradstreet’s survey of 10,000 businesses in 32 countries, published in October 2026, only 28% reported broad or strong returns from AI. Among businesses whose data was fully ready for AI, 81% did.
What needs to be decided before an AI pilot starts?
Six decisions, each written down and signed by the person who owns it:
| Decision | What to write down | Who signs |
|---|---|---|
| Use case | One process with a measurable cost and a team that wants the change | Business sponsor |
| Ownership | The leader accountable for the result, and the team that will run the system after launch | Business owner |
| Success criteria | One metric, its current value from your own data, the target to scale and the level at which you stop | Business owner and finance |
| Data and access | The systems and data the pilot uses, with access confirmed for production | Data and system owners |
| Path to production | Integrations, security review, operations and the budget after the pilot | IT and security |
| Decision | The date of the go or no-go call and the possible outcomes | Sponsor |
The list is short because each item closes a known way to fail. In a Dataiku and Harris Poll survey of 600 CIOs, run in December 2025 and January 2026, 85% said gaps in traceability or explainability had already delayed or stopped AI projects from reaching production.
Who should own an AI pilot?
The business leader whose numbers the system will change should own it, with a named team ready to run the system after launch.
A pilot owned by the team that builds it has nobody waiting for it at the end. A system that drafts quotes or sorts support cases changes someone’s daily work, so the owner of that work has to agree to the change before the build starts. Name the people who will run the system too, because operating it is a job that starts on the day it goes live.
How do you set success criteria for an AI pilot?
Pick one metric close to the P&L, measure its current value from your own data, and agree the target and the stop level before anyone builds.
Cost per case, time to quote and first-time-right rate all work, because someone already tracks them. Build the benchmark from real cases in your own systems, with the correct answers agreed by the people who do the work today. Every demo after that gets judged against it. Criteria written after the results come in tend to describe whatever the pilot managed to do.
Why should a pilot run on real data and real systems from the start?
A pilot that skips integration, security and real data meets those problems later, when they cost more to fix.
In the same Dun & Bradstreet survey, only 8% of businesses said their data was fully ready to support AI at scale. None of them show up in a demo built on a clean export. Connect the pilot to the systems it will use in production, bring security in during the first weeks, and build testing, monitoring and operations in from the start. The first version then needs hardening rather than a rebuild.
Put the production phase in the budget at the same time. A pilot that succeeds and then waits a quarter for new funding loses its team and its sponsor.
How should the go or no-go decision work?
Set the date and the possible outcomes before the pilot starts, then read the numbers against the criteria you signed.
Three outcomes cover almost every case: scale it, fix one named gap by a set date, or stop. A clean stop is a result, since it frees the budget and keeps the benchmark, the data access and the integrations for the next use case. The outcome to avoid is the open-ended extension, where the pilot keeps running because nobody agreed what would end it.
What does a pilot built for production look like in practice?
It looks like the first phase of a live system, with the same owner, data and success criteria all the way through.
BOI is an AI deployment studio. We agree success criteria before we build, with a benchmark made from your own data, and deliver a working version within weeks. Testing, security and operations are built in from the start, the system runs in your own cloud, and your team receives the code, tests, agent instructions, runbooks and dashboards. For a global industrial manufacturer, systems built this way now process about 45,000 support emails every night. See how we build production-grade AI systems inside your architecture, and read why enterprise AI adoption stalls at scale for the organizational side.
If you have a pilot planned for next quarter, write down the six decisions this week and see which ones nobody can sign yet.
Summarize this article
Share
Emili Castro
Marketing
