MMOOZZGG.com

AI assistant

In short

AI pilots fail because of three gaps: no metric, no link to company systems, and no owner. MIT Project NANDA found that 95% of organizations got no measurable return from generative AI, and Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027. The main takeaway is that a pilot should cover one process, one channel, and one metric written into the contract.

Key ideas

  • MIT Project NANDA found that 95% of organizations got no measurable return from generative AI.
  • Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027.
  • The three gaps named in the article are no metric, no link to company systems, and no owner.
  • A pilot works with one process, one channel, and one metric written into the contract.
  • The article lists five questions to ask any vendor before you sign.

Prepared by AI from the article

Why AI pilots fail and how to run one that does not

NewsConsulting

Easy3 min read

Why AI pilots fail comes down to three gaps: no metric, no link to company systems, and no owner. MIT Project NANDA found that 95% of organizations got no measurable return from generative AI. Gartner expects over 40% of AI agent projects to be canceled by the end of 2027.

On this page · 7 sections

Overview

A pilot is a short, small test of one process before a larger rollout. A good one gives you a number after 14 days, not a presentation.

What did MIT find about AI pilots?

MIT found that most money went into pilots that never reached daily work. The 2025 report “The GenAI Divide” is based on interviews in 52 organizations, 153 surveys of executives, and more than 300 public projects. It covers the period from January to June 2025.

  • Over 80% of companies tried general tools such as ChatGPT and Copilot, and about 40% rolled them out.
  • Custom systems did worse: 60% evaluated them, 20% reached a pilot, and 5% reached production.
  • The main cause was not infrastructure or regulation. The systems did not remember feedback, did not adapt to context, and did not improve over time. Staff lost trust after the first errors.
  • Most budgets went to sales and marketing, where results are easy to show. Back-office automation often paid back better.

What does Gartner predict for AI agent projects?

Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027. In its note of 25 June 2025, it names three reasons: rising costs, unclear business value, and weak risk control.

Gartner also sees “agent washing”. Vendors relabel old chatbots and scripted automation as agents. Of the thousands of vendors that call their products agents, it counts about 130 real ones. A chatbot is a program that exchanges messages with a person in a chat window, on a website or in an app.

The market is not going away. Gartner expects agents to make 15% of day-to-day work decisions by 2028 and to appear in a third of enterprise software. Projects without a metric are the ones that drop out.

What are the most common mistakes in an AI pilot?

Three mistakes repeat in the pilots we review.

  1. No owner and no definition of success. A team buys a bot subscription, connects it to the website, and never writes down what counts as a result.
  2. Too wide a start. The team launches on all channels and processes at once. When the agent makes one mistake, everyone switches it off everywhere.
  3. No access to company data. A knowledge base is a set of selected and prepared company documents that an AI assistant searches for answers: policies, instructions, a catalog, and answers to common questions. Without it, and without a CRM link, the agent gives general answers until customers stop writing.

What do we see in reviews in Russia?

The same three mistakes. A company bought a bot subscription, connected it to the site, named nobody responsible and did not write down what counts as a result. It launched on all channels and processes at once, and when the agent erred in one, switched it off everywhere. It gave the agent no access to the CRM and knowledge base, so it answered in general words until customers stopped writing.

Russia adds a fourth: the data went to a foreign service, and the lawyer closed the project, not the owner.

How should a pilot be designed?

A pilot works when it has one process, one channel, and one metric written into the contract. Typical metrics are response time, the share of conversations saved in the CRM, or the share of questions closed without a person.

Our 14-day pilot follows five rules.

  • The term is 14 days from contract to report. The price and the term are fixed in the contract.
  • The agent connects to your CRM and knowledge base on day one. Without that, it is a demo, not a pilot.
  • One of your employees spends about 30 minutes a day reviewing answers, and the rules improve as you go.
  • The final report has three numbers and a decision: stop, keep the agent on support, or build a full AI department with the pilot credited.
  • If the project does not pay back, the report says so.

What should you ask any vendor before a pilot?

Ask five questions before you sign.

  • Which metric will you track, and where is it written?
  • Which of our systems will the agent connect to during the pilot?
  • Who on our side is responsible, and how much time will it take?
  • What happens to rules, data, and access rights after the pilot?
  • What happens if the metric is not reached?

Questions and answers

Is it true that 95% of AI pilots fail?

MIT NANDA reported in 2025 that 95% of organizations saw no measurable return, and only 5% of custom AI tools reached production. The finding describes pilots with no metric and no link to working systems.

How can you tell an AI agent from a repackaged bot?

Ask which systems the agent is connected to, what it does without a person, and which metric is in the contract. Gartner counts only about 130 real agent vendors among thousands.

How do you run an AI pilot that gives a number?

Take one process, one channel, and one metric. Write the metric into the contract and connect the agent to the CRM and knowledge base from day one.

Contact us