MMOOZZGG.com

AI assistant

In short

AI agent conversion rate is judged with five metrics: median time to first reply, share closed without a person, share of requests that reach the CRM, share of handovers to a person, and deals. The article explains how to record a baseline before the start and which numbers mislead. The main point: count only requests that were given to the agent.

Key ideas

  • Use the median reply time, not the average, because one email answered after three days distorts the average.
  • The share of requests that reach the CRM is the main metric for sales.
  • Before the start, take a month of conversations and count the median reply time and the CRM share.
  • Compare the month before, the first week and the last five days, with three columns for each metric.
  • The most common mistake is counting all requests, including those never given to the agent.

Prepared by AI from the article

AI agent conversion rate: how to measure it with 5 metrics

GuideAdvertising and marketing

Medium4 min read

“The agent handled 3,000 dialogs” says nothing about money. Below are the five metrics of AI agent conversion rate we use to judge a pilot, how to measure them before the start so you have something to compare with, and which numbers usually lie.

On this page · 10 sections

1. Time to first reply

The median, not the average: one email answered after three days spoils the average, while the median shows what a typical client sees. Measure it in the CRM or messenger for a month before the pilot and during the pilot. This is the agent’s first win: a minute instead of hours.

2. Share of requests closed without a person

How many dialogs the agent brought to a result by itself: answered, booked, created a deal. Count only those where the client did not write the same question again. For support, our benchmark from the home page is 80% of typical questions; the pilot shows the exact share for your process.

3. Share of requests that reach the CRM

The main metric for sales. Before the agent, managers enter some requests into the CRM, and the rest live in chats. With the agent, each qualified request is a deal with a history. It is counted as the ratio of deals to requests.

4. Share of handovers to a person

Lower is not always better. Too low a share means the agent takes on what it should not; too high means the rules are not written down. In a pilot, this number usually falls by the end of the second week, and that is normal.

5. Deals and money

How many deals from requests the agent handled reached payment. It is counted last, because the deal cycle is longer than the pilot. So in the contract the pilot metric is the first four, and we look at the money a month or two later.

Which numbers lie

If you are deciding how to measure the performance of a chatbot or an agent, start with what not to count. “Number of dialogs”: an agent can run thousands of useless ones. “Customer satisfaction” from a “rate this answer” button: only a few people press it. “Time saved” without the 1.3 factor and without subtracting support. Check what the contractor reports on before you sign.

How to measure before the start

Take a month of conversations and count the median reply time and the share of requests in the CRM, by hand or with an export. It takes an hour, and it is the only way to say honestly later what changed. We do this with you in the first three days of the pilot.

What the pilot report looks like

The report has three columns for each metric: the month before the pilot, the first week and the last five days. We do not count the first days, because the agent is learning. An example of the structure without numbers: median time to reply; share of requests closed without a person; share of requests in the CRM; share of handovers to a person and their reasons by group; deals from the agent’s requests on the report date. Under the report is a list of changes over two weeks: what changed in the rules and why.

The same report becomes the basis of support: the same five numbers every month, so you can see whether the agent improves or degrades.

The most common mistake

Counting the metric over all requests, including those not given to the agent. If complaints and disputed cases go to people by the rules, they should not spoil the share closed without a person. Count by the groups of requests the agent handles, and show separately how many went to people and why. That keeps the number honest and comparable from month to month.

In short

Five metrics: median time to reply, share closed without a person, share of requests in the CRM, share of handovers to a person, and deals. Measure them before the start on a month of conversations, write the first four into the contract and look at the money a month or two later. Do not trust the number of dialogs or button ratings.

Publications

Contact us