AI Strategy

Your AI ROI Numbers Are Lying to You

By Alyssa Caridi · Last updated July 6, 2026

• AI ROI
• AI Measurement
• CMO
• AI Strategy

In February 2024, Klarna announced its AI assistant was doing the work of 700 customer service agents and would add $40 million to profit (Klarna). Fifteen months later the CEO said the cost-cutting had gone too far and began rehiring people (Entrepreneur). The number was accurate the day it went out. It simply was not the whole picture, and the correction took more than a year to surface. That lag is the real risk in AI ROI reporting. Emergn's 2025 survey of more than 700 senior leaders found a quarter admit their status reports read rosier than the facts support, and IBM found only 29% of executives can confidently measure AI ROI at all. The figure on your slide has usually been softened before you see it.

What does a softened AI report actually look like?

Here is the move in its smallest form. A marketing team runs a content-automation pilot with one goal agreed up front: cut cost per published asset by 30% in the quarter. The quarter closes with cost per asset down 9%, well short of target. The status update does not say that. It says adoption is strong, the team is saving about six hours a week, and output is up 40%. Every one of those statements is true. None of them is the number the pilot was funded to move. Three quiet substitutions carried the report from red to green:

• Usage stood in for cost. "Adoption is strong" answers a question nobody funded.
• Hours saved stood in for dollars returned. Six hours a week feels like ROI without being priced as ROI.
• Output volume stood in for the target. "Up 40%" is impressive and unrelated to the 30% cost goal that vanished.

In the program reviews I run with clients, this is the fastest tell: a status deck that leads with adoption metrics and never shows the target the program was funded to hit. The deck is not dishonest. It answers easier questions than the one that was asked.

Why does the number get softened on the way up?

None of this requires anyone to lie. It requires ordinary incentives and thin visibility. Each person in the reporting chain has a reason to present progress: budget already committed, headcount defended, a personal bet on the tool. Emergn's research found that only 32% of leaders could produce a complete, real-time view of every live AI program on demand, so most reporting is assembled after the fact, when the story is easier to shape. Sunk cost does the rest. 27% of leaders had watched a program continue purely because of what had already been spent (Resultsense, reporting on Emergn). Klarna is the pattern at scale. The 700-agent figure shipped in a press release within a month. The correction took more than a year and a Bloomberg interview to arrive, and only after service quality slipped enough for customers to notice. Optimistic numbers move quickly through an organization. The facts that would revise them tend to sit in an inbox.

Is your problem measuring AI ROI or reporting it honestly?

Two separate failures are stacking here, and telling them apart decides what you fix. The first is capability. Most teams cannot measure AI return cleanly: IBM's 2025 CEO study of 2,000 executives found only 25% of AI initiatives delivered the ROI leaders expected (IBM, 2025), and MIT's 2025 review of enterprise generative AI found 95% of pilots produced no measurable impact on profit and loss (MIT, via Fortune). The second failure is candor. Even when a real number exists, a quarter of leaders admit it reaches the board in more flattering shape than the facts support. Fixing only the first gives you better dashboards that still get softened before leadership sees them. Fixing only the second gives you honest reports of numbers no one trusts. The verification layer below sits on top of measurement and targets both.

How do you measure AI ROI honestly?

Stop trying to produce a single ROI number. The honest output is a ledger kept in three tiers, sorted by how sure you are, and the tiers never get added together. The discipline that resists gaming is refusing to let a confident dollar and a hopeful dollar sit in the same total.

Tier one: what you can bank

Money that has already changed the P&L. A role you did not backfill, a freelance line you cut, a tool you retired, a retainer you signed for a service AI now lets you sell. This is the only tier you are allowed to call return, because it is the only tier a CFO could audit. Most reports never populate it, because most "savings" are still sitting in tier three wearing a tier-one label.

Tier two: what you can attribute

Results that are real but rest on a comparison. A campaign that converted better than a matched holdout, retention that moved against a clean baseline. These count, but only with the method printed next to the number. "Conversion up 12% versus control" is a tier-two claim. "Conversion up 12%" with no control is a coincidence you are hoping to bill for.

Tier three: what you are projecting

Weighted pipeline from work AI helped you win or pitch faster, plus any freed capacity you have not yet turned into cash. Everything here is risk-adjusted by your real win rate and labeled as projection. It informs decisions. It is never reported as return until it lands in tier one.

How do you count freed capacity?

You do not count it as value at all until you route it. Freed hours are potential, worth zero on their own. They become return only when they resolve into one of two things: a cost you actually removed, or billable and new-business work that actually got booked. So the question for every block of saved time is where it went, and the honest default answer is that it evaporated into slack unless you can name the invoice or the eliminated cost it became. Six hours a week saved across a team is a headline. The dollars those hours turned into, or the admission that they turned into nothing, is the measurement.

Subtract the cost of running the AI

The license and build cost are obvious. The one that hides is human oversight. The hours your team spends reviewing, re-prompting, and fixing AI output are a direct charge against the hours it saved. A workflow that saves six hours and adds four hours of quality control saved two. Ramp counts too, because the first quarter of any tool is mostly learning curve. A gross number that skips these overstates the return, often by enough to turn a real loss into a reported win.

Measure the outcome, not the activity

Adoption, output volume, and hours saved are leading indicators. They tell you the tool is being used. They cannot stand in for the lagging number, which is cost removed or revenue booked. Match your measurement window to when that lagging value actually matures. Content built this quarter shows up in pipeline one or two quarters out, so a 30-day ROI read on a content tool is measuring effort, not effect.

Two rules keep the ledger honest once it exists. Agree the kill criterion before launch, the result that would end the program, so tier-three optimism cannot quietly fund a project that has already failed. And give the ledger to someone whose budget does not depend on it looking good, because one degree of separation between the owner and the scorekeeper removes most of the softening. A ledger kept this way describes what happened instead of what everyone hoped happened, which is the point of keeping it. It does not make the AI perform better. It lets you see clearly enough to move budget toward the programs that are actually paying back, which is where the return has been waiting the whole time.