Hello. This is Alex Bakker with what’s important in the IT and business services industry this week.
If someone forwarded you this briefing, consider subscribing here.
What You Need to Know
A useful (if oversimplified) model for thinking about economics is that time is money. Implications of this way of thinking include: money is stored time, saved money is saved time and spending more money essentially equates to putting more time on a problem. Of course, just like nearly all exchanges, the ratio of time to money is not perfectly one to one, but the idea that time is money sits at the core of how we prioritize spending, investment and work.
In the technology industry especially, these exchanges are incredibly important, since technology both consumes time in the building of it and produces tools that save time. Organizations, therefore, often try to evaluate how much time it will take to build and run a tool versus how much time the tool will save. When they look at their input time in terms of money and compare it to time saved in terms of money, they end up with an ROI case.
You may have noticed that I’ve intentionally ignored words like value, productivity and efficiency, but it should be obvious that those words for describing the exchange rate of time to money and back again.
The Need for New Calculations
Today, AI is breaking some of these core assumptions and, while organizations aren’t talking about it, they are struggling with the fact that the thinking described above doesn’t work anymore.
Let’s consider two scenarios that an enterprise might have to compare in their invoice processing:
Scenario One: A member of the finance team receives an invoice, applies some judgement, goes through an approval process, enters data into software and eventually pays the invoice. Assuming this process is reasonably modern, the software has helped optimize the time spent, and the organization incurred costs in the form of both human and system time spent on the process, with notionally lower cost than the work would have required had it totally lacked tools and technology.
Scenario Two: An enterprise has spent some money on agentic AI invoice processing software. The agent receives the invoice, applies some AI-generated judgement to evaluate the invoice, uses the software to document its evaluation and pays the invoice. In this case, the organization spent agent time to process the invoice, incurred some costs for the tools and notionally saved some costs relative to doing the process manually.
Even if we make the assumption that both tools produced the exact same outcome, the challenge is you cannot definitively say either of these statements:
- Scenario Two was faster than Scenario One, or
- Scenario Two was cheaper than Scenario One
In fact, we don’t really have any information about the time/money tradeoff of the scenarios because we don’t know how many tokens the AI consumed or how much it costs to run it.
The True Cost of AI
For those of you who have been hiding under a rock, AI processes requests in tokens (see OpenAI’s explanation), which includes instructions (prompts) as well as any internal reasoning, recursion, etc. But a token has no time-dependence. Yes, AI models tend to have a token rate – the speed of response – but they use different numbers of tokens to accomplish similar tasks. And, because of the varying assemblies of prompts, logic, context and other tools used in building agents, it’s difficult to relate the token cost of a query to its output.
The problem really is that agentic costs are driven primarily by token consumption, and the non-agentic costs are driven primarily by time (human + machine time). This means we can’t balance the equation to directly compare the scenarios.
In the simplified example above, it may seem like a trivial distinction, but even a simple example can be extrapolated to pose some challenging questions. For example, if Scenario One takes one hour and net costs the company $20, and Scenario Two costs $20 in tokens, then was it worth it? The answer, economically, is it depends. We don’t know how much time the agent took because the costs in tokens don’t ever convert into time.
We said earlier that a core of the economic thinking was built on the understanding that money was stored up time – and all valuation methodologies tend to come back to that same thinking. Return on investment, internal rate of return, net present value, total cost of ownership – all of these are different ways of quantifying value in terms of time. This doesn’t mean you can’t compare, but it makes the business case much harder. Instead of building a business case, you’re obligated to do some R&D first.
What Gets Measured, Gets Managed
The only way to compare AI work to non-AI work is to profile the AI in production. This is the crux of the problem: the only way to figure out the value of your money spent on AI is to convert the work to AI first and then measure the time and result of the work done by AI. Empiricism, in this case, is the only approach.
We’ve been saying for a long time that there are no fast followers in AI. AI use cases tend to be spread out across many different ideas and tend to be very company specific. What this means is that businesses are struggling to predict if their AI use case will work, and the industry lacks blueprints for “first time right” AI deployments. Right now, the only way to find out is to try.
About the author
