# AI Investment Without a Plan: Where Budgets Leak in the First Year of the Hype

Author: Marin Niehues, Niehues Consulting · Published: 2024-06-20 · AI Strategy & Business Case
Source: https://www.niehuesconsulting.de/en/blog/ai-investment-without-a-plan/

AI budgets leak when no measurable business outcome, no owner and no operating plan exist before the money is spent. The spend then flows into licences and pilots that are used but change no figure in the company. This article shows the five places where this happens, a calculation method for the economics and a checklist for tying every release to a result.

Since ChatGPT appeared in November 2022, almost every executive team has faced the same situation: interest in the business units is high, vendors make many offers, and competitors seem to move faster. Releasing money in that situation is understandable. Releasing it without a plan means that twelve months later the invoice is on the table and the effect is not.

**An AI investment is well placed when, before release, three things are settled: which metric changes by what amount, who is accountable for it, and who runs the solution from day one.**

## How widespread is AI in business, and why does it rarely pay off at once?

According to the statistical office of the European Union, 8 per cent of enterprises with at least ten employees used AI technologies in their business in 2023. Adoption is therefore still low, yet many executive teams feel strong competitive pressure. That tension creates the urge to buy quickly and to clarify the purpose later.

Research shows at the same time that the benefit is real but conditional. A field study of more than 5,000 customer support agents at a software company found that an AI assistant raised the number of issues resolved per hour by 14 per cent on average, and by far more for novices than for experienced agents. An experiment with 453 professionals in occupations with writing tasks found that completion time fell by about 40 per cent and rated quality rose by 18 per cent. Both studies measure effects at the level of individual tasks or a clearly defined workplace.

Between an effect on a task and an effect on the profit and loss account lies a path the organisation has to walk. In 1987 the economist Robert Solow observed that the computer age was visible everywhere except in the productivity statistics. The observation applies to AI again: twenty minutes saved per day become a result only when someone decides what the time is used for.

## In which five places do AI budgets leak?

AI budgets leak in five typical places: licences bought without a use case, pilots without a route to operation, data work without an owner, missing enablement of users and spending without a metric.

The first is the licence bought in advance. A company provides access for all employees because vendors bundle packages and nobody wants to be last. If the usage statistics show after three months that only a quarter of the accounts are used regularly, nobody can say which work has changed. Licence costs continue, and the decision to renew or withdraw is taken without a yardstick.

The second is the pilot without a route to operation. A prototype can be built in a few weeks and demonstrated convincingly in a presentation. Whether it holds up in daily work shows only once real data, real system connections and real users are involved. Anyone who funds the pilot without planning the operation pays for a demonstration. Since 2015 the paper by Sculley and colleagues on hidden technical debt in machine learning systems has described how the model code makes up only a small part of the overall system, while most of the work lies in data handling, configuration and operation.

The third is data work without an owner. Almost every application needs data that does not yet exist in the required form. If nobody in the business unit is responsible for quality and upkeep, a data project consumes budget and leaves behind stocks that are out of date after a year.

The fourth is missing enablement. A tool whose possibilities users do not know is applied to tasks it suits poorly, or not at all. The field study on support agents shows that the effect varies strongly with the experience of users. This argues for budgeting enablement as part of the investment.

The fifth is spending without a metric. If benefit is recorded only as satisfaction or perceived time saving, no calculation can be drawn up, and in the next budget round the AI spend competes with initiatives whose effect is documented.

**The five leaks at a glance**
Leak | How to recognise it | Countermeasure before release
Licence in advance | Usage rate below 50 per cent, no named use case | Release per use case and user group, extension after proof
Pilot without a route to operation | Prototype runs on sample data, nobody owns operation | Name the operations owner before the start
Data work without an owner | Data quality is not quantified in the presentation | Data owner in the business unit, sample check before the start
Missing enablement | Access exists, use concentrated on a few people | Enablement and process changes included in the budget
Spending without a metric | Benefit reported as a mood picture | Metric, baseline and target value before release

## How do you calculate the return on an AI investment?

An AI investment pays off when the realised annual benefit exceeds the total cost of licence, integration, operation and enablement. The calculation has four steps, and the third decides the outcome.

In the first step the theoretical benefit is determined: number of users times hours saved per week times working weeks times cost rate per hour. In the second step the total costs over twelve months are assembled. In the third step the benefit is multiplied by a realisation rate, which states what share of the time saved actually turns into additional output, quality or avoided cost. In the fourth step the difference is formed and submitted to the business unit for confirmation.

The following example is illustrative and shows the logic for an assistant in a department with 200 users.

**Worked example for 200 users**
Item | Assumption | Amount per year
Theoretical benefit | 200 users, 1 hour per week, 46 weeks, 60 euros per hour | 552,000 euros
Realisation rate | 20 per cent of the time saved becomes additional output | 110,400 euros
Licences | 30 euros per user per month | 72,000 euros
Integration, operation, enablement | Assumption of IT and the business unit | 30,000 euros
Result | Realised benefit minus total cost | plus 8,400 euros

The calculation shows that the investment looks excellent at the theoretical benefit and only narrowly positive at a realisation rate of 20 per cent. At 15 per cent it would be negative. This sensitivity makes the realisation rate the most important assumption, and it can be substantiated only in the business unit: a team that plans and reallocates the freed time reaches higher values than a department with nothing new in mind for it. Why expectations of new technology first overshoot and distort budget decisions is described in the article on the [AI hype cycle](/en/blog/ai-hype-cycle-progress-versus-announcement/).

## Which questions should a release answer?

A release should answer six questions before the money flows. The checklist suits every investment above a threshold the executive team sets.

1. Which metric changes, by what amount, and who in the business unit is accountable for it?
2. What baseline does the metric have today, and who measured it?
3. Which data does the application need, in what quality is it available, and who maintains it in operation?
4. Who runs the solution from day one, and what capacity is reserved for it?
5. How are users enabled, and which processes change?
6. After what period and at what result will the initiative be extended, changed or ended?

If the answer to one question is missing, the release is granted in stages: first a budget for clarifying the open points, then the budget for the prototype, then the budget for operation. This approach costs little time and prevents money being spent on assumptions nobody has tested. How decision-makers turn pressure into a plan is described in the article on [AI activism at board level](/en/blog/ai-activism-board-pressure-into-plan/).

## How do you tell whether success stories hold up?

Success stories about AI hold up when they name a metric, a baseline, a period and the state of operation. Reports that give only participant numbers, usage rates or satisfaction show activity but no result. This distinction helps in the budget round as much as in assessing vendor claims. An article on [checking AI success stories](/en/blog/ai-case-studies-check/) deepens the criteria.

For internal reporting a fixed template with four lines per initiative is enough: metric, baseline, current value and state of operation. It makes initiatives comparable and shows the executive team early which spending has an effect and which only causes cost.

## Investment planning with Niehues Consulting

[Niehues Consulting](/en/) is a consultancy for AI management and organisational design from a delivery perspective, based in Düsseldorf. Founder Marin Niehues brings AI into operation, in projects from VC-funded scale-ups to DAX 40 groups and in more than 20 countries on six continents. In the AI management service area the work covers business analysis, requirements, the business case, the selection and sequencing of applications, and portfolio steering.

The approach follows the Niehues Delivery Model with three phases. Three of its test questions matter most for the investment decision: which business outcome should change measurably? Is the business case confirmed with real values? Which metric shows after 90 days that the solution is landing? Scope and terms of an engagement are discussed in the free initial consultation, which you can arrange in 30 minutes through the [contact form](/en/#kontakt).

**Conclusion**
AI budgets leak in five places: licences without a use case, pilots without a route to operation, data without an owner, missing enablement and spending without a metric. The countermeasure is a staged release with six questions and a calculation whose realisation rate the business unit confirms. That turns the pressure to act quickly into a plan the executive team can substantiate in the next budget round.

**Sources**
- Eurostat: 8% of EU enterprises used AI technologies in 2023, Eurostat News, 2024
- Brynjolfsson, Li, Raymond: Generative AI at Work, NBER Working Paper, 2023
- Noy, Zhang: Experimental evidence on the productivity effects of generative artificial intelligence, Science, 2023
- Sculley et al.: Hidden Technical Debt in Machine Learning Systems, NeurIPS conference paper, 2015
- Solow, Robert: We'd better watch out, New York Times Book Review, 1987

## Frequently asked questions

### How much should a company invest in AI?

The amount follows the benefit of the prioritised applications and at the start equals the cost of one to three initiatives. A fixed share of revenue justifies no decision, whereas a budget per initiative does. Release it in stages and raise it once the effect is proven.

### Why do so many AI projects deliver no measurable benefit?

Many AI projects deliver no measurable benefit because neither a metric nor an owner nor an operating plan was set before the start. The prototype shows what is technically possible, and nobody is responsible for its use in daily work. Time saved also becomes a result only when the business unit reallocates the freed time.

### How do you calculate the return on investment of an AI solution?

The return on investment is the realised annual benefit minus total cost, divided by total cost. Total cost covers licence, integration, operation and enablement. The benefit is multiplied by a realisation rate that the business unit confirms.

### How can you tell that AI licences go unused?

You recognise unused licences by the usage rate per account that the vendor shows in its admin console. If it is below 50 per cent after three months, a named use case or enablement is usually missing. Withdrawing accounts and concentrating on user groups with a use case lowers costs quickly.

### Should a company build data first or test applications first?

Both run in parallel, and the use case determines which data is built. Data work without a named use case ties up budget without benefit. Check by sample for each initiative whether the data suffice for the use case before you start a data project.

### Who decides on AI budgets in the executive team?

The executive team decides on criteria and total budget, the business unit owns the metric, and IT owns feasibility. A joint release meeting with a recorded decision prevents initiatives from starting without an owner.

### When is an AI pilot worthwhile?

A pilot is worthwhile when it tests an assumption of the business case in the real process with real data and has a stop criterion. A pilot without a question to answer delivers only a demonstration. Define before the start which result leads to continuation and which to termination.