# The AI Hype Cycle: How Executives Tell Progress from Announcement

Author: Marin Niehues, Niehues Consulting · Published: 2023-10-23 · AI Strategy & Business Case
Source: https://www.niehuesconsulting.de/en/blog/ai-hype-cycle-progress-versus-announcement/

Executives recognise progress when an AI capability is available to buy, has been measured in their own workflow and shows a benefit that appears in a metric. An announcement, a demonstration or a preview does not yet prove that. This article places the hype cycle's phase model in context, offers five maturity levels as a testing tool and uses published studies to show what an effect looks like once it exists.

Since the release of ChatGPT at the end of November 2022, new announcements have reached the executive team every week: more capable language models and image generators, assistants inside office software, enterprise versions for business use. Each announcement comes with the claim that it will change the industry, and each raises the question in the organisation whether it must act. This article provides a yardstick for placing such announcements within a few minutes.

**An announcement is a promise, whereas progress is a measured difference in your own operation. Five maturity levels lie between them, and the investment decision belongs at the top of that staircase.**

## What is the AI hype cycle?

The hype cycle is a phase model describing the typical course of attention to a new technology in five stages: a trigger, a peak of inflated expectations, a trough of disillusionment, a slope of realistic application and a plateau of productivity. The model is a rule of experience and not a forecast. It does not predict when a given technique reaches which stage, and it applies to each application, not to a technology as a whole. The same language technology can sit on the plateau for text summarisation while its use in decisions with high potential for harm is still at the start.

A related observation, attributed to the futurist Roy Amara, holds that the short-term effect of a technology is overestimated and the long-term effect underestimated. Economic history knows the pattern. The economic historian Paul David showed in 1990, using electrification, that electric power brought factories little productivity for decades as long as operators replaced steam drive one for one. The gain arrived when they redesigned workshops and workflows. For executives this is the most important lesson of the cycle: the technology is the smaller part of the work, and the larger part lies in the workflow in which it is used.

## How can you place an AI announcement on five maturity levels?

An AI announcement can be placed on five maturity levels that ask what the vendor actually puts on the table. The higher the level, the more evidence stands behind the statement.

**Five maturity levels from announcement to proven benefit**
Level | What exists | Typical source | Consequence for executives
1 Announcement | A promise with a timetable | Press release, keynote | Observe, plan nothing on it
2 Demonstration | A showing under ideal conditions | Video, trade fair, live demo | Ask about data and error rate
3 Preview | Access for selected users | Waiting list, beta | Test on a small scale, make no promises to customers
4 Generally available | An orderable product with price and support | Price list, documentation | Start a pilot with a clear metric
5 Benefit proven | A measured effect in a comparable operation | Study with a control group, own measurement | Plan the rollout and calculate the business case

With this placement, any announcement of recent months can be judged. GPT-4 has been available through paid access and an interface since March 2023 and therefore sits at level 4, as does ChatGPT Enterprise since August 2023. New image-understanding functions and assistants for office applications are announced or being rolled out, and count as level 2 or 3 depending on access. Anyone who names the level has already answered the executive team's most important question: can I order it tomorrow, and who has measured it under my conditions?

## What does proven effect look like?

Proven effect appears in studies with a control group that report an effect in a metric and name its limits. Three studies from 2023 show what level 5 means.

A field study of more than 5,000 customer-service staff, which Brynjolfsson, Li and Raymond published as a working paper in April 2023, measured how many issues are resolved per hour when a generative assistant suggests replies. On average the number rose by 14 per cent, by 34 per cent for novices and barely at all for highly experienced staff. An experiment by Noy and Zhang with writing tasks drawn from professional work, published in Science in summer 2023, cut the time taken by around 40 per cent, and independent raters judged quality about 18 per cent higher. A study presented in September 2023 by Dell'Acqua and colleagues with 758 management consultants showed that participants using GPT-4 completed 12.2 per cent more tasks and worked 25.1 per cent faster on tasks inside the technology's capability frontier. On a task outside that frontier they were 19 percentage points less likely to be correct than the control group.

Two lessons sit in these results. First, the effects are real and strong when task, tool and workflow fit together. Second, the frontier of capability is uneven, so an application on the same platform helps in one case and harms in the next. Each study is confined to one type of task, which means a result from customer service justifies no assumption about invoice checking. Executives gain a requirement for every vendor claim from this: it names the task and the metric as well as the comparison group and the limits.

## Which test questions separate progress from announcement?

Six test questions separate progress from announcement in practice, and each can be asked without technical training. They suit board meetings, vendor conversations and requests from business units.

1. Which maturity level does the announcement have, and where is the evidence for that level?
2. Which task is improved, and which metric measures the improvement?
3. Was there a comparison group without the technology, and who measured?
4. Which data did the demonstration use, and does it match the data in your own operation?
5. Who is accountable for the application from day one, and who maintains it?
6. What does operation cost per user or per transaction, and when is break-even reached?

An announcement that answers all six questions has reached level 5 and belongs in the plan. If it answers only the first, it stays under observation. This separation prevents the most frequent mistake in the phase of inflated interest, which is to release budget for level 1 and 2 because competitors are talking about the same announcement. How pressure of this kind arises is described in the article [AI FOMO: Why companies invest wrongly out of fear of missing out](/en/blog/ai-fomo-companies-investing-wrongly/). A view of the first twelve months after ChatGPT is in [ChatGPT in the enterprise: a sober look one year after the hype](/en/blog/chatgpt-in-business-sober-review/).

## How do executives set up a watch list?

A watch list assigns every AI topic in the company to a maturity level and defines what spend of time and money is justified at each level. One page is enough: the application on the left, the level in the middle, on the right the next piece of evidence that would change the level and the date of the next review. Updated quarterly, it becomes a calm basis for decisions. Applications at levels 4 and 5 receive pilot budget, those at level 3 a small test, and levels 1 and 2 remain a matter of market observation by a named person.

The list also answers the question of who in the company evaluates announcements. Without an owner, announcements reach the executive team through chance channels, and the loudest voice sets the agenda. With a named role that grades announcements, an orderly flow emerges that prepares decisions.

## What does this mean for planning with Niehues Consulting?

[Niehues Consulting](/en/) is a consultancy for AI management and organisation design from a delivery perspective, based in Düsseldorf. Founder Marin Niehues brings AI into operation, in projects ranging from VC-funded scale-ups to DAX 40 corporations and in more than 20 countries on six continents. In the field of AI management, the work includes business analysis, the business case, and the selection and sequencing of applications, which are exactly the steps that move an announcement from level 3 to level 5.

The approach follows the Niehues Delivery Model. Three test questions matter for placing announcements: which business outcome should change measurably? What is the baseline, and how do we measure success? Does the prototype prove the benefit in the real workflow with real data? The scope and terms of an engagement are discussed in the free initial consultation, which you can arrange through the [contact page](/en/#kontakt).

**Conclusion**
Telling progress from announcement means asking for the maturity level, the metric and the comparison group. Studies from 2023 show real effects but also an uneven frontier, which justifies pilots with measurement. A watch list with a named owner turns the pressure of announcements into a calm, quarterly decision.

**Sources**
- Brynjolfsson, Li, Raymond: Generative AI at Work, NBER Working Paper 31161, 2023
- Noy, Zhang: Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence, Science, 2023
- Dell'Acqua et al.: Navigating the Jagged Technological Frontier, working paper, 2023
- David, Paul A.: The Dynamo and the Computer, American Economic Review, 1990

## Frequently asked questions

### What is the hype cycle for AI?

The hype cycle describes the typical course of attention to a new technology in five phases, from the trigger through the peak of inflated expectations and the trough of disillusionment to the plateau of productivity. It is a rule of experience and not a forecast. It helps executives adjust their expectations to the phase an application is in.

### Is AI overhyped?

For some applications there are solid measurements, for others only demonstrations, so there is no single answer for the technology as a whole. Customer service and text work have been measured in studies, whereas autonomous decisions with high potential for harm have not. Placing each application on a maturity level is more reliable than a verdict on the whole market.

### How do I know whether an AI announcement shows real progress?

A real advance names a task, a metric, a comparison group and the limits. If one of these elements is missing, it is an announcement or a demonstration. The six test questions in this article settle the matter in a few minutes.

### How much productivity does AI add according to research?

Studies from 2023 report gains from 14 per cent more issues resolved in customer service to around 40 per cent less time on writing tasks. The values apply to the task studied and cannot be transferred to other work. In the service study, novices gained clearly more than experienced staff.

### Should companies test every new AI feature?

No, a company tests what has reached at least level 3 and fits a prioritised application. Testing every novelty ties up business units in trials without a metric. A named observer filters the announcements and proposes only applications with evidence for a test.

### How quickly do AI decisions become outdated?

Technical capabilities improve at short intervals, whereas the requirements for data, processes and ownership stay stable. Decisions built on these prerequisites keep their value. Decisions for a particular model or vendor should remain exchangeable and are reviewed quarterly.

### Who in a company should assess AI announcements?

A named person with access to the executive team and the business units, ideally from AI management or delivery responsibility. This person places each announcement on a maturity level and keeps the watch list. The executive team decides on pilot budget on that basis.

### What is the cost of missing an AI trend?

Waiting for an application at level 1 or 2 costs little, because applications that reach level 4 can be adopted with the right preparation. A head start comes from workflows, data and ownership that can be prepared before a product matures. An organisation that creates these prerequisites can adopt a capability as soon as it is proven.