AI Review of 2023: What Companies Delivered and What Stayed Announcement

The AI balance sheet of 2023 has two halves: the documented effect lies in single work steps in text, research and customer contact, while the redesign of whole processes stayed an announcement. About one in eight German companies used AI, and the gap between large companies and the Mittelstand is wide.

This article draws up the balance of a year that began with the launch of ChatGPT at the end of 2022 and in which GPT-4 and a race between models set the headlines. It separates documented results from announcements and gives you a checklist for drawing up the balance of your own company.

How many companies use AI?

In Germany around one in eight companies used artificial intelligence in 2023. On 27 November 2023 the Federal Statistical Office reported a share of 12 per cent in a survey of companies with ten or more employees. The figure rises with company size: 10 per cent among small companies with 10 to 49 employees, 16 per cent among medium-sized ones with 50 to 249 employees and 35 per cent among large companies with 250 or more employees.

The figures show that the wave of announcements has so far reached only a small part of the economy. The same survey names accounting, controlling and financial administration as the most frequent field of use, at 25 per cent of the companies using AI, followed by IT security at 24 per cent. Among the technologies, speech recognition (43 per cent), automation of workflows or decision support (32 per cent) and text mining (30 per cent) lead. Companies without AI cited a lack of know-how most often as an obstacle, at 72 per cent. What is missing, above all, is people who can bring an application into operation.

What was actually implemented in 2023?

What was documented in 2023 was mainly the effect of applications in which a language model speeds up a single work step: drafting texts, summarising documents or suggesting replies in customer service. These applications share three features. They can be used without rebuilding existing systems, a person checks every result, and the benefit shows directly in working time.

Two scientific studies put numbers on this picture. A field study published by the US National Bureau of Economic Research (NBER) analysed data on 5,179 customer support agents who were given a generative AI assistant. The number of issues resolved per hour rose by 14 per cent on average, and by 34 per cent for novices and less experienced staff. An experiment with 453 professionals, published in Science in 2023, had occupation-specific writing tasks completed with and without ChatGPT. Average time on task fell by 40 per cent, and the quality rated by assessors rose by 18 per cent. The authors point out, however, that they did not check the texts for factual accuracy.

Both studies measure work steps. Whether the time gained becomes more revenue, lower cost or better quality is decided by the operation that reallocates the time. This is why many pilots of 2023 showed good individual results and nevertheless stayed invisible in the company's own accounts. A sober assessment of a year with ChatGPT is given in the article ChatGPT in the company.

What stayed an announcement?

Everything that rested on the promise that models would handle whole processes autonomously and without error stayed an announcement in 2023. That includes fully automated decisions in customer contact and AI as a replacement for entire departments, as well as platforms that work with company knowledge straight away without any data work.

Four reasons explain the gap. First, language models produce plausible-sounding but occasionally wrong answers, which is a risk for any process without human review. Second, a company's knowledge is scattered across documents and systems as well as the heads of its staff, and making it accessible is work. Third, every application in steady-state operation needs owners for maintenance and support, which many projects had not planned. Fourth, many organisations lacked a calculation that confirms the benefit with real values.

A product such as Microsoft 365 Copilot, available to businesses since November 2023, changes little about these four points, since it eases access and does not take over the rollout in operations. What is included at market launch is described in the article Microsoft 365 Copilot. The model race of the year, most recently with the unveiling of Gemini in early December, also has smaller consequences for most companies than the coverage suggests. Why comparing models is secondary for the decision in a company is shown in the article Gemini versus GPT-4.

How do you tell whether an AI announcement is implementation or promise?

You tell by five test criteria: is there a named operation, a measured metric, a process with human review, a statement on the data and a statement on running costs? The more answers are missing, the more likely it is an announcement. The checklist suits press releases and vendor presentations as well as internal proposals. How executives separate progress from hype is explored further in the article on the AI hype cycle.

How do you draw up the balance in your own company?

You draw it up in four steps that can be completed in half a day with the executive team and department heads.

1. Record the inventory: list every AI application in use in the company, including tools that employees introduced themselves.
2. Test each application: apply the five criteria of the table. An application counts as implemented only when operation and metric exist.
3. Sort the announcements: every initiative without operation and without a metric stayed an announcement in 2023. It forms the starting point for next year's plan.
4. Close the gaps: for each initiative, fix the owner, the baseline and the route into operation. Then decide which initiatives continue.

How boards turn the pressure to act now into a robust plan is described in the article on AI activism.

AI review with Niehues Consulting

Niehues Consulting is a consultancy for AI management and organisational design from a delivery perspective, based in Düsseldorf. Founder Marin Niehues brings AI into operations, in projects from VC-funded scale-ups to DAX 40 groups and in more than 20 countries on six continents. In the service area of AI management, the work runs from business analysis and requirements through the business case to the selection and sequencing of applications and the steering of the portfolio.

The approach follows the Niehues Delivery Model with its three phases. Three of its test questions matter for an annual review: Which business outcome should change measurably? Is the business case confirmed with real values? Which metric shows after 90 days that the solution is landing? Scope and terms of an engagement are discussed in the free initial consultation, which you can arrange in 30 minutes through the contact form.