# ChatGPT in Business: A Sober Review Almost a Year After Launch

Author: Marin Niehues, Niehues Consulting · Published: 2023-10-12 · AI Strategy & Business Case
Source: https://www.niehuesconsulting.de/en/blog/chatgpt-in-business-sober-review/

ChatGPT delivers measurable benefit in business on clearly defined writing tasks with human review, and it is unsuitable as a source of facts without a cross-check. Studies from 2023 show efficiency gains of 14 to 40 per cent, yet the same studies show that errors outside the model's capability frontier sound convincing. Executives therefore need a grid that sorts tasks into use classes and ties approval to how well the result can be checked.

This article takes stock after almost a year. It sets out the solid evidence, classifies the available offerings and contains a grid of four use classes for approving applications in everyday business. It provides an overview and does not constitute legal advice.

**The benefit of ChatGPT arises at the task that the tool takes over. Anyone who sorts tasks by checkability gets the gains from the studies and avoids their errors.**

## What does ChatGPT demonstrably achieve in business?

Generative AI assistants such as ChatGPT raise productivity on writing tasks by 14 to 40 per cent in studies, and the effect is largest for less experienced staff. A field study by Brynjolfsson, Li and Raymond of more than 5,000 customer support agents at a software company, which examined an assistant built on GPT models, found an average rise of 14 per cent in issues resolved per hour, and 34 per cent for novices. An experiment by Noy and Zhang with 453 professionals found working time cut by around 40 per cent with higher-rated quality. The effect partly evens out performance differences between employees, because the tool makes the approach of strong colleagues available to others.

A third study from September 2023 limits this finding. In a study of 758 consultants at an international consulting firm, participants with access to GPT-4 solved tasks inside the model's capability frontier around 25 per cent faster and at higher rated quality. On a task outside that frontier they produced 19 percentage points fewer correct solutions than the group without AI. The authors speak of a "jagged" frontier: which task lies on which side is hard to see from outside. For a company this means approval should follow the checkability of the result, while the impression a reply makes says nothing about whether it is right.

## Where are the limits of ChatGPT in business?

The limits of ChatGPT lie in invented content, in a knowledge base with a cut-off date and in uneven quality across tasks. Research calls invented but plausible-sounding statements hallucinations, and a survey by Ji and colleagues (2023) classifies causes and countermeasures. The model computes likely sequences of words. It does not check whether a statement is true.

Three rules for everyday use follow. First, figures, names and source references from model replies never go unchecked into decision papers or customer letters. Second, every application needs a person who can judge the result on the merits: whoever cannot judge a task cannot check its solution either. Third, confidential company data should be processed only in offerings whose terms of use govern how inputs are used. For ChatGPT Enterprise, which the provider introduced at the end of August 2023, the provider states that inputs are not used to train its models. A further route is access through the Azure OpenAI Service, generally available since January 2023, whose terms companies agree with the provider individually.

## Which offerings exist, and what do they cost?

In October 2023 four routes to access are available for everyday business: the free version of ChatGPT, the paid offering ChatGPT Plus, ChatGPT Enterprise and access through programming interfaces such as the Azure OpenAI Service. The provider quotes a price of 20 US dollars per user per month for ChatGPT Plus. No public price list exists yet for ChatGPT Enterprise, and terms are offered individually.

A rough calculation shows when a paid account pays off. At a full cost of 60 euros per working hour, around 20 euros a month corresponds to about 20 minutes of working time. If a person saves more than 20 minutes a month through regular use, the licence pays for itself on paper. The caveat lies in the words "on paper": saved time has value only if the business unit plans it for other tasks. How quickly licence costs grow without usage is shown in the article on [AI FOMO](/en/blog/ai-fomo-companies-investing-wrongly/): with 20 per cent active users, the cost per user rises fivefold.

## Which four use classes are there?

Four use classes sort ChatGPT applications by how well the result can be checked and by the damage a mistake would cause. The grid answers whether an application is approved, accompanied or put on hold.

**Four use classes for ChatGPT in business**
Use class | Examples | Checkability of the result | Approval
A Drafting | Text drafts, outlines, rewording, summaries of your own documents | The author sees at once whether the result fits | Approve, review by the user is enough
B Research | First orientation on a topic, ideas, explanations of terms | Statements must be checked against sources | Approve, with the rule that facts are confirmed from primary sources
C Decision preparation | Analyses, assessments, figures for papers | Checking requires expertise and data | Accompany, with a named reviewer and documented method
D Automatic outputs | Replies to customers without review, automated decisions | No check before the output takes effect | Put on hold until the checking procedure and the damage from errors are settled

Most applications in office and administrative work fall into classes A and B and can be approved with little effort. Class C requires working instructions that say who checks and with what. Class D begins where the reply reaches customers or business processes without human review, and it presupposes a quality assurance that still has to be built for a language model.

## How do companies introduce ChatGPT in an orderly way?

Companies introduce ChatGPT in an orderly way by starting with one to three applications from classes A and B, fixing an owner and a metric, and deciding after 90 days. The sequence follows four steps. First, business units record what employees already use the tool for, because use often begins before the company regulates it. Next, the management defines the use classes and states which data belong in which offering. In the third step individual teams start with clearly defined tasks, such as drafting quotations or preparing meeting notes, and measure processing time and quality against the baseline. In the fourth step, leadership decides after 90 days which applications to extend.

Enablement matters as much as the licence. The studies show the largest gains among employees who use the tool with clear tasks and feedback. Staff need examples of good prompts, a rule for checking results and a contact person for questions.

**Checklist before approving an application**
- Which use class does the task belong to?
- Who can judge the result on the merits?
- Which data flow in, and do they belong in this offering?
- Which metric shows after 90 days whether the application works?
- Who is accountable for the application in the business unit?

## ChatGPT in business with Niehues Consulting

[Niehues Consulting](/en/) is a consultancy for AI management and organisation design from a delivery perspective, based in Düsseldorf. Founder Marin Niehues brings AI into operations, in projects ranging from VC-funded scale-ups to DAX 40 corporations and in more than 20 countries across six continents. In the AI management field the work runs from business analysis and requirements through the business case to the selection and sequencing of applications.

The approach follows the Niehues Delivery Model. Three checks matter most for ChatGPT: Which business result should change measurably? Does the prototype prove the benefit in the real workflow with real data? Are users enabled and processes adapted? For language models the third question decides whether licences turn into use. Scope and terms of an engagement are discussed in the free initial consultation, which you can arrange via the [contact form](/en/#kontakt).

**Conclusion**
After almost a year ChatGPT is a tool with proven benefit for checkable writing tasks. Anyone who approves applications by use class, names an owner and a metric, and enables staff uses the gains and limits the errors.

**Sources**
- Brynjolfsson, Li, Raymond: Generative AI at Work, NBER Working Paper 31161, 2023
- Noy, Zhang: Experimental Evidence on the Productivity Effects of Generative AI, Science, 2023
- Dell'Acqua et al.: Navigating the Jagged Technological Frontier, Harvard Business School Working Paper 24-013, 2023
- Ji et al.: Survey of Hallucination in Natural Language Generation, ACM Computing Surveys, 2023

## Frequently asked questions

### Is ChatGPT safe to use for business?

ChatGPT is suitable for business when tasks are approved by how well their results can be checked and confidential data enters only offerings with defined terms of use. For drafts and research a simple framework is enough. For automatic outputs without review the precondition is missing, because nobody checks the reply before it takes effect.

### Is ChatGPT worth it for companies?

ChatGPT is worth it where active users save more working time than the licence costs and the business unit reuses that time. Studies show productivity gains of 14 to 40 per cent on clearly defined writing tasks. Unused licences remove the benefit and multiply the cost per active user.

### How much does ChatGPT cost for business?

The provider quotes 20 US dollars per user per month for ChatGPT Plus. For ChatGPT Enterprise terms are offered individually. Whether a paid account pays off is decided by real usage: a gain arises only when active users save more working time than the licence costs.

### What are hallucinations in ChatGPT?

Hallucinations are statements that sound plausible but are untrue or unsupported. They arise because language models compute likely sequences of words and do not verify truth. Countermeasures are source checks by people and tasks whose results are easy to judge.

### Can employees enter confidential data into ChatGPT?

Confidential data should enter only offerings whose terms of use govern how inputs are used. For business offerings such as ChatGPT Enterprise the provider states that inputs are not used for model training. Management decides which types of data belong in which offering.

### How do you roll out ChatGPT in a company?

A rollout works with one to three applications from use classes A and B, one owner per application and a metric for 90 days. Examples of good prompts and a rule for checking results are added. Only then is use extended to further teams.

### What is the difference between ChatGPT Plus and ChatGPT Enterprise?

ChatGPT Plus is a paid offering for individual users, whereas ChatGPT Enterprise is a business offering that the provider introduced at the end of August 2023. According to the provider, Enterprise includes administration features and no use of inputs for model training. Terms are agreed individually.