An agent in customer service does three things very well and one thing not at all, and the difference is not a question of models but of authority. On top of that, since 2 August 2026 a duty applies that used to be a matter of taste: the customer must be told they are talking to a machine.
Since 2 August 2026 One Question Has Been Settled
Until recently it was a matter of taste whether a service chat says that it is a machine. Some operators found it honest, others found it bad for business. That question has since been settled by law in force.
Article 50(1) of the AI Act, Regulation (EU) 2024/1689, reads: „Providers shall ensure that AI systems intended to interact directly with natural persons are designed and developed in such a way that the natural persons concerned are informed that they are interacting with an AI system, unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect, taking into account the circumstances and the context of use."
Article 113 sets the date of application at 2 August 2026 and lists exceptions. As amended by the Digital Omnibus Regulation (EU) 2026/1744 of 8 July 2026, Chapters I and II apply from 2 February 2025, several further chapters from 2 August 2025, Articles 102 to 110 from 27 July 2026, and the high-risk rules in Chapter III only from 2 December 2027 (Annex III) or 2 August 2028 (Article 6(1)). Article 50 sits in Chapter IV, and Chapter IV appears in none of those exceptions. Article 50 has therefore applied since 2 August 2026. Only for systems that generate synthetic content and were already on the market does the labelling duty in Article 50(2) run until 2 December 2026.
This paragraph addresses providers, not everyone who deploys a piece of software. Whoever merely buys the chat is usually a deployer; which duty applies in an individual case is a legal question. The text itself also carves out the case in which the machine nature is obvious anyway.
In practice this changes little about what has to be done on your own site. The notice belongs where the customer types, not in the privacy policy. What compliance with the regulation otherwise costs I wrote up separately in what EU AI Act compliance actually costs.
Language Agents in Customer Service: What the Statistics Show
How common is this in the first place? The German Federal Statistical Office surveys which AI technologies are in use, most recently for the 2025 reporting year, as of 24 November 2025. The shares refer exclusively to enterprises that use artificial intelligence, and that reference is missing from almost every quotation.
- Text mining: 52 percent
- Generation of images, video or audio: 52 percent
- Speech recognition: 42 percent
- Natural language generation, including program code: 35 percent
- Automation of workflows and decision-making: 27 percent
- Machine learning: 23 percent
Because 26 percent of the surveyed enterprises use AI at all, 42 percent of those come to just under eleven percent of all surveyed enterprises for speech recognition. For natural language generation it is around nine percent, for workflow automation around seven.
A service chat built on language models is therefore still a clear minority in Germany. Anyone building one in autumn 2026 is early. And since the same survey starts at ten persons employed, it says nothing at all about smaller businesses.
The Three Things an Agent Really Does Well in Service
From running our own operation, and after several failed attempts, three tasks hold up.
First, understanding and classifying. A message arrives, often in poor German, with the order number somewhere in the middle and three separate requests in one paragraph. The agent breaks that into case, request and urgency and attaches it to the right record. This is the task with the best ratio of value to risk, because a mistake here is a wrong drawer, not a wrong statement.
Second, looking things up in your own systems. Where is the parcel, what does the returns policy say, when was the last contact. That is a lookup with a checkable answer. The agent reads from the system instead of remembering. What it does not find, it does not say.
Third, preparing a reply that a human sends. The draft is ready, with the right data, in the right tone, in the customer's language. The human reads, changes one line, sends. That costs twenty seconds instead of four minutes, and those twenty seconds are where mistakes get caught.
None of these three tasks changes the world; they sort, look up and phrase. Every error stays inside the house as long as a human sits in between.
The One Thing It Cannot Do: Take On Responsibility
The boundary is not a question of model quality. An agent cannot make a promise that somebody stands behind, because a promise is a decision, not a piece of information.
The difference is clearest in a goodwill case. „Our returns policy gives you 14 days" is information, and a machine can supply it because it is written in a document. „We will make an exception for you" is a decision with a price, and that price is paid from an account someone is answerable for. A system allowed to produce such sentences hands out money without a mandate.
This is not a theoretical risk. A language model produces the most plausible next sentence, and in a friendly service dialogue the most plausible sentence is very often a concession. The model is not doing anything wrong, it is doing something kind in a place where it has no authority. I wrote about this pattern in more detail in my piece on where agents escape their guardrails.
The Handover Is Not an Emergency Exit, It Is the Product
Most service chats treat the handover to a human as an admission of failure and only trigger it when the agent is stuck. That is the wrong construction, and it fails on a technical property that is discussed too rarely.
A language model's self-assessment is not a metric. A model that says it is confident is not thereby more reliable than one that says nothing. Hanging the handover on a confidence threshold hangs it on a number that measures nothing. It is the same mistake as a monitor reporting green because it checked nothing.
The handover therefore depends on the class of case, not on the state of the model. Four classes always go to a human with us, no matter how clear the case looks:
- Money: refunds, credits, discounts, goodwill
- Deadlines: anything where a date carries a legal consequence
- Complaints: as soon as somebody is unhappy rather than merely asking
- Repeat contact: a second message about the same matter
The fourth is the one most often missing and the one that helps most. Anyone writing a second time did not get a solution the first time. An agent that tries again simply doubles the disappointment.
How to Measure Whether It Works
The usual metric is the automation rate, meaning the share of cases handled without a human. It is the worst one available, because it improves when you answer cases badly instead of handing them over. A metric that improves as quality drops measures the opposite of what it should.
Three measures are useful, the ones I work with in my own operation, and the third is the most honest.
Time to the first substantive reply. Not to the acknowledgement, but to a sentence that engages with the request. An agent improves this reliably, and it is the real gain for the customer.
Reopening rate. The share of cases reopened after an agent reply. It reveals what the automation rate hides: an answer that closes a case without solving it.
Draft edit rate. The share of prepared replies a human changes before sending. It is uncomfortable because it judges the agent directly, and that is why it warns earliest. If it rises, the agent is getting worse or the cases are changing. If it falls to zero, nobody is proofreading any more, and the release button is just a click.
What the Operation Needs Before the Agent Arrives
The most common reason a service agent disappoints has nothing to do with the agent. It amplifies the order that already exists, and where there is none, it invents one.
If the returns policy is written nowhere and lives in the heads of two colleagues, then it is not available to the machine. The machine will answer anyway, because that is how it is built. It will formulate the most plausible rule, and it will sound right. The expensive answer is therefore the credible one, not the obviously wrong one.
So before the first agent there is a task that sounds like paperwork and is the actual work. Write down the ten most frequent cases, with the rule that applies to each, and with a sentence on what happens when you deviate. That takes a morning. After it the agent is a tool. Before it, it is a generator of plausible sentences about rules nobody has decided.
Anyone planning this for the Christmas season should know the timetable. Whatever is not in place on 1 November will not be in place during the season. On that I wrote about how agents support the retail peak, and how I use AI in my own working day is on the page about AI in daily work.
Frequently Asked Questions
Does a chatbot have to say that it is not human?
Article 50(1) of Regulation (EU) 2024/1689 requires providers to design AI systems intended for direct interaction with people so that the person is informed they are interacting with an AI system. The text exempts cases where this is obvious anyway. Article 50 sits in Chapter IV and therefore applies from 2 August 2026. Which duty falls on an individual deployer is a legal question.
How many enterprises use language technology?
According to the German Federal Statistical Office for 2025, among enterprises that use AI, 42 percent use speech recognition and 35 percent natural language generation. The shares refer to AI users. Since 26 percent of all surveyed enterprises use AI at all, that comes to just under eleven and around nine percent of all surveyed enterprises.
Which tasks suit an agent in customer service?
Three tasks hold up reliably: understanding incoming messages and attaching them to the right record, looking things up in your own systems instead of remembering, and preparing a reply that a human sends. None of them change the world; they sort, look up and phrase. An error stays inside the house as long as a human sits in between.
When must an agent hand over to a human?
Not when the model seems unsure, because a language model's self-assessment is not a metric. The handover belongs to the class of case: money, meaning refunds, credits or goodwill, deadlines with legal consequences, complaints, and any second contact about the same matter. That last class is the one most often missing and the one that helps most.
Why is the automation rate the wrong metric?
Because it improves when cases are answered badly instead of handed over. Three other measures are useful: time to the first substantive reply, the share of cases reopened after an agent reply, and the share of drafts a human edits before sending. The last one warns earliest.
What has to be prepared before deploying a service agent?
Your own rules have to be written down. An agent amplifies the order that exists, and where none exists it formulates the most plausible rule. That is the expensive case, because an invented rule sounds credible. One morning spent on the ten most frequent cases, with the applicable rule and the exception path, turns the generator into a tool.
Warm regards,
Dennis Weidner
Note: AI tools supported me in writing this article, and some images were edited with AI. I stand behind its content and every statement with my name.
