Agents that run5 min read
Confidence threshold and the human fallback
A confidence threshold is the certainty below which an AI agent stops acting and hands the case to a person. Above the threshold the agent classifies, routes and creates the task, and logs why. Below it, the agent creates no task, escalates the case to a named person with its reasoning attached, and logs the handover. There is no state in between. That single rule is what "human in the loop" means when it is written into an operating agreement rather than a slide.
Human in the loop is a mechanism, not a reassurance
The phrase is used to calm buyers: a person is involved somewhere. As a design it has to answer three questions precisely. When does the agent hand over? To whom? What does the person receive? If any of the three is answered with "it depends", the loop does not exist. It becomes a person reviewing everything, which saves nothing, or a person reviewing nothing, which is an unsupervised agent with a comforting name.
How the threshold is set, and by whom
The threshold is agreed with the Fachbereich, the department that owns the Vorgang (one intake, one task, one destination), before the Pilotbetrieb starts, and tuned during its four productive weeks on real volume. It is never a vendor default. Three inputs decide it:
- The cost of a wrong assignment in this Vorgang. A misrouted brochure costs a forward; a misrouted deadline letter costs more.
- The share of cases the department can review by hand each day without the process slowing down.
- The confidence distribution the agent actually produces on the department's documents in week two.
The right value is the one the department can defend to its own management: at this certainty we let the agent act, below it we look ourselves, and here is why.
What too high and too low look like in practice
| Threshold | What the department experiences | What the log shows |
|---|---|---|
| Too high | Most cases land on a person's desk; the agent saves little and is resented within a month. | Escalation rate far above the misassignment rate found in review. |
| Too low | Almost everything is processed automatically; errors surface later as complaints. | Misassignments in review that carried confidence just above the threshold. |
| Defensible | The department reviews a predictable share each day; the rest runs. | Escalations and misassignments both rare and both explainable case by case. |
What the person receives at handover
A handover is only useful if the person can decide faster than from scratch. Every escalated case arrives with the same five things: the document itself, the classes the agent considered with their confidence, the class it would have chosen, the rule or example that decision rested on, and a one-line reason for the doubt. The person decides, the decision is written to the decision log, and the case becomes a labelled example for correction.
This is also where the loop closes. A department that decides ten escalated cases a day is not doing the agent's work; it is teaching the agent the edge of its own process, which is exactly the knowledge no vendor has.
The fallback protects the organisation, not the vendor
The question after any wrong assignment is not whether the agent erred but what happened next. With a threshold, the answer is one of two sentences: the agent was confident and wrong, here is the log entry and the correction; or the agent was unsure, handed over, and a person decided. Both are defensible to a supervisor, an auditor or a citizen. "The AI decided" is not.
The intervention path runs the other way too. A named person can stop, correct or take over the agent at any time, and the path is documented before go-live. Operation entirely within the EU, responsibility from Düsseldorf. GDPR-compliant processing with an AVV included. A complete decision log available as evidence for EU AI Act audits. A defined human intervention path, documented and triggerable at any time.
How the threshold changes during operation
Weeks four to seven of the Pilotbetrieb are productive operation on real volume, and the threshold is tuned there against the agreed hit rate. Afterwards it moves only when the organisation changes: a new document type, a renamed department, a supplier with a new layout. Each change is a correction, logged, and each is reported in the monthly evaluation. What the threshold is on a given day is never a secret; it is the first line of the operating report.
Frequently asked questions
Who decides the confidence threshold?
The department that owns the Vorgang, together with us, before the Pilotbetrieb starts. We bring the confidence distribution the agent produced on real documents in week two; the department brings the cost of a wrong assignment and its review capacity. The value is then tuned during the four productive weeks and recorded in the operating agreement.
Does human in the loop mean a person checks every case?
No. A person decides the cases below the threshold, which should be a small and predictable share once the agent is tuned. Reviewing everything is not a loop; it is manual processing with an extra step. The point of the threshold is to make the reviewed share explicit and defensible.
What does an escalated case look like?
The document, the candidate classes with their confidence, the class the agent would have chosen, the rule or example behind it, and a one-line reason for the doubt. The person's decision is logged and becomes a training example. Handover takes seconds, not a re-read from scratch.
Can a person override the agent when it was confident?
Yes, at any time. The human intervention path is documented before go-live: a named person can stop the agent, correct an assignment or take over a case. Every override is logged with who, when and why, and feeds the next correction cycle.
What happens to the threshold after the Pilotbetrieb?
It stays as agreed unless the organisation changes. A new document type, a renamed department or a changed supplier layout can shift the confidence distribution; each adjustment is a logged correction and appears in the monthly evaluation. The current value is always the first line of the operating report.
The decidable next step
If a recurring intake in your organisation is sorted by hand today, the confidence threshold for it can be agreed in week one of a Pilotbetrieb: one Vorgang, fixed price agreed before the start, seven weeks with four of them productive operation on real documents. Request a first conversation. How operation works as a whole is in What operating an AI agent actually means.