When should an AI support agent hand off to a human?

The worst AI support experience is not a wrong answer. It is a confident wrong answer followed by three more attempts, with no way to reach a person. The second worst is an agent that gives up on everything, so every "how do I reset my password" lands in your inbox anyway.

Handoff is the design problem in the middle of those two failures. This post is how we think about it, how it is implemented in Svellyo, and how to tune it for your own product in about twenty minutes.

The rule that matters most

Before thresholds and phrases: the visitor must always be able to reach a person, and it must be obvious how.

Every Svellyo widget shows a "Talk to a person" link under the composer while the AI is handling a conversation. Tapping it moves the chat to your team immediately, no confirmation, no "are you sure". If your team is offline, the visitor is told when to expect a reply and asked for an email so nothing is lost.

Everything else in this post is about handing off before the visitor has to ask. But the escape hatch comes first, because it turns a bad AI answer from a dead end into a minor delay, and that is what makes it safe to let the AI answer at all.

Four signals we use

The agent evaluates four things on every reply. Any one of them can trigger a handoff.

1. Confidence

Every answer comes with a score between 0 and 1. It is the model's own estimate of how well the retrieved knowledge supports the reply, and in practice it tracks reality well: high scores come with a clear source article, low scores come with the agent reaching for something adjacent.

Below your threshold, the agent does not send its draft. It tells the visitor it is bringing in a teammate, and the conversation moves to Needs human in your inbox. The default threshold is deliberately conservative. Lower it when the agent hands off answers you would have been happy with; raise it when it guesses.

2. Trigger phrases

Some requests should never be answered by a bot regardless of confidence. "Talk to a human", "real person", "speak to someone" are there by default. Add your own: "cancel my account", "refund", "I was charged twice", "legal" are common additions. Phrase matching is case-insensitive and applies to every visitor message.

3. Turn limits

If a visitor has gone five rounds with the AI and is still asking, the conversation is not going well, whatever the confidence score says. Set a maximum number of AI replies per conversation and the agent hands off when it is reached. Three to five is right for most products. Leave it unset for a documentation-heavy product where long AI conversations are normal.

4. The model's own judgment

Finally, the agent can decide on its own that a human is needed. The structured output it returns includes a needsHuman flag and a reason. It sets that flag for things like account-specific problems it cannot see, requests for exceptions, anger, or anything that reads like a complaint. This catches the cases no keyword list anticipates.

What the visitor sees

Handoff should feel like a promotion, not a failure. The message we use is "Connecting you with the team", followed by your expected reply time when you are online, or your away message when you are not. The visitor's previous AI exchanges stay visible so they do not have to repeat themselves, and the person picking up sees the same thread with the AI's confidence and sources attached.

Do not make the visitor re-explain. The single most common complaint about bot handoffs in every support tool we looked at was being asked "how can I help?" by a human after a page of typing to a bot. Your teammate reads the transcript; that is the whole point of having one.

What your team sees

In the inbox, a handed-off conversation shows the reason: "Low confidence", "Visitor asked for a person", "Trigger phrase: refund", or the model's own note such as "Visitor reports a double charge". You get an email, and a Slack or Discord message if connected. The Needs human filter collects all of them.

Reply, and the AI steps aside for that thread. When you are done, click Hand back to AI and the agent resumes with full context, including your replies. That last part is why handoff is cheap to trigger: it is not a one-way door.

Tuning it in twenty minutes

1. Open Settings, AI and the test chat next to it.
2. Paste in twenty real questions from your inbox. Include the easy ones and the ones you never want a bot to touch.
3. For each, note the confidence and whether it handed off.
4. Good answers being handed off: lower the threshold by 0.05 and retry.
5. Guesses getting through: raise the threshold, then write the article the agent was missing. A guess is almost always a retrieval gap, not a threshold problem.
6. Any question that should always reach you: add its key phrase to the trigger list.
7. Set a turn limit of four unless your product genuinely needs long AI conversations.

Then open Analytics a week later and look at "Why chats were handed off". If "low confidence" dominates, write articles. If "visitor asked for a person" dominates, your AI answers are probably fine but visitors do not trust them yet; make the source citations visible and consider a friendlier agent name than "Bot".

Three settings that change the whole feel

Auto-escalate when offline. With this on, chats outside business hours skip the AI and go straight to the message form. Turn it off if you want the AI to cover nights, which for most products it should. Turn it on for products where a wrong answer at 3 a.m. is worse than a delay.

Collect email on handoff. When the agent hands off and nobody is online, ask for an email in the widget. Almost everyone provides it, and it turns an abandoned chat into a reply you can send in the morning.

Reply language. Automatic by default, so a French visitor gets French answers from your English knowledge base. Pin one language only if your team cannot follow up in others.

What we chose not to build

We do not do sentiment gauges, "frustration scores" or multi-step bot flows with buttons. They add configuration and rarely change outcomes. A confidence score, a phrase list, a turn limit and a visible escape hatch cover almost every case, and the model's own judgment covers the rest. The complexity budget went into retrieval instead, which is where answers actually come from.

The short version

Show the "talk to a person" link always. Hand off on low confidence, on phrases you choose, after a few turns, and when the model itself says so. Keep the transcript so nobody repeats themselves. Make handing back one click. Then spend your time writing the articles that the handoff reasons tell you are missing.