Site-Specific AI Chatbots: The Engineering Behind the Five Qualities

Most write-ups about "good" AI chatbots stay at the level of tone and copy. That's useful, but it skips the part I care about: every one of those qualities maps to a concrete engineering decision, and most of them are expensive.

Nielsen Norman Group recently published usability research on site-specific chatbots — the little assistants bolted onto a retailer or SaaS site, not the frontier models people chat with directly. They found five qualities that separate the bots people trust from the ones people abandon: handoff willingness, flexibility, proactivity, emotional responsiveness, and transparency (NN/g, July 2026).

I read it as a backend engineer who builds RAG pipelines, and I want to translate each quality into the architecture that produces it. Because a bot doesn't fail to escalate because its designer forgot — it fails because nobody wired the escalation path. The gap between a good bot and a frustrating one is usually a gap in the system diagram.

1. Handoff willingness → escalation is a first-class code path

NN/g's finding is blunt: users don't yet see chatbots as equivalent to human agents, and gatekeeping the "talk to a person" request destroys trust. One participant described the loop as "a hamster wheel kind of spinning around and around." Their good example (Wyze) handed over a phone number immediately; the bad one (Wilson) deflected three escalation requests first.

From an engineering standpoint, handoff is not a fallback — it's a routing decision that needs at least three independent triggers:

Explicit intent. If the user types "agent," "human," or "representative," route immediately. No re-qualifying questions.
Confidence collapse. When your retrieval layer returns low-confidence results — or the bot fails to resolve the same intent 2–3 times — escalate instead of guessing.
Sentiment. Detected frustration or anger should offer a human proactively.

The part teams underbuild is the context package. A good handoff transfers the full chat history, any collected data (order number, email), and the CRM profile so the customer never repeats themselves (eesel AI). That's a serialization and state-transfer problem, and it's exactly the kind of thing that gets cut when the deadline slips.

Alex's take: Build escalation before you build cleverness. A bot that answers 70% of questions and hands off the other 30% cleanly beats a bot that attempts 100% and traps people in a loop on the hard 30%. Escalation is your error-handling layer — treat skipping it the way you'd treat a bare except: pass.

2. Flexibility → retrieval scope and honest uncertainty

NN/g frames flexibility as handling adjacent questions and topic shifts without forcing users into the bot's rigid script. They put it on a spectrum: "too narrow" misses obvious adjacencies (an outdoor retailer's bot not recognizing hiking pants as in-scope), while "too broad" answers things it shouldn't. The target is "appropriately scoped."

Under the hood, "scope" is your retrieval index plus your guardrails. Too narrow usually means your knowledge base is thin or your chunking severs related content. Too broad means you're letting the model answer from parametric memory when retrieval comes back empty — which is precisely where hallucinations live.

The recommended defenses are things I already reach for on any retrieval project: hybrid retrieval (BM25 keyword search fused with vector embeddings via Reciprocal Rank Fusion), reranking the top candidates with a cross-encoder, and semantic chunking that keeps a rule and its exception in the same chunk (Red Gate / Simple Talk). That last one is underrated: fixed-size splitting will happily put "refunds are allowed" in one chunk and "unless purchased on promotion" in another, and the model will confidently state a policy that doesn't exist.

3. Proactivity → cheap on the frontend, dangerous on the backend

Proactivity — clarifying questions, scannable suggestion buttons, product links with images — is where NN/g is most UX-flavored. Williams Sonoma's bot asking about machine type and budget before recommending an espresso maker is good product design.

But proactivity has a backend trap. NN/g's own "clarification proactivity" rule is to ask for information only when the answer can actually be acted upon. Their bad example is a bot that walked a user through availability questions and then admitted it couldn't check availability at all. That's not a copywriting miss — that's the UI promising a capability the system doesn't have. Proactive suggestions should be generated from what your tools and retrieval can actually deliver, not from a prompt that guesses at plausible next steps.

Alex's take: Proactive buttons are the easiest place to write a check your backend can't cash. If a suggestion chip implies an action, there should be a real tool call behind it. Otherwise you've built a demo, not a product.

4. Emotional responsiveness → guardrails, not personality

NN/g is refreshingly restrained here: acknowledge the situation, don't claim to feel emotions the user never expressed, and never let empathy substitute for actually solving the problem. Their bad example (Lacoste's "I understand your disappointment" when no disappointment was stated) is a bot performing empathy at someone.

I read this as a guardrail requirement more than a tone requirement. The constraint "don't invent the user's emotional state" is the same class of rule as "don't invent a refund policy" — it's an instruction to stay grounded in what was actually provided. And the deeper point matches the RAG literature: emotional polish can't paper over a wrong or missing answer. Grounding and uncertainty estimation are what keep responses safe; tone is the wrapper, not the fix (Parloa).

5. Transparency → identity, capability, rationale, and it's now the law

NN/g breaks transparency into four parts: identity (disclose you're AI), capability (be specific about limits, not a vague "I can't help with that"), rationale (explain judgment calls, especially refusals), and privacy (say why you need the email, in the conversation).

Two of these are directly implementable from the retrieval layer. Capability transparency and rationale transparency are basically source transparency — if your answer is grounded in retrieved passages, you can cite them, and "I don't have access to your purchase history, but I can connect you to someone who does" becomes a truthful statement about system state rather than a canned line. Citation grounding also creates a self-correcting pressure where omission is cheaper than a confident fabrication (Red Gate).

Identity transparency, meanwhile, stopped being optional. Under the EU AI Act's Article 50, systems that interact with people must disclose that they're AI, at the latest at the first interaction — and those transparency obligations apply from 2 August 2026, with fines up to €15M or 3% of global turnover (EU AI Act, Article 50; Sidley). If you ship a bot into the EU, the "it's an AI" badge is now a compliance line item.

The pattern underneath

Reading all five together, four of them collapse into the same two engineering problems: retrieve well, and know when you don't know. Flexibility, honest capability, rationale, and the refusal to fabricate emotion are all downstream of grounded retrieval plus a real confidence gate that escalates instead of improvising. Handoff is the escape hatch that catches whatever the gate rejects.

Alex's take: If I could only build two things into a site chatbot, they'd be a confidence threshold that routes low-certainty queries to a human, and a clean context handoff when it does. Every other quality on this list gets easier once the bot is willing to say "I'm not sure — let me get you someone who is." The frustrating bots aren't the ones that don't know things. They're the ones that won't admit it.