The Same Answer, Three Explanations: Role-Aware AI Is a Design Problem

Here's a failure mode I keep seeing: a team ships an AI feature, wires up a single "why?" explanation, and calls it explainable. Then the analyst wants the retrieved evidence, the executive wants a one-line "can I trust this," and the end user just wants to know whether to act on it. One explanation, three audiences, and it satisfies none of them.

Nielsen Norman Group published a piece on exactly this — Crafting AI Explanations for Every Role in Your Enterprise — and their core claim is worth repeating: enterprise AI can't use a one-size-fits-all explanation, because the people who need to trust the system have different responsibilities. They frame explainability as design work, not a technical afterthought. I agree with the framing. I want to push it one layer down, into what the backend actually has to produce for any of that design to be possible.

The NN/g framework, briefly

NN/g splits enterprise AI users into three roles and pairs each with a kind of explanation:

Role	What they own	Explanation they need
Governance leads	Practices, security, compliance	Global — how the AI decides across situations
Builders	Platforms, config, integration	Local — why this output, tied to inputs
Domain experts	Workflow context, judgment calls	Plain-language — grounded in familiar decisions, no jargon

Global vs. local is the standard XAI split, and it isn't NN/g-specific: global explanations describe overall model behavior, local explanations describe why one specific prediction happened. What NN/g adds is the reminder that "good explanation" is audience-relative — a technical reader may want the detailed, near-mathematical version while a non-technical reader wants the intuitive one. Same underlying decision, different rendering.

Where I want to reframe it

NN/g writes for designers, so it stops at the interface. But every one of those explanations is a rendering of state the system either captured or didn't. You can't design a local explanation for a builder if the pipeline threw away which inputs drove the output. You can't hand governance a global view if you never logged decisions in a queryable shape.

So my working model: the model produces an answer plus an explanation substrate; the audience determines the projection. The substrate is one thing. The projections are many. If the substrate is thin, no amount of UX polish saves you.

Concretely, the substrate I try to emit alongside any AI output:

Retrieved sources — what evidence the answer stood on, with stable IDs.
A confidence signal — some honest measure of how sure the system is.
A reasoning trace — the steps or intermediate decisions, when they exist.
Structured logs — the same facts, persisted, so governance can query trends and not just eyeball one response.

Verbosity is then a knob, not a rewrite. The executive gets the confidence badge and one cited claim; the analyst expands the full source list and trace; the governance dashboard aggregates across thousands of these. One substrate, three projections.

RAG makes this concrete

This is where retrieval-augmented generation stops being a buzzword and starts being the cleanest example of explainability I know. In a RAG system, citing the retrieved chunk is the local explanation. The evidence isn't reconstructed after the fact with a post-hoc attribution method — it's the literal input the generation stood on.

That's a real advantage over trying to explain a raw model. The retrieval step gives you the source-of-truth for free; your job is to not lose it. Henry Ruiz's write-up on citations in RAG systems lays out the mechanics well: keep chunk metadata (title, url, year, a stable reference number) attached through retrieval and generation, and instruct the model to cite by that number — "Only cite a source when you are explicitly referencing it." Anchoring answers to concrete, traceable evidence is what reduces hallucination in the first place.

One distinction from that piece I want to flag, because it bites in production: correctness is not faithfulness. A cited source can genuinely support a claim (correct) while the citation is pinned to the wrong sentence, or the model paraphrased past what the source actually says (unfaithful). If you only check that a link exists, you'll ship confident-looking answers that fall apart the moment the analyst clicks through. The citation has to point at the span that earned it.

Roughly, the shape I aim for:

{
  "answer": "Diabetic patients should be coded E11.9 when unspecified. [1]",
  "citations": [
    {"ref": 1, "chunk_id": "icd10-e11-0007", "source": "ICD-10-CM 2026", "span": [412, 498]}
  ],
  "confidence": 0.82,
  "retrieval": {"strategy": "hybrid", "k": 8, "reranked": true}
}

That object renders three ways without regeneration: a badge for the executive, a clickable [1] for the domain expert, the full chunk-and-span for the analyst, and a row in a table for governance to aggregate.

Confidence is the part everyone gets wrong

Surfacing a number is easy. Surfacing an honest, calibrated one is not. The UX guidance here is consistent and sensible: confidence signals should help users calibrate trust, and — as one design guide puts it — "the interface should never imply more certainty than the system can deliver." When signals are inconsistent, users either over-trust or disengage entirely, and both are expensive.

The hard truth underneath: a model's self-reported confidence is often poorly calibrated, so verbalized confidence can actively mislead. So I treat the displayed signal as a product decision, not a raw model output — derived from things I can defend (retrieval score agreement, whether multiple sources concur, reranker margin) rather than asking the LLM "how sure are you?" and printing the answer. And the strongest trust signal in that same UX research isn't a number at all — it's editability: let the user review, adjust, or override. Control beats a confidence bar.

Alex's take

(Opinion, clearly labeled.)

Two things I believe more strongly than the sources will commit to:

First, explanation quality is a data-model problem before it's a UI problem. Most "we need explainability" conversations should start at the schema — are you even persisting the sources, scores, and traces? — not at the component library. There's research arguing that current XAI methods don't actually give end users the explanations they ask for, and that transparency plus rigorous validation earns trust better than bolted-on explanation techniques. That matches my instinct: a truthful cited source beats a fancy saliency map for a domain expert every time.

Second, log for the audience that isn't in the room yet. The governance/global view is the one teams skip, because there's no user tapping their foot for it on day one. But it's the one you can't backfill — if you didn't persist decisions in a queryable shape, that history is just gone. Cheap insurance: write every AI decision, its citations, and its confidence to a structured, append-only log from day one, even if nothing reads it yet.

The reframe I'd leave you with: stop asking "is our model explainable" and start asking "does our system emit enough for us to explain it to whoever's asking." Explainability is a design problem, yes — but design can only project what the backend bothered to keep.