User Experience and Human Factors

Executive summary

User experience for AI is not cosmetic polish—it is trust engineering. Users who cannot see uncertainty, sources, limitations and escalation paths will either over-trust (accept harmful outputs) or under-trust (abandon a useful tool). An AI Solution Engineer must design interfaces that calibrate confidence: show when the system is unsure, cite evidence with inspectable sources, make correction and override frictionless, refuse safely when policy or confidence thresholds fail, and embed human-in-the-loop at the points where accountability and regulation require it. Accessibility, progressive disclosure, conversational UX and error states are not separate topics—they are the mechanisms through which trust is built or destroyed. This topic teaches you to produce UX principles checklists, key flows, trust copy guidelines and escalation patterns you can defend to risk, legal and frontline users—not wireframes that look impressive in a demo but fail in production.

When to use this page

• Before designing or reviewing any user-facing AI surface: chat, copilot panel, embedded assist, agent workflow UI.
• When pilot adoption stalls despite “good” model metrics—often a trust or workflow-fit problem, not a model problem.
• When risk, legal or DPO asks how users know an answer might be wrong and how they escalate harmful output.
• When accessibility audits flag WCAG gaps on dynamic AI content (live regions, focus management, contrast on confidence badges).
• When rebuilding after an incident where staff pasted unverified AI text into customer-facing channels.
• When comparing vendor UX patterns—evaluate trust calibration, not animation quality.
• When writing acceptance criteria for sprint stories that touch prompts, citations, or override flows.
• Pair with Adoption and change for training, champions and operating-model embed—not UI alone.

Executive view

Ask whether the UI makes wrong answers look as confident as right ones. Demand evidence: override rates, escalation volume, refusal rates, accessibility audit status—not demo screenshots.

Decision lens: Can a frontline user complete the task with appropriate scepticism in under 60 seconds, including finding a source and reporting an error?

Technical view

Treat UX as part of the control stack: confidence thresholds, citation rendering, trace IDs, ACL-aware source links, and human-in-loop gates are implementable requirements—not copywriter preferences.

Pair UI patterns with eval harness and logging; every “Report wrong” action should emit structured telemetry for quality loops.

Why this matters

Most enterprise AI failures visible to regulators and customers are human factors failures, not parameter tuning failures. A contact-centre agent pastes an invented refund policy into a live chat. A clinician accepts a confident but incomplete summary. A caseworker cannot find which document supported a recommendation and defaults to trusting the screen. In each case the model may have been “average” while the interface lied by omission—no uncertainty, no source, no one-click correction, no safe refusal state.

The cost is measurable. A UK insurer’s internal policy assistant launched with chat-style “I’m sure” copy and no citations; within six weeks, 14% of sampled sessions contained at least one answer contradicted by the corpus (manual QA). Override UI existed but required 11 clicks and opened a generic IT form with no trace ID—reporting rate was 0.3% of sessions. After redesign (confidence bands, inline citations, two-click “this was wrong” with auto-attached trace), contradiction rate in QA fell to 4% and reporting rose to 6.2%—feeding a retraining and prompt-fix backlog that prevented external complaints. Adoption rose from 31% to 58% in the same queue because supervisors trusted the tool enough to mandate shadow use.

Poor AI UX also destroys programme economics. If users double-check everything manually, you save no time. If they trust blindly, you inherit liability. Calibrated trust is the middle path: fast when evidence is strong; cautious when it is not; always correctable.

This topic connects to Responsible AI and Governance (policy and harm), AI Evaluation and Quality Assurance (measuring UX-linked quality), Stakeholder Management (user council input), and Adoption and change (training and embed). Topic 28 covers executive narrative; topic 23 covers what users actually see and do.

Learn

User-centred design (UCD) for AI—not generic UX with a chat box

Classic UCD still applies: understand tasks, contexts, errors and recovery before pixels. AI adds probabilistic output—the primary user question is not “where is the button?” but “should I believe this?”

UCD discovery questions for AI surfaces:

Question	Why it matters	Example finding
What decision does the user make after reading the output?	Sets required confidence and HITL	Approve customer refund up to £500
What is the cost of a wrong answer?	Sets refusal threshold and citation depth	Regulatory mis-sale; £40k fine risk
How do users verify today without AI?	Citations must match trusted sources	SharePoint policy PDF + supervisor
What breaks flow under time pressure?	Error states and progressive disclosure	90-second SLA on chat
Who is accountable for the final text?	Human-in-loop placement	Named agent, not “the AI”

Task-first layout: Start from the job story—“When I’m on a call and the customer disputes a cooling-off period, I want the exact clause and effective date so I can read it aloud confidently.” Design citations, date metadata and read-aloud formatting before conversational pleasantries.

Trust calibration—confidence, hedging and false certainty

Trust calibration means the interface communicates epistemic status honestly. Users should not need ML literacy to know when to pause.

Patterns that work:

Pattern	User reads	Implementation note
Confidence band	High / Medium / Low with colour + text	Map from retrieval score + eval classifier; never binary green/red only
Hedged copy	“Likely…”, “Draft for your review”	Template-controlled; avoid fake personality
Evidence strength	“3 sources agree” vs “1 weak match”	Show count and agreement
Known limits	“Not in corpus: billing disputes pre-2022”	Corpus coverage surfaced proactively
Refuse state	“I can’t answer safely—here’s what to do”	Linked runbook, not dead end

Anti-patterns:

• Uniform “I’m confident” on every response
• Percentages without explanation (“87% confident”—87% of what?)
• Animations that imply thinking equals correctness
• Hiding low confidence behind “show more”

Copy principle: Separate assistant voice (helpful tone) from epistemic voice (what we know). The assistant can be warm; the epistemic line must be precise.

Citations UI—inspectable, ACL-aware, copy-safe

Citations are not footnote decoration—they are verification affordances. Users and auditors must open the source, see the passage in context, and confirm ACL.

Citation UI requirements:

1. Inline markers linked to source list (numbered [1][2])
2. Snippet highlight in source viewer with surrounding paragraph
3. Metadata: document title, version, effective date, owner
4. ACL enforcement: if user lacks access, show “Restricted document—request access” not silent omission
5. Copy behaviour: copying answer optionally includes citation block for paste into case notes
6. Stale detection: badge if corpus version < policy published date

Worked UI string (good):

Answer (Medium confidence): Cooling-off is 14 days from policy start for retail lines. [1][2]

Sources
[1] Retail Policy v3.2 §4.1 (effective 1 Jan 2026) — "14 calendar days..."
[2] FAQ: Cooling-off exceptions (updated 12 Mar 2026)

Not found in corpus: commercial lines, renewals.
Review before customer send — you are accountable for final text.

Worked UI string (bad):

I'm sure! The cooling-off period is 14 days. Trust me 😊

Escalation and feedback—two-click correction with trace

Users will not report errors if reporting is punishment or bureaucracy. Escalation UX must be faster than opening a blank ticket.

Minimum viable feedback flow:

1. User clicks “This is wrong” or thumbs-down on the answer block
2. Optional category: factual error / missing context / harmful / other
3. Optional free text (≤500 chars)
4. System auto-attaches: session ID, trace ID, prompt version, retrieval IDs, model version, user role, timestamp
5. Confirmation: “Report #7842 sent to Policy QA—typical response 2 business days”
6. User can continue working with manual fallback link

Integration targets: ServiceNow, Jira, Zendesk, internal “AI quality” queue—not anonymous spreadsheet.

Metrics to instrument:

Metric	Healthy signal	Warning signal
Reports per 1k sessions	2–15 (domain dependent)	<0.5 (UI broken) or >80 (quality crisis)
Time to submit report	<30 seconds median	>2 minutes
Repeat reporters	Power users engaged	Same user 50+/day (UI abuse or rage)
Resolution loop closure	Fix deployed + user notified	Reports vanish into void

Human-in-the-loop (HITL) design—where humans must decide

HITL is not “human reviews everything”—that kills ROI. HITL is placed at accountability boundaries defined by risk, policy and workflow.

HITL placement matrix:

Risk tier	Example workflow	UI pattern
Low	Internal FAQ draft	Edit-before-send optional
Medium	Customer email draft	Mandatory preview + send button
High	Medical, legal, credit decision	AI draft; human Approve gate; no auto-send
Prohibited	Autonomous eligibility decision	AI may suggest; system refuses auto action

UI elements for HITL:

• Draft watermark on all generated text until approved
• Approve / Edit / Reject with required reason on reject
• Audit trail visible to supervisor: who approved, what changed
• Rollback to last approved template for regulated content
• Dual control for high-value actions (second approver)

Phrase to avoid: “The AI decided.” Phrase to use: “Draft generated—your approval required before customer sees this.”

Accessibility (WCAG)—dynamic AI content is not optional

AI UIs often fail accessibility because content is injected asynchronously. Regulated employers and public sector procurement increasingly require WCAG 2.2 AA on internal tools—not only customer sites.

Critical accessibility requirements for AI surfaces:

Area	Requirement	AI-specific note
Perceivable	Text alternatives for non-text confidence icons	Don’t rely on colour alone for High/Med/Low
Operable	Keyboard access to citations, approve, report	Chat widgets often trap focus
Understandable	Plain language; consistent refusal patterns	Avoid jargon in error messages
Robust	Live regions announce new answers	Screen reader users must hear confidence + “draft”
Focus management	Focus moves to new answer container	Prevents “silent” updates
Timing	No time limits on review unless essential	Agents need time to read sources
Motion	Respect prefers-reduced-motion	Disable typing animations

Live region example (conceptual):

<div aria-live="polite" aria-atomic="true">
  Policy answer ready. Medium confidence. 2 sources. Review required before send.
</div>

Accessibility acceptance test: Complete a task using keyboard only + screen reader (NVDA/VoiceOver) including cite source, report wrong, approve send.

Conversational UX—threads, context and scope boundaries

Conversational UI is powerful when users explore ambiguous questions; dangerous when it implies omniscient dialogue.

Conversational patterns:

• Scope banner: “Ask about UK retail policies only—not claims litigation.”
• Session memory indicator: “Using context from this case (#4421)” with clear reset
• Suggested prompts grounded in allowed tasks—not open “ask me anything”
• Turn limits for high-risk domains with graceful handoff to human
• No faux empathy on refusals—“I can’t help with that” not “I’m sad I can’t”

Progressive disclosure: Show short answer + “Show reasoning and sources” expander. Reduces cognitive load for experts; protects novices from wall-of-text.

Error states, empty states and safe failure

AI systems fail differently from CRUD apps: timeout, retrieval miss, policy block, model refusal, rate limit, partial context.

Error state catalogue:

Failure	User message	Recovery action
Retrieval miss	“No approved sources found for this question.”	Suggest rephrase + link to manual search
Low confidence	“Answer below quality threshold—not shown.”	Show top sources only; no synthesis
Policy block	“This request isn’t permitted (PII export).”	Explain rule + alternative workflow
Timeout	“Taking longer than usual—try narrower question.”	Retry + ticket link
Harmful content filter	“Blocked by safety policy.”	Escalate to supervisor code
Service down	“Assistant unavailable—use runbook B.”	Status page + ETA if known

Empty state: On first open, show what the tool can and cannot do with three example prompts—reduces junk queries and false expectations.

Onboarding and progressive trust building

Onboarding is not a video tour—it is calibration training in the UI.

Onboarding layers:

1. 60-second scope + accountability (“You approve all customer text”)
2. Interactive exercise: find intentional error in sample answer; practice “Report wrong”
3. Supervisor sign-off for production access in high-risk roles
4. Office hours badge for first two weeks
5. Metrics dashboard for team: adoption, override rate, report rate

Trust ladder: Week 1 shadow mode (suggestions only) → Week 2 edit-before-send → Week 3 timed send with spot QA → steady state per risk tier.

Anthropomorphism and unsafe persona design

Anthropomorphism increases engagement—and over-trust. Regulators and employee relations teams increasingly challenge “friendly AI colleague” framing in high-stakes work.

Do:

• Name the product functionally: “Policy Assistant”
• Use first person sparingly for system actions: “I found 2 sources”
• Attribute accountability to humans in UI chrome

Don’t:

• Imply consciousness, emotion or friendship
• Use human avatar implying human judgement
• Simulate typing delay to suggest “careful thinking”
• Hide that output is generated

Union-sensitive contexts: Avoid surveillance cues (keystroke timers, “productivity scores” tied to AI usage) in the same UI as assist—adoption will collapse.

Mobile, embedded and multi-surface consistency

Users encounter AI in CRM side panels, mobile field apps, Teams bots and desktop web. Trust patterns must be consistent or users re-learn scepticism per surface.

Surface	UX constraint	Trust implication
CRM embed	Narrow width	Citations collapse to drawer; must remain one tap
Mobile field	Offline periods	Show last-sync corpus date; refuse if stale
Teams bot	Thread noise	Pin confidence + sources in card attachment
Voice	No visual skim	Spoken confidence + “verify in app” deep link

Measuring UX quality—not only CSAT

Pair qualitative UX review with operational metrics:

Metric	Definition	Target example (pilot)
Task success	User completes workflow with acceptable output	≥85%
Time on task	Median time vs baseline manual	−20% without error rise
Override rate	User edits AI draft before send	8–25% (too low = blind trust)
Report rate	Wrong-answer reports per 1k sessions	>2
Refusal rate	Safe refuses / all queries	Domain-specific
Citation click-through	Users open ≥1 source	≥40% on high-risk answers
Abandon rate	Session exit after answer	<15%

Frameworks and methods

Trust UX checklist (eight pillars)

Use before every gate review:

1. Scope visible — user knows domain limits
2. Confidence visible — not uniform certainty
3. Sources inspectable — ACL-safe citations
4. Accountability explicit — human approves customer-facing text
5. Correction easy — report wrong ≤2 clicks + trace
6. Refusal safe — policy blocks with next step
7. Accessible — WCAG-oriented keyboard + screen reader path
8. Failure graceful — timeouts and outages have runbook links

NIST-inspired human-AI teaming UX mapping (simplified)

Map UX controls to teaming functions:

Teaming function	UX control
Observe	Show sources and reasoning summary
Orient	Confidence + known gaps
Decide	Approve / edit / reject
Act	Send / apply with audit log
Learn	Report wrong closes loop

CALM copy framework (Confidence, Accountability, Limits, Mechanism)

Write UI strings with four elements:

• Confidence level stated
• Accountability on user for external send
• Limits of corpus / policy
• Mechanism to verify (citations) and correct (report)

Progressive disclosure tiers

Tier	Content	Audience
T0	Answer + confidence badge	All users
T1	Sources list + snippets	Default expand for novices
T2	Retrieval scores, chunk IDs	Power users / QA
T3	Trace JSON export	Engineering / incident

Never show T3 to frontline by default—support burden explodes.

Design critique rubric (score 0–2 each, max 16)

Criterion	0	1	2
Confidence honest	Always sure	Sometimes	Calibrated
Citations usable	None	Links broken/partial	ACL + snippet
Correction friction	>5 clicks	3–5 clicks	≤2 clicks
Refusal helpful	Blank	Generic	Runbook linked
HITL placement	Auto-send	Optional preview	Mandatory gate
Accessibility	Untested	Partial	Keyboard+SR pass
Error recovery	Dead end	Retry only	Fallback workflow
Onboarding	None	Video only	Interactive calibrate

Score ≤8 → block pilot expand; 9–12 → fix before scale; ≥13 → proceed with monitoring.

UX ↔ eval loop integration

Every UX pattern should tie to eval or logging:

UX element	Backend hook
Confidence badge	Classifier threshold in eval JSON
Report wrong	Labelled example queue
Refusal	Logged as abstention metric
Approve button	Human label for RLHF / audit
Citation click	Implicit relevance feedback

Co-design with users and risk teams

UX that survives production is co-designed, not handed off from a vendor design system.

User council UX sessions (90 minutes):

1. Walk through today’s task without AI (shadow recording if permitted)
2. Paper-prototype answer, citation and report-wrong flows
3. Rank trust elements: confidence vs citations vs approve gate
4. Capture failure stories from past tools (“why we stopped using X”)

Risk team review checklist:

• Can user identify AI-generated vs template text?
• Is approve gate non-bypassable via keyboard shortcut hack?
• Does copy avoid implying legal/medical advice where restricted?
• Are refusal messages aligned with policy language legal approved?

Document UX decisions in lightweight UX decision record (UXR) linked to ADRs.

Internationalisation and plain language

Global programmes need plain language at B1 reading level for frontline roles—not policy jargon compressed by the model.

Issue	UX response
Multi-language answers	User selects working language; corpus language badge shown
Locale dates/numbers	Format per locale in citations
Legal term consistency	Controlled vocabulary in UI labels
Translation of AI output	Label machine-translated content; lower default confidence

Test with non-native speakers in target market—not only HQ English reviewers.

Hypercare UX monitoring (first 30 days)

Week	UX focus	Action trigger
1	Report-wrong volume, error states	Spike in timeout → infra
2	Citation click-through	<20% on high-risk → citation UI fix
3	Override rate by site	Outlier site → local training
4	Accessibility complaints	Any P1 → hotfix sprint

Hypercare ends when metrics stable 2 consecutive weeks—not on calendar alone.

Real-world scenarios

Scenario A — Insurance contact centre: false certainty and citation rebuild

Context: UK general insurer; 3,200 contact-centre agents; RAG over 2,400 policy documents. Pilot queue 800 agents; baseline median handle time 11.4 minutes on policy clarification calls.

UX failure mode: Vendor demo UI used chat bubbles with green checkmarks on every answer; no citations above fold; “Copy to customer” one click with no preview gate.

Incidents (6-week pilot): 23 customer complaints citing wrong cooling-off advice; internal QA sample 14% contradictory answers; 0.3% report rate.

Intervention:

• Replaced certainty icon with High/Medium/Low bands tied to retrieval + eval score
• Mandatory preview panel before paste to customer chat
• Inline citations with SharePoint deep links and effective dates
• Report wrong → auto-ticket to Policy QA with trace ID
• Refusal when confidence < threshold: show sources only, no synthesis

Outcome (next 8 weeks, n=1,100 calls):

• QA contradiction rate 4.1%
• Report rate 6.2% per 1k sessions
• Override (edit before send) 19%—healthy scepticism
• Median handle time 9.1 minutes (−21%) vs baseline
• Customer complaints on policy advice zero in pilot queue

Lesson: Handle time improved after trust UX hardened—not before. Speed without calibration is liability.

Scenario B — NHS trust clinical admin: accessibility and HITL for summaries

Context: Acute trust; 420 medical secretaries summarising referral letters into EPR notes. Pilot assist for draft summaries only; CQC and clinical governance oversight.

UX requirements driven by risk:

• Mandatory approve before note enters EPR
• WCAG 2.2 AA on desktop Citrix environment
• Live region announcements for new drafts
• Red banner on all drafts: “Not reviewed—do not use for clinical decisions without clinician sign-off”
• Refusal for missing key fields (allergies, medications) with checklist UI

Accessibility work: Fixed focus trap in vendor widget; added keyboard shortcut Alt+R to open citations drawer; high-contrast confidence badges (not colour-only).

Outcome (12-week pilot, n=680 letters):

• Median prep time 18 min → 13 min (−28%)
• 100% drafts passed human approve gate (by design)
• Secretary task success 91% (completed note without rework)
• Accessibility audit AA pass on critical path
• Clinical governance sample 0 critical patient safety edits missed by approver

Numbers for steering: Projected £340k/year capacity equivalent at £22/hour loaded cost; UX/HITL build £95k one-off.

Scenario C — Public sector housing repairs: mobile field app and offline trust

Context: Regional housing association; 1,400 field operatives; mobile app suggests repair codes and parts from job history + manuals. Union concern: “black box” recommendations affecting job plans.

UX approach:

• Show-your-work drawer: top 3 manual excerpts + photos from similar jobs (anonymised)
• Operative override one tap with reason codes (union co-designed)
• Offline badge: “Corpus synced 09:42—some policies may be stale”
• No anthropomorphic avatar; functional label “Repair suggester”
• Escalation to scheduler when confidence Low—call scheduler button

Outcome (16-week pilot, 2 regions, 380 operatives):

• First-visit fix rate 74% → 81%
• Override rate 22%—used in tuning, not suppressed
• Union neutral statement after co-design
• Report-wrong rate 3.8% per 1k jobs
• Abandon rate 9% (healthy vs 28% in first UI without sources)

Scenario D — Financial services AML analyst: escalation to investigation workflow

Context: Tier-1 bank; 220 AML analysts; assist drafts SAR narrative sections from case files. Legal insists: no autonomous filing; 4-eyes review.

UX failure (initial vendor): Chat-style “I completed your SAR” button nearly caused regulatory breach—demo culture.

Redesign:

• Workflow steps mirror existing case system—not free chat
• Each section: Draft → Analyst edit → Second reviewer approve
• Citations to transaction IDs and policy clauses
• Escalate to MLRO fixed action with pre-filled context
• Refusal when source documents redacted beyond analyst clearance

Outcome:

• Draft prep time −35% on pilot cases (n=190)
• Zero auto-submissions
• MLRO escalation time −18% due to structured context
• Analyst trust survey: 72% “I understand when not to trust the draft” vs 31% pre-redesign

Scenario E — Retail ecommerce: accessibility complaint drives redesign

Context: EU retailer; 120 customer service agents; product return policy assist; launched without screen reader testing.

Failure: Blind agent filed equality complaint week 3; citations keyboard-inaccessible; confidence colour-only.

Fix (4-week sprint): WCAG remediation, live regions, text labels on bands, focus order documented.

Outcome:

• AA pass on retest; complaint closed with union witness
• Adoption among agents with assistive tech 44% → 71%
• Legal exposure avoided; steering approved scale with £45k accessibility line in budget

Lesson: Accessibility defects are adoption and legal risks—gate before scale.

Practice exercises

Primary exercise — Rewrite trust copy for RAG answers (45 minutes)

Brief: HR policy assistant for 4,000 employees. Three answer states required: High confidence, Medium confidence, Refuse (topic outside corpus). Audience: employees on mobile; WCAG required.

Tasks:

1. Write UI strings for each state using CALM framework (≤120 words each).
2. Design citation block format (mock text) with version and effective date.
3. Specify Report wrong flow in ≤5 bullet steps with trace fields listed.
4. Write onboarding banner (3 bullets) stating scope and accountability.
5. Define one accessibility acceptance criterion per state (keyboard or screen reader).

Acceptance criteria:

• No false certainty phrases (“I’m sure”, “guaranteed”)
• Refuse state includes next step (human contact or manual search)
• Accountability line present on any customer- or manager-visible output
• Medium confidence visibly different from High (text + non-colour cue)

Stretch exercise — End-to-end trust UX spec for regulated launch (half day)

Brief: Insurance claims adjuster copilot; 2,500 users; customer-facing emails generated from drafts; CISO requires ACL on citations; union consultation on monitoring.

Tasks:

1. Complete eight-pillar trust checklist with pass/partial/fail and remediation.
2. Draw HITL placement matrix for four workflow steps (research, draft email, approve, send).
3. Error state catalogue: minimum 6 failure types with user message + recovery.
4. Metrics dashboard wireframe (text): 8 metrics with targets and red thresholds.
5. Design critique rubric score self-assessment with evidence.
6. 15-minute usability test script for 3 frontline users (tasks + success criteria).

Acceptance criteria:

• Design critique score ≥13 or explicit blockers listed
• No auto-send path documented
• Escalation integrates ticket ID and trace
• Union concern addressed in UX (no covert monitoring cues)

Questions you should be able to answer

1. How does the UI show uncertainty without requiring ML literacy?
2. How does a user open an inspectable citation with ACL enforced?
3. What happens in the UI when confidence is below threshold?
4. How does a user report a wrong answer in two clicks or fewer?
5. What trace data attaches automatically to a user report?
6. Where is human-in-the-loop mandatory vs optional in your workflow?
7. How does the interface state user accountability for customer-facing text?
8. What does the refuse state show instead of a synthesized answer?
9. How do error states handle timeout, retrieval miss and policy block differently?
10. What WCAG requirements apply to dynamically injected AI answers?
11. How does onboarding calibrate trust in the first session?
12. What UX metrics indicate blind over-trust vs healthy scepticism?
13. How are citations and confidence consistent across CRM embed and mobile?
14. What anthropomorphism risks exist in your current copy and visuals?
15. How does UX feedback connect to eval harness and prompt fixes?

Operating model — UX in the delivery cadence

UX is not a one-off design phase—it runs on the same rhythm as engineering and change.

Cadence	UX activity	Owner	Output
Sprint	Trust copy review for new stories	UX + ASE	Strings in repo
Fortnightly	Citation ACL test per role	QA + security	Test log
Monthly	Override/report review	Product + change	Backlog priorities
Gate	Design critique rubric score	ASE	Pass/fail record
Incident	Harmful output UI post-mortem	UX + legal	UXR update

Link UX defects to RAID like any blocker—a broken citation link is not polish if adjusters cannot verify policy.

Trust copy banned phrases (starter list)

Remove from production UI:

• “I’m sure” / “guaranteed correct”
• “As an AI language model…” (breaks task focus without adding control)
• “Trust me”
• “This replaces your judgement”
• “100% accurate”
• Emoji-only confidence indicators
• Human persona name implying human accountability

Replace with CALM patterns from the Learn section.

Scenario walkthrough — one answer, three UI versions

Same retrieval result (Medium confidence, 2 sources, billing exception not in corpus):

Version A (fail): Green check; “The cooling-off period is 14 days.” Copy button only.

Version B (pass): “Medium confidence — verify before customer send.” Citations [1][2]; “Billing exceptions not in corpus—escalate to Team B.” Report wrong button.

Version C (pass+): Version B plus mandatory preview panel before paste to customer channel; approve checkbox.

Use this walkthrough in design reviews—stakeholders instantly see trust delta without ML detail.

Accessibility test script (30-minute lab)

Run before each gate with real assistive tech:

1. Tab through: ask field → submit → answer → first citation → report wrong → approve (if HITL)
2. Screen reader: confirm live region announces confidence and draft status
3. Zoom 200%: no horizontal scroll on citation drawer
4. High contrast mode: confidence band still readable
5. Record pass/fail per step with screenshot IDs for audit pack

Fail any step on critical path → amber gate minimum until fixed.

Negative cases — when UX fails

The confidence theatre

Symptom: Green checkmarks, typing animation, “I’m happy to help!” on every answer; confidence always “high.”

Impact: Wrong answers accepted; incidents; regulator asks “how did staff know?”

Fix: Calibrated bands; show retrieval weakness; refuse when below threshold.

Citation wallpaper

Symptom: Links present but broken, generic, or point to whole PDF page 1—not supporting clause.

Impact: Users ignore citations; audits fail; trust collapses after first broken link.

Fix: Snippet highlights; version metadata; ACL tests in UAT per role.

Report into the void

Symptom: “Feedback” opens blank email or generic IT portal; no trace ID.

Impact: Report rate near zero; quality team blind; same errors repeat.

Fix: Structured report with auto context; ticket ID confirmation; closure loop visible to users.

Auto-send by accident

Symptom: Single click sends AI text to customer or EPR; no preview.

Impact: Regulatory breach; patient safety near-miss; programme halt.

Fix: Mandatory preview + approve; draft watermark; undo window where policy allows.

Accessibility afterthought

Symptom: Mouse-only citations; no live regions; colour-only confidence.

Impact: Excludes staff; legal equality risk; workaround double-work negates ROI.

Fix: Keyboard path UAT; screen reader test; WCAG checklist in definition of done.

Chat for everything

Symptom: Free-form chat on high-risk workflow; no scope banner; no step alignment.

Impact: Prompt injection surface; users ask out-of-scope; hallucinations on edge cases.

Fix: Workflow-aligned UI; scope limits; suggested prompts; structured steps.

Demo UX shipped to production

Symptom: Sales demo skin with consumer branding; no HITL; features disabled in prod confuse users.

Impact: Expectation gap; adoption cliff after week 2.

Fix: Pilot UX = production UX; gate on trust checklist not screenshot.

Output

Expected outputs for this topic:

• UX principles checklist — eight pillars scored per surface
• Key screens/flows — research, answer, cite, approve, report, refuse, error
• Trust copy guidelines — CALM templates; banned phrases list
• Escalation specification — report flow, trace fields, ticket integration
• Accessibility test record — keyboard + screen reader pass/fail per flow
• UX metrics dashboard definition — override, report, citation CTR, task success

Practice checklist

• I can explain trust calibration without naming a vendor or model first
• I completed the primary copy rewrite exercise in writing
• I scored a surface on the design critique rubric (≥13 or blockers documented)
• I specified HITL gates for at least one regulated workflow
• I linked UX metrics to eval/logging hooks
• I know which stakeholders (risk, union, users) must review UX before scale
• I opened Adoption and change for embed and training alignment

Related playbook content

• Adoption and change — training, champions, embed and operating rhythm
• Responsible AI patterns — policy, harm and human oversight
• Discovery and problem framing — task analysis before UI design
• Stakeholder Management — user council and union engagement
• AI Evaluation and Quality Assurance — linking UX to eval loops
• How to use this Learning Map
• 8D Framework

Key takeaways

• AI UX is trust engineering: uncertainty, sources, correction and refusal are control features—not copy tweaks.
• False certainty destroys programmes faster than weak models; calibration enables both safety and speed.
• Human-in-the-loop belongs at accountability boundaries—design approve gates before customer or EPR send.
• Accessibility on dynamic AI content is a delivery requirement, not a backlog nice-to-have.
• Measure override and report rates—too low means blind trust; too high means quality crisis.
• Pair every UX pattern with telemetry that feeds eval and prompt improvement loops.

What to do next

1. Run the primary exercise on your current or fictional RAG UI strings.
2. Score your pilot UI on the eight-pillar checklist before expanding users.
3. Continue with Integration and Enterprise Systems.
4. Align embed and training with Adoption and change.
5. Open the interactive playbook for workshop templates.