AI Evaluation and Quality Assurance

Executive summary

AI evaluation and quality assurance define how you know the system is safe to ship and stays safe in production. Match methods to risk class: offline eval on golden sets, human review, LLM-as-judge (with known limits), adversarial red teams, regression gates in CI/CD, and online eval (feedback, shadow traffic, A/B). An AI Solution Engineer refuses launch when metrics miss bars—e.g. citation correctness 71% against a 90% gate for customer-influencing servicing—and designs evaluation as ongoing discipline, not a one-time UAT sign-off.

When to use this page

• Before any production release of LLM, RAG or agent features—regardless of pilot applause.
• When changing models, prompts, retrieval, tools or corpora and you need regression proof.
• When stakeholders conflate demo quality with measured faithfulness, safety or task completion.
• When defining risk-based bars: marketing draft assistant ≠ credit decision support ≠ clinical summarisation.
• When building observability that connects offline golden sets to online drift signals.
• Pair with Prompt and Context Engineering, Retrieval-Augmented Generation and Agentic AI and Workflow Automation for domain-specific metrics.
• Deep dive: Production-Grade Evaluation Strategies for LLMs, RAG, Agents, and Multi-Agent Systems.

Executive view

Ask for the **release gate table**: metric, threshold, owner, last run result. Challenge launch when eval scope doesn’t cover critical failure modes or when only vendor benchmarks exist.

Decision required: Does measured quality meet the bar for this risk class—and what happens automatically if online metrics degrade post-launch?

Technical view

Build stratified golden sets; automate schema and citation checks; use LLM judges only with human calibration; wire CI regression; sample production for human review; align online metrics to offline dimensions.

Document known judge biases and failure modes. Eval is code—version datasets, prompts and harnesses together.

Why this matters

Enterprise AI fails in two acts: Act 1—ship on demo vibes; Act 2—discover in production that citations were wrong, agents looped, or summarisers omitted exclusions. Evaluation is how you prevent Act 2 or detect it in hours, not quarters.

Different risk classes demand different bars. A marketing copy assistant might tolerate occasional tone drift with human edit before publish. A policy Q&A influencing customer refunds needs citation correctness ≥90% and refusal when evidence missing. Credit decision support may require human-only Decide with AI as read-only retrieval—eval focuses on leakage and omission, not “helpfulness.” Treating one bar as universal either blocks low-risk innovation or green-lights high-risk harm.

The AI Solution Engineer owns eval strategy, golden dataset design, release quality gates, human review operating model, and the bridge from offline CI scores to online monitoring. Weak programmes run fifty manual chats before go-live. Strong programmes run hundreds of labeled cases every PR, red-team injection suites, calibrated LLM judges, weekly human sampling with inter-rater agreement, and automatic rollback triggers when faithfulness drops.

Evaluation also protects teams from endless subjective prompt debates: if golden case #147 regressed, the change doesn’t merge—discussion ends with data. When leadership asks “is it good enough?” you answer with metrics, risk class and residual risk acceptance—not “we tested it internally.”

Connect to Evaluation and observability playbook, RAG for retrieval metrics, and Responsible AI for risk classification.

Learn

Evaluation dimensions — what to measure

Dimension	Definition	Applies to
Accuracy / correctness	Output matches ground truth or expert label	Classification, extraction
Faithfulness	Claims supported by provided context	RAG, summarisation
Citation correctness	Cited sources actually support claims	RAG, compliance
Answer relevance	Addresses user question	All generative
Refusal appropriateness	Declines when evidence insufficient	RAG, advice
Safety	No policy-violating content	Customer-facing
Fairness	Parity across protected groups where applicable	HR, credit, marketing
Task completion	Multi-step goal achieved	Agents
Tool correctness	Right tool, valid args, auth respected	Agents
Latency p95/p99	User experience	All
Cost per success	FinOps efficiency	All at scale
Robustness	Performance under paraphrase, noise	All

Pick 3–5 primary metrics per feature aligned to risk—not twelve dashboards no one gates on.

Offline evaluation — golden sets and harnesses

Offline eval runs fixed datasets through the system in CI or batch jobs before release.

Golden set properties:

• Stratified — cover happy path, edge, negative, adversarial, ACL
• Labeled — expected answer pattern, required citations, or acceptable set
• Versioned — git tag with dataset changelog
• Sized to risk — 100–500+ cases for medium/high; smaller only for low-risk with documented gaps
• Maintained — owner refreshes on corpus/policy change

Harness components:

1. Input fixture (query + identity + context mocks if needed)
2. Execute full pipeline (not model-only unless isolated test)
3. Automated checks: JSON schema, citation ID validity, regex rules
4. Metric scorers: exact match, fuzzy, embedding similarity, NLI faithfulness, LLM judge
5. Report: pass/fail per case, aggregate metrics, diff vs baseline branch

Regression policy: Compare PR branch to main; fail if primary metric drops > agreed delta (e.g. 2 points) or below absolute floor.

Golden set design — coverage, not volume alone

Coverage matrix example (RAG policy assistant):

Bucket	% of set	Example
Factual lookup	30%	“Cooling-off period for motor?”
Multi-chunk synthesis	20%	Compare two endorsements
Negative/unanswerable	15%	Question outside corpus
ACL forbidden	10%	Restricted memo query as junior role
Injection/adversarial	10%	Override in pasted text
Ambiguous/conflict	10%	Two clauses tension
Format/edge	5%	Empty query, very long paste

Each case has ID, risk tag, expected behaviour (not always single gold string).

Sourcing labels: SME workshops, production sampled failures, red team, synthetic augmentation (flag synthetics).

Avoid: Duplicate near-identical cases inflating pass rate; all labels from one enthusiastic SME; static set never updated after first launch.

LLM-as-judge — power and limits

LLM-as-judge uses a model to score outputs against rubric—scalable but biased.

Known limits:

Limit	Effect	Mitigation
Position bias	Prefers first answer in pairwise compare	Swap order, aggregate
Leniency	Scores “pretty good” too high	Calibrate to human scores
Self-preference	Favours same model family	Use independent judge model
Rubric drift	Judge prompt changes shift scores	Version judge prompts
Citation blind spots	Scores faithfulness without span check	Add deterministic citation verifier
Cost	Judge adds inference spend	Sample + human anchor

Calibration protocol:

1. Human-rate 100 cases on 1–5 scale per dimension
2. Run judge on same cases; compute correlation (target ≥0.8 Spearman on primary metric)
3. Adjust rubric until calibrated—or demote judge to triage only
4. Re-calibrate on model upgrade

Never: LLM-as-judge as sole gate for high-risk without human anchor set.

Human evaluation — when people remain essential

Human eval for: new feature baselines, judge calibration, high-risk sampling, dispute resolution, fairness audits.

Operating model:

• Raters: SMEs with documented guidance; secondary review on disagreements
• Rubrics: Observable criteria, examples of 1 vs 3 vs 5
• Inter-rater agreement: Cohen’s kappa or percent agreement; target ≥0.7 for launch
• Throughput: Sample size calculators—don’t guess “we reviewed 20”
• Blinding: Raters don’t see vendor name or model version when comparing
• Feedback loop: Failed cases become golden set IDs

Queue design: Route production outputs by risk score to human review (e.g. compensation intent, low citation confidence).

Adversarial and red-team evaluation

Red team proactively attacks: injection, jailbreak, tool misuse, ACL bypass, data exfil.

Deliverables: Attack case library in CI; severity classification; fix verification rerun.

Cadence: Before major release; quarterly for customer-facing; after incident.

Integrate with Responsible AI and security testing.

Regression gates in CI/CD

Gate types:

Gate	Trigger	Example fail
Schema	Every PR	Parse rate <99%
Golden primary	Every PR	Faithfulness <90%
Golden full	Nightly	Citation <88%
Latency smoke	Every PR	p95 >6s
Cost estimate	Release branch	>20% over budget
Red team subset	Weekly	Any Sev1 unmitigated

Promotion flow: Dev → staging with full golden → canary online → full rollout if online within bounds.

Artifact retention: Store eval reports per release for audit (model version, dataset hash, scores).

Online evaluation — production truth

Offline ≠ online. Users behave differently; corpora drift; load affects latency.

Online methods:

• Explicit feedback — thumbs, reason codes (biased toward angry users—weigh carefully)
• Implicit signals — escalation, edit distance before send, time-to-task, abandonment
• Shadow mode — new model runs parallel without user impact; compare metrics
• A/B test — controlled rollout with statistical power plan
• Human review sample — continuous 1–5% stratified by intent
• Drift detection — embedding distribution shift, zero-citation rate spike

Online/offline linkage: Each primary offline metric should have online proxy (e.g. offline faithfulness ↔ human review fail rate on sample).

Rollback triggers: Automated if online proxy breaches for N hours (e.g. citation missing >8%).

Risk-based bars — not one threshold for all

Risk tiers (illustrative):

Tier	Example use	Illustrative gates
Low	Internal brainstorm	Schema valid; safety smoke
Medium	Internal policy Q&A	Faithfulness ≥90%; citation ≥90%; ACL 100%
High	Customer servicing influence	Above + human sample ≥95% accept on critical intents
Critical	Credit/medical/legal Decide	Human-only decision; AI read-only eval for leakage

Document residual risk when bar not met—explicit accept or defer, never vague “monitor.”

Agent and workflow evaluation specifics

Beyond text quality:

• Task completion rate — did scenario achieve goal state?
• Tool sequence correctness — optimal or acceptable path?
• Budget compliance — steps/tokens within cap?
• HITL compliance — no bypass of gates in test harness?

Use simulation environment with mocked tools recording calls—never prod write in eval.

Frameworks and methods

EVAL-MATCH — choose methods per feature

Step	Question
Exposure	Who sees output? Customer or internal?
Value of error	Financial/regulatory cost?
Automation level	HITL or auto-send?
Lifecycle	How often corpus/process changes?
Metrics	Which 3 primary dimensions?
Artifacts	Golden, judge, human, red team?
Thresholds	Numeric gates per tier?
Cadence	CI, weekly, quarterly?
Handoff	Online monitors and rollback?

Release gate table template

Metric	Threshold	Dataset	Owner	Last run	Block release?
Faithfulness	≥90%	GOLD-v3	ASE lead	91.2%	Y
Citation correctness	≥90%	GOLD-v3	Risk	88.1%	Y
ACL forbidden pass	100%	ACL-40	Security	100%	Y
p95 latency	≤5.5s	LAT-smoke	Eng	5.1s	Y

Publish table in steering packs.

Golden case ID schema

GOLD-{domain}-{bucket}-{seq} e.g. GOLD-MOTOR-NEG-014

Tags: risk, bucket, added_version, source (prod_fail, sme, redteam)

Judge rubric snippet (faithfulness)

Score 1 if any factual claim is unsupported or contradicts CONTEXT.
Score 3 if mostly supported but one minor unsupported detail.
Score 5 if all factual claims clearly supported by CONTEXT spans.
Ignore style. Do not use outside knowledge.
CONTEXT: {{retrieved}}
ANSWER: {{output}}

Version rubric; store with harness.

Online/offline alignment worksheet

Offline metric	Online proxy	Alert threshold
Faithfulness	Human sample fail %	>12% weekly
Citation correctness	Zero-citation rate	>8% daily
Refusal appropriateness	Escalation on negative class	+5 pts vs baseline

Real-world scenarios

Scenario A — UK insurer: launch blocked at 71% citation correctness

Context: Customer servicing policy Q&A; leadership wanted launch before quarter-end. Offline eval: faithfulness 89%, citation correctness 71% (LLM-judge + human sample n=80). Gate: 90% both for Tier High customer influence.

Decision: No launch. Root cause: model paraphrased without attaching correct source_id; post-gen validator not deployed.

Intervention (4 weeks):

1. Structured output with required citations array
2. Deterministic verifier: each citation must contain claim substring or NLI pass
3. Golden set expanded +40 citation-edge cases from SME
4. Human calibration: judge aligned to human (ρ 0.83)

Measurable outcomes:

• Citation correctness 71% → 94%; faithfulness 89% → 92%
• Launch approved; 90-day production human sample fail 6.2% (within model)
• Customer complaint rate on policy wording no spike vs baseline
• Steering documented residual risk accept on 6% sample fail with HITL on compensation intents

Lesson: Faithfulness without citation check is insufficient for regulated Q&A.

Scenario B — Retail bank: regression gate catches model upgrade regression

Context: Team upgraded foundation model for cost savings. Nightly golden: task completion 86% → 79%; JSON parse 99.2% → 96.1%; refusal on negative set 94% → 81%.

Gate: Block release; auto-alert to ML lead.

Intervention: Prompt v2.3 adjusted shots; schema strict mode; partial rollback to prior model for agent planner only.

Measurable outcomes:

• Post-fix task completion 85% (within 1 pt tolerance)
• Parse 99.5%; refusal 95%
• Avoided estimated £1.2m incident cost (historical servicing error run-rate extrapolation)
• Model upgrade saved 18% inference cost at final approved config

Lesson: CI regression gates pay for themselves on first caught upgrade.

Scenario C — NHS trust: human review loop for clinical admin

Context: Clinical pathway summarisation; cannot rely on automation alone. Eval design: offline golden 120 cases; 100% human SME review on pilot; production 10% stratified sample + 100% low-OCR-confidence flag.

Metrics: Faithfulness, omission of exclusion criteria, citation to verified pathway only.

Measurable outcomes:

• Pilot human agreement κ 0.78 on faithfulness
• Production omission incidents 0 in 6 months (governance audit)
• Nurse reviewer time 8 min/case average; funded in run-rate
• Online proxy: edit distance on nurse edits ↓ 34% over pilot (quality improving)

Lesson: High-risk eval centres humans in loop with measurable sampling discipline.

Scenario D — E-commerce: online A/B vs offline mismatch

Context: Product description generator; offline BLEU/ judge scores +5% on new prompt. Online A/B: conversion −0.3%, return rate +0.2% (statistically significant at 2 weeks).

Analysis: Offline set didn’t cover size/fit ambiguity complaints; model added confident specs not in source attributes.

Intervention: Add NEG-FIT bucket to golden; gate on attribute grounding check; rollback winning offline variant.

Measurable outcomes:

• Offline attribute grounding 76% → 93% after golden expansion
• Second A/B: conversion neutral; returns normalised
• Documented offline/online gap in eval playbook for merchandising team

Lesson: Online metrics decide business success; offline must evolve from online failures.

Practice exercises

Primary exercise — Release gate table (60 minutes)

Brief: Internal HR policy Q&A RAG assistant; medium-high risk; SSO; 5,000 employees; HITL on compensation-related answers.

Tasks:

1. Classify risk tier with rationale.
2. Choose 3 primary metrics and thresholds.
3. Write release gate table (≥6 rows including latency and ACL).
4. Design golden coverage matrix with bucket percentages.
5. Specify LLM judge use with calibration plan and known limits.
6. Define one online proxy and rollback trigger per primary metric.

Acceptance criteria:

• Thresholds numerically stated per tier
• ACL gate is 100% block release
• Human review role defined for compensation intents
• Judge not sole gate for high-risk dimensions

Stretch exercise — Full eval strategy document (full day)

Brief: Customer-facing mobile tariff FAQ RAG (telecom); GDPR; marketing conduct; target 200k queries/month.

Tasks:

1. EVAL-MATCH worksheet completed.
2. Golden set outline 200 cases with ID schema and sourcing plan.
3. CI pipeline stages diagram (PR vs nightly vs pre-release).
4. Red-team 25 cases across injection and ACL.
5. Human review ops model (sample rate, rubric, kappa target).
6. Online shadow deployment plan for two weeks before A/B.
7. Steering-ready one-page residual risk statement if any gate at 88% vs 90% target.

Acceptance criteria:

• FinOps cost-of-eval estimated (judge + human hours)
• Online/offline alignment table complete
• Rollback automation described—not manual “we’ll watch”

Questions you should be able to answer

1. What risk tier is this feature and who classified it?
2. What three primary metrics gate release?
3. What does the golden set contain—size, buckets, version?
4. How were labels sourced and when last refreshed?
5. What automated checks run on every PR?
6. Where is LLM-as-judge used and how was it calibrated?
7. What are judge limits for this domain?
8. What human review happens before and after launch?
9. What red-team coverage exists and cadence?
10. What regression delta fails a merge?
11. What online proxies track each offline metric?
12. What rollback trigger fires automatically?
13. How do you evaluate agents differently from single-shot RAG?
14. What happens if score is 88% vs 90% gate—ship, HITL, or defer?
15. Where are eval reports stored for audit?

Negative cases — when evaluation fails

Failure mode	Symptom	Prevention
Demo UAT	20 manual chats	Golden set + CI
Vanity metrics	High BLEU, wrong business outcome	Task + online A/B
Judge-only	False confidence	Human calibration
Stale golden	Policy change undetected	Owner + refresh SLA
Happy path only	Production surprises	Negative/adversarial buckets
No ACL tests	Retrieval leak	Forbidden suite 100% gate
Metric soup	No gate	EVAL-MATCH 3 primaries
Offline worship	Online harm	Online proxies + rollback
Unversioned datasets	Irreproducible scores	Git tag datasets
Post-hoc eval	Launch then measure	Gates before prod

Case study: the launch memo. Team cited “internal testing positive” without numbers. Week 2: citation errors drive +15% escalations. Retro: no golden set; no citation metric. Rebuild eval; second launch after 94% citation gate.

Case study: judge loved hallucinations. LLM judge scored verbose answers high; humans scored low. Correlation 0.52. Judge demoted to triage; human anchor 100 cases per release.

Case study: golden set leakage. Test queries copied from training examples in fine-tune. Inflated scores. Held-out production sample revealed gap. Strict dedup and prod-sourced labels.

Operating-model notes

Role	Responsibility
Eval lead (ASE)	Strategy, gates, golden ownership
Domain SME	Label quality, rubric authorship
ML engineer	Harness, CI integration
Risk / compliance	Risk tier, audit retention
Product	Online metrics, A/B design
Ops	Rollback execution on alert

Meeting cadence:

• Weekly eval standup during build: failing cases review, golden additions
• Pre-release gate review with risk sign-off for Tier High+
• Monthly production eval sample readout to steering

Dataset governance and versioning

Golden datasets are artefacts under change control:

• Store in git LFS or dedicated dataset registry with semver
• Changelog entry: cases added/removed, reason, labeler
• Prohibit direct edit without PR; CI runs diff summary (case count per bucket)
• Retain n−3 dataset versions for reproducibility audits

PII in golden sets: Synthetic or redacted only; if real prod samples used, access restricted and DPIA updated.

Statistical rigour for online experiments

A/B tests need power analysis before launch:

• Define MDE (minimum detectable effect) on primary business metric
• Estimate variance from pilot or analogous feature
• Run duration calculator—avoid stopping early on noise

Guardrails metrics (auto-stop if breached): safety rate, escalation rate, latency p99.

Document peeking policy—who can view interim results and when early stop allowed.

Fairness and subgroup evaluation

Where decisions affect people (HR, credit, marketing, pricing):

• Stratify golden and online sample by protected attributes where lawful and available
• Report parity gaps on primary metrics with confidence intervals
• Escalate if gap exceeds policy threshold—even if aggregate passes

Fairness eval is not one-off; re-run on model upgrade and demographic traffic shift.

Safety and toxicity layers

Combine:

• Blocklist / classifier pre-output
• LLM judge on safety rubric
• Human red team on jailbreak suite
• Online reporting workflow

Track false refusal rate on benign queries—safety tuning that blocks legitimate use erodes adoption.

Eval cost management

Method	Cost driver	Control
Full golden LLM judge	Cases × judge tokens	Nightly not every PR
Human label	SME time	Sample + pre-label assist
E2E staging	API call volume	Parallelise with cap
Prod shadow	Double inference	1–5% sample

Publish eval opex alongside inference opex—prevents eval cut when budgets tighten wrongly.

Scenario E — Regulator exam: reproducible eval evidence pack

Context: UK bank FCA skilled persons review requested evidence that GenAI servicing assistant was tested before launch.

Evidence pack assembled:

• Risk tier memo (Tier High)
• GOLD-v4 manifest with hash; 240 cases bucketed
• CI reports for release tag v2.1.0 showing all gates pass
• Human sample protocol and κ scores
• Red-team summary (42 cases, 0 Sev1 open)
• Online monitoring definition linking to offline metrics
• Change log since launch with re-eval results per change

Outcome: No findings on testing methodology; 3 minor observations on label refresh SLA—remediated with quarterly corpus sync trigger.

Lesson: Eval artefacts must be exam-ready at release time—not reconstructed under exam stress.

Scenario F — Start-up acquisition: inheriting zero eval

Context: Enterprise acquired fintech with customer chatbot; no golden set; incident history of wrong fee quotes.

90-day eval remediation:

• Week 1–4: Forensic sample 500 prod transcripts; label failures; build GOLD-v1 180 cases
• Week 5–8: CI gates on schema + citation; block new features until baseline faithfulness measured (62%)
• Week 9–12: Remediation sprint; faithfulness 89%; HITL on fee-related intents until 93%

Measurable outcomes:

• Fee misquote incidents 7/month → 0 by month 5 post remediation
• Eval harness cost £45k one-off; avoided estimated £2.1m conduct exposure (internal model)
• Acquired product integrated into group release gate template

Lesson: Acquisitions should eval-diligence like code diligence—missing harness is technical debt with regulatory interest.

Connecting eval to incident management

When production incident traced to AI output:

1. Add incident transcript to golden as source=prod_incident
2. Re-run harness—confirm fail on current prod version
3. Fix + verify pass
4. Post-incident: expand bucket coverage (e.g. add FEE-QUOTE class)

Close loop in eval changelog referenced in incident ticket.

Continuous improvement loop

Production signal → Triage → New golden case → Harness fail → Fix → Gate pass → Deploy → Monitor

Weekly triage of:

• Top 10 thumbs-down with reason codes
• Escalations tagged “AI wrong”
• Parse/citation monitor breaches

SME time-box 2 hours/week during stabilisation phase—prevents golden rot.

Eval dimensions reference card (quick lookup)

Feature type	Primary metrics	Secondary
RAG Q&A	Faithfulness, citation, refusal	Latency, cost
Extraction	Field F1, schema pass	—
Classification	Accuracy, calibration	Fairness subgroups
Summarisation	Faithfulness, omission rate	Length compliance
Agent	Task completion, tool accuracy	Budget compliance
Generation (marketing)	Grounding, brand rubric	Conversion (online)

Pin reference card in team wiki; link from CI report headers.

Third-party and vendor model eval

When using vendor-managed models or RAG:

• Contractual SLA on latency; not on your domain accuracy—you own golden eval
• Request change notification for model updates → automatic re-run trigger
• Shadow new vendor version minimum 48 hours before cutover

Never accept vendor benchmark on their dataset as your gate.

Documentation for auditors — minimum eval dossier

1. Risk classification memo
2. Golden dataset manifest and methodology
3. Release gate table with signed results
4. Sample human review records (redacted)
5. Red-team scope and outcomes
6. Online monitoring plan and sample dashboard
7. Change management: eval re-run evidence for last 3 releases

Store under document control ID; map to VALUE gate evidence list.

Pairwise comparison eval for model selection

When choosing between prompt variants or models:

• Run pairwise human or judge comparison on same inputs
• Swap A/B order to reduce position bias
• Require statistical significance on win rate before declaring winner (e.g. 95% CI excluding 50%)

Avoid cherry-picking 10 wins from 100 ties—report full matrix.

Regression triage workflow

When CI fails:

1. Identify failing case IDs—cluster by bucket (injection vs ACL vs faithfulness)
2. Diff prompt/retrieval/model vs last green build
3. If >5 cases same bucket, likely systematic; if scattered, maybe flake—rerun once
4. Assign owner: prompt vs retrieval vs model
5. Block merge until green or explicit risk acceptance with HITL compensating control

Document in PR template “eval delta explanation” field mandatory on failure override.

Stretch metrics for mature programmes

After primary gates stable, add:

• Calibration — confidence scores match empirical accuracy
• Edit burden — human edits per output character
• Time-to-answer conditional on correctness
• Customer effort score correlation

Mature eval connects quality to experience and economics, not only pass/fail.

Read-aloud checklist before AI feature go-live

1. Risk tier documented and signed
2. Release gate table attached with all blockers green
3. Golden dataset version hash in release notes
4. Human review path live for required intents
5. Online monitors and rollback triggers configured in prod
6. Red-team Sev1 findings closed or accepted with compensating control
7. Eval dossier stored in document control
8. Support team briefed on known failure modes and escalation

Steering receives one-page summary; engineering attaches full harness report.

Playbook cross-links for delivery

Implementation patterns for eval harnesses, dashboards, LLM-as-judge setup and production monitoring are in Evaluation and observability. Use this Learning Map page to assess engineer competency; use the playbook to build the actual CI jobs and observability stack on engagements.

Summary — eval method selection by risk tier

Tier	Offline golden	LLM judge	Human review	Red team	Online
Low	Smoke 30	Optional	Spot	Annual	Basic feedback
Medium	100–300	Calibrated	Pre-launch + 2% sample	Quarterly	Proxy metrics
High	200–500	Calibrated, not sole gate	Pre-launch + 5–10%	Each release	A/B or shadow
Critical	Read-only scope	Minimal	Continuous SME	Continuous	Real-time alerts

Adjust counts upward for customer-facing scale and regulatory scrutiny—table is minimum, not maximum.

Eval anti-patterns quick reference

Avoid: demo UAT as gate; judge without calibration; golden set never updated; online launch without rollback; single metric obsession while citation fails; eval cut first when budgets tighten on Tier High features.

Related playbook content

• Production-Grade Evaluation Strategies for LLMs, RAG, Agents, and Multi-Agent Systems — layered eval, RAG/agents, release gates and tooling
• Evaluation and observability — harness patterns, dashboards and runbooks
• RAG — retrieval-specific metrics
• Agentic AI — task completion eval
• Prompt and Context Engineering — prompt regression testing
• Retrieval-Augmented Generation — faithfulness and ACL eval
• Agentic AI and Workflow Automation — agent budgets and tool eval
• Responsible AI and Governance — risk classification and red team
• MLOps, LLMOps and Observability — deployment and monitoring integration
• Performance and FinOps — cost-per-success metrics
• How to use this Learning Map — reference depth standards
• VALUE gate — evidence bar before scale investment
• 8D Framework — Demonstrate stage eval evidence

Output

Expected outputs for this topic:

• Eval strategy — EVAL-MATCH worksheet with methods per lifecycle stage
• Golden dataset outline — buckets, IDs, size, sourcing and refresh owner
• Release quality gate table — metrics, thresholds, owners, block-release flags
• Judge calibration report — human correlation and rubric version (if used)
• Human review operating model — sample rates, rubric, agreement targets
• CI/CD regression spec — PR vs nightly jobs and fail conditions
• Online/offline alignment — proxies, alerts and rollback triggers

Practice checklist

• I classified risk tier and chose three primary metrics with numeric thresholds
• I designed golden set coverage including negative, ACL and adversarial buckets
• I specified automated PR gates separate from nightly full runs
• I documented LLM-as-judge limits and calibration—or avoided judge for sole gate
• I defined human review before launch and ongoing sampling
• I linked offline metrics to online proxies with rollback triggers
• I completed release gate table suitable for steering review
• I filed eval artefacts with dataset version and harness hash

Supplemental scenario — banking FAQ release gate regression

Context: Retail bank FAQ RAG; tier 2 risk; monthly model provider updates; steering requires ≥95% golden pass before release.

Release gate table (excerpt):

Metric	Threshold	PR gate	Nightly	Block release?
Golden pass rate	≥95%	Smoke 50 cases	Full 420	Yes
Citation valid	≥92%	Yes	Yes	Yes
IST injection block	≥95%	10 cases	80 cases	Yes
Latency p95	≤5.5s	No	Yes	Warn only

Regression caught pre-release: Prompt v4.3 dropped citation field on 7% of German queries—PR gate failed at 93.1% pass; rollback to v4.2; root cause static English few-shot overflowed pack for DE locale.

Human review: 5% random sample + 100% of payment-adjacent intents; inter-rater agreement 0.86 on rubric v2.

Online proxy: citation_missing_rate alert if >4% over 1h → auto-canary rollback.

Key takeaways

• Evaluation matches risk—one size fits none; marketing ≠ credit ≠ clinical.
• Golden sets must be stratified, versioned and maintained—not static happy paths.
• LLM-as-judge scales triage but requires human calibration and known bias limits.
• Regression gates in CI catch model, prompt and retrieval regressions before users do.
• Human review remains essential for high-risk and for calibrating automation.
• Online eval validates business outcomes; offline gates must link to production signals and rollback.

What to do next

1. Complete the release gate table exercise for the HR policy assistant brief.
2. Continue to Performance and FinOps for cost-per-success and latency SLOs.
3. Open Evaluation and observability playbook to implement harnesses.
4. Revisit Retrieval-Augmented Generation to align retrieval eval with your gates.