AI and data foundations

Executive summary

Stage 2 is where investment rationale from Stage 1 meets technical reality. Before you design cloud landing zones or negotiate vendor contracts, you must answer: Is the data usable? Is the problem a prediction, retrieval, or generation task? Which model class fits cost, latency, privacy, and quality constraints? Can we evaluate success before scaling? Completing this stage means you can gate pilots on data readiness, choose classical ML versus LLM paths with evidence, and produce evaluation plans—not slide decks that assume perfect corpora and frontier models.

Outcome

Primary outcome: You can determine whether a prioritised use case is technically feasible—with explicit judgement on data readiness, problem type, model approach, retrieval or agent needs, and evaluation discipline.

Secondary outcomes:

• Produce Stage 2 core deliverables (data inventory, readiness score, problem-type choice, model selection matrix, eval plan) without vendor-first framing.
• Stop or defer pilots when data, metrics, or evaluation design would produce false confidence.
• Explain trade-offs to executives in business language (freshness, recall, latency, residency)—not only architecture diagrams.

Stage 2 does not require production deployment. It requires you to know what would break in production before spend accelerates.

Who Stage 2 is for

Profile	Typical gap	Stage 2 fix
Strong Stage 1, weak data	Great value case; no corpus audit	Topics 09, 13
ML engineer, weak GenAI eval	Strong training pipelines; no RAG/agent QA	Topics 11–14, 21
GenAI enthusiast, weak ML	Defaults to LLM for tabular prediction	Topics 10, 11
Architect jumping ahead	VPC design before data inventory	Topic 09 first; Stage 3 after exit

Entry criteria

Start Stage 2 when all of the following are true:

• Stage 1 exit criteria met—or documented sponsor risk acceptance for missing value hypothesis (logged, rare).
• At least one prioritised use case with owner, baseline metric, and scope boundary from topic 05 backlog.
• You can name the primary data sources (even if unvalidated) and decision type the AI must support.

Do not start Stage 2 when:

• Discovery produced only a demo script with no scored opportunity.
• No one owns data access requests (IT ticket black hole likely).
• Success criteria are unfalsifiable ("better employee experience").

Minimum entry artefact: one-page use case brief from Stage 1—problem, metric, population, constraints, exclusions.

Exit criteria

Declare Stage 2 complete only when all exit checks pass:

Artefacts (minimum set)

Artefact	Source topic	Pass test
Data-source inventory + readiness score	09	Freshness, quality, ownership, sensitivity scored; go/no-go rationale
Lineage sketch for pilot path	09	Ingest → store → feature/RAG → consumer documented
Problem-type and metric definition	10	Primary metric matches business cost of errors; leakage risks named
Model selection matrix	11	Weighted criteria; residency and cost explicit
Prompt/context strategy note (draft)	12	System role, guardrails, output shape for pilot
RAG or retrieval feasibility note	13	Chunk strategy, ACL model, freshness SLA—or "N/A" with reason
Agent scope boundary (if applicable)	14	Human-in-the-loop points; tool allow-list
Evaluation plan outline	21	Offline + online metrics; red-team scope; acceptance thresholds

Knowledge checks

• Answer ≥10/12 randomly chosen questions from topics 09, 10, and 11 without notes.
• Explain one negative case from Stage 2 (data theatre, wrong problem type, eval gap) with client-relevant example.
• Defend classical ML vs LLM choice for two different use cases in 60 seconds each.

Behavioural checks

• You have gated at least one fictional or real pilot on data readiness—not only model choice.
• A peer reviewer confirms artefacts are filed in pattern library with sector tag.

Exit is not "I watched a transformer explainer." Exit is artefacts reviewed.

Capability map

Stage 2 spans seven numbered topics (09–14, 21). Topics 12–14 and 21 extend GenAI delivery; topic 09–11 are foundations you cannot skip.

Topic 09 — Data Engineering and Data Architecture

Capability: Judge data readiness for AI—stores, pipelines, quality, lineage, feature layers, unstructured ingest; gate RAG and ML on freshness and ownership.

You know it when: You produce a readiness score that blocks a pilot when corpus is stale or ownerless—and sponsors accept the gate.

Core deliverables: Data-source inventory, quality assessment, readiness score, lineage sketch.

Deep link: Data Engineering and Data Architecture

Topic 10 — Machine Learning Foundations

Capability: Classify problem types, choose metrics aligned to error costs, recognise overfitting and leakage, and decide classical ML vs LLM without defaulting to GenAI.

You know it when: You refuse accuracy as headline metric for imbalanced fraud detection and specify recall at precision floor with eval design.

Core deliverables: Problem-type choice, metric definition, eval plan outline.

Deep link: Machine Learning Foundations

Topic 11 — Generative AI and LLM Fundamentals

Capability: Explain transformers, tokens, context, fine-tuning options; select models on quality, latency, cost, privacy, residency, tools, multimodal trade-offs.

You know it when: You route PII-heavy servicing to regional deployment with masking—and document when frontier model is worth the risk.

Core deliverables: Model selection matrix, deployment pattern choice.

Deep link: Generative AI and LLM Fundamentals

Topic 12 — Prompt and Context Engineering

Capability: Design system prompts, few-shot structure, output schemas, and context budgets for reliable pilot behaviour.

You know it when: Same model behaves predictably across eval set after prompt/version control—not ad hoc chat tweaks.

Deep link: Prompt and Context Engineering

Topic 13 — Retrieval-Augmented Generation

Capability: Architect chunking, embedding, retrieval, ACL-aware search, and freshness for grounded answers.

You know it when: You can explain why hybrid search beats naive vector-only for policy numbers and named entities.

Deep link: Retrieval-Augmented Generation · RAG Architecture · Graph RAG Architecture

Topic 14 — Agentic AI and Workflow Automation

Capability: Bound agent scope, tool use, human approval points, and failure recovery—avoid unbounded autonomy in regulated flows.

You know it when: Agent design doc lists forbidden actions and escalation paths before build.

Deep link: Agentic AI and Workflow Automation

Topic 21 — AI Evaluation and Quality Assurance

Capability: Define offline and online eval, red-teaming, regression suites, and acceptance thresholds that gate scale.

You know it when: Steering committee sees pass/fail criteria before pilot launch—not after complaints arrive.

Deep link: AI Evaluation and Quality Assurance

Topics in this stage (quick index)

9. Data Engineering and Data Architecture
10. Machine Learning Foundations
11. Generative AI and LLM Fundamentals
12. Prompt and Context Engineering
13. Retrieval-Augmented Generation — companions: RAG Architecture, Graph RAG Architecture
14. Agentic AI and Workflow Automation
15. AI Evaluation and Quality Assurance

Recommended practice sequence

Sequence A — Standard (5 weeks, ~8–10 hours/week)

Week	Focus	Outputs due
1	Topic 09 + Data & Knowledge Guide skim	Data inventory + readiness score for practice use case
2	Topic 10	Problem-type memo + metric definition + leakage register
3	Topic 11 + Model landscape review	Model selection matrix (2 options, weighted)
4	Topics 12–13	Prompt draft + RAG feasibility (chunk/ACL/freshness)
5	Topics 14–21	Agent boundary (if needed) + eval plan with thresholds

Sequence B — Live engagement crash (2 weeks)

Use when staffed mid-pilot with weak feasibility artefacts:

1. Days 1–2: Topic 09 corpus audit and go/no-go score.
2. Days 3–4: Topics 10–11 problem type and model path decision.
3. Week 2: Topics 13 + 21 retrieval eval and acceptance criteria.
4. Parallel: Block architecture (Stage 3) until readiness and eval exist in writing.

Rule: No production commitment until eval plan has named thresholds and data owner sign-off.

Sequence C — ML-heavy vs GenAI-heavy paths

ML-heavy (fraud, churn, forecasting): Week 1 topic 09 (structured data), week 2 topic 10 deep dive, week 3 topic 21 eval; topics 11–14 skim for integration points only.

GenAI-heavy (knowledge assistant, drafting): Week 1 topic 09 (unstructured ingest), week 2 topics 11–13, week 3 topics 12 + 21; topic 10 for "when not to use LLM" guardrails.

Integration with Guide and 8D

Stage 2 capability	Guide / reference
Data readiness, pipelines, corpus	Data & Knowledge
ML vs GenAI patterns	AI Patterns
Model catalog and trade-offs	Model landscape
Methodology spine	8D Framework — Design, Develop
Quality bar before scale	VALUE gate (feasibility evidence)

Deep-link rule: Learning Map teaches judgement; Guide walks delivery steps. Link and practise artefacts—do not duplicate full Guide chapters in notes.

Common failure modes

Failure mode	How it shows up	Fix
Skip data readiness	RAG on SharePoint dump with stale PDFs	Topic 09 score; gate on freshness and owner
LLM for everything	Tabular churn prediction via prompt	Topic 10 problem-type decision tree
Metric tourism	Accuracy on 1% fraud rate	Business-weighted metrics; precision/recall trade-off
Eval after launch	"We'll measure CSAT later"	Topic 21 thresholds before pilot
Context window fantasy	Entire policy corpus in prompt	Topic 11 + 13 retrieval architecture
Agent autonomy creep	Tool use without approval on money movement	Topic 14 scope boundary
Vendor benchmark worship	Public leaderboard ≠ your corpus	Custom eval set from topic 21
Lineage blind spot	Cannot explain wrong answer source	Topic 09 lineage sketch
Privacy afterthought	PII in logs and prompts	Topic 11 residency + masking pattern
Stage 3 before Stage 2	Landing zone design before data inventory	Block architecture review

Stage 2 anti-pattern: the confident pilot

A team launches a GenAI assistant in six weeks. Demo works on three golden questions. Production sees 2,400 weekly users; 34% of sampled answers cite deprecated policy; compliance escalates; programme paused at £280K spend with no scale criteria.

Stage 2 antidote: Readiness score + eval plan + acceptance thresholds signed before user-facing launch. Pilot on bounded intent set with weekly regression—not open-ended chat.

Stage 2 anti-pattern: leakage luxury

Offline model shows 94% AUC on credit default prediction. Production performs at 71% within one quarter. Post-mortem: training data included post-default behaviour labels. Team trusted notebook metrics without topic 10 leakage review.

Stage 2 antidote: Temporal split, feature audit, and pre-registration of eval protocol in topic 21.

Manager review rubric (Stage 2 gate)

Dimension	Red	Amber	Green
Data readiness	No inventory	Inventory without scores	Scored; go/no-go documented
Problem type	"AI will figure it out"	Type named, metric weak	Type + metric + leakage risks
Model choice	Single vendor default	Two options, no weights	Weighted matrix + residency
RAG/agents	Scope unbounded	Scope draft	ACL, freshness, HITL explicit
Evaluation	Post-hoc CSAT only	Metrics listed, no thresholds	Offline + online + red-team plan
Classical vs LLM	Always LLM	Ad hoc choice	Documented decision tree

Green on ≥5/6 dimensions → recommend Stage 3 start for that workstream.

Competency proof

When claiming Stage 2 complete:

1. Submit artefact pack (table above) to peer or manager.
2. 45-minute challenge session: data gates, metrics, model trade-offs, negative cases.
3. Optional: Competency test prompts for Stage 2.
4. File anonymised artefacts in team pattern library.

What comes next

Stage 3 asks: How do we host, integrate, and secure this solution in the enterprise? Do not advance until Stage 2 exit criteria are met—or document explicit risk acceptance on data and eval gaps (logged with sponsor).

Output

Stage 2 portfolio deliverables (engagement pack minimum):

• Data-source inventory and readiness score (topic 09)
• Lineage sketch for pilot data path (topic 09)
• Problem-type choice and metric definition (topic 10)
• Model selection matrix (topic 11)
• Prompt/context strategy draft (topic 12)
• RAG feasibility note or N/A rationale (topic 13)
• Agent scope boundary or N/A rationale (topic 14)
• Evaluation plan with acceptance thresholds (topic 21)

Stage 2 completion checklist

• Data readiness scored; at least one gate criterion enforced
• Problem type and primary metric documented with business error costs
• Leakage and overfitting risks named for prediction use cases
• Model selection matrix includes latency, cost, privacy, residency
• RAG/agent scope bounded with ACL and human approval points
• Eval plan exists before pilot launch—not after
• I can explain when not to use an LLM for a given problem
• Artefacts filed in pattern library; peer review completed

Key takeaways

• Stage 2 outcome is feasibility with evidence, not model enthusiasm—data and eval gate spend.
• Topics 09–11 are non-negotiable foundations; 12–14 and 21 operationalise GenAI safely.
• Classical ML often beats LLM on structured prediction—defaulting to GenAI is a failure mode.
• Readiness scores and eval thresholds are negotiation tools with sponsors—not bureaucracy.
• Common failure is confident pilot without corpus governance; antidote is Stage 2 artefacts before scale.
• Stage 3 architecture without Stage 2 invites expensive rework when data and metrics collapse.

What to do next

1. If new to the map, read How to use this Learning Map for study method (90–180 min per topic).
2. Begin with Data Engineering and Data Architecture—no model work until data is scored.
3. Pair topic 09 with the Data & Knowledge Guide for delivery patterns.
4. When exit criteria pass, continue to Architecture and cloud.