RAG Engineering

Guide · Enterprise AI Solution EngineeringPage 7 of 18Overview → … → RAG

Executive summary

RAG lets models answer from approved enterprise knowledge. Design the full pipeline—from ingestion and chunking through hybrid retrieval, security trimming, citations and evaluation—so answers are grounded, access-controlled and measurable.

When to use this page

• Use when the agreed pattern is retrieval-augmented QA or grounded generation.
• Use after Data and knowledge readiness and metadata are clear.
• Prefer simpler search or extraction if generation is not required.

Executive view

Fund corpus governance, access control and evaluation—not only embeddings and a chat UI.

Decision required: Which approved sources may be retrieved, and what is the refuse threshold?

Technical view

Own pipeline stages end to end; preserve structure and access metadata in chunks; verify security trimming with negative tests.

Evaluate retrieval and citation accuracy, not generation quality alone; include no-answer cases in the golden set.

Owners

Owner · AI Solution Engineer / RAG lead

Contributors · Knowledge owners, data engineering, security, evaluation owner, AI engineering

What RAG solves

Retrieval-augmented generation allows a model to answer using approved enterprise knowledge.

A RAG pipeline typically includes:

1. Ingestion
2. Parsing
3. Cleaning
4. Chunking
5. Metadata enrichment
6. Embedding
7. Indexing
8. Query transformation
9. Retrieval
10. Reranking
11. Context assembly
12. Generation
13. Citation
14. Evaluation

Ingestion

Support PDFs, Office documents, web pages, wikis, databases, APIs, SharePoint, file stores and knowledge platforms. Prefer systems of record with owners over unmanaged drives.

Chunking

Chunking affects retrieval quality. Strategies include fixed size, semantic, section-based, paragraph-based, document-aware, table-aware and parent-child.

Avoid splitting content without preserving heading, source, page, section, version and access classification.

Retrieval strategy

Possible approaches: keyword search, dense vector search, hybrid search, metadata filtering, knowledge graph retrieval, multi-query retrieval and query decomposition.

Hybrid search plus metadata filters is a strong default for enterprise corpora with overlapping terminology.

Architecture depth

Use the Learning Map companions when the engagement needs system design beyond this playbook checklist:

• RAG Architecture — index topologies, intent routing, iterative multi-hop, context assembly contracts, latency budgets and simple-vs-deep decision matrices.
• Graph RAG Architecture — minimal graph schema, hybrid expand-plus-chunks, curator ops, relationship eval and adopt-or-defer memos.

Decision prompts: Which intent classes fail on hybrid RAG alone? Is GraphRAG justified by a measured recall gap and funded curator hours—or should you deepen vector topology first? What p95 budget and ACL fail-closed path are non-negotiable?

Reranking

Reranking improves ordering when many results appear relevant, queries are ambiguous, or documents share similar terminology.

Access-controlled retrieval

The system must not retrieve content the user cannot access. Security trimming should occur before context reaches the model.

Citations

Every grounded answer should record source document, section, page or paragraph, retrieval score, version and timestamp.

Risk

Answers without citations, untrimmed retrieval, or evaluating only generation quality create trust and compliance failures.

Control

Enforce security trimming before context reaches the model, and require citations plus refuse behaviour when retrieval confidence is low.

RAG evaluation

Measure retrieval precision and recall, context relevance, faithfulness, answer relevance, citation accuracy, unsupported-claim rate and no-answer accuracy.

Case study: financial-services assistant

Ingest only approved product and policy libraries. Chunk by section with product and jurisdiction metadata. Hybrid retrieve, filter by the user’s role and region, rerank, generate with mandatory citations, and refuse when retrieval confidence is low.

Output

End-to-end pipeline design with owned stages, chunking and citation schemas, hybrid retrieval + filter justification, security-trimming test plan, and a golden set including no-answer cases.

Common failure modes

• Dumping unchunked PDFs into embeddings
• No access trimming before generation
• Answers without citations
• Evaluating only generation quality, not retrieval
• Ignoring stale or superseded document versions

Solution Engineer checklist

Solution Engineer checklist

• Pipeline stages owned end to end
• Chunking preserves structure and access metadata
• Hybrid retrieval + filters justified
• Security trimming verified with negative tests
• Citation schema defined
• Golden set includes no-answer cases

Practical exercise

Build a ten-question golden set for your corpus: five answerable, three ambiguous, two that must refuse. Run them after every chunking or retrieval change.

Key takeaways

• RAG quality depends on ingestion, chunking, retrieval and evaluation—not only the model.
• Security trimming must happen before generation.
• Citations and refuse behaviour are product requirements.
• Evaluate retrieval and no-answer cases, not demos alone.

What to do next

1. For system design depth, open RAG Architecture and Graph RAG Architecture.
2. Continue with Agentic AI systems only if tool use beyond retrieval is needed.
3. Otherwise proceed to Security and privacy.
4. Align metrics with Evaluation and observability.