Graph RAG Architecture
Executive view
GraphRAG is a **product and operating-model** commitment: ontology ownership, weekly curator time and eval on relationship queries—not only a new database SKU. Defer if there is no named curator budget.
Decision required: Does relationship-query recall lag vector-only by enough to justify graph build and maintenance cost—and who owns edge quality?
Technical view
Design light schema before heavy ontology; extract with provenance; retrieve by subgraph expansion plus chunk evidence; enforce ACL on nodes and edges; evaluate wrong-edge and missing-edge failure modes.
Route relationship vs lookup queries; keep hybrid vector path for factual lookup. Fail closed when graph confidence is low.
Why this matters
Vector RAG excels at “find passages like this question.” It struggles when the answer is a path: Contract A amends Schedule B which sets TariffBand C under regulator notice D. Embeddings may retrieve each document in isolation and still miss the chain. Teams then either hallucinate the link or return incomplete answers that fail legal review.
GraphRAG’s failure mode is different but equally expensive: wrong edges look authoritative (“applies_to” attached to the superseded amendment), unmaintained ontologies drift from reality, extraction jobs silently drop relations after a parser change, and ACL on documents is not mirrored onto graph nodes—so traversal leaks restricted neighbours.
The AI Solution Engineer’s job is to prove the gap, design a minimal viable graph, wire hybrid retrieval (traversal + chunks), fund curation, and gate release on relationship-query recall and citation correctness—or to defer with a written memo. Weak programmes buy a graph database first. Strong programmes start from eval slices and curator hours.
Connect to RAG Architecture for non-graph multi-hop patterns, Retrieval-Augmented Generation for the baseline contract, and RAG playbook for engagement workshops.
Preview fits the page width — use the expand icon for the full flowchart.
Learn
Graph RAG types and search modes
GraphRAG is not one algorithm. Choose a serve mode from evidence, cost and curator capacity.
Mode ladder: Basic → Local → Global → DRIFT-style → Agentic Graph RAG
Preview fits the page width — use the expand icon for the full flowchart.
Public Microsoft GraphRAG documents Basic / Local / Global / DRIFT; enterprise hybrids add agentic routing.
| Mode | What it retrieves | Best for | Cost / latency |
|---|---|---|---|
| Basic | Standard top-k vector chunks | Simple semantic FAQ inside a graph-enabled platform | Low |
| Local | Entity neighbourhood + linked text chunks | “What connects A to B?”, entity-centric facts | Low–medium |
| Global | Community / theme summaries over the graph | Corpus-wide themes, “what are the main risk clusters?” | High (map-reduce style) |
| DRIFT-style | Iterative mix of global then local refinement | Exploratory multi-hop with follow-ups | Medium–high |
| Hybrid enterprise | Graph expand + vector + BM25 fused | Regulated contracts / BOM / policy cross-refs (this playbook default) | Tuned by router |
| Agentic Graph RAG | Agent selects basic / local / global / tools | Mixed relationship + tool workflows | Highest — needs budgets |
Default for this playbook: hybrid enterprise (router → bounded expand + chunks + ACL). Adopt Global/DRIFT only with labeled theme/exploratory query classes and FinOps approval.
Agentic Graph RAG — when the agent sits on the graph
Flow: agent planner → choose vector / graph mode / tool → critique evidence → re-plan if weak (capped) → cited answer or refuse.
Preview fits the page width — use the expand icon for the full flowchart.
| Pattern | Role of the agent | Guardrails |
|---|---|---|
| Router-only | Classifier picks Local vs Basic — no free loop | Prefer this first |
| Single-agent Graph RAG | Agent may re-expand or switch mode once | Cap rounds; log mode switches |
| Corrective | Agent refuses or re-queries if path incomplete | Path + chunk citation required |
| Multi-agent | Separate planner / expander / critic | Rare; justify vs single-agent |
Do not let an agent unbounded-traverse the graph. Hop caps, node caps, ACL on expand, and curator ownership still apply. Link agent budgets to Agentic AI and Agentic RAG types in 13.1.
Further reading: Microsoft GraphRAG (Local / Global / DRIFT / Basic).
When GraphRAG is justified — and when to defer
Adopt signals (need several, not one slogan):
- Labeled relationship / multi-hop query class exists (≥30–60 queries)
- Vector-only or deep iterative RAG shows recall gap (commonly >10–15 points) on that class
- Source corpus has explicit cross-references (amends, supersedes, references, applies_to, part_of)
- Named ontology/curator capacity (hours/week) funded in the runbook
- Business value of correct chains exceeds build + maintenance cost
Defer signals:
- Flat FAQ or single-document lookup dominates traffic
- No curator; “we will fix edges later”
- Ontology wars (enterprise data model unagreed)
- advanced RAG iterative retrieve-then-read already closes the gap
- Pilot corpus < few hundred docs with rare links
| Question type | Prefer |
|---|---|
| “What is the cooling-off period?” | Hybrid vector RAG |
| “Which live contracts still cite Schedule 7?” | GraphRAG (or structured DB) |
| “Summarise this policy section” | Vector / parent-child |
| “Supplier → component → contract path” | GraphRAG |
| “My account balance” | Tool / API — never graph or RAG |
Graph construction — schema, extraction, versioning
Light schema first. Prefer a small typed graph over a sprawling enterprise ontology.
Example entity types: Contract, Amendment, Schedule, Policy, Notice, Product, Clause, Supplier, Component.
Example edge types: references, amends, supersedes, applies_to, part_of, governed_by.
Construction pipeline: Use the Build lane in the architecture flowchart above for the end-to-end extract → HITL → graph → chunk-link path. Every edge needs provenance; every node used in generation needs a chunk link.
Extraction principles:
- Provenance — every node/edge stores
source_id, locator, extractor version, confidence - Idempotent upserts — stable entity keys (e.g. contract number + jurisdiction)
- Version edges —
supersedes/amended_bywith effective dates; never silently overwrite history - Human-in-the-loop on low-confidence or high-risk edge types (legal, clinical)
- Reject orphans — entities without chunk links cannot be sole evidence for generation
Ontology vs light schema: Start with light schema tied to query classes. Expand types only when eval shows systematic miss. Full enterprise ontology projects often stall GraphRAG for quarters.
Pitfall: LLM extraction without sampling QA. A 5% wrong applies_to rate becomes a 40% wrong-answer rate on multi-hop questions that traverse two edges.
Storage and hybrid retrieval
GraphRAG in enterprise is almost always hybrid:
| Store | Holds | Role |
|---|---|---|
| Graph DB | Nodes, typed edges, properties | Traversal / neighbourhood expand |
| Chunk / vector / BM25 index | Text evidence + embeddings | Passage retrieve and citation spans |
| Metadata catalogue | ACL, tenant, effective dates | Filters on both stores |
Canonical retrieve pattern:
- Route query: relationship vs factual lookup vs refuse/tool
- If relationship: seed entities (NER / linker / user entities) → expand subgraph (bounded depth, e.g. 1–3 hops) with ACL filters on nodes/edges
- Collect linked chunks for nodes on the path
- Optional: vector retrieve for supporting narrative not captured as edges
- Rerank / pack path summary + chunks for generation
- Cite node IDs and chunk locators; verify post-generation
The Serve lane in the architecture flowchart is the canonical path. Keep this sequence for runtime detail:
Preview fits the page width — use the expand icon for the full flowchart.
Query router: Rules (keywords: “which contracts reference”, “superseded by”) + small classifier. Lookup traffic must stay on cheap hybrid vector path—do not expand graphs for “what is VAT on X?”.
Bounds: Max hop depth, max nodes, timeout. Unbounded expansion is a latency and leakage risk.
Generation and citations for multi-hop answers
Citation contract for GraphRAG:
- Every relational claim maps to an edge or path (edge IDs) and ≥1 chunk span supporting that edge when the edge was text-extracted
- Factual elaborations use chunk citations as in standard RAG
- If path incomplete → refuse or partial answer with explicit gap (“no
applies_tolink from Notice Y to Policy Z in graph”)
Faithfulness: LLM-as-judge on prose is insufficient. Add path consistency checks: mentioned entities must appear on retrieved subgraph; claimed relations must match edge types.
UI/API: Show path breadcrumb (Contract → Amendment → Schedule) plus document locators. Auditors replay paths, not only paragraphs.
Operations — curator budget, drift, ACL, re-extract
| Ops concern | Practice |
|---|---|
| Curator | Named role; weekly hours in runbook (e.g. 4h/week) |
| Drift | Diff extract job output; alert on edge-count shocks |
| Re-extract | Versioned jobs; dual-write; offline relationship eval before cutover |
| ACL | Node/edge inherit document ACL; filter at expand time; forbidden traversal suite |
| Tombstones | Retracted docs remove or mark edges unusable within SLA |
| Cost | Graph infra + extract compute + curator — in business case |
Without curator → defer. Topic 13’s rule stands: graphs rot faster than text indexes.
Worked example: Energy retailer funded 4 hours/week ontology curator after relationship recall jumped; without that line item, legal rejected go-live.
Eval specific to graphs
| Metric | What it catches |
|---|---|
| Relationship-query recall | Missing paths / under-extraction |
| Relationship precision | Wrong edges in answers |
| Path citation correctness | Theatre citations |
| Wrong-edge rate | Systematic extract/schema bugs |
| Lookup regression | Graph path accidentally hurting FAQ quality |
| Latency p95 on relationship class | Unbounded expansion |
| Curator hours actual vs plan | Operating-model denial |
| ACL traversal suite | Leak via neighbour nodes |
Golden set design: Stratify relationship queries by hop count (1 / 2 / 3+), by edge type, and include negative paths (“must not connect A to B”). Keep vector-only baseline scores in the decision memo.
Release gate example: Relationship recall ≥85% on 60-query slice; wrong-edge in sampled answers ≤2%; ACL suite 100%; lookup-class faithfulness not degraded >2 pts vs pre-graph baseline.
Decision memo artefact — adopt or defer
Mandatory one-pager before build:
- Problem — query class and business impact
- Baseline — vector / deep RAG metrics on that class
- Gap — numeric delta and sample failures
- Proposed schema — entity/edge types (minimal)
- Ops — curator hours, owners, re-extract cadence
- Cost — build + run vs value
- Gates — metrics that allow production
- Defer criteria — what would make us stop or not start
- Decision — Adopt / Defer / Time-box spike (e.g. 4 weeks)
Aligns with the GraphRAG decision memo exercise in topic 13.
Entity linking and seed resolution
Graph expand fails if seeds are wrong. Seed resolution is its own design surface:
| Step | Practice |
|---|---|
| Detect | NER / gazetteer / user-selected entities in UI |
| Link | Map surface forms to stable entity IDs (contract numbers, policy codes) |
| Disambiguate | Prefer metadata constraints (jurisdiction, product line) before expand |
| Fail soft | If linker confidence < τ → vector path or ask clarifying question — do not expand top-3 fuzzy IDs |
Pitfall: Expanding on the wrong “Schedule 7” across jurisdictions creates precise-looking wrong paths. Prefer refuse/clarify over confident traversal.
Property graphs vs RDF — engagement choice
Most enterprise GraphRAG deliveries use a property graph (nodes/edges with JSON-like properties) because engineering teams already operate that model and latency is predictable. RDF/OWL stacks suit organisations with existing semantic layers and SPARQL skills—but ontology governance cost is higher. Choose based on operating model, not purity:
- Property graph + light schema → default for ASE-led GraphRAG spikes
- RDF → when enterprise already has a governed knowledge graph team and query class maps cleanly to SPARQL
Document the choice in the decision memo; switching mid-flight is a programme reset.
Security model for traversal
Document ACL is necessary but not sufficient.
- Inherit — node ACL = union or strictest of source document ACLs that created it (pick one policy; document it).
- Edge visibility — if either endpoint is invisible to the principal, edge is not traversable.
- Neighbour suppression — do not return neighbour labels the user cannot access (even as “hidden related”).
- Forbidden suite — scripted identities attempt to reach restricted nodes via alternate paths.
- Audit — log seed IDs, expanded node IDs, and filtered-out counts for forensics.
Cost model sketch
| Line item | Notes |
|---|---|
| Graph infrastructure | HA store, backups, environments |
| Extract compute | Batch + incremental; GPU if using heavy models |
| Curator labour | Hours × loaded cost — often dominates |
| Eval maintenance | Relationship suite growth |
| Dual-path serving | Vector stack remains; graph is additive |
If curator + extract > value of relationship-class deflection, defer. GraphRAG is rarely cheaper than better chunking.
Anti-patterns catalogue
| Anti-pattern | Failure | Replace with |
|---|---|---|
| Graph as default index | FAQ latency/cost | Router |
| Unbounded hops | Timeouts / leakage | Depth and node caps |
| Edges without provenance | Unauditable answers | Source locator mandatory |
| Auto-extract email/chat | Noise | System-of-record docs only |
| No lookup regression | FAQ quality drop | Dual-class gates |
| Schema from vendor demo | Unused types | MVG from query class |
| “Curator later” | Rot | Defer or fund now |
Frameworks and methods
Minimal viable graph (MVG)
- Pick one query class
- Define ≤8 entity types and ≤6 edge types
- Extract from a pilot corpus slice
- Human-review 100 edges
- Build hybrid retrieve for that class only
- Compare to deep RAG baseline
- Expand schema only with new failed query themes
Edge quality review checklist
- Stable entity key collision rate acceptable
- Effective dating on supersedes correct on sample
- ACL inherited on nodes from source docs
- Low-confidence edges quarantined
- Chunk links present for generation evidence
Router policy table
| Signal | Path |
|---|---|
| Explicit relation language + known entity types | Graph expand + chunks |
| SKU / FAQ / definition | Hybrid vector only |
| Live operational data | Tools / API |
| Ambiguous | Conservative vector first; optional graph if linker high confidence |
Real-world scenarios
Scenario A — Energy retailer: contract–tariff dependencies
Context: B2B energy; answers needed linking master supply agreements, amendments and time-of-use tariff schedules. Vector-only RAG missed “which contracts still reference deprecated Schedule 7?” Relationship-query recall 41% on 60-query slice.
Intervention:
- Entity graph: Contract, Amendment, Schedule, TariffBand with
supersedes/referencesedges. - Hybrid: graph traversal for relationship queries + chunk retrieval for text evidence.
- Query router: classify relationship vs factual lookup (rules + small classifier).
- Curator 4 hours/week; relationship eval in CI.
Measurable outcomes:
- Relationship-query recall 41% → 87%
- Legal review time for contract audit −38%
- Incorrect tariff citation rate 9% → 2%
- Lookup-class faithfulness held within 1 pt of baseline
Lesson: GraphRAG earns its keep on multi-hop enterprise questions—with maintenance budget.
Scenario B — Pharma: regulatory notice to SOP linkage
Context: Quality organisation must answer “which SOPs are impacted by regulator notice Y?” Vector search returned notice text and random SOPs sharing vocabulary (equipment names) without true impacted_by links. Audit finding: AI assistant unsafe for change-control impact analysis.
Intervention:
- Light schema: Notice, SOP, Clause, ProductFamily; edges
amends,impacted_by,owned_by. - Extraction from controlled change-control packs only (not whole SharePoint).
- Mandatory human confirm on new
impacted_byedges before index. - Generation refuses if path lacks approved edge; UI shows pending edges separately for authors.
Measurable outcomes:
- Impact-analysis precision on golden 38% → 91%
- Mean time to draft impact pack −44%
- Zero critical audit findings on AI-assisted packs in two cycles
- Spike decision: do not auto-extract from email—too noisy
Lesson: For regulated impact graphs, human confirmation on critical edge types is part of the architecture.
Scenario C — Manufacturing: supplier–BOM–contract multi-hop
Context: Discrete manufacturer asks “which active contracts cover suppliers of component X used in SKU Y?” Vector RAG retrieved supplier PDFs and BOM exports separately; buyers still spent hours joining spreadsheets. Relationship recall on a 50-query slice 33%.
Intervention:
- MVG:
SKU,Component,Supplier,Contractwithuses,supplied_by,covered_byedges. - Seeds from SKU/component master data IDs (not free text alone).
- Expand depth 2; join clause chunks from contract PDF for citation.
- Nightly incremental extract from PLM + contract repository; curator reviews new
covered_byedges (2h/week). - Router: only procurement intent class hits graph; engineering FAQ stays vector.
Measurable outcomes:
- Relationship recall 33% → 84%
- Mean time to assemble sourcing pack −51%
- Wrong-supplier citation incidents 7 → 1 per quarter
- Lookup FAQ faithfulness unchanged within 1.5 pts
Lesson: Prefer master-data entity IDs as seeds; free-text-only linking underperforms on industrial identifiers.
Practice exercises
Primary exercise — GraphRAG decision memo (2–3 hours)
Brief: National retailer; returns-policy corpus plus supplier contracts. Leadership heard “GraphRAG” at a conference.
Tasks:
- Write 15 relationship queries and 15 lookup queries.
- Hypothesise vector-only failure modes on the relationship set.
- Draft MVG schema (entities/edges) for returns↔supplier obligations only.
- Complete the adopt/defer decision memo with assumed metrics and curator hours.
- Design ACL traversal tests (store colleague must not see supplier pricing nodes).
Acceptance criteria:
- Memo has numeric adopt threshold and defer recommendation if curator unpaid
- Schema ≤8 / ≤6 types
- Router policy separates lookup from relationship
Stretch exercise — Hybrid retrieve design (full day)
Brief: Adopt approved for contract–schedule graph (energy-style).
Tasks:
- Draw build pipeline and online sequence (mermaid or equivalent).
- Specify hop limits, timeouts and pack contents for the LLM.
- Define relationship eval suite (metrics + gates).
- Write re-extract cutover runbook.
- Produce ops RACI including curator KPI.
Acceptance criteria:
- Citations require path + chunk
- Forbidden traversal suite listed
- Lookup regression gate defined
Questions you should be able to answer
- What eval gap justifies GraphRAG versus iterative Advanced/Modular RAG?
- Who is the curator, and how many hours/week are funded?
- What is the minimal schema, and which query class does it serve?
- Which serve mode is default (Basic / Local / Global / DRIFT / hybrid)—and why?
- How do edges carry provenance and effective dates?
- How does the router prevent graph expansion on FAQ traffic?
- What are max hop depth and node caps?
- How is ACL enforced on graph expand?
- What is the citation contract for relational claims?
- Which metrics gate relationship release vs lookup regression?
- How do you detect extract drift after a parser change?
- When do critical edges require human confirmation?
- Is Agentic Graph RAG in scope—what caps and mode-switch rules apply?
- What does the decision memo say is out of scope for v1?
- How does this interact with Agentic RAG types in 13.1?
- What is the rollback if relationship precision collapses post-release?
Negative cases — when GraphRAG fails
| Failure mode | Symptom | Prevention |
|---|---|---|
| Graph without curator | Silent wrong answers | Fund hours or defer |
| Ontology boil-the-ocean | No go-live | MVG + one query class |
| Wrong edges | Confident multi-hop errors | Sample QA + precision gates |
| ACL on docs only | Leak via neighbours | Filter nodes/edges at expand |
| Always-on graph path | Latency/cost on FAQ | Router |
| Global search for FAQ | Cost spike, slow p95 | Reserve Global for theme class |
| Unbounded agent on graph | Traverse storm | Mode caps + hop/node limits |
| Citation theatre | Pretty paths, weak spans | Require chunk evidence |
| Extract shock | Edge count −40% overnight | Diff alerts + dual-write |
| Email as graph source | Noise edges | Restrict source systems |
| No lookup regression | FAQ quality tanks | Dual-class eval |
| Unbounded expansion | Timeouts / over-retrieval | Hard caps |
Case study: edges without chunks. A consultancy generated answers from graph properties alone (“Contract status=active”) without retrieving the governing clause; status field was stale ETL. Architecture rule: relational claims need path and supporting chunk from system of record document.
Case study: deferred correctly. A university FAQ bot showed only 4% relationship queries; spike estimated £180k graph build. Decision memo: defer; invest in parent-child + hybrid. Relationship complaints stayed negligible.
Operating-model notes
| Role | Responsibility | KPI |
|---|---|---|
| Ontology curator | Edge quality, schema changes | Wrong-edge rate; review backlog |
| ASE | Hybrid retrieve, router, citations | Relationship gates |
| Data engineering | Extract jobs, idempotency | Job success; drift alerts |
| Search / graph platform | Store health, ACL predicates | Expand p95 |
| Security | Traversal forbidden suite | Zero leakage |
| Eval owner | Relationship + lookup suites | Gate breaches blocked |
| Corpus owner | Authoritative docs for extract | Freshness SLA |
Re-extract triggers: Schema change; major corpus load; extractor model upgrade; audit finding on wrong edge; regulatory event requiring freeze.
Spike playbook (four weeks)
When the decision is “time-box,” run a fixed spike—not an open research programme:
| Week | Outcome |
|---|---|
| 1 | Relationship golden set (≥40) + vector/deep baseline scores |
| 2 | MVG schema + extract on pilot slice + 100-edge human review |
| 3 | Hybrid retrieve prototype + citation contract wired |
| 4 | Gates measured; adopt/defer memo signed |
Stop rules: curator unpaid; precision on reviewed edges <90%; no measurable recall lift ≥10 pts on the target class.
Integration with enterprise search and MDM
GraphRAG should not invent a parallel master data universe. Prefer:
- Entity IDs from MDM / PLM / contract systems as graph keys
- Documents from ECM as chunk evidence
- Enterprise search for lookup classes; graph only for routed relationship classes
Duplicate “AI-only” entity stores diverge within months. Architecture diagrams must show the system of record for each entity type.
Interview and partner challenge lines
Be ready to answer:
- “Show me the relationship-class lift versus iterative RAG.”
- “Who owns edge quality next quarter?”
- “What happens when extract confidence is 0.6 on a legal
amendsedge?” - “Prove a restricted user cannot traverse to a forbidden neighbour.”
- “Why is this not just a SQL join on structured tables?” (Answer: unstructured cross-refs dominate; if it is structured, prefer the database.)
Worked example — adopt memo (compressed)
Problem: Buyers cannot reliably answer which contracts cover suppliers of BOM component X.
Baseline: Vector + iterative deep RAG relationship recall 33% (n=50).
Gap: −51 pts vs target 84%; business impact = 2.1 FTE equivalent in manual join work.
MVG: SKU, Component, Supplier, Contract; edges uses, supplied_by, covered_by.
Ops: Curator 2h/week; extract nightly from PLM + ECM; human confirm on new covered_by.
Cost: Infra + extract + curator < annual value of pack time saved (model attached).
Gates: Relationship recall ≥80%; wrong-edge ≤2%; ACL traversal 100%; lookup Δ faithfulness ≤2 pts.
Decision: Adopt MVG for procurement intent only; FAQ remains vector. Review in 90 days.
When structured data should win
If the “graph” is already a well-keyed relational model (ERP contract lines, BOM tables) with stable joins, prefer SQL/API tools over NLP extraction into a parallel graph. GraphRAG earns its place when unstructured cross-references (PDF amendments, narrative notices) carry the relations. Misclassifying a SQL problem as GraphRAG wastes a quarter and still leaves the warehouse underused.
Related playbook content
- Retrieval-Augmented Generation — baseline RAG, ACL, citations, GraphRAG summary
- RAG Architecture — topologies and multi-hop without a graph
- Prompt and Context Engineering — packing paths and evidence
- AI Evaluation and Quality Assurance — golden sets and regression
- Data Engineering and Architecture — pipelines and quality
- Privacy, Legal and Compliance — lawful basis for graphing personal data
- Responsible AI and Governance — risk class and oversight
- MLOps, LLMOps and Observability — jobs, drift, rollback
- Performance and FinOps — expand latency and cost
- RAG — engagement playbook
- Architecture — Layer 6 knowledge and retrieval
- Evaluation and observability — metrics and gates
- How to use this Learning Map — reference depth standards
Practice checklist
- I quantified a relationship-query gap versus vector/deep RAG
- I proposed an MVG schema tied to one query class
- I funded or explicitly deferred curator hours
- I designed hybrid expand + chunk evidence with hop caps
- I defined path-plus-chunk citation rules and refuse behaviour
- I included ACL traversal tests and lookup regression gates
- I wrote an adopt/defer memo stakeholders can sign
- I linked back to Retrieval-Augmented Generation and RAG Architecture
Discussion
Comments
Share feedback or questions about this page. No account required.
Loading comments…