Skip to main content

Graph RAG Architecture

Executive view

GraphRAG is a **product and operating-model** commitment: ontology ownership, weekly curator time and eval on relationship queries—not only a new database SKU. Defer if there is no named curator budget.

Decision required: Does relationship-query recall lag vector-only by enough to justify graph build and maintenance cost—and who owns edge quality?

Technical view

Design light schema before heavy ontology; extract with provenance; retrieve by subgraph expansion plus chunk evidence; enforce ACL on nodes and edges; evaluate wrong-edge and missing-edge failure modes.

Route relationship vs lookup queries; keep hybrid vector path for factual lookup. Fail closed when graph confidence is low.

Why this matters

Vector RAG excels at “find passages like this question.” It struggles when the answer is a path: Contract A amends Schedule B which sets TariffBand C under regulator notice D. Embeddings may retrieve each document in isolation and still miss the chain. Teams then either hallucinate the link or return incomplete answers that fail legal review.

GraphRAG’s failure mode is different but equally expensive: wrong edges look authoritative (“applies_to” attached to the superseded amendment), unmaintained ontologies drift from reality, extraction jobs silently drop relations after a parser change, and ACL on documents is not mirrored onto graph nodes—so traversal leaks restricted neighbours.

The AI Solution Engineer’s job is to prove the gap, design a minimal viable graph, wire hybrid retrieval (traversal + chunks), fund curation, and gate release on relationship-query recall and citation correctness—or to defer with a written memo. Weak programmes buy a graph database first. Strong programmes start from eval slices and curator hours.

Connect to RAG Architecture for non-graph multi-hop patterns, Retrieval-Augmented Generation for the baseline contract, and RAG playbook for engagement workshops.

13.2 Graph RAG architecture flowchart

Preview fits the page width — use the expand icon for the full flowchart.

Learn

Graph RAG types and search modes

GraphRAG is not one algorithm. Choose a serve mode from evidence, cost and curator capacity.

Mode ladder: Basic → Local → Global → DRIFT-style → Agentic Graph RAG

Graph RAG serve mode ladder

Preview fits the page width — use the expand icon for the full flowchart.

Public Microsoft GraphRAG documents Basic / Local / Global / DRIFT; enterprise hybrids add agentic routing.

ModeWhat it retrievesBest forCost / latency
BasicStandard top-k vector chunksSimple semantic FAQ inside a graph-enabled platformLow
LocalEntity neighbourhood + linked text chunks“What connects A to B?”, entity-centric factsLow–medium
GlobalCommunity / theme summaries over the graphCorpus-wide themes, “what are the main risk clusters?”High (map-reduce style)
DRIFT-styleIterative mix of global then local refinementExploratory multi-hop with follow-upsMedium–high
Hybrid enterpriseGraph expand + vector + BM25 fusedRegulated contracts / BOM / policy cross-refs (this playbook default)Tuned by router
Agentic Graph RAGAgent selects basic / local / global / toolsMixed relationship + tool workflowsHighest — needs budgets

Default for this playbook: hybrid enterprise (router → bounded expand + chunks + ACL). Adopt Global/DRIFT only with labeled theme/exploratory query classes and FinOps approval.

Agentic Graph RAG — when the agent sits on the graph

Flow: agent planner → choose vector / graph mode / tool → critique evidence → re-plan if weak (capped) → cited answer or refuse.

Agentic Graph RAG control loop

Preview fits the page width — use the expand icon for the full flowchart.

PatternRole of the agentGuardrails
Router-onlyClassifier picks Local vs Basic — no free loopPrefer this first
Single-agent Graph RAGAgent may re-expand or switch mode onceCap rounds; log mode switches
CorrectiveAgent refuses or re-queries if path incompletePath + chunk citation required
Multi-agentSeparate planner / expander / criticRare; justify vs single-agent

Do not let an agent unbounded-traverse the graph. Hop caps, node caps, ACL on expand, and curator ownership still apply. Link agent budgets to Agentic AI and Agentic RAG types in 13.1.

Further reading: Microsoft GraphRAG (Local / Global / DRIFT / Basic).

When GraphRAG is justified — and when to defer

Adopt signals (need several, not one slogan):

  • Labeled relationship / multi-hop query class exists (≥30–60 queries)
  • Vector-only or deep iterative RAG shows recall gap (commonly >10–15 points) on that class
  • Source corpus has explicit cross-references (amends, supersedes, references, applies_to, part_of)
  • Named ontology/curator capacity (hours/week) funded in the runbook
  • Business value of correct chains exceeds build + maintenance cost

Defer signals:

  • Flat FAQ or single-document lookup dominates traffic
  • No curator; “we will fix edges later”
  • Ontology wars (enterprise data model unagreed)
  • advanced RAG iterative retrieve-then-read already closes the gap
  • Pilot corpus < few hundred docs with rare links
Question typePrefer
“What is the cooling-off period?”Hybrid vector RAG
“Which live contracts still cite Schedule 7?”GraphRAG (or structured DB)
“Summarise this policy section”Vector / parent-child
“Supplier → component → contract path”GraphRAG
“My account balance”Tool / API — never graph or RAG

Graph construction — schema, extraction, versioning

Light schema first. Prefer a small typed graph over a sprawling enterprise ontology.

Example entity types: Contract, Amendment, Schedule, Policy, Notice, Product, Clause, Supplier, Component.

Example edge types: references, amends, supersedes, applies_to, part_of, governed_by.

Construction pipeline: Use the Build lane in the architecture flowchart above for the end-to-end extract → HITL → graph → chunk-link path. Every edge needs provenance; every node used in generation needs a chunk link.

Extraction principles:

  1. Provenance — every node/edge stores source_id, locator, extractor version, confidence
  2. Idempotent upserts — stable entity keys (e.g. contract number + jurisdiction)
  3. Version edgessupersedes / amended_by with effective dates; never silently overwrite history
  4. Human-in-the-loop on low-confidence or high-risk edge types (legal, clinical)
  5. Reject orphans — entities without chunk links cannot be sole evidence for generation

Ontology vs light schema: Start with light schema tied to query classes. Expand types only when eval shows systematic miss. Full enterprise ontology projects often stall GraphRAG for quarters.

Pitfall: LLM extraction without sampling QA. A 5% wrong applies_to rate becomes a 40% wrong-answer rate on multi-hop questions that traverse two edges.

Storage and hybrid retrieval

GraphRAG in enterprise is almost always hybrid:

StoreHoldsRole
Graph DBNodes, typed edges, propertiesTraversal / neighbourhood expand
Chunk / vector / BM25 indexText evidence + embeddingsPassage retrieve and citation spans
Metadata catalogueACL, tenant, effective datesFilters on both stores

Canonical retrieve pattern:

  1. Route query: relationship vs factual lookup vs refuse/tool
  2. If relationship: seed entities (NER / linker / user entities) → expand subgraph (bounded depth, e.g. 1–3 hops) with ACL filters on nodes/edges
  3. Collect linked chunks for nodes on the path
  4. Optional: vector retrieve for supporting narrative not captured as edges
  5. Rerank / pack path summary + chunks for generation
  6. Cite node IDs and chunk locators; verify post-generation

The Serve lane in the architecture flowchart is the canonical path. Keep this sequence for runtime detail:

Graph RAG online sequence

Preview fits the page width — use the expand icon for the full flowchart.

Query router: Rules (keywords: “which contracts reference”, “superseded by”) + small classifier. Lookup traffic must stay on cheap hybrid vector path—do not expand graphs for “what is VAT on X?”.

Bounds: Max hop depth, max nodes, timeout. Unbounded expansion is a latency and leakage risk.

Generation and citations for multi-hop answers

Citation contract for GraphRAG:

  • Every relational claim maps to an edge or path (edge IDs) and ≥1 chunk span supporting that edge when the edge was text-extracted
  • Factual elaborations use chunk citations as in standard RAG
  • If path incomplete → refuse or partial answer with explicit gap (“no applies_to link from Notice Y to Policy Z in graph”)

Faithfulness: LLM-as-judge on prose is insufficient. Add path consistency checks: mentioned entities must appear on retrieved subgraph; claimed relations must match edge types.

UI/API: Show path breadcrumb (Contract → Amendment → Schedule) plus document locators. Auditors replay paths, not only paragraphs.

Operations — curator budget, drift, ACL, re-extract

Ops concernPractice
CuratorNamed role; weekly hours in runbook (e.g. 4h/week)
DriftDiff extract job output; alert on edge-count shocks
Re-extractVersioned jobs; dual-write; offline relationship eval before cutover
ACLNode/edge inherit document ACL; filter at expand time; forbidden traversal suite
TombstonesRetracted docs remove or mark edges unusable within SLA
CostGraph infra + extract compute + curator — in business case

Without curator → defer. Topic 13’s rule stands: graphs rot faster than text indexes.

Worked example: Energy retailer funded 4 hours/week ontology curator after relationship recall jumped; without that line item, legal rejected go-live.

Eval specific to graphs

MetricWhat it catches
Relationship-query recallMissing paths / under-extraction
Relationship precisionWrong edges in answers
Path citation correctnessTheatre citations
Wrong-edge rateSystematic extract/schema bugs
Lookup regressionGraph path accidentally hurting FAQ quality
Latency p95 on relationship classUnbounded expansion
Curator hours actual vs planOperating-model denial
ACL traversal suiteLeak via neighbour nodes

Golden set design: Stratify relationship queries by hop count (1 / 2 / 3+), by edge type, and include negative paths (“must not connect A to B”). Keep vector-only baseline scores in the decision memo.

Release gate example: Relationship recall ≥85% on 60-query slice; wrong-edge in sampled answers ≤2%; ACL suite 100%; lookup-class faithfulness not degraded >2 pts vs pre-graph baseline.

Decision memo artefact — adopt or defer

Mandatory one-pager before build:

  1. Problem — query class and business impact
  2. Baseline — vector / deep RAG metrics on that class
  3. Gap — numeric delta and sample failures
  4. Proposed schema — entity/edge types (minimal)
  5. Ops — curator hours, owners, re-extract cadence
  6. Cost — build + run vs value
  7. Gates — metrics that allow production
  8. Defer criteria — what would make us stop or not start
  9. Decision — Adopt / Defer / Time-box spike (e.g. 4 weeks)

Aligns with the GraphRAG decision memo exercise in topic 13.

Entity linking and seed resolution

Graph expand fails if seeds are wrong. Seed resolution is its own design surface:

StepPractice
DetectNER / gazetteer / user-selected entities in UI
LinkMap surface forms to stable entity IDs (contract numbers, policy codes)
DisambiguatePrefer metadata constraints (jurisdiction, product line) before expand
Fail softIf linker confidence < τ → vector path or ask clarifying question — do not expand top-3 fuzzy IDs

Pitfall: Expanding on the wrong “Schedule 7” across jurisdictions creates precise-looking wrong paths. Prefer refuse/clarify over confident traversal.

Property graphs vs RDF — engagement choice

Most enterprise GraphRAG deliveries use a property graph (nodes/edges with JSON-like properties) because engineering teams already operate that model and latency is predictable. RDF/OWL stacks suit organisations with existing semantic layers and SPARQL skills—but ontology governance cost is higher. Choose based on operating model, not purity:

  • Property graph + light schema → default for ASE-led GraphRAG spikes
  • RDF → when enterprise already has a governed knowledge graph team and query class maps cleanly to SPARQL

Document the choice in the decision memo; switching mid-flight is a programme reset.

Security model for traversal

Document ACL is necessary but not sufficient.

  1. Inherit — node ACL = union or strictest of source document ACLs that created it (pick one policy; document it).
  2. Edge visibility — if either endpoint is invisible to the principal, edge is not traversable.
  3. Neighbour suppression — do not return neighbour labels the user cannot access (even as “hidden related”).
  4. Forbidden suite — scripted identities attempt to reach restricted nodes via alternate paths.
  5. Audit — log seed IDs, expanded node IDs, and filtered-out counts for forensics.

Cost model sketch

Line itemNotes
Graph infrastructureHA store, backups, environments
Extract computeBatch + incremental; GPU if using heavy models
Curator labourHours × loaded cost — often dominates
Eval maintenanceRelationship suite growth
Dual-path servingVector stack remains; graph is additive

If curator + extract > value of relationship-class deflection, defer. GraphRAG is rarely cheaper than better chunking.

Anti-patterns catalogue

Anti-patternFailureReplace with
Graph as default indexFAQ latency/costRouter
Unbounded hopsTimeouts / leakageDepth and node caps
Edges without provenanceUnauditable answersSource locator mandatory
Auto-extract email/chatNoiseSystem-of-record docs only
No lookup regressionFAQ quality dropDual-class gates
Schema from vendor demoUnused typesMVG from query class
“Curator later”RotDefer or fund now

Frameworks and methods

Minimal viable graph (MVG)

  1. Pick one query class
  2. Define ≤8 entity types and ≤6 edge types
  3. Extract from a pilot corpus slice
  4. Human-review 100 edges
  5. Build hybrid retrieve for that class only
  6. Compare to deep RAG baseline
  7. Expand schema only with new failed query themes

Edge quality review checklist

  • Stable entity key collision rate acceptable
  • Effective dating on supersedes correct on sample
  • ACL inherited on nodes from source docs
  • Low-confidence edges quarantined
  • Chunk links present for generation evidence

Router policy table

SignalPath
Explicit relation language + known entity typesGraph expand + chunks
SKU / FAQ / definitionHybrid vector only
Live operational dataTools / API
AmbiguousConservative vector first; optional graph if linker high confidence

Real-world scenarios

Scenario A — Energy retailer: contract–tariff dependencies

Context: B2B energy; answers needed linking master supply agreements, amendments and time-of-use tariff schedules. Vector-only RAG missed “which contracts still reference deprecated Schedule 7?” Relationship-query recall 41% on 60-query slice.

Intervention:

  1. Entity graph: Contract, Amendment, Schedule, TariffBand with supersedes / references edges.
  2. Hybrid: graph traversal for relationship queries + chunk retrieval for text evidence.
  3. Query router: classify relationship vs factual lookup (rules + small classifier).
  4. Curator 4 hours/week; relationship eval in CI.

Measurable outcomes:

  • Relationship-query recall 41% → 87%
  • Legal review time for contract audit −38%
  • Incorrect tariff citation rate 9% → 2%
  • Lookup-class faithfulness held within 1 pt of baseline

Lesson: GraphRAG earns its keep on multi-hop enterprise questions—with maintenance budget.

Scenario B — Pharma: regulatory notice to SOP linkage

Context: Quality organisation must answer “which SOPs are impacted by regulator notice Y?” Vector search returned notice text and random SOPs sharing vocabulary (equipment names) without true impacted_by links. Audit finding: AI assistant unsafe for change-control impact analysis.

Intervention:

  1. Light schema: Notice, SOP, Clause, ProductFamily; edges amends, impacted_by, owned_by.
  2. Extraction from controlled change-control packs only (not whole SharePoint).
  3. Mandatory human confirm on new impacted_by edges before index.
  4. Generation refuses if path lacks approved edge; UI shows pending edges separately for authors.

Measurable outcomes:

  • Impact-analysis precision on golden 38% → 91%
  • Mean time to draft impact pack −44%
  • Zero critical audit findings on AI-assisted packs in two cycles
  • Spike decision: do not auto-extract from email—too noisy

Lesson: For regulated impact graphs, human confirmation on critical edge types is part of the architecture.

Scenario C — Manufacturing: supplier–BOM–contract multi-hop

Context: Discrete manufacturer asks “which active contracts cover suppliers of component X used in SKU Y?” Vector RAG retrieved supplier PDFs and BOM exports separately; buyers still spent hours joining spreadsheets. Relationship recall on a 50-query slice 33%.

Intervention:

  1. MVG: SKU, Component, Supplier, Contract with uses, supplied_by, covered_by edges.
  2. Seeds from SKU/component master data IDs (not free text alone).
  3. Expand depth 2; join clause chunks from contract PDF for citation.
  4. Nightly incremental extract from PLM + contract repository; curator reviews new covered_by edges (2h/week).
  5. Router: only procurement intent class hits graph; engineering FAQ stays vector.

Measurable outcomes:

  • Relationship recall 33% → 84%
  • Mean time to assemble sourcing pack −51%
  • Wrong-supplier citation incidents 7 → 1 per quarter
  • Lookup FAQ faithfulness unchanged within 1.5 pts

Lesson: Prefer master-data entity IDs as seeds; free-text-only linking underperforms on industrial identifiers.

Practice exercises

Primary exercise — GraphRAG decision memo (2–3 hours)

Brief: National retailer; returns-policy corpus plus supplier contracts. Leadership heard “GraphRAG” at a conference.

Tasks:

  1. Write 15 relationship queries and 15 lookup queries.
  2. Hypothesise vector-only failure modes on the relationship set.
  3. Draft MVG schema (entities/edges) for returns↔supplier obligations only.
  4. Complete the adopt/defer decision memo with assumed metrics and curator hours.
  5. Design ACL traversal tests (store colleague must not see supplier pricing nodes).

Acceptance criteria:

  • Memo has numeric adopt threshold and defer recommendation if curator unpaid
  • Schema ≤8 / ≤6 types
  • Router policy separates lookup from relationship

Stretch exercise — Hybrid retrieve design (full day)

Brief: Adopt approved for contract–schedule graph (energy-style).

Tasks:

  1. Draw build pipeline and online sequence (mermaid or equivalent).
  2. Specify hop limits, timeouts and pack contents for the LLM.
  3. Define relationship eval suite (metrics + gates).
  4. Write re-extract cutover runbook.
  5. Produce ops RACI including curator KPI.

Acceptance criteria:

  • Citations require path + chunk
  • Forbidden traversal suite listed
  • Lookup regression gate defined

Questions you should be able to answer

  1. What eval gap justifies GraphRAG versus iterative Advanced/Modular RAG?
  2. Who is the curator, and how many hours/week are funded?
  3. What is the minimal schema, and which query class does it serve?
  4. Which serve mode is default (Basic / Local / Global / DRIFT / hybrid)—and why?
  5. How do edges carry provenance and effective dates?
  6. How does the router prevent graph expansion on FAQ traffic?
  7. What are max hop depth and node caps?
  8. How is ACL enforced on graph expand?
  9. What is the citation contract for relational claims?
  10. Which metrics gate relationship release vs lookup regression?
  11. How do you detect extract drift after a parser change?
  12. When do critical edges require human confirmation?
  13. Is Agentic Graph RAG in scope—what caps and mode-switch rules apply?
  14. What does the decision memo say is out of scope for v1?
  15. How does this interact with Agentic RAG types in 13.1?
  16. What is the rollback if relationship precision collapses post-release?

Negative cases — when GraphRAG fails

Failure modeSymptomPrevention
Graph without curatorSilent wrong answersFund hours or defer
Ontology boil-the-oceanNo go-liveMVG + one query class
Wrong edgesConfident multi-hop errorsSample QA + precision gates
ACL on docs onlyLeak via neighboursFilter nodes/edges at expand
Always-on graph pathLatency/cost on FAQRouter
Global search for FAQCost spike, slow p95Reserve Global for theme class
Unbounded agent on graphTraverse stormMode caps + hop/node limits
Citation theatrePretty paths, weak spansRequire chunk evidence
Extract shockEdge count −40% overnightDiff alerts + dual-write
Email as graph sourceNoise edgesRestrict source systems
No lookup regressionFAQ quality tanksDual-class eval
Unbounded expansionTimeouts / over-retrievalHard caps

Case study: edges without chunks. A consultancy generated answers from graph properties alone (“Contract status=active”) without retrieving the governing clause; status field was stale ETL. Architecture rule: relational claims need path and supporting chunk from system of record document.

Case study: deferred correctly. A university FAQ bot showed only 4% relationship queries; spike estimated £180k graph build. Decision memo: defer; invest in parent-child + hybrid. Relationship complaints stayed negligible.

Operating-model notes

RoleResponsibilityKPI
Ontology curatorEdge quality, schema changesWrong-edge rate; review backlog
ASEHybrid retrieve, router, citationsRelationship gates
Data engineeringExtract jobs, idempotencyJob success; drift alerts
Search / graph platformStore health, ACL predicatesExpand p95
SecurityTraversal forbidden suiteZero leakage
Eval ownerRelationship + lookup suitesGate breaches blocked
Corpus ownerAuthoritative docs for extractFreshness SLA

Re-extract triggers: Schema change; major corpus load; extractor model upgrade; audit finding on wrong edge; regulatory event requiring freeze.

Spike playbook (four weeks)

When the decision is “time-box,” run a fixed spike—not an open research programme:

WeekOutcome
1Relationship golden set (≥40) + vector/deep baseline scores
2MVG schema + extract on pilot slice + 100-edge human review
3Hybrid retrieve prototype + citation contract wired
4Gates measured; adopt/defer memo signed

Stop rules: curator unpaid; precision on reviewed edges <90%; no measurable recall lift ≥10 pts on the target class.

Integration with enterprise search and MDM

GraphRAG should not invent a parallel master data universe. Prefer:

  • Entity IDs from MDM / PLM / contract systems as graph keys
  • Documents from ECM as chunk evidence
  • Enterprise search for lookup classes; graph only for routed relationship classes

Duplicate “AI-only” entity stores diverge within months. Architecture diagrams must show the system of record for each entity type.

Interview and partner challenge lines

Be ready to answer:

  • “Show me the relationship-class lift versus iterative RAG.”
  • “Who owns edge quality next quarter?”
  • “What happens when extract confidence is 0.6 on a legal amends edge?”
  • “Prove a restricted user cannot traverse to a forbidden neighbour.”
  • “Why is this not just a SQL join on structured tables?” (Answer: unstructured cross-refs dominate; if it is structured, prefer the database.)

Worked example — adopt memo (compressed)

Problem: Buyers cannot reliably answer which contracts cover suppliers of BOM component X.
Baseline: Vector + iterative deep RAG relationship recall 33% (n=50).
Gap: −51 pts vs target 84%; business impact = 2.1 FTE equivalent in manual join work.
MVG: SKU, Component, Supplier, Contract; edges uses, supplied_by, covered_by.
Ops: Curator 2h/week; extract nightly from PLM + ECM; human confirm on new covered_by.
Cost: Infra + extract + curator < annual value of pack time saved (model attached).
Gates: Relationship recall ≥80%; wrong-edge ≤2%; ACL traversal 100%; lookup Δ faithfulness ≤2 pts.
Decision: Adopt MVG for procurement intent only; FAQ remains vector. Review in 90 days.

When structured data should win

If the “graph” is already a well-keyed relational model (ERP contract lines, BOM tables) with stable joins, prefer SQL/API tools over NLP extraction into a parallel graph. GraphRAG earns its place when unstructured cross-references (PDF amendments, narrative notices) carry the relations. Misclassifying a SQL problem as GraphRAG wastes a quarter and still leaves the warehouse underused.

Practice checklist

  • I quantified a relationship-query gap versus vector/deep RAG
  • I proposed an MVG schema tied to one query class
  • I funded or explicitly deferred curator hours
  • I designed hybrid expand + chunk evidence with hop caps
  • I defined path-plus-chunk citation rules and refuse behaviour
  • I included ACL traversal tests and lookup regression gates
  • I wrote an adopt/defer memo stakeholders can sign
  • I linked back to Retrieval-Augmented Generation and RAG Architecture

Discussion

Comments

Share feedback or questions about this page. No account required.

Loading comments…