Cassie Shum has spent the last three years building agentic systems at RelationalAI, a company focused on relational knowledge graphs. In a recent talk at QCon, she laid out a case for why knowledge graphs should serve as the structural foundation of production AI agent systems, not merely as a retrieval optimization trick.
The argument is timely. As LLMs grow more capable and context windows expand, many teams assume that stuffing enough data into a prompt solves the problem. Shum agrees that retrieval from graphs has lost some of its edge. But she says that misses the point entirely.
GraphRAG Is Not the Endgame
Six months before this talk, Shum presented at QCon New York on GraphRAG, the technique of retrieving structured information from knowledge graphs to augment language model outputs. By the time she updated her slides, the landscape had shifted. LLMs had gotten significantly better at extracting useful context from raw text. The retrieval advantage that graphs once offered had narrowed.
"I'm not seeing that differentiator as much anymore," she admitted. "I don't think it's dead, but not in the way you think."
Her revised position: knowledge graphs matter because they do things retrieval alone cannot. They track how decisions evolve over time. They map dependencies across teams and domains. They give agents something more reliable than a scraped wiki page from 2019 that nobody has updated.
Why Prompts Are Not Enough
Long context windows are impressive, but they have a consistency problem. One engineer's prompt produces a different result than another's. There is no shared state, no audit trail, no way to trace why an agent made a particular recommendation.
Shum pointed to a concrete example from a financial institution she works with. If you ask a generic agent what compliance regime applies to a given transaction, it will search the internet and give you a generic answer. If you ask it what Tier 1 means in the context of that specific organization, it will guess. The answers are different depending on who asks, when they ask, and what phrasing they use.
A knowledge graph, by contrast, stores the organization's actual definitions, policies, and relationships between them. The agent queries the graph instead of improvising. The answer is consistent across teams and over time.
Tracking Institutional Memory
One of the less obvious problems Shum identified is institutional knowledge decay. Every organization has wiki pages with rules from five years ago that nobody remembers creating. That knowledge becomes dead code, reimplemented in different places, contradicted by newer policies, and eventually just accepted as tribal knowledge that lives in engineers' heads.
Agents cannot do anything with that. They cannot distinguish between an active policy and one that was quietly deprecated in a Slack thread eighteen months ago.
A knowledge graph can encode not just what the current state of a decision is, but how it got there. When was the rule created? What changed it? Who approved the change? That lineage is the difference between an agent that gives you a plausible answer and one that gives you the right answer.
The Four Patterns
Shum shared four patterns she has observed while building internal developer tools on top of knowledge graphs at RelationalAI. The talk framed these as lessons from her own team's experience rather than prescriptive architecture.
The first is using the graph as a shared context layer across teams. When multiple groups depend on each other's domains, the graph maps those dependencies explicitly. This replaces the common problem of the distributed monolith, where microservices create the illusion of independence while hidden coupling causes cascading failures.
The second is treating the graph as a development harness. Shum built internal tooling on top of a knowledge graph so that developers could query their own organizational context instead of searching through old Confluence pages. The graph becomes the source of truth for how the system is supposed to work, not just how it currently runs.
The third involves auditability. Because the graph captures relationships and changes over time, teams can trace an agent's recommendation back to the specific knowledge it drew from. If the agent made a mistake, you can see which part of the graph was wrong and fix it there instead of patching prompts.
The fourth is domain integration. Different teams have different rules, different vocabularies, different ways of describing the same concept. The graph forces a shared vocabulary and makes the connections between domains explicit rather than implicit.
The Moat Is the Graph, Not the Model
Shum's central claim is straightforward: the competitive advantage in AI systems is not the model. Models improve every quarter. Fine-tuning and domain-specific training are commoditizing. What does not commoditize easily is the structured, curated knowledge of how your organization actually works.
"I believe it's actually the domain knowledge and the graph that you put together from your domain," she said.
For teams building agentic systems, the implication is that investing in a knowledge graph is not a side project or an academic exercise. It is the infrastructure that determines whether your agents give you consistent, auditable, organization-specific answers or whether they hallucinate generic ones that happen to sound right.
The tools and frameworks will keep changing. The models will keep improving. But the graph, if you build it well, is yours.