Blog Post

Microsoft Foundry Blog
13 MIN READ

Knowledge graphs in Foundry IQ improve evidence recall with fewer tokens

MattGotteiner's avatar
MattGotteiner
Icon for Microsoft rankMicrosoft
Nov 17, 2026

Foundry IQ introduces knowledge graphs in Foundry IQ, with entities and topics. Knowledge graphs improved evidence recall by 43% while using 52% fewer tokens, compared to using RAG alone.

By: the Foundry IQ knowledge graph team

We’re introducing knowledge graphs in Foundry IQ to help agents answer questions that connect facts across documents or gathering evidence across a collection. Knowledge graph entities structures facts and their relationships for precise queries. Knowledge graph topics organize documents into a browsable topic hierarchy and helps retrieval find evidence across different subjects. Below, we show how these approaches complement existing retrieval, with evaluations of answer quality, evidence coverage, and token use.

Key results and takeaways

  • Knowledge graph entities enables correct answers for structured graph queries while reducing token usage. Entity graph search answered 93.5% of queries correctly versus 26.1% for RAG and used 64.3% fewer tokens.
  • Knowledge graph topics turns your corpus into a browsable topic map and helps search find evidence across it. On thematic questions, topic graph search improved evidence recall by up to 13.2 percentage points over classic vector search.
  • The approaches are complementary and additive. Use classic retrieval for direct passage matches, entities for exact typed facts, and topics for cross-theme evidence synthesis and corpus navigation. Layer them when a question requires precision and breadth, and achieve better retrieval with fewer tokens (in our example, improving evidence recall by 43% while using 52% fewer tokens).

Why knowledge graphs matter for retrieval

Retrieval-augmented generation (RAG) works well when a few relevant passages contain the answer. It struggles when the answer depends on exact facts spread across many documents or on themes spanning an entire corpus. For example, in a set of Microsoft earnings call transcripts, “What capital expenditures did Microsoft report in each quarter?” requires finding specific facts across many separate calls. Knowledge graphs in Foundry IQ, now available in public preview, offers two capabilities for these kinds of questions:

  • Knowledge Graph Entities extract facts from your documents as entities and relationships so you can query them directly.
  • Knowledge Graph Topics group your documents into a hierarchy of topics.

Used alongside RAG, they cover questions that RAG alone struggles to answer.

Knowledge graph entities: structured facts for precise answers

Knowledge graph entities transform facts from your documents into a structured graph you can query. Each fact is represented as a relationship between two entities. For example, an earnings call that says “Azure is part of Microsoft’s Intelligent Cloud” becomes “Microsoft offers Azure”, and “Azure belongs to Intelligent Cloud”. The same entity is linked across every call that mentions it, so a question such as “Who offers Azure?” can be answered by querying the graph rather than searching for relevant passages.

 To decide which facts to extract, you define an ontology: a schema of the entities in your domain (such as organizations, products, and segments) and how they connect (such as an organization offering a product). An LLM reads your documents and extracts the facts matching your ontology.

Because answers come from complete, structured facts, agents can handle questions spanning many documents with far fewer tokens than traditional RAG

Efficiency and capability improvements

We compared knowledge graph entities with RAG on 6 classes of questions about SEC filings, from counting and ranking to finding shared connections, gaps, and negative claims (what a document says does not apply). For each answer, we measured correctness, how complete the supporting evidence was, and how many tokens it took.

Entities answered 93.5% of questions correctly compared to 26.1% for RAG while using 64.3% fewer tokens. The extra tokens RAG spent did not improve its answers. The gap was largest when a question depends on every relevant fact. RAG found some of the right evidence, 72.0% of the needed facts on average, but it found all the facts in only 26.8% of queries. For counts, overlaps, or distinct connections, one missing fact makes the answer wrong. RAG answered no negative-claim questions correctly and only 17.6% of counting questions. Entities answered every question correctly in 3 of the 6 categories: counting, source tracing, and patterns and outliers.


Query class

Method

Correct
answers

Complete
Evidence Recall

Mean
Evidence Recall

Mean
tokens/run

All classes
Aggregate across the 6 query classes

Entities

93.5%

95.7%

97.3%

4,725.4

RAG

26.1%

26.8%

72.0%

13,230.8

Negative claims
Which X is stated not to apply?

Entities

83.3%

100.0%

100.0%

4,409.2

RAG

0.0%

0.0%

70.4%

13,235.9

Counting and calculating
Which X has the most Y?

Entities

100.0%

100.0%

100.0%

5,420.7

RAG

17.6%

17.6%

63.2%

13,695.3

Source tracing
Which source supports X?

Entities

100.0%

100.0%

100.0%

4,379.9

RAG

28.6%

28.6%

73.8%

13,455.7

Patterns and outliers
Which X doesn't fit the pattern?

Entities

100.0%

100.0%

100.0%

4,338.5

RAG

44.4%

44.4%

80.0%

12,495.3

Shared connections
Which Xs do Y and Z have in common?

Entities

66.7%

66.7%

66.7%

4,321.2

RAG

33.3%

33.3%

74.1%

13,418.7

Gaps and differences
Which X is missing from Y?

Entities

75.0%

75.0%

93.8%

4,022.5

RAG

50.0%

58.3%

89.6%

12,369.1

Questions that depend on a complete set of facts expose a limitation of RAG: retrieving more passages and using more tokens does not guarantee finding every fact. For example, if you ask which products two companies both offer, a retriever that finds most of the relevant passages can still miss one product and get the answer wrong. Knowledge graph entities are the better fit when the answer depends on how facts are connected, not just on whether relevant text exists somewhere in the documents.

Entities are not always necessary. On questions where the answer is stated directly in text, such as fact lookups, timelines, acquisitions, and corporate structure, the two methods perform comparably (Entities 98% correct, RAG 96%), but RAG uses 3.1 times more tokens.

Importance of ontology design

An ontology determines which facts are extracted from the source documents, and therefore which facts the system can use to answer questions. Defining a good ontology has historically been one of the most labor-intensive parts of building an enterprise knowledge graph. An agent skill we provide now uses knowledge graph topics to generate an initial ontology automatically. Refining that initial ontology for your documents and the questions you expect to ask shapes what the graph contains.

To show how much refinement can matter, we compared three versions of the ontology on the same collection of SEC filing documents: the starting point generated with knowledge graph topics, a first refinement, and the final ontology. The final ontology defined 29% more entity types and 72% more relationship types than the starting point, and extracted 40% more distinct facts, averaged over five extraction runs.

Metric

Starting Point

First refinement

Final Ontology

Entity types

38

43

49

Relationship types

78

100

134

Mean facts extracted

11,462.0

14,920.8

16,224.4

Mean distinct facts extracted

5,474.6

7235.2

7656.2

SEC filings often repeat the same fact, for example across reporting periods, so the same fact can be extracted more than once. "Facts extracted" counts every extraction; "distinct facts" counts each unique fact once.

Most of the new facts came from the first refinement. The starting point captured facts the system could reliably recognize. The first refinement defined what a complete financial fact should include. With 5 new entity types and 22 new relationship types, that step delivered 81% of the total increase in distinct facts. The final ontology added more definitions (6 entity types and 34 relationship types) but relatively few new facts. Instead, it reorganized how financial facts are represented, adding context and structure.

The takeaway is that a generated ontology is a useful starting point, and a single round of refinement guided by the questions you expect to ask delivered most of the gain in our SEC example. Later rounds added few new facts and mainly improved how the facts are structured.

Knowledge graph topics

Knowledge Graph Topics helps you see what’s in a corpus of documents and find information across it. It groups related content into a hierarchy of topics, organizing broad subjects into more specific subtopics, like a table of contents for the corpus.  Sitemap makes that hierarchy browsable, while Topic Graph Retrieval uses it to answer thematic queries, questions that require connecting information across topics or several documents.

A sitemap works like a table of contents. The example below is built from 49 Microsoft earnings transcripts. Topic descriptions explain what each area covers, and topic proportions show how much of the corpus it represents.

 

Sitemap illustration of Microsoft earnings transcripts dataset. Knowledge Graph Topics turns 49 earnings transcripts into a browsable table of topics and subtopics

A useful sitemap should cover the corpus broadly and accurately describe its topics. To measure coverage, we check how many of the small text passages used to index documents, called chunks, are represented in the topic hierarchy. Across seven datasets, this chunk coverage averaged 99.6%. That shows how much source content is organized, not whether every fact or theme is captured. To check label quality, an LLM judge reviewed 420 sampled topics and found that titles and descriptions generally reflected their topics well, with descriptions grounded in the source content.

Dataset

Chunk Coverage (%)

Total topics

SEC

99.2

3,468

Customer - oil and gas

99.9

1,186

Customer - insurance

100.0

787

DAYI

99.9

2,462

FDA

99.5

2,735

MIML - English banking subset

99.8

993

Leukemia Papers

99.0

3,702

Overall

99.6

15,333

Chunk coverage and topic counts for Sitemaps generated across seven datasets. Overall coverage is the unweighted dataset average; topic counts are summed up.

Topic graph retrieval

Some questions can be answered by locating a specific piece of evidence. Thematic queries can’t. They require connecting evidence across a corpus, such as explaining how risks interact, what drives business trade-offs, or how competing constraints affect a decision. For these queries, several relevant passages may still tell only one part of the story. Topic Graph Retrieval uses the topic hierarchy to guide the search toward complementary evidence across relevant topics while preserving strong direct matches.

Example of a thematic query: What operating trade-offs shape how Activision Blizzard makes, sells, and keeps games playable?

A useful answer needs to connect game development, distribution costs, reliance on platforms, and ongoing operations. More passages about distribution alone won’t fill those gaps. This example tests whether Topic Graph Retrieval finds evidence across these areas.

Does the answer explain…

Classic Vector Search

Topic Graph SEARCH

Digital distribution economics

✅

✅

Dependence on platform owners

✅

✅

Keeping live games running

✅

✅

Internal studios versus external developers

➖

✅

Financial risk from unsold physical copies

➖

✅

✅ = directly explains this subject; ➖ = does not directly explain it.  

Which parts of Activision Blizzard's operating model each search type explains (same question, 30 retrieved passages per method)

The image below compares how the two methods retrieve information for the same question. Each grid shows 50 passages in rank order. Each dot is a passage, and its number and color identify its topic. The shaded rows are the top 30 passages, where we compare the methods. A reference fact is a point a complete answer should cover. A black ring marks the first passage in each ranking that supports a reference fact, and one passage can support more than one reference fact. Later passages may build on or repeat that information without another ring.

Top 50 passages retrieved for the Activision Blizzard question, by method. Each dot is one passage, ordered by rank from left to right, then down; its number and color show its topic. A saturated, ringed dot marks the first passage to support a reference fact, fully or partially. Pale dots add no new support and may be unassessed. In the top 30, topic graph spans 17 topics and supports all 7 reference facts; classic vector spans 5 topics and supports 5 reference facts.

In the top 30 passages, topic graph search covers 17 topics and all 7 reference facts, compared with 5 topics and 5 of 7 facts for classic vector search. The additional topics supply the missing pieces of the answer.

Beyond MMR: Diversifying with a corpus-wide topic map

Maximal Marginal Relevance (MMR) reduces repetition by selecting passages that balance relevance with lower similarity to one another. Topic graph search instead uses a corpus-wide topic map to guide selection across relevant subject areas. Because the topic groups are precomputed, it avoids MMR’s passage-to-passage comparisons.

We compared topic graph, MMR, and classic vector search using the same thematic queries and evidence budget across three datasets. Topic graph search achieved the highest evidence recall on all three datasets, outperforming MMR by 16.3 percentage points overall and classic vector search by 9.8 percentage points.

Evidence Recall between vector search, MMR and topic graph. Higher is better. Each row uses the same queries and evidence budget.

We compared classic vector search with topic graph search on thematic queries across six datasets. We measured evidence recall, which is the share of reference facts that the retrieved documents covered. Topic graph search improved evidence recall by 7.8 percentage points overall across all six datasets.

Evidence Recall 

Dataset

Classic Vector Search

Topic Graph Search

Delta 

Customer

62.4

75.6

+13.2

FDA

59.9

63.5

+3.6

DAYI

57.7

68.6

+10.9

Leukemia Papers

67.3

74.4

+7.2

MIML

77.5

81.7

+4.2

SEC

88.7

93.3

+4.6

Overall
(Query-weighted)

71.2

79.1

+7.8

Evidence recall for classic vector search and topic graph search on thematic queries across six datasets. Scores are percentages; deltas are topic graph search minus classic vector search in percentage points.

Evidence from more distinct documents

Classic vector search can concentrate its results on similar passages from a small set of documents. For thematic queries, more chunks from the same few documents add depth without covering more of the question. Topic graph search complements classic vector search by bringing evidence from different subject areas into the candidate pool.

As the chart shows, topic graph search retrieves 19.8%–43.1% more distinct source documents within the same evidence budget. This gives answer synthesis a broader set of sources.

Average distinct parent documents per query, at the same top-50 chunk cutoff.  Broader document reach is not itself proof of relevance

Putting the Three Retrieval Approaches Together

To see how the three approaches work together, consider an analyst using Microsoft's July 2024 and January 2025 earnings calls to ask how the company responded when AI demand outgrew its infrastructure.

RAG handles the directly stated facts: demand exceeded capacity, Azure growth was guided at 28–29%, and capital spending would rise. Knowledge graph entities then finds the matching capacity-constraint records across both calls. Knowledge graph topics summarizes how the story developed across later calls: relief timing slipped from June to December 2025, and capacity was allocated across the portfolio rather than to Azure alone.

Together, the three recovered 92% of the 24 required facts, compared with 64% for RAG alone, while using 52% fewer tokens.

Configuration

Mean Evidence Recall

Mean Facts Recovered (of 24)

Mean Facts gained

Mean Tokens used

RAG

64.2%

15.4

—

212,718

RAG + Entities

81.2%

19.5

+4.1

147,781

RAG + Entities + Topics

92.0%

22.1

+2.6

102,288

Choosing the Right Retrieval Approach

RAG remains the right starting point when the answer is stated directly in a passage. Add knowledge graph entities when a question depends on exact facts and how they are connected, such as counts, overlaps, or matching records. Add knowledge graph topics when the answer depends on evidence spread across many documents or reporting periods. The approaches are complementary, and knowledge graph topics can also generate the starting ontology required for knowledge graph entities.

The results above show where each helps. Entities answered 93.5% of questions correctly versus 26.1% for RAG, Topic graph search improved evidence recall over classic vector search by 7.8 percentage points overall, and in the Microsoft example the three together recovered 92% of the required facts versus 64% for RAG alone.

Appendix

Datasets

Customer

Customer-provided corporate and member-document collections in English, covering domains such as oil and gas corporate reports and health-insurance member documents.

FDA

FDA drug-label and clinical documents in English

DAYI

A Chinese medical QA corpus in Simplified Chinese

Leukemia Papers

Leukemia research papers used for questions that connect findings across studies.

MIML

A multi-industry, multi-language corporate document benchmark in English, French, and Simplified Chinese.  

SEC

SEC filings of US public companies in English

Metrics

Each question has annotated ground-truth nuggets: individual facts that together describe the information needed for a complete answer.

Metric

What it measures

Evidence Recall

Percentage of ground-truth nuggets that are covered by the documents retrieved.

Complete Evidence Recall

Percentage of queries where 100% of the ground-truth nuggets are covered by the documents retrieved.

Represented chunks (%)

Percentage of input text chunks represented in the topic hierarchy.

Evaluation Methodology

  1. Table 1: We compared a knowledge base with a single content only index (using hybrid search and semantic ranker) against a knowledge base using an entity knowledge source. GPT-5.6-Luna was used for graph fact extraction, NL2GQL translation, and answer synthesis.
  2. Table 2: We used knowledge graph topics and an agent skill to generate a “starting point” ontology. For the “first refinement”, we prompted the agent to refine the starting point ontology so that it could represent and answer common questions asked about SEC filings. For the “final ontology”, we prompted the agent to iterate on the ontology to improve the structure, find missing entities and relationships, and make improvements to formalize the ontology. We then generated five graphs using each ontology and measured the total number of raw facts extracted, and then using an LLM, determined the number of distinct facts (using both semantic similarity and looking up and cross-referencing source text).
  3. Table 3: We measured chunk representation and topic counts across each existing graph. For label quality, we uniformly sampled 60 topics per corpus build (420 topics across the seven displayed rows) and regenerated their titles and descriptions with GPT-5.6 Luna. The generator uses medium reasoning in the default table and no reasoning in the alternative table; results are reported separately. A separate GPT-5-mini judge, using medium reasoning, assessed anonymous candidate labels against bounded source excerpts.
  4. Table 5: We compare classic vector search with topic graph search. The topic graph search pipeline used the search API with balanced topic retrieval to enable topic-guided retrieval with reranking enabled. Selected passages were packed into a 10,000-token evidence budget and passed to GPT 5.6 Luna for answer synthesis. Reported scores are averaged over three runs. Deltas are expressed in percentage points, calculated as topic graph search minus classic vector search.

 

 

 

 

Updated Nov 18, 2026
Version 2.0
No CommentsBe the first to comment