A confident wrong answer costs more than no answer at all. Enterprise agents cross that line the moment they guess instead of grounding in your real data. This post shows how Foundry IQ turns GPT-RAG into an agent that answers only from what your business actually knows, with citations you can check.
Enterprise AI has a trust problem, and it is not about fluency. A model will answer almost any question with confidence, whether or not it actually knows. The moment a legal or support team catches it inventing a policy that does not exist, they stop trusting it for anything that matters. What makes an enterprise agent useful is not how well it writes. It is whether every answer comes from your own knowledge and shows where it came from.
That shifts the hard part of the work from the model to the grounding, and the industry now treats it that way. The trajectory is clear: by 2030, universal semantic layers will be treated as critical infrastructure, alongside data platforms and cybersecurity. The shared, governed layer of meaning that lets an agent answer in your business terms is moving from nice-to-have to non-negotiable foundation. The differentiator is no longer the base model. It is how well the system is grounded in what only your organization knows, and how tightly that grounding is governed.
Why Ungrounded AI Fails Enterprises
A raw model answers from a fixed snapshot of public text. That is fine for general knowledge and useless for your policies, your contracts, and your product specifics, none of which were in its training data. Ask it something particular to your business and it still answers, just from the nearest pattern it has seen. That is how you get a confident paragraph citing a policy you never wrote.
The failure has three parts, and they compound. Coverage: the model does not have your internal knowledge, so instead of admitting the gap it answers from its training and hallucinates, a fluent, confident paragraph that happens to be wrong. Traceability: even when it happens to be right, nothing points to a source, so no one can verify it and no auditor will accept it. Permissions: a system that pools every document into one searchable pile will surface a salary letter or an unreleased contract to whoever asks, because the model has no idea who is on the other side.
Grounding fixes all three at once, and it is more than pasting documents into a prompt. Done properly it means retrieving the right passages from your live corpus, answering only from them, showing the citation, and enforcing each user's permissions at the moment of retrieval.
What Grounding Actually Takes
Fixing all three takes a dedicated knowledge layer, the part of the system that retrieves the right passages from your content and hands the model only what it can cite. GPT-RAG is an open-source solution accelerator you deploy in your own Azure subscription, and its knowledge layer is Microsoft Foundry IQ. When a user asks a question, the agent does not recall an answer from training. It runs agentic retrieval over your own content.
Agentic retrieval breaks the question into parts, searches your sources in parallel, reranks the candidates for relevance, and answers only from the passages that come back, each carrying a citation. That is what separates it from a basic search-and-stuff pipeline, and it is what makes an answer both grounded and verifiable.
One question becomes focused subqueries, runs in parallel across your sources, and is reranked so the model answers only from the top cited passages
A Concrete Example: One Question, One Cited Answer
Take the question straight from the demo. A signed-in user opens the agent and asks: “In which model year did VW introduce the electronic fuel injection system?”
A large model might guess this correctly from something it read on the public web, and that guess is exactly the dependency we want to remove. In the demo the answer lives in a specific engineering document we control, and the whole point is that the agent answers from that document rather than from whatever the model happens to remember. So instead of guessing, it retrieves the relevant passage and composes a short, direct answer: VW introduced the system on its 1975 models. Next to the answer sits a blue citation, vw-fuel-system.pdf, the exact source the answer was built from.
Two things happened there that a plain search box cannot do. The answer is grounded, so it comes from your document rather than the model's memory, and it is cited, so the reader can open the source document behind it and confirm it. It is also permission-aware: retrieval ran as that specific user, so the document was only in play because they were allowed to see it. Change the user and the same question can return a different set of sources, or none, with no special prompting.
One question, answered end to end, grounded in the source and shown with its citation.
What Foundry IQ Actually Does
Foundry IQ is the knowledge layer that sits behind your agents. Instead of every application building and maintaining its own retrieval stack, the chunking, the indexing, and a separate connector for each source, you define a Knowledge Base once and reuse it. It runs on Azure AI Search, and Microsoft ships it as a first-party, SLA-backed retrieval service, so you inherit a supported engine rather than hand-rolled plumbing. As of Build 2026, Foundry IQ knowledge bases are generally available.
A Knowledge Base connects one or more knowledge sources, and those sources come in two flavors. Indexed sources are your content pre-processed into a searchable index. Point the Knowledge Base at your corpus, the files in Blob Storage and the content indexed in AI Search, and that corpus becomes the ground truth the agent answers from. Remote sources are live systems the Knowledge Base queries at retrieval time instead of indexing up front. Because sources are configuration and not code, extending what the agent knows is a matter of connecting another source, not rebuilding the app.
Those remote sources go beyond documents. The same Knowledge Base can connect a Fabric ontology that captures how your business entities relate, Fabric data agents that answer from live systems, and Work IQ for a person's Microsoft 365 context, so structured, up-to-the-minute signals ground the agent alongside your indexed files.
When you are ready to wire this up yourself, GPT-RAG's grounding sources overview is the operator guide. It walks through each supported source kind and the concrete steps to enable it on your Knowledge Base.
The intelligence is in how it retrieves. Basic retrieval throws your raw question at a single index and hopes the top hits are relevant. Agentic retrieval treats the question as a task: it decomposes a complex query, searches the sources in parallel, reranks the candidates by relevance, and hands the model a focused, well-ordered set of passages to work from. Microsoft continues to ship quality improvements to this path, and it is the layer that turns “the model said so” into “here is the source.”
Permissions are enforced at query time. The agent forwards the signed-in user's identity to retrieval as an on-behalf-of token, so the Knowledge Base trims results to what that person is allowed to see before the model receives anything. This is document-level security enforced on the retrieval side, not a filter the model could quietly skip. Two people can ask the identical question and get answers built from two correctly scoped views of the same corpus.
Beyond Documents: Where Work IQ and Fabric IQ Fit
Foundry IQ is one member of a family that Microsoft groups under Microsoft IQ, its intelligence layer for connecting knowledge to agents. Three grounding domains matter for an enterprise agent, because they answer three different kinds of question. The split is what separates an agent that knows “what is our refund policy” from one that also knows “what is on my calendar” and “what did we sell last quarter.”
Foundry IQ grounds on your authoritative content: the policies, product documentation, and reference material you curate into a corpus. It is the shared organizational source of truth, and it is what the demo above runs on.
Work IQ grounds on how your people actually work. It reaches into Microsoft 365, the mail, meetings, chats, files, and the org graph, to answer questions about a person's own work, like “what meetings do I have about Project Falcon” or “who owns the billing integration.” Every result is trimmed to what that signed-in user is already permitted to see.
Fabric IQ grounds on the state of the business. It sits over governed analytics in Microsoft Fabric and OneLake and answers questions about your numbers and your business entities, like “what were Q3 sales by region” or “how is Contoso related to Shipment 8842.” The answer comes back tied to a live semantic model, not a spreadsheet someone exported last month.
These are complementary, not competing, and you have more than one way to bring them into your agent. The choice comes down to how tightly a domain needs to sit inside your Knowledge Base retrieval.
Reach for the Foundry IQ path when you want one grounded answer that fuses everything at once. Along with your documents, a Knowledge Base can connect a Fabric ontology, Fabric data agents, and Work IQ as additional knowledge sources. The agent then makes a single retrieval call across content, structured business context, live systems, and a person's Microsoft 365 work, all with the same citations and on-behalf-of security. This is the right default when documents are the anchor and the other signals should co-retrieve with them.
Reach for Fabric IQ as a tool call when analytics is a separate question, not part of the same retrieval. Fabric IQ is a first-party grounding tool inside the Microsoft Foundry Agent Service. The agent decides to invoke it when the user asks something quantitative, and it queries governed analytics in OneLake alongside what the Knowledge Base returns. Use this when the analytics answer stands on its own and shouldn't be blended into every document lookup.
Reach for Work IQ via Agent-to-Agent (A2A) when you want to delegate a task rather than pull raw context. The agent hands off a natural-language request like “summarize my recent emails about Project Contoso,” and Work IQ handles retrieval, reasoning, and response synthesis against the user's Microsoft 365 data on their behalf. Use this when Work IQ's own reasoning over M365 is the answer you want, not just its retrieved snippets.
Foundry IQ stays the central knowledge layer for your content. Fabric IQ and Work IQ (both in public preview) extend that with analytics and workplace context when a use case calls for it, and you can still point Foundry IQ at a single source when that is all you need.
The Azure Services Doing the Work
Underneath the interface, GPT-RAG is the orchestrator. It owns the whole flow: it authenticates the user, forwards their identity to retrieval, calls the Foundry IQ Knowledge Base, and shapes the grounded, cited answer that comes back. It leans on a few Azure services to do this, and one of them does most of the retrieval work.
Azure AI Search hosts the Foundry IQ Knowledge Base. It holds the connection to your sources, runs agentic retrieval, enforces per-user permissions on the query, and returns passages with the metadata that becomes citations. The grounding story rests on this component.
The orchestrator is the coordinator. It takes the user's question and identity, acquires the on-behalf-of token, and calls the retrieve API. The passages that come back become the only material the model is allowed to answer from, and each one carries a source label that turns into a clickable citation. It then calls a chat model through a Microsoft Foundry endpoint to compose the final answer from those passages and nothing else.
Microsoft Entra ID is what makes the security real rather than cosmetic. People sign in, and their identity is what flows through the on-behalf-of token into retrieval, so document-level trimming runs against a real principal. The application itself runs on Azure Container Apps, with its dependencies reached over the network rather than exposed to the public internet.
One retrieval path: GPT-RAG on Container Apps, the Foundry IQ Knowledge Base on Azure AI Search, and the IQ knowledge sources it grounds onBuilt for the Enterprise
A grounded answer is worth little if the system holding your most sensitive content is not one your security team will approve. That is where most demos quietly stop.
GPT-RAG is built to deploy on the Azure AI Landing Zone, Microsoft's reference architecture for AI workloads under a Zero Trust posture.
Every dependency, from AI Search and Microsoft Foundry to storage, is reached through a private endpoint inside a virtual network. Public traffic enters through a gateway with a web application firewall, and outbound traffic is forced through a firewall. Nothing talks to the public internet unless you deliberately allow it.
Pair that network isolation with document-level security at retrieval and you get the property enterprises actually need: the perimeter is locked down, and even inside it, every answer still respects the permissions your files already carry.
Try It Yourself
Getting a grounded agent running is a single azd up. The command provisions the Azure resources, deploys the orchestrator and the UI, and hands you a working agent. Point the Knowledge Base at your own documents, sign in, and ask your first question. For production, deploy GPT-RAG on the Azure AI Landing Zone for the network-isolated topology. The deployment guide has the preflight checks and the step-by-step, from a basic deployment to the network-isolated one.
From a Cited Answer to a Trusted Agent
The payoff is not a single impressive answer. It is what changes when trust stops depending on the model's confidence and starts depending on evidence. When every answer arrives grounded in your content, cited to its source, and scoped to the person asking, the agent moves out of the demo category and into the small set of systems a compliance team can approve. Foundry IQ makes that shift possible, and Work IQ and Fabric IQ extend it from your documents to your work and your numbers.
Deploy GPT-RAG, point it at your corpus, and ask it something only your data can answer.
References
Microsoft