Microsoft Developer Community Blog

4 MIN READ

Building Knowledge-Grounded AI Agents with Foundry IQ

Microsoft

Mar 19, 2026

Foundry IQ now integrates with Foundry Agent Service via MCP (Model Context Protocol), enabling developers to build AI agents grounded in enterprise knowledge.

This integration combines Foundry IQ’s intelligent retrieval capabilities with Foundry Agent Service’s orchestration, enabling agents to retrieve and reason over enterprise data.

Key capabilities include:

Auto-chunking of documents
Vector embedding generation
Permission-aware retrieval
Semantic reranking
Citation-backed responses

Together, these capabilities allow AI agents to retrieve enterprise knowledge and generate responses that are accurate, traceable, and aligned with organizational permissions.

Why Use Foundry IQ with Foundry Agent Service?

Intelligent Retrieval

Foundry IQ extends beyond traditional vector search by introducing:

LLM-powered query decomposition
Parallel retrieval across multiple sources
Semantic reranking of results

This enables agents to retrieve the most relevant enterprise knowledge even for complex queries.

Permission-Aware Retrieval

Agents only access content users are authorized to see.

Access control lists from sources such as:

SharePoint
OneLake
Azure Blob Storage

are automatically synchronized and enforced at query time.

Auto-Managed Indexing

Foundry IQ automatically manages:

Document chunking
Vector embedding generation
Indexing

This eliminates the need to manually build and maintain complex ingestion pipelines.

The Three Pillars of Foundry IQ

1. Knowledge Sources

Foundry IQ connects to enterprise data wherever it lives — SharePoint, Azure Blob Storage, OneLake, and more.

When you add a knowledge source:

Auto-chunking — Documents are automatically split into optimal segments
Auto-embedding — Vector embeddings are generated without manual pipelines
Auto-ACL sync — Access permissions are synchronized from supported sources (SharePoint, OneLake)
Auto-Purview integration — Sensitivity labels are respected from supported sources2. Knowledge Bases

2. Knowledge Bases

A Knowledge Base unifies multiple sources into a single queryable index. Multiple agents can share the same knowledge base, ensuring consistent answers across your organization

3. Agentic Retrieval

Agentic retrieval is an LLM-assisted retrieval pipeline that:

Decomposes complex questions into subqueries
Executes searches in parallel across sources
Applies semantic reranking
Returns a unified response with citations

Agent → MCP Tool Call → Knowledge Base → Grounded Response with Citations

The retrievalReasoningEffort parameter controls LLM processing:

minimal — Fast queries
low — Balanced reasoning
medium — Complex multi-part questions

Project Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                    FOUNDRY AGENT SERVICE                            │
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────────────────┐ │
│  │   Agent     │───▶│ MCP Tool    │───▶│  Project Connection     │ │
│  │ (gpt-4.1)   │    │ (knowledge_ │    │  (RemoteTool + MI Auth) │ │
│  └─────────────┘    │ base_retrieve)   └─────────────────────────┘ │
└─────────────────────────────│───────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────────────┐
│                    FOUNDRY IQ (Azure AI Search)                     │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │  MCP Endpoint:                                               │   │
│  │  /knowledgebases/{kb-name}/mcp?api-version=2025-11-01-preview│   │
│  └─────────────────────────────────────────────────────────────┘   │
│                              │                                      │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────┐ │
│  │ Knowledge       │  │ Knowledge       │  │ Indexed Documents   │ │
│  │ Sources         │──│ Base            │──│ (auto-chunked,      │ │
│  │ (Blob, SP, etc) │  │ (unified index) │  │  auto-embedded)     │ │
│  └─────────────────┘  └─────────────────┘  └─────────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘

Prerequisites

Enable RBAC on Azure AI Search

az search service update --name your-search --resource-group your-rg \ --auth-options aadOrApiKey

Assign Role to Project's Managed Identity

az role assignment create --assignee $PROJECT_MI \ --role "Search Index Data Reader" \ --scope "/subscriptions/.../Microsoft.Search/searchServices/{search}"

Install Dependencies

pip install azure-ai-projects>=2.0.0b4 azure-identity python-dotenv requests

Connecting a Knowledge Base to an Agent

The integration requires three steps.

Connect Knowledge Base to Agent via MCP

The integration requires three steps:

Create a project connection — Links your AI Foundry project to the knowledge base using ProjectManagedIdentity authentication
Create an agent with MCPTool — The agent uses knowledge_base_retrieve to query the knowledge base
Chat with the agent — Use the OpenAI client to have grounded conversations

Step 1: Create Project Connection

import requests
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

credential = DefaultAzureCredential()
PROJECT_RESOURCE_ID = "/subscriptions/.../providers/Microsoft.CognitiveServices/accounts/.../projects/..."
MCP_ENDPOINT = "https://{search}.search.windows.net/knowledgebases/{kb}/mcp?api-version=2025-11-01-preview"

def create_project_connection():
    """Create MCP connection to knowledge base."""
    bearer = get_bearer_token_provider(credential, "https://management.azure.com/.default")
    
    response = requests.put(
        f"https://management.azure.com{PROJECT_RESOURCE_ID}/connections/kb-connection?api-version=2025-10-01-preview",
        headers={"Authorization": f"Bearer {bearer()}"},
        json={
            "name": "kb-connection",
            "properties": {
                "authType": "ProjectManagedIdentity",
                "category": "RemoteTool",
                "target": MCP_ENDPOINT,
                "isSharedToAll": True,
                "audience": "https://search.azure.com/",
                "metadata": {"ApiType": "Azure"}
            }
        }
    )
    response.raise_for_status()

Step 2: Create Agent with MCP Tool

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition, MCPTool

def create_agent():
    client = AIProjectClient(endpoint=PROJECT_ENDPOINT, credential=credential)
    
    # MCP tool connects agent to knowledge base
    mcp_kb_tool = MCPTool(
        server_label="knowledge-base",
        server_url=MCP_ENDPOINT,
        require_approval="never",
        allowed_tools=["knowledge_base_retrieve"],
        project_connection_id="kb-connection"
    )
    
    # Create agent with knowledge base tool
    agent = client.agents.create_version(
        agent_name="enterprise-assistant",
        definition=PromptAgentDefinition(
            model="gpt-4.1",
            instructions="""You MUST use the knowledge_base_retrieve tool for every question.
Include citations from sources.""",
            tools=[mcp_kb_tool]
        )
    )
    
    return agent, client

Step 3: Chat with the Agent

def chat(agent, client):
    openai_client = client.get_openai_client()
    conversation = openai_client.conversations.create()
    
    while True:
        question = input("You: ").strip()
        if question.lower() == "quit":
            break
        
        response = openai_client.responses.create(
            conversation=conversation.id,
            input=question,
            extra_body={
                "agent_reference": {
                    "name": agent.name,
                    "type": "agent_reference"
                }
            }
        )
        
        print(f"Assistant: {response.output_text}")