1 million documents to 300+agents: Building an enterprise-scale Microsoft 365 Copilot connector

1 million documents. 100,000+ monthly users. 300+ Copilot Studio agents. Here's what we learned building the Microsoft 365 Copilot connector behind them.

Most enterprise knowledge lives in portals and repositories that people trust but rarely visit in the flow of work. That's the gap this connector was built to close.

At Microsoft, we built an enterprise-scale Microsoft 365 Copilot connector to make internal documentation available as grounded context across supported Copilot experiences. The connector runs in production today, serving teams across engineering, support, and field roles.

Who this post is for: platform teams, developers, and connector owners who are past prototype and thinking about production. We cover the design choices, security and governance considerations, schema decisions, operational patterns, and retrieval-quality practices that helped us make enterprise knowledge easier to discover and reuse through Microsoft 365 Copilot.

Understand why a Microsoft 365 Copilot connector matters

New to Microsoft 365 Copilot connectors? Start with the Microsoft 365 Copilot connectors overview and the Copilot connectors API documentation.

The real power of a Microsoft 365 Copilot connector isn't the ingestion — it's what becomes possible once your content is in the platform.

The moment your connector is live and configured for supported Microsoft 365 Copilot experiences, your content can become available as grounded context in those experiences. That means:

• Copilot Chat — users ask questions in natural language and get answers grounded in your content, alongside emails, files, and Teams messages. No portal visit required.
• Copilot in Microsoft 365 apps — in Copilot for Word, PowerPoint, and Excel (where supported), your knowledge can appear as context when users draft, summarize, or build presentations.
• Copilot Studio agents — users with appropriate permissions and licensing can build custom agents scoped to your connector's content. In our case, 300+ agents were created by teams within Microsoft — incident response assistants, onboarding copilots, domain-specific Q&A bots — all without additional work from us.
• Copilot CLI and developer tools — Copilot CLI is the command-line experience for developers working in a terminal. Engineers get answers in their terminal or IDE, grounded in the same authoritative content.
• The Copilot Retrieval API — teams with appropriate access can programmatically query the indexed knowledge to build custom experiences we never anticipated.

The key insight is that a connector is not just an ingestion pipeline. When you invest in clean schema, reliable ingestion, and the right access controls, supported Copilot experiences can reuse that content without requiring a separate integration for every surface.

A quick note on terminology. This post describes a Microsoft 365 Copilot sync connector (previously known as a Graph connector). Sync connectors ingest and index your content into the Microsoft 365 substrate, which is different from federated connectors that query content in place.

Plan for production gaps

The public documentation gets you from zero to a working connector. Getting from a working connector to a production one uncovered four gaps we hadn't planned for: tenant configuration, security and compliance readiness, discoverability, and shared throttling limits.

Validate tenant configuration early

The documented path starts with creating a connector in your tenant and ingesting data. In our environment, we had to validate architecture before full production, and hit a few things that weren't in the docs. Watch out for:

• Cross-tenant data flow. There is no supported model for cross-tenant data flow between Entra ID tenants today. We resolved this by consolidating connector operations into a single tenant.
• "Draft" state limbo. Connectors can get stuck in "Draft" state despite correct permissions. Escalating through the platform team was the fastest unblock.
• Ownership gaps. Multiple platform teams have overlapping responsibilities and no single owner. Identify your escalation path before you need it.
• Dev tenant lifecycles. Dev tenants may have undocumented auto-deletion policies. Don't rely on a dev tenant as a long-lived staging environment.

Start security and compliance early

Registering an app and configuring permissions is the easy part. For a cross-tenant connector at enterprise scale, we also needed:

• A threat model.
• A privacy review.
• A Responsible AI review.
• Design reviews from platform teams.
• Security exception approvals.

No template existed. No checklist. No predefined path. This was a significant effort that started well before we wrote code.

Make connector content discoverable

Even with successful ingestion, Copilot may not prioritize your content. Emails and user files rank higher by default. Connector content ranking varies across clients. There's no visibility into why a result was or wasn't surfaced.UserPrincipal support — the ability to attach a user identity to an indexed item so it's ranked with the same signals as that user's emails and files — is an important platform improvement for us. We plan to adopt it in the near future, with the goal of making connector content show up more reliably alongside a user's other work in Copilot results.

Design for throttling and shared limits

At approximately 1 million documents, throttling became an important design consideration. We handled it with retry logic, exponential back-off, monitoring for 429 responses, and operational alerts so we could detect ingestion slowdowns quickly.

The Microsoft 365 Copilot connectors platform has a global rate limit shared across all connectors in a tenant. If another connector spikes ingestion, yours gets throttled — and the 429 doesn't tell you why or who's consuming the budget. Today, there's no visibility into the shared quota, no priority system, and no way to reserve capacity, so it's worth designing your pipeline to tolerate unexpected back-pressure.

Build the connector for scale

Our architecture focused on one principle: keep the source content separate from the ingestion compute. That choice made the pipeline easier to retry, easier to monitor, and easier to evolve as the documentation platform changed.

Separate data from compute

A core design principle was keeping the data layer completely separate from the ingestion compute. The documentation platform stores content in Azure Blob Storage — that's the source of truth. The ingestion pipeline never owns or duplicates that data. Instead, it reacts to changes and transforms on the fly.

• Blob Change Feed — Azure Blob Storage's change feed emits events whenever documents are created, updated, or deleted. This is the trigger for all ingestion work — no polling, no scheduled full crawls.
• Azure Function (event-driven) — an Azure Function listens to the change feed and processes each event. It reads the blob, transforms the content, enriches metadata, and pushes the item to the Microsoft Graph connectors API. The function is stateless — all state lives in the storage layer and the Microsoft Graph platform.
• Storage as the boundary — the function has read access to blob storage but never writes back. Data flows one direction: storage → function → Graph API. This keeps the pipeline simple to reason about and safe to retry — reprocessing a blob event is always idempotent.
• Graph API ingestion — items are pushed with retry logic, back-off on 429s, and observability via Application Insights.

This separation means:

• The documentation platform can evolve its storage independently — new formats, new metadata, reorganizations — without touching the ingestion pipeline.
• The pipeline scales horizontally via Azure Functions consumption plan — spikes in content updates don't require capacity planning.
• Failures are isolated — a bad blob doesn't block other ingestion, and the change feed provides natural retry semantics.

Iterate on schema design

We treated schema design as the highest-leverage decision. Through four iterations, we significantly improved retrieval quality — entirely through schema and metadata refinement:

• Mapped document types (TSGs, runbooks, architecture docs, onboarding guides) to distinct property sets.
• Wrote detailed connection descriptions optimized for Copilot's ranking.
• Added custom properties for freshness and ownership signals.
• Tested with real user queries, not synthetic data.

Use secure authentication and deployment patterns

A few patterns kept the deployment story simple across environments:

• Managed Identity for service-to-service communication in production.
• Infrastructure-as-code for consistent deployment across commercial and sovereign clouds.
• Environment-specific configuration abstracted from the start.

Measure retrieval quality

One benefit of onboarding to the Microsoft 365 Copilot platform was access to the Search Evaluation framework used by the Copilot team. This helped us measure retrieval quality, including precision and recall against real user queries, without building evaluation infrastructure from scratch. We used this framework to test pipeline changes and catch regressions before they reached users.

Monitor connector health

A healthy connector is a prerequisite for everything else, so we built:

• Alerting on ingestion failures and throttling spikes.
• Recovery automation for re-ingestion scenarios.

See the impact

Once the connector was live, three things stood out: the portal became a platform, an ecosystem of agents formed on top of it, and we kept the controls we needed while gaining new ways to measure quality.

From portal to platform

The documentation platform moved from being a destination people visited to a knowledge source that powers AI experiences across Microsoft. Users no longer need to navigate to the portal — knowledge surfaces directly in Copilot Chat, Copilot in Word, Copilot in VS Code, and Azure Copilot. Consumption is ambient and contextual, embedded in existing workflows.

An ecosystem of agents

The core value proposition of the Microsoft 365 Copilot platform is this: push your content once, and it's enabled across multiple AI surfaces. That's exactly what happened.

300+ Copilot Studio agents now build on the connector. Content surfaces in Copilot Chat, Copilot CLI, Work IQ (Microsoft's workplace intelligence experience), and other AI-powered developer tools — all from a single ingestion pipeline. Teams within Microsoft created:

• Incident response assistants that help surface relevant troubleshooting guides.
• Onboarding copilots that pull architecture docs contextually.
• Domain-specific Q&A agents scoped to their team's content.
• Support augmentation workflows with human oversight.

Because we invested in getting the connector right once — clean schema, proper access controls, reliable ingestion — supported Copilot experiences reuse that content automatically. None of the 300+ agents required additional integration work from our team.

Measurable quality and preserved governance

Alongside the new capabilities, the shift preserved what mattered most: existing controls and a way to measure what we were doing.

• Programmatic access at scale. The Copilot Retrieval API lets teams with appropriate access query the indexed knowledge programmatically, without the documentation team's involvement. Single ingestion, multi-surface consumption.
• Governance stayed centralized. Despite AI integration, the connector continues to rely on existing Microsoft 365 security and access controls. Governance policies and access restrictions remain centrally managed. This was critical for sovereign cloud deployment and regulated environments.
• Quality became measurable. With evaluation frameworks in place, we can quantify answer quality and continuously improve — something that wasn't possible when the platform was purely a static portal.

Apply these recommendations when building connectors

If you are planning a Microsoft 365 Copilot connector, these are the practices we would prioritize before moving from prototype to production:

• Start security and compliance conversations before writing code — budget significantly more time than you expect.
• Treat schema design as your highest-leverage decision — test with real Copilot queries early and iterate aggressively.
• Build evaluation from day one — you need to measure whether Copilot is actually using your content effectively.
• Build health monitoring for the connector itself — not just the pipeline.
• Design for the ecosystem — if you build it well, people will build on it in ways you didn't anticipate.

Make the investment worthwhile

The public docs for Microsoft 365 Copilot connectors cover the "what." This post covers the "how" — and more importantly, the "why it's worth it."

Security approvals took months. Schema iteration took four rounds. Operational readiness took ownership none of us had budgeted for. But the payoff is real: your organization's knowledge becomes accessible through AI, grounded in authoritative content and available across supported Microsoft 365 Copilot experiences.

The platform is powerful. Getting there takes work. In our first year of production, 300+ agents were built on the connector — none of which required additional integration work from our team.

Get started

• Microsoft 365 Copilot connectors overview
• Work with the Copilot connectors API
• Create and manage connections
• Register and manage schema
• Add Copilot connectors as a knowledge source in Copilot Studio
• Microsoft Graph throttling guidance
• Azure Blob Storage change feed

Join the conversation

Have you built a Microsoft 365 Copilot connector at scale? We'd love to hear what you learned — share your experience in the comments below, and let us know which patterns worked (or didn't) in your environment.

Did you know? The Microsoft 365 Roadmap is where you can track upcoming updates across Microsoft 365 apps and services. Microsoft 365 Copilot release notes are where you can review generally available Copilot features by platform. Check both resources regularly for the latest status, and note that roadmap dates are tentative and subject to change.