Adopting Copilot: Improving Response Quality via Connectors

Internally, Microsoft has been adopting Copilot across every person and every function. Our goal is to learn from our experiences to improve the product for everyone. Being a very large global enterprise, Microsoft has lots of content spread across thousands of applications and tools. Yet every Copilot user expects Copilot to be accessing all of the content so that responses are complete and accurate. How can we enable Copilot to access all that content?

Connectors

Microsoft has provided Copilot Connectors and Power Platform Connectors. With the adoption of MCP, it seems likely further capabilities will be provided as well. Internally, Microsoft has been implementing both Connector options. My team is responsible for configuring and maintaining the Copilot Connectors, and I work with the team supporting Power Platform Connectors.

Following initial trials in 2024, we began scaling out in 2025. At this point, we have ~100 Copilot Connections and 40 Power Platform Connectors; delivering content which is included in 15% of all copilot responses, >1.8M citations per month.

Key Concept: Copilot response success is based on successful RAG

As the picture above shows, most Copilot responses are based on augmentation (RAG). That augmentation comes from doing searches across all the available content and bringing the right results back to Copilot for processing – “grounding”. Successful grounding is based on 3 interdependent pillars - good relevance, good content quality, and good completeness.

• Relevance is handled within the product. There’s not much that an admin can do about it.
• Content Quality is at the control of everyone in the company plus the management policies adopted. That’s worth a separate discussion.
• Completeness consists of including all the content that users expect. Not all the content that exists, but the content they expect to be there. This realization had a big impact on the approach we took.

Our Approach

How to identify the content users expect to be available? This is a company of >300,000, across every discipline, around the globe. Our AAD team provided a report of all applications and the authentication count for each of them over 90 days. From that, it’s possible to identify the most popular apps.

Some of those are included in Copilot out of the box (SharePoint, Outlook, Teams, Power BI). Others are not (ADO, Dynamics, and all the other unlabeled slices). Those which were not included became targets for connections.

• For ADO, Microsoft has a published Copilot Connector.
• For Dynamics, Microsoft published Power Platform Connectors.
• For many of the other slices, they are third parties or internal apps. We are using published Copilot Connectors where they exist and creating custom connectors (using the SDK) where they don’t.

In addition to using authentication volume to determine which content to pursue, and in what priority, we also looked at common business functions that would benefit everyone in the company. For instance, obtaining technical support (Help Desk) and enabling product engineering (core to Microsoft’s business). The result is a continually evolving prioritized list of content to pursue.

Execution

This post will focus on Copilot Connections, not Power Platform Connectors.

For each target content, it’s easy to research whether a Copilot Connection exists via Learn.Microsoft.com. Where they exist, we then evaluate whether the connector would meet our security, privacy, and other compliance requirements. If that was positive, we then evaluate the content itself.

1. Is the content of interest to a significant portion of the company? There are limits to how much content can be connected in and there is ongoing maintenance overhead. So let’s make sure the content is worthwhile.
2. Is it the authoritative source for this information? For instance, don’t connect in a marketing demo instance that could result in fake data appearing as authentic in responses.
3. Is the content owner willing to have the content available in Copilot?
4. Is the source content managed such that:
   • Obsolete content is removed reasonably quickly.
   • There is a feedback mechanism for reporting erroneous content and that content is updated quickly.
   • Permissions to individual entries in the source content are managed so that only appropriate employees have access. And access changes are implemented promptly.
5. If the content is coming from a third party application and if any content from that application is confidential or personal, is the third-party app/vendor approved for confidential/personal information?

If the evaluation is positive, then configure the connection, in partnership with the content owner.

• The first time configuring any type of connector has always been a challenge. After the first one, it gets much easier.
• When initially creating a connection, always limit visibility to just a few people for validation. After they validate the content, then make the connection generally available.
• We then add the connection to our published list. This list is available to everyone in the company, supporting agent creators everywhere.

Maintenance

My team is responsible for configuring new connections, maintaining existing ones, and taking questions from agent creators and content owners around the company. We expect our connection inventory to continue growing.

• We’ve published a site with information about connectors, showing the ones currently available, and providing a request process for additional ones.
• We’ve received nearly 200 requests so far this calendar year.
• We receive 30-40 questions from agent creators each month.

Operationally, we check the health of each connection daily. Highly recommend opening each connection and looking at the full details, don’t just look at the summary screen - we’ve discovered some failure cases which are not shown on the summary view. This feedback has been given to the product team for future improvements in reporting.

We were able to get some custom reporting showing usage by connection. Such reporting will be delivered in the product, though I don’t know a date. This has been very useful, as those connections which aren’t getting meaningful use can be identified and removed, reducing ongoing maintenance.

Results

As mentioned at the beginning, 15% of all Copilot responses now use connected content. That percentage has grown steadily as we have been adding more connections. Additionally, connections are now used in hundreds of Copilot Agents around the company.

Interesting insight has come from looking at the usage of these connectors.

• It was expected, given the heavy use of Azure DevOps inside Microsoft, that it would receive significant usage. Similarly, Service Now powers our Help Desk and other functions, so expected it to have significant use.
• Intranet usage is surprising. These connections are various Microsoft public Internet sites (included as a result of user requests). The result is very strong usage. Leading to a recommendation to consider connecting sites that have strong business value to your company, even though a simple Internet search might obtain the same information.
• The Non-MS category encompasses the custom connections we’ve built. It’s likely every enterprise has some high-value internal content. Building a custom connector is not hard, so recommend doing so.