Forum Widgets
Latest Discussions
Custom Engine CoPilot with CoPilot Studio
Hi Experts, Need your input on this topic please. Microsoft at multiple presenations, events and documentation mentions Custom Engine CoPilot can be developed using CoPilot Studio (Low Code Approach) like here https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/overview-custom-engine-agent#low-code-approach As per documentation, "Custom Engine CoPilot" can have our own orachastrator (any LLM apart from M365 built in LLM), Can you help me to understand how the "Custom Engine CoPilot" can be built using "CoPIlot Studio" with our own LLM? Thanks in advanceFile Formats to upload too restricted
Hello, I want to upload an .svg file but it won't let me. I am trying to use an .svg in a PowerPoint but it only points to my uploaded file, I can't upload it for my device. Then it tells me it is unsupported. File formats supported by Microsoft 365 Copilot - Microsoft Support Help?Penetration Test Asks Questions About Copilot Access to SharePoint Online
An article by a company specializing in penetration tests raised some questions about how attackers might use Copilot for Microsoft 365 to retrieve data. The article is an interesting read and reveals how Copilot can reveal data in password protected Excel worksheets. However, many of the issues raised can be controlled by applying available controls, and the biggest worry is lhow the account being used to run Copilot came to be compromised! https://office365itpros.com/2025/05/12/copilot-for-microsoft-365-pentest/How to Enhance Copilot Usage Data
Copilot usage data can be pretty sparse, but it’s easy to enhance the data to gain extra insight into how Microsoft 365 Copilot is used within a tenant. In this case, an administrator wanted to have department and job title information available for each Copilot license holder, so we combined the Copilot usage data with details of Entra ID user accounts with Copilot licenses to create the desired report. https://office365itpros.com/2025/05/09/copilot-usage-data-accounts/
Microsoft Copilot Embedded in the SharePoint
Hi experts, I have developed a Copilot Agent to assist to search the documents in the SharePoint document libraries. In that case I have registered the App in Azure and set the API Permission as follows. Microsoft Graph -> Application Permission 1.1. Sites.FullControl.All 1.2. Files.Read.All 1.3. Sites.Read.All 1.4. User.Read.All 2. SharePoint -> Application Permission 2.1. Sites.FullControl.All 2.2. Sites.Read.All 2.3. User.Read.All 3. Created Client Secret 4. I did the following settings in Copilot Studio under Authentication. 4.1. I have selected "Authenticate manually" and turn "Require users to sign in" off. 4.2. Then, gave the Client ID and Client secret. 4.3. Set the "profile openid Sites.Read.All Files.Read.All" under Scopes and Save. 4.4. After all the configurations are done, I get the embed code and insert it into SharePoint page. My objective is to give the better user experience to the internal users, and I do not need to use this "Sign in" option. Even I set all above settings, it is asking me to "Sign in". How can I avoid this matter? I really appreciate it if someone can help me out on this regard. Thanks in advance, Wiki.Building Copilot Agents with User Experience in Mind
Understand where and how the user will be accessing the Copilot agent Create multiple-modal experiences for your Copilot using voice To enhance the accessibility and user-friendliness of your copilot agent, consider developing a voice agent for voice interactions. This hands-free experience is not only inclusive but also particularly beneficial in customer scenarios where associates are using their hands and need to interact with the agent quickly and effectively without typing. Before deploying the agent to your user base, ensure thorough testing in the work environment to validate a good end-user experience, especially considering the loudness of the environment. For step-by-step instructions on testing the voice agent experience, you can view this blog Determine the proper channel deployment Selecting the appropriate channel for deployment is crucial. Ensure you reach users where they perform their work, whether it's through M365 Copilot, M365 Copilot chat, embedding into a website, or other channels. It's essential to deploy the agent to the channels that best suit the use case. Creating multilingual agents Creating multilingual agents is essential for providing a personalized and inclusive experience. By enabling your agent to interact in multiple languages, you can cater to a diverse user base and ensure effective communication without language barriers. This feature enhances accessibility, user satisfaction, and expands the agent's reach in global contexts. For more specific information on multilingual agents see below. Take Time During the Configuration to Set Up the Agent for Success Creating a name for the agent In a world where multi-agents are on the rise, it's essential to be as descriptive as possible in the name and description for the intended use. This helps users understand the agent's purpose and functionality. Planning out your multi-agent scenarios early and defining clear names is especially important to avoid confusion. Add starter prompts Starter prompts guide users on how to interact with the agent effectively. These prompts should be clear and concise. When selecting starter prompts, consider defining the most common use cases for the agent. For example, if you are creating an IT Support agent, you might include prompts like "create a support ticket" or "check request status" to cover the most asked questions. Provide description and general instructions A detailed description of the agent's capabilities and intended use helps users understand what they can expect from the interaction. This description also assists the language model in identifying and utilizing your agent for specific tasks or situations. Ensure that the description is concise yet informative. For the instructions (declarative and generative mode), you should be as detailed as possible. Include the purpose, guidelines on what the agent should and should not do, tone, and the skills the agent will possess. Below are resources that provide detailed instructions on how to construct the verbiage effectively. Add key knowledge resources Adding key knowledge sources to the agent is crucial for its effectiveness and reliability. By integrating relevant documentation and connecting to essential business systems, the agent can provide accurate and timely information, ensuring users receive the support they need. These connections enable the agent to perform specific tasks efficiently, leveraging up-to-date data and resources. This not only enhances the user’s experience but also ensures the agent remains a valuable tool in various scenarios, from customer support to internal business operations. Offer alternative ways to answer questions Connecting to a live agent Provide users with the option to connect to a live agent for more complex queries or issues that the agent cannot resolve. Customizing the escalation topic is another way to track potential opportunities for agent improvement. Creating a step in the topic to record the escalation to ensure it is captured for agent performance is key to ensuring agent success and continued feedback on areas of improvement. Create adaptive cards Using adaptive cards in a Copilot agent enhances user interaction by providing a visually appealing and interactive interface. They facilitate structured data collection, ensuring consistency and ease of processing. Adaptive cards offer customization and flexibility, allowing them to be tailored to various scenarios and use cases. They improve communication by presenting information clearly and concisely, reducing ambiguity. Additionally, adaptive cards integrate seamlessly across different platforms and devices, ensuring consistent user experience. Overall, they significantly enhance the functionality and user experience of Copilot agents. Provide documentation references or links In the agent's response, include links to additional resources. Offer links to documentation or external sites where users can find more detailed information. Automate, Automate, Automate Enhancing efficiency with automation Using automation in Copilot agents (agent flows and agent actions) can significantly improve efficiency and reduce manual workload by identifying key processes that can be automated. Incorporating messages that inform users when the agent is processing their request helps manage user expectations and enhances the overall experience. By automating repetitive tasks and providing real-time updates, Copilot agents can streamline workflows, allowing users to focus on more complex and value-added activities. This approach not only boosts productivity but also ensures smoother and more transparent interaction with the agent. Monitor Agents for Insights Actively review the analytics for the agent Regularly reviewing analytics is essential for assessing the agent's performance and identifying areas for improvement, ensuring a high-quality user experience. By analyzing metrics such as engagement outcomes, knowledge source usage, action success rates, and user feedback, you can gain valuable insights into how effectively your agent is meeting user needs. Resources like Microsoft Copilot Studio Analytic, Topic Analytic Autonomous agent health, and Application Insights telemetry provide detailed guidance on measuring and enhancing agent performance. For more information see the discussion on monitoring performance and effectiveness of M365 Copilot agents. Collect Feedback Integrate feedback mechanisms Use adaptive cards to collect user feedback after every interaction with your agent. This allows you to assess and enhance the quality of agent responses while ensuring user satisfaction. Implementing this feedback mechanism helps in obtaining valuable insights into how well your agent is performing and identifying areas for improvement. Conclusion To create an effective Copilot agent, focus on providing easy accessibility by developing voice interactions and multilingual capabilities. Ensure you set up the agent properly in a world of multi-agents to decrease complexity and avoid confusion. Be creative in how you answer prompts, using adaptive cards and alternative ways to respond. Automate crucial business processes associated with the agent's purpose to enhance efficiency and reduce manual workload. Finally, actively collect feedback and apply it to continuously improve the user experience, ensuring the agent remains a valuable tool. A special thanks to IvicaIvancic ((3) Create your first custom AI enabled Copilot Studio Voice Agent – step-by-step guide | LinkedIn) and TrevorNorcross (Monitoring performance and effectiveness of Microsoft 365 Copilot agents | Microsoft Community Hub) for their incredible support in allowing me to link to their insightful blogs.Monitoring performance and effectiveness of Microsoft 365 Copilot agents
When asked by a systems integrator partner about the best ways to "monitor" M365 Copilot agents (and this comes up frequently), I always follow up with questions to clarify what, exactly, they are trying to monitor. Are we trying to monitor performance, looking at speed and potential bottlenecks? Are we trying to find out who is using the agent, how often it's being abandoned, and how effective the functionality appears to be for end users? Microsoft offers multiple means of "monitoring" M365 Copilot agents, based on the intent of the question. By understanding what tools are available to the administrator, implementer, agent maker, or product owner, it is possible to gain deeper insight into what does (and doesn't) work well in the custom agent solutions. MONITORING PERFORMANCE Monitoring the performance of custom M365 Copilot agents is essential to ensure they operate efficiently and end users aren't waiting unnecessarily long a specific outcome. One of the primary aspects to monitor is the speed of the agents. This involves tracking how quickly the agents respond to user queries and perform their designated tasks. Tools like the Copilot Control System provide comprehensive analytics and reporting capabilities that allow administrators to view core usage telemetry and track the speed of agent responses. By analyzing these metrics, you can identify any delays or slowdowns in agent performance and take corrective actions to optimize their speed. Identifying and addressing bottlenecks is another critical aspect of performance monitoring. Bottlenecks can occur at various points in the agent's workflow, such as during data retrieval, processing, or response generation. Using tools like the PVA test framework for load testing and performance monitoring can help you simulate different scenarios and identify potential bottlenecks. Additionally, connecting via DirectLine API allows for programmatic testing of Copilot Studio agents, helping you pinpoint specific areas where performance may be lagging. By addressing these bottlenecks, you can ensure that your agents operate smoothly and efficiently. MONITORING USAGE AND EFFECTIVENESS Monitoring the usage, analytics, and effectiveness of custom M365 Copilot agents is crucial for ensuring they deliver optimal performance and meet organizational needs. One of the primary tools for tracking usage is the Microsoft 365 admin center, which provides detailed reports on agent activity. These reports become available within 72 hours of the end of the day and can be filtered by different periods, such as the last 7 days, 30 days, 90 days, or 180 days. The agent usage report captures metrics on active agents, user interactions, and adoption rates, helping administrators understand how frequently agents are being used and identify any patterns or trends in their usage. Analytics play a significant role in measuring the effectiveness of custom M365 Copilot agents. The Copilot Control System offers comprehensive analytics and reporting capabilities that allow administrators to view core usage telemetry and track license utilization. By analyzing these metrics, organizations can gain insights into how well their agents are working for end users, and identify areas for improvement. For example, the Analytics tab in Copilot Studio provides detailed reports on agent effectiveness, including metrics such as escalation rate, abandon rate, and resolution rate. These reports help administrators understand the impact of agents on user productivity and business outcomes, enabling them to make data-driven decisions to enhance agent performance. Effectiveness can also be monitored through user feedback and interaction logs. The upgraded analytics page in Copilot Studio allows administrators to review user feedback, knowledge source use, and the outcomes of conversations between agents and users. This helps identify any recurring issues or areas where agents may not be performing as expected. Additionally, reviewing historical activity and logs of AI inference or decision-making can help address inconsistencies and optimize agent performance. By proactively monitoring these aspects, organizations can ensure that their custom M365 Copilot agents deliver accurate and timely responses to users. CONCLUSION By understanding various tools available for monitoring custom M365 Copilot agents, stakeholders can ensure their solutions are performing well and effectively solving problems for the business users in the organization. I would not be surprised to see additional focus on these areas in future evolution of the Copilot and Copilot Studio products and administration tools, as this is one of the most asked-about topics in my discussions with global systems integrator partners in our ongoing partnership discussions.
