Event banner
Event details
108 Comments
When developing a RAG system on your own, many things can go wrong: format of the files including images or tables, or scanned pages that prevent a nice parsing of the information, errors in the retrieval, errors in the generation of the responses, differences between the expected level of detail of such responses from the user's perspective, hallucinations.
How do deal with these difficulties? how do you enable the owner of the site to evaluate the performance of the agent? how can she curate the documentation that is exposed through the agent so the responses are properly grounded?- cjtan
Microsoft
We are continuously improving the quality of responses from the SharePoint agents based on how customers are using and the feedback that is given for responses. Be sure to use the Thumbs up/down buttons in the message response window when you see something that works or doesn't work as expected. As an administrator, the information from this feedback can be viewed through Microsoft Admin Center, as well.
Because SharePoint agents have a configuration for "Instructions", you can direct the agent more specifically with instructions to help make the responses more predictable; for example, directions that instruct agent not to use its latent knowledge and only use the selected circumstances. You can use the File Statistics on the .agent file to understand if the agent is being used.
Thanks for the reply. So to understand how the agent is performing I have to look at the feedback provided by the users with the thumbs up/down. Do you plan to add other metrics for evaluating this performance in a more offline way, like metrics for evaluating the faithfulness or groundedness of the responses and the documents, or the relevancy of the responses regarding the questions posed?
Is there a recommendation how sharepoint pages and documents should be structured so that the agent can access the knowledge best?
jolenetamNo particular recommendations, as it is as straightforward as selecting the documents that you want the SharePoint agent to be grounded on. But please do send feedback if your experience isn't what you're expecting. Our engineering team is continuously working to improve the experience.
We have been for a while working on improving RAG applications and it is quite a topic. There are technical terms involving performance that are not common knowledge of final users. How do you envision to enable final users to understand if the RAG the agent is doing is working or not and if not why and what to do about it?
This AMA is chat only, so you are seeing what there is to see. Bummer, huh?
- Sarah_Gilbert
Community Manager
We totally get where you’re coming from! While a chat-only format might feel limiting at first, it’s actually designed to be as inclusive and efficient as possible. This way, everyone—no matter their time zone or tech setup—can participate, and all the insights from the session are captured in writing for future reference.
We hope you’ll find value in this format and the opportunity to engage directly with our experts. Thanks for sticking with us! 😊
It's a chat-based event. No audio or video. Just comments here, to my understanding.
Is it possible that the copilot agent in SharePoint sites has not reached our tenant yet? I cannot see it at this time although I am a site owner with a copilot license.
- cjtan
The agent in SharePoint sites is still rolling out this week. We expect that this will be completed by mid-January.
We are seeing it light up slowly in our tenant, some users see it in some sites. This has been expanding since Monday. We have tens of thousands of sites and tens of thousands of users, so it is very hit and miss still. Hoping by Monday everyone will see it 🤞 !
Based on my last meeting with Microsoft I got the information that they are still rolling out this feature and should be available till the end of January. I have same issue. :)
Are there any limitations for aspx pages, such as their length?
When will the integration and editing capabilities via Copilot Studio be available? What is the estimated launch date for Actions?
Can you extend SharePoint Agents to non-SharePoint sources such as Confluence?
- mithunas
We can't extend as of now. We do have Copilot Studio integration in the works for later in the year. We will support all additional sources when Copilot Studio supports those sources.
- cjtan
Not at this time. We are working on adding the capability to extend SharePoint Agent to Copilot Studio (in private preview now), which is where you can add non-SharePoint sources, such as Confluence.
Is the pricing known for users who will use the SharePoint agent without a license?
Can the SharePoint agent search across a hub, or must sites be entered individually?
- mithunas
As of now, we don't have that available but that is something we have in our roadmap for later in the year
Is the SharePoint Agents promotional offer automatically enabled in EDU tenants?
- jolenetam
SharePoint agents should be enabled in EDU tenants for Copilot licensed users or if the tenant has the pay-as-you-go meter enabled. Please let us know if this is not the experience you're seeing.
