Ignite 2024: SharePoint agents now in general availability

AdamHarmetz- you said you wanted some feedback - here you go:

We really want to use SharePoint agents as they offer some great insights into our documents, etc. However, there are far too many issues currently as listed below. Note that we’ve been asking for points 1-4 since the early days of the private preview (and not just us asking). There were some early sessions from Microsoft that said these controls would be in place, but nothing yet.

1. We need the ability to disable across the tenant and only make them available for a select few people for testing purposes. We have only been able to do this by Microsoft implementing a backend configuration on their end which is not ideal but the only option for now. There should be a disable option for any large/new features rolling out by Microsoft (we are fighting to get Site themes disabled as it goes completely against our corporate brand strategy, but no option at the moment).
2. For our existing Copilot users - Microsoft said we can disable access to SharePoint agents via the Copilot license - however, that will not only affect the SharePoint agents, but also anything related to Copilot and SharePoint – so not an option.
3. We need the ability to only enable on select SharePoint sites. If we were to enable SharePoint agents now, they would be on every single SharePoint site in our tenant (we currently have well over 100k sites) – not an option.
4. We need the ability to control who can create agents on the site. It should be owners, and only users the owners provide permission to. Currently, it’s the owners plus every single person that has Edit rights which in some cases could be 1000+ users across a single site. This is not an option for us as we would require anyone building agents to have some level of training first – and it is not viable with the current implementation.
5. If an owner approves an agent, it automatically moves the file from its original location to Site Assets > Copilots > Approved. This keeps the existing permissions on the agent file if it had unique permissions but will change the permissions if it was originally inheriting from the library – which can be an issue if the original library has unique permissions. In addition, it removes any metadata we might have had in the original library where it was located. It also means users may now be going into our Site Assets library which is not ideal.
6. Once an agent has been approved the only way to un-approve it is to move it out of the Approved folder it was moved to when you approved it. This one took me a while to figure out – I was expecting there to be an un-approve option like how I approved it. This isn't great UX.
7. If a user creates an agent or moves it into Site Assets > Copilots > Approved, it is automatically approved (even if you’re not an owner). This confirms that approved just means it lives in the Approved folder – which is a major design flaw. I tested this theory by moving one of my previously approved agents back into the library where it was created, and it was now set as not approved. And I moved an unapproved agent using a test user that wasn’t an owner, and it auto-approved the agent – which is not great to say the least.
8. If you edit an agent that is approved, it’s still approved (doesn’t move into an un-approved state).
9. There is no way to hide the Agent link under the New menu – It looks like you can remove it (untick it to not show) but it just adds it back in there regardless.
10. As soon as anyone creates an agent it shows up under Recent in the Copilot menu even if it’s just for testing, not finished, etc. There is no way for an owner to remove it from here or prevent it from showing until you are ready for users to start using it. Similar to Recommended agents – this is not something the owner has any control over that I can see.
11. Pay as you go is way too overarching – there is no way to scope this across sites, agents, anything. And there is no way to track or get analysis on usage costs per agent, etc. Can only see by user and that’s across everything.
12. Also, the Pay as you go cost is not viable. Microsoft have kept saying $0.01 per message, but what they didn’t tell you is that each question/answer is an interaction and consists of 32 messages – so the cost for every question/answer is $0.32 (x32 more than what was mentioned).
13. There isn’t a straightforward way to see what an agent is using as its source. Remember, it starts at the site or library/list level, but can be extended to other sites libraries, etc. And if anyone with Edit rights is creating them, how do you govern this?
14. There is added complexity with a SharePoint Agent that has been extended with Copilot Studio – as it’s now managed within the Power Platform governance model, so you have a quasi-SharePoint agent that's a Copilot Studio agent.
15. If you extend the agent with Copilot Studio, even users with a Copilot License are charged per transaction (they weren’t charged before). I'm fairly certain this is the case, but happy to be corrected.
16. There is no way to edit the default Site agent that is automatically provisioned on every single site. There will be cases where you only want to target certain parts of the site.

Apologies for the extensive list of feedback but hoping this can help to improve the current implementation and allow us to roll it out in our tenant. Also, hopefully useful for others looking at SharePoint Agents.