SOLVED

Retention Policies and SharePoint sites

Iron Contributor

We have five year retention policies covering almost everything, including Teams. When a site is deleted, e.g. by deleting a team, all files contained in the site are retained. But what about the site (including lists etc.)?

 

If a user wants to recover a site (after 30 days in the recycle bin has passed), is it only the files that can be recovered? If so, is it possible to identify which files were stored in the site? 

51 Replies
Thanks for clarifying!!

@Tony Redmondthat's a very clear explanation, worthy of being upgraded to an docs article, kudo's. Could you explain the following scenario:

  1. I created a 'Teams' Team falling under 2 retention policies retaining content forever (SharePoint, O365 Group and Team Channel messages are all covered)
  2. played around with the Team for a month or so, all resources (Groups Mailbox, SharePoint site, Channel messages) are touched and created
  3. now I want to really, really delete the Team and all associated resources, leaving no trace what so ever

What to do?

 

What I did was:

  1. delete the Azure group (Remove-AzureADGroup)
  2. permanently delete the Azure Group (Remove-AzureADMSDeletedDirectoryObject)
  3. set an exclusion in the SharePoint site retention policy (Set-CCRetentionCompliancePolicy)

The Teams and Group resources seem to be gone entirely, but the SharePoint site will complain 'This site has a compliance policy set to block deletion' even though it also says 'We couldn't find the Office 365 group connected to this site.'. How to delete the SPO site, in this situation? Should I have started with adding exclusions for the Office 365 Group, Team and SPO site to the retention policies? Should I switch the policies off (won't this impact other sites covered with the policy?

 

Hope you can help me

@SjoerdV Sorry for the delay in replying. I've been away.

 

I believe that you might have "broken" the links that connect the Office 365 group and its associated resources by deleting the Azure AD group using PowerShell. It might have been better to:

 

1. Remove the site from the retention policy.

2. Wait for SPO to process the command from the SCC to remove the policy from the site.

3. Remove the team (this forces a notification to all workloads).

4. Accelerate the 30-day removal process by hard-deleting the soft-deleted group after waiting for about a day to ensure that all workloads have been informed about the deletion.

 

 

@SjoerdV 

Sorry for the delay in replying. I've been away.

 

I believe that you might have "broken" the links that connect the Office 365 group and its associated resources by deleting the Azure AD group using PowerShell. It might have been better to:

 

1. Remove the site from the retention policy.

2. Wait for SPO to process the command from the SCC to remove the policy from the site.

3. Remove the team (this forces a notification to all workloads).

4. Accelerate the 30-day removal process by hard-deleting the soft-deleted group after waiting for about a day to ensure that all workloads have been informed about the deletion.

 

 

Thanks for the heads-up, I'll adjust procedures to align with you suggestion. It kind of worries me that
A) you can get in an undisirable state this easily, which will also not resolve itself (what to do with that sharepoint site that won't go anywhere?)
B) the amount of time that is needed between steps severly hinders routine maintenance

The whole retention architecture feels kind of flaky or an afterthought at best. Seems like it should be an entirely seperate architectural layer (both application and storage) but it isn't. I'll try to cope with it anyway :)

@SjoerdV It's more like these are loosely-coupled workloads where synchronization must happen to keep everything aligned. With that in mind, it makes sense to take account of the need for synchronization before doing anything to remove data.

@Tony RedmondJust an update on this. I am still working with Microsoft how to get a grip on SharePoint sites with Retention policies (excluded or not). As it is right now step 2: "Wait for SPO to process the command from the SCC to remove the policy from the site." is never happening. So sites can never be removed.

 

Also retention policies for SharePoint Sites still only support 100 Sites, as we have thousands with varying retention needs this number is way to low.

 

As it is right now SharePoint and Retention is still not a very 'agreeable' subject in my book.

 

As long as this thread remains open, I will update when MS delivers a working solution.

 

P.S. I found a thread with the same issues I am seeing BTW, I thought I share that as well: https://answers.microsoft.com/en-us/msoffice/forum/all/inability-to-delete-sharepoint-team-site/c5e4...

Let me ping some people I know in Microsoft about this...

@Tony Redmond to date I have had 4 tickets opened on this subject on multiple tenants with a default turnover time of several weeks each. Still there seems to be an issue with sites that can not be deleted because of a 'phantom hold' somewhere. I even saw that MS internally has some cmdlets that can detect these holds and remove them, but every support case starts out with: "please run: Remove-SPOSite ...", which obviously fails.

This even happens on sites that are not attached to a Group or Team, so it's really a SharePoint issue with the retention policy mechanism. I am being told everywhere that excluding a site from a retention policy should be enough to clear the hold, but it clearly is not.

 

For the future: Sites will always be in need of being deleted

 

If you have an update, I'd really appreciate your feedback

@SjoerdV   I don't have anything particular to share right now. I have a thread going with some folks in the SharePoint team that I hope to pursue at Microsoft Ignite next week. The problem of course is that the conference might become chaotic and I lose all sense of reality, in which case I probably won't get this done.

I appreciate anything you can do, but first and foremost have fun at ignite!

Hello @SjoerdV 

I just wanted to chime in on this conversation because we are facing the same issues. 

 

A user created either a Group from Outlook or a Team in MS Teams. I am not sure which, but whichever it was, it also created a SharePoint site. It was created as an experiment and then they later deleted the Group or Team, or both -- more than 30 days ago.  

 

When this site is selected in the SP admin center, there are two notices above the properties pane:  

 

clipboard_image_1.png

 

 

If we try to delete the site, we get the popup "Can't delete site"  message with the details of "A compliance policy is currently blocking this site deletion"

 

For lack of a better term, I am currently calling this site an "Orphaned" site.

 

The Default data governance publishing policy, which applied to Exchange, OneDrive, SharePoint and Office 365 Groups is currently off.  Separate policies for SharePoint, OneDrive and Exchange were created six or seven months ago.   The policy covering SharePoint sites remains published, but it was edited about one month ago to turn off SharePoint sites in the "Locations applied" section.  I am guessing this effectively turns off retention for all SharePoint sites, and was done so that we could clean up some sites that needed to be removed.  I wasn't the admin that did this, so I am just assuming that after removing the sites, the plan was to turn it back on.  Some sites were able to be removed, but some still could not be deleted, so the retention policy for SharePoint sites was not re-enabled.

I also created a new policy more than 24 hours ago with specific sites  (one of which is the "Orphaned" site that we can't delete.   I don't want the policy to delete the sites for me, so the settings look like this:

clipboard_image_0.png

I waited 24 hours, but the site is still flagged as having a compliance policy that prevents deletion.  We don't have any other published policies that cover SharePoint.  (It is very possible I don't understand how to best create a policy for sites that we want to remove by setting it to a 1 day retention period. )

I am in agreement with @Tony Redmond    @SjoerdV   concerning his opinion regarding the mechanisms for SharePoint & Retention.   It seems like documentation on how to do it properly is lacking. If it is out there, I'd love to see it.  I find the process arduous and difficult to follow. 

In the future, we will follow the advice in the post about "What would have been better...", but in the meantime, is there any procedure in a doc or technet post that we could follow and not have to contacting support and open a ticket?  

I was mixed up on who I was agreeing with and held the opinion that, "As it is right now SharePoint and Retention is still not a very 'agreeable' subject in my book." I thought it was @Tony Redmond  . I see it was @SjoerdV 

 

Sorry about that.  There is a longer post that is awaiting moderation approval.  Until that shows, this reply may not make sense. 

@Scot Bickell @Tony Redmond I just got informed by the MS Support Team that Microsoft is working on a fix but that it will take some time because of the complexity of the problem. They do understand the issue, so I reckon we just have to wait a bit longer. 

HI @SjoerdV

Thank you for the update.   

 

I have an update on my attempts to remove the site I've been trying to delete.   

 

Here's a complete timeline :  
10/31/2019 - 7 pm EDT:   Unable to delete site due to retention policy issue, but unable to see any active policies that would apply to the site.  Continued to look and research for the next two days.

 

11/2/2019 -  2 pm EDT:  Created new custom retention policy (illustrated in previous post)

 

11/3/2019  - 3 pm EST:  Still unable to delete site -  Changed custom retention policy status to Off & still unable to delete.

 

11/4/2019  - 2 pm EST  Still unable to delete site.  Continue researching this. 

 

11/5/2019 - about 8:45 am EST  (Today)  -  Removed custom retention policy.

 

11/5/2019  about 8:46 am EST - Saw that that the only yellow warning on site was the group not found warning: 

clipboard_image_0.png


11/5/2019  8:47 am EST - Successfully deleted site and it now shows in Deleted sites listing.  

I wish I had attempted to check on the status of the site before removing the policy, so I am unsure if more time just needed to pass since turning the retention policy off, or if removing the policy did something to jiggle loose any policies previously associated with the site.   This at least tells me I can delete a site, even if it becomes orphaned from the originally connected Group.    I'll try that out the next time I run into this. 

Good luck to the engineers who have to fix this. I don't envy their task on such intricate integrations between all these Office 365 products.  

@Scot Bickell I spoke about the problem to @Mark Kashman of Microsoft. Flagging it here for him to get some context.

SharePoint Retention got another complicating dimension added today. Private Channels have their own site collection! Didn’t see that coming :-): https://docs.microsoft.com/en-us/MicrosoftTeams/private-channels
Great post! Looking forward to your follow-up on 'management topics', I hope you could point out the ramifications for retention as well.

Looks like the engineers who worked on the site URL renaming feature have not been talking to the engineers who work on retention policies:

 

I wanted to test out the new feature that allows changing a site's URL.    The doc mentions a list of Effects, but they might want to add to that list of effects to include Retention Policy interactions.

 

In order to be able to edit the URL, I had to add the site to the list of Excluded sites in the Retention policy.  Once the policy took effect, I was able to edit the URL.  

After the URL has been changed, the old URL still shows in the list of excluded sites, and if you try to change it and enter the new URL, you get an error upon saving the policy:

The Client Error message: 

 

No exact match was found. The URL "https://...sharepoint.com/sites/....  may be invalid, you don't have permission to access this location, or the location is not indexed by Search.

 

clipboard_image_1.png


Details show a 500 exception

 

Request: /api/RetentionCompliance Status code: 500 Exception: Microsoft.Exchange.PswsClient.PswsException Diagnostic information: {Version:17.00.3679.019,Environment:NCUPROD,DeploymentId: removed for privacy,InstanceId:WebRole_IN_1,SID:removed for privacy,CID:removed for privacy} Time: 2019-11-13T18:18:49.8887659Z

It is also possible that I need to allow time for things to reindex under the new URL, so I will try it again in about 24 hours.