Issues with implementing SPO/ODFB Retention Policies based on Last Modified Date

%3CLINGO-SUB%20id%3D%22lingo-sub-827631%22%20slang%3D%22en-US%22%3EIssues%20with%20implementing%20SPO%2FODFB%20Retention%20Policies%20based%20on%20Last%20Modified%20Date%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-827631%22%20slang%3D%22en-US%22%3E%3CP%3ESince%20this%20is%20a%20common%20means%20of%20implementing%20retention%2Fdestruction%20policies%2C%20I'd%20like%20to%20share%20some%20issues%2Ffrustrations%20I've%20had%20and%20hopefully%20spur%20some%20discussion.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1)%26nbsp%3B%3CSTRONG%3EThe%20Sync%20Client%2FWindows%20Explorer%20loophole%3C%2FSTRONG%3E.%26nbsp%3BWhen%20users%20upload%20a%20document%20to%20ODFB%20or%20SharePoint%20via%20the%20browser%2C%20the%20modified%20date%20(crawled%20property%3A%20ows_Modified%2C%20which%20is%20the%20property%20respected%20by%20retention%20policies%2C%20and%20is%20the%20one%20displayed%20in%20the%20interface%20in%20SPO%2FODFB)%20is%20automatically%20updated%20to%20the%20time%20of%20upload.%20However%2C%20when%20documents%20are%20uploaded%20via%20the%20Sync%20Client%20in%20Windows%20Explorer%2C%20the%20modified%20date%20simply%20inherits%20the%20existing%20internal%20document%20modified%20date.%20This%20can%20lead%20to%20older%20documents%20being%20deleted%20immediately%20upon%20upload%20if%20the%20internal%20modified%20date%20is%20prior%20to%20the%20destruction%20policy%20date.%20More%20significantly%2C%20it%20is%20confusing%20to%20the%20end%20user%20to%20have%20two%20different%20retention%20experiences%20based%20on%20the%20method%20of%20upload.%20There%20needs%20to%20be%20a%20way%20to%20ensure%20the%20modified%20date%20is%20consistently%20applied%20across%20both%20methods%20of%20upload.%20It%20is%20not%20realistic%20to%20expect%20users%20to%20know%20not%20to%20upload%20via%20the%20Sync%20client.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E2)%26nbsp%3B%3CSTRONG%3EContent%20Search%20and%20Last%20Modified%20Date.%26nbsp%3B%3C%2FSTRONG%3EThe%20managed%20property%20LastModifiedTime%20is%20mapped%20out%20of%20the%20box%20to%20show%20internal%20document%20date%20crawled%20properties%20instead%20of%20the%20ows_Modified%20property.%20In%20practical%20terms%20this%20means%20that%20searching%20SPO%2FODFB%20for%20documents%20last%20modified%20more%20than%20five%20years%20ago%20will%20return%20many%20documents%20which%20in%20fact%20have%20a%20more%20recent%20ows_Modified%20date%2C%20and%20therefore%20it%20is%20not%20an%20accurate%20way%20to%20audit%20documents%20subject%20to%20destruction.%20If%20you%20%3CA%20href%3D%22https%3A%2F%2Fblogs.technet.microsoft.com%2Fsposupport%2F2018%2F01%2F27%2Fdifferences-between-the-created-or-modified-date-in-search-results-compared-to-listlibrary-view%2F%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Echange%20the%20mapping%20of%20the%20LastModifiedTime%20managed%20property%3C%2FA%3Eit%20will%20work%2C%20but%20you%20first%20need%20to%20reindex%20every%20single%20site%2C%20including%20each%20ODFB%20site%2C%20in%20the%20tenant.%20Oddly%2C%20there%20is%20a%20system%20managed%20property%20in%20SharePoint%20search%2C%20%3CSTRONG%3ELastModifiedTimeForRetention%3C%2FSTRONG%3E%2C%20which%20is%20mapped%20only%20to%20ows_Modified%20crawled%20property%2C%20and%26nbsp%3Bthus%26nbsp%3B%3CEM%3Eshould%26nbsp%3B%3C%2FEM%3Ework%2C%20but%20it%20is%20not%20recognized%20by%20Content%20Search%20in%20testing%20-%20searching%20against%20it%20returns%20every%20item%20in%20the%20search%20scope.%20Ideally%20there%20would%20be%20a%20way%20to%20search%20ows_modified%20in%20Content%20Search%20OOTB%20without%20resorting%20to%20a%20full%2C%20cumbersome%20reindex%20in%20order%20to%20understand%20documents%20subject%20to%20retention%20policies.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWould%20be%20interested%20to%20hear%20others'%20experiences%20and%20thoughts%20on%20implementing%20such%20a%20policy.%20Thanks!%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-827631%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ECompliance%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Occasional Contributor

Since this is a common means of implementing retention/destruction policies, I'd like to share some issues/frustrations I've had and hopefully spur some discussion.

 

1) The Sync Client/Windows Explorer loophole. When users upload a document to ODFB or SharePoint via the browser, the modified date (crawled property: ows_Modified, which is the property respected by retention policies, and is the one displayed in the interface in SPO/ODFB) is automatically updated to the time of upload. However, when documents are uploaded via the Sync Client in Windows Explorer, the modified date simply inherits the existing internal document modified date. This can lead to older documents being deleted immediately upon upload if the internal modified date is prior to the destruction policy date. More significantly, it is confusing to the end user to have two different retention experiences based on the method of upload. There needs to be a way to ensure the modified date is consistently applied across both methods of upload. It is not realistic to expect users to know not to upload via the Sync client.

 

2) Content Search and Last Modified Date. The managed property LastModifiedTime is mapped out of the box to show internal document date crawled properties instead of the ows_Modified property. In practical terms this means that searching SPO/ODFB for documents last modified more than five years ago will return many documents which in fact have a more recent ows_Modified date, and therefore it is not an accurate way to audit documents subject to destruction. If you change the mapping of the LastModifiedTime managed property it will work, but you first need to reindex every single site, including each ODFB site, in the tenant. Oddly, there is a system managed property in SharePoint search, LastModifiedTimeForRetention, which is mapped only to ows_Modified crawled property, and thus should work, but it is not recognized by Content Search in testing - searching against it returns every item in the search scope. Ideally there would be a way to search ows_modified in Content Search OOTB without resorting to a full, cumbersome reindex in order to understand documents subject to retention policies.

 

Would be interested to hear others' experiences and thoughts on implementing such a policy. Thanks!

0 Replies