Forum Discussion
Property Demotion leads to infinite import cycles when integrating with SharePoint
Hi,
we have built an integration with SharePoint that imports watches and imports files from connected SharePoint sites into our product for further processing. After that process is complete, we push some of the processing results back to a SharePoint list as metadata. In order to keep the extracted data up to date, even when file content changes after the initial upload/import, we subscribe to changes in the SharePoint site and re-import the file if its hash has changed since the last import.
This process worked really well until we started processing some of the Microsoft file types (e.g. .docx, .msg, etc.). Apparently SharePoint Online will include metadata from list columns into these files in a process called "Property Demotion". Since this changes the file content, it triggers a re-import of the file. Due to the non-deterministic (AI-generated) nature of some of the metadata we extract, this re-import will likely result in different metadata values that are exported back to SharePoint - which of course triggers property demotion again, leading to an infinite cycle.
What we have tried so far:
- Disable property demotion -> Not possible in SharePoint online as far as we know.
- Only export metadata if anything actually changed -> Only solves the issue if metadata is stable w.r.t. document content, which does not hold for our use-case.
- Skip re-imports occurring X minutes after the last metadata export -> While this works most of the time, it is fragile (what if the property demotion is delayed?) and worsens user-experience (what if a user edits the file within the X minute window?)
Has anyone encountered the same issue before? How did you end up handling it?
Thanks,
Johannes