so I'm developing in SPO and got into an issue with >5k list items. Fixed that with creative filtered views, then another issue sprang up.
The list in question now has 7500 or so items in it, and we need to report on the data, and therefore need to have a complete data set. So using the creative filtered views (quarterly dated) I exported all the data by opening each view in turn and exporting to excel, then cut and pasted from all to first sans table headings and viola I got a complete list.
I'm now attempting to make this a little more robust so thinking of building a Flow to sync modified or new items to a SQL table.
Using the on premises data connector I was able to insert row and update row just fine to SQL. To determine if the flow run was inserting or updating I put a column in the list with a number, and tested if that value was equal to 0, if so I inserted, else updated.
The problem with this approach is that somewhere in the flow I it was necessary to update the list to increment the count, and if you are following this logic you may have just figured out that this creates a recursive loop.
Then I switched to using a query or stored proc in sql, and got an error saying the data connector didn't permit use of a query.
I think I could do this with Layer2 (have used that in the past), also had good luck running codeplex SSIS SharePoint list source and destination connector for something similar. In my current contract I need to not purchase 3rd party s/w, and have not been given MSDN (VS) to play with.
Anybody got any suggestions?
Thanks in advance!
