MGDC for SharePoint FAQ: Which dates to use for the Files dataset?

The SharePoint Files dataset became publicly available in Microsoft Graph Data Connect (MGDC) on August 20, 2024. With its release came a unique pricing model ( $0.75 per 50,000 objects, significantly lower than the standard $0.75 per 1,000 objects for other MGDC datasets) and a unique update frequency (data is refreshed weekly, not daily).

Archimedes sculpting a file

Weekly Refresh

Unlike other SharePoint datasets, the Files dataset is refreshed only once a week, over the weekend. This timing helps avoid performance conflicts with tenant activity, as the dataset is typically massive. You can learn more about dataset behavior in the MGDC for SharePoint FAQ: Dataset types and features.

So, which dates should you use for your data pulls? There are two main scenarios: full data pulls and delta pulls.

Full Data Pulls

For a complete snapshot, treat the SharePoint Files dataset like other state-based datasets (Sites, Groups, Permissions). Use the same start and end date to indicate a full pull. Since MGDC data is always delayed by two days, you’ll want to target the most recent refresh.

For instance, if today is Wednesday, October 8, 2025 (2025-10-08), set both your start and end date to Monday, October 6, 2025 (2025-10-06). This aligns with the weekend refresh and ensures you're pulling the latest available data as soon as possible.

Calendar

Note: Time values don’t matter for MGDC for SharePoint. Feel free to leave them at 00:00:00 for both start and end date.

Delta Pulls

To keep your dataset current, use delta pulls. These capture all file additions, updates, and deletions. Details on how delta datasets work are available in the MGDC for SharePoint FAQ: How can I use Delta State Datasets.

Since the dataset updates weekly, pulling deltas more often than that won’t yield new data. Stick to a Wednesday schedule, targeting the previous Monday’s data.

In summary, here's a good strategy for using deltas with Files:

• Initial full pull: Run on Wednesday, using Monday’s date for both start and end.
• Ongoing delta pulls: Every Wednesday, set the end date to the most recent Monday and the start date to the Monday before that.

For more guidance and documentation, visit MGDC for SharePoint.