Forum Discussion

hrusha1's avatar
hrusha1
Copper Contributor
Jun 06, 2024

Copy activity in Azure Data Factory silently fails

I am using the ADF Copy Activity to extract an Office365 mailbox to Blob Storage. I have the pipeline configured and running but it seems that it extracts incomplete data:

The source inputs a filter on date ranges. Specifically I am filtering on recievedDateTime.

 

Whenever I use a time range larger than 1 month, I am getting very little data in the blob storage (less than 20 Mb) and the line count is always a suspiciosely round number (600, 800,...).

When I use smaller intervals- I as much as 200 Mb per month of data with a 'rgular' non round number of a few thousand emails.

I know for sure this is not accidental- there is no reduced activity in the mailbox at the 'problematic' time range.


I am new to ADF, so reaching out to how can I even troubleshoot this?

I've tried enabling logs but I do not see anything interesting there (as if they are not being written).

My backup plan is to perform the extraction in small intervals and perhaps automate this process but looking for something that "just works"

No RepliesBe the first to reply

Resources