Session log is now available in copy activity
By
Published Dec 01 2020 08:00 PM 2,522 Views
Microsoft

 

Benefit of using session log in copy activity:

 

With session log enabled in copy activity, you can log your copied file names, which can help you to further ensure the data is not only successfully copied from source to destination store, but also consistent between source and destination store by verifying the copied file names with their metadata in the log files. When you enable fault tolerance setting in copy activity to skip faulty data, the skipped files and skipped rows can also be logged. For more information, you can get details from here.

 

How to get started:

 

You can create a pipeline with a copy activity, and then configure the copy activity to enable session log as below:

session log.png

 

The sample log file is as below:

 

sessionlogfile.png

From the log file above, you can see sample1.csv has been skipped because it failed to be verified to be consistent between source and destination store. You can get more details about why sample1.csv becomes inconsistent is because it was being changed by other applications when ADF copy activity is copying at the same time. You can also see sample2.csv has been successfully copied from source to destination store.

1 Comment
Copper Contributor

Hi @Ye Xu ,

 

We're using a copy activity with a REST sink, any plans to expand the logging to include that?
Currently the logging only logs info on the source of the activity but not the sink.

 

Thanks

Version history
Last update:
‎Dec 01 2020 08:07 PM
Updated by: