Feb 08 2021 09:57 PM
Hi,
How can you limit the scans that are done on the execution of a Data Factory pipeline?
Currently it seems you cannot select how frequently, the scan executes every time a pipeline is executed.
We would want the functionality to limit this when there have been no significant design changes.
Feb 09 2021 10:32 AM
@DebbieH The feature is to get operational information at the time of factory runtime. Such as status, rows impacted and much more data quality aspects in the future. We cant get such operational info at the definition time. May I know the requirement to limit the lineage capture?
Feb 15 2021 07:01 PM
Hi @ChandruS , if the cost to acquire the information is less than the value for decision making we may want to turn the feature off or limit how often it runs. For example the information captured on a weekly basis may be sufficient and limited benefit to getting the information at each execution.
Feb 16 2021 12:50 AM