Forum Discussion
DebbieH
Feb 09, 2021Brass Contributor
Limiting scans being captured every time Data Factory pipeline is executed
Hi,
How can you limit the scans that are done on the execution of a Data Factory pipeline?
Currently it seems you cannot select how frequently, the scan executes every time a pipeline is executed.
We would want the functionality to limit this when there have been no significant design changes.
- DebbieHBrass Contributor
Hi ChandruS , if the cost to acquire the information is less than the value for decision making we may want to turn the feature off or limit how often it runs. For example the information captured on a weekly basis may be sufficient and limited benefit to getting the information at each execution.
- ChandruSMicrosoftThanks for checking. At this point in Azure preview, we don't have any implications on cost for tracking the lineage once or every time the pipeline runs. We will keep in mind though at any point if this becomes a cost factor.