Currently we are working to come up with a scheduling architecture for azure data factory pipelines .
As part of that one of the major requirements that we have is automated dependency detection based on scheduling metadata.
One of the options that we also evaluated is ADF triggers , however that looks good for our requirements at a small scale . On a mass scale for example around 1800 jobs it looks very effort intensive to manage dependencies via triggers .
Hence we are looking at some options whereby if we can control and automate dependency using azure synapse tables and t-SQL procedures and trigger ADF pipelines using t-SQL.
Looking for other scheduling tools that might meet this criteria . Open to guidance and suggestions.
