The Azure Data Factory team is happy to announce another round of new improvements to the monitoring experience. These improvements and new features are based off community feedback and votes, social media feedback, and survey results.
- First up, more columns are available for sorting. You can now sort on Pipeline Name, Run End, and Status. Additionally when viewing triggers you can now sort on Trigger Name and Status. Note: you may need to clear your browser cache to see these updated columns.
- Viewing the pipeline consumption report should now display a loading icon for large number of activities in a pipeline. Previously this data was streamed in but was not clear when the consumption information was fully calculated.
- Groupings of pipeline runs (for reruns) will now only fetch data when that grouping is expanded. Previously all re-runs were fetched for the user interface even if they were never viewed. This should greatly increase performance for monitoring pages that had multiple pipeline re-run groupings.
- Monitoring time zone settings are now saved in browser cache (along with your configured columns).
- In the pipeline activity runs page, pipelines with large number of activities will be able to fetch all records. Previously, if this list took longer than 60 seconds it may time out and not return all activities. Now you will be able to advance to further pages to fetch the next pages of activities.
We hope this next set of monitoring improvements helps to make your monitoring experience smoother, quicker, and more productive. We are always open for feedback on what’s next for monitoring in ADF and you can provide suggestions or vote via our Ideas forum Data factory · Community (azure.com).