By default, Azure Data Factory reports lump sum charges for billing, meaning that at the factory level, we add up charges across all pipelines within a factory, and tell you how much you have spent on these pipelines. In many cases, these aggregate numbers should suffice. But in others, these numbers lack the clarity and transparency that we thrive to provide customers. For instance, if you are running data pipelines for multiple teams, you may want to determine the cost for each pipeline, for proper book-keeping and/or charge backs.
Now, Azure Data Factory will help you with this endeavor, with built-in per pipeline detailed billing view. Moreover, we built the feature on top of the Azure Billing and Cost Analysis platform, allowing you to stay with the cost and budget management tool that you are familiar with to identify spending trends and spot where overspending might have occurred.
By opting in to the feature, you will have a separate line item for each of your pipelines. Charges associated with a pipeline will be grouped together under the pipeline name, giving you a clear view of the cost of operations. You also have the chance to get the aggregate view for your factories, as you can filter the charges by factory name in the Azure billing report.
NOTE: There will be one entry for each pipeline in your factory. Please be particularly aware if you have excessive amount of pipelines in the factory, as it may significantly lengthen your billing report.
NOTE: the change only impacts how bills are emitted going forward, and does not change past charges. Please give some time before the change populate to billing report: typically, the change is updated within 1 day.
You need to opt-in to this feature for every factory you want detailed billing for. To turn on the per pipeline detailed billing feature, (1) go to Azure Data Factory portal; (2) under the Manage tab, select Factory settings in the General section; (3) select Show billing report (preview) by pipeline; and (4) publish the change.
This setting is not included in the exported ARM templates from your factory, meaning that Continuous Integration and Delivery (CI/CD) will not overwrite billing behaviors for the factory. This neat trick allows you to set different billing behaviors for development, test, and production environments, even when they share the same pipeline definitions.
These charges will file under a fallback line item for your factory:
For now, Dataflows running on Azure Integration runtime with a Time-to-Live (TTL) setting will file under a fallback line item for your factory. We are iterating to improve the experience for our users.
For all existing private preview customers that previously onboarded to the feature's private preview whitelist: we will continue to honor our commitment and allow detailed billing for your factories. However, for the time being, you may notice some discrepancies in the setting: even when the factory setting states billing by factory, you may see detailed billing in your billing report.
For existing private preview customers, we recommend the following steps:
We will gradually turn off the feature whitelist, as we progress to Public Preview and GA stages. Going forward, your factory settings will be the single source of control for the detailed billing behavior. Thank you so much for your understanding!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.