Get tips for visibility and management of your cloud spend in Azure using your endpoint of choice, from the Azure Cost Management portal, Power BI for custom views and reporting, to API-based integration that feeds cost data into your existing financial system. Matt McSpirit, Senior Program Manager for Microsoft Azure, joins Jeremy Chapman to share how you can configure your Azure environment to meet specific cost reporting needs.
Whether you’re from the IT team or the finance team, choose from a number of options:
- Azure Cost Management in the Azure portal- Gives you the most comprehensive access to cost management views and reports.
- Azure Cost Management Power BI app- Great visibility with pre-built views out-of-the-box to track your Azure spend at the account, subscription, and resource group level.
- Power BI desktop connector- For more custom reporting, take our base Power BI template files and customize by building your own views.
- API-based integration- For ultimate control and customization, use APIs directly to integrate the data within your internal apps and financial systems.
QUICK LINKS:
01:29 — See your options
03:32 — Assess spend by department or business unit
05:22 — Configure cost tracking at granular workload level
07:14 — Establish management groups or tags retrospectively
08:38 — Options for Power BI
10:57 — Leverage APIs
11:51 — Wrap up
Link References:
Watch our episode on cloud economics essentials to understand the fundamentals of cloud costs at https://aka.ms/MechanicsAzureCloudEconomics
Get tips on naming your tags at https://aka.ms./CAFTagging
Download our cost management app at https://aka.ms/ACMApp
For guidance on the Azure Cost Management connector in Power BI desktop, go to https://aka.ms/ACMPowerBI
Learn more about configuring resources in Azure with the Azure setup guide at https://aka.ms/AzureSetupGuide
Find more tools, templates, and guidance at https://azure.com/cloudeconomics
Unfamiliar with Microsoft Mechanics?
We are Microsoft’s official video series for IT. You can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft.
- Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries?sub_confirmation=1
- Join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
- Watch or listen via podcast here: https://microsoftmechanics.libsyn.com/website
Keep getting this insider knowledge, join us on social:
- Follow us on Twitter: https://twitter.com/MSFTMechanics
- Follow us on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
Video Transcript:
- Up next, I’m joined by Azure expert Matt McSpirit, to look at lesser-known tips for visibility and management of your cloud spend in Azure using your endpoint of choice, from the Azure Cost Management portal, to Power BI for custom views and reporting to API-based integration that feeds cost data into your existing financial system. So Matt, it’s never a dull moment with you. Welcome back to the show.
- Well, it’s good to keep things interesting.
- And you always do. In fact, on today’s topic, it falls into the category of something that we get a lot of questions on. We did a recent episode actually with you on cloud economics essentials, to really explain the fundamentals of cloud costs, which if you missed that one, you can catch it at aka.ms/MechanicsAzureCloudEconomics. That said, to track your cloud costs in a way that makes sense for your organization is an area that a lot of people I think want to get a handle on.
- Yeah, you’re right. And in fact, this is an area where finance and technology teams can often find themselves disconnected. And there are a few parts to the challenge here. So on the one hand, there’s the issue of how costs are tracked. For example, you may need to map costs against your organizational structure or regions, or even against a specific timeline. And alternatively, there’s the issue of specificity and granularity of spend, which is important, especially if there are shared apps and workloads in use across your organization and you need to understand how costs should be distributed. And as with everything data related, the insights you get are only as good as the reporting foundation you set. Now, the trick here is that you really want to look at how you configure your Azure environment to meet specific cost reporting needs.
- Isn’t half the problem also how people are able to consume the data in a way that makes sense for whoever’s monitoring and reporting against that cloud spend?
- Yeah, that’s definitely the case. And the good news is, is we give you a number of options to get what you need, whether you’re from the IT team or the finance team. Now as an IT team, Azure Cost Management in the Azure portal gives you the most comprehensive access to cost management views and reports. Next, if you prefer Power BI, our Azure Cost Management Power BI app gives you great visibility with prebuilt views out-of-the-box to track your Azure spend at the account, subscription, and resource group level, by service and much more. And for more custom reporting, you can use our Power BI desktop connector to take our base Power BI template files and customize it by building your own views. Then lastly, for ultimate control and customization, you can also cull our APIs directly to integrate the data within your internal apps and financial systems.
- So you can consume cost management data in the portal via Power BI, also directly from the APIs. And you also mentioned, there’s a trick in this case to getting all the right data for your needs.
- Yeah, there’s definitely a level of planning that you want to do first to make sure that costs are broken down in the way you need them to be. And this starts by understanding a few basic constructs for tracking your spend. The first of which is a subscription, which you can use to delineate departments or cost centers in your organization. And underneath each subscription are your resources, which are organized in resource groups. Now, importantly, tags work to track specific consumption and can be assigned for your subscriptions, resource groups, and all the way down to specific resources. Now next, overseeing all of this, management groups can help you define a hierarchy over your subscriptions and resources. For example, using this construct, you can group sets of subscriptions and their resources for cost management, for example by business unit. And you can configure role-based access to offer privileged views over cost analysis at the department or down to the individual-user level.
- Great, so can you walk us through then a few scenarios for configuring the right cost management foundation in Azure, and why don’t we start with the most popular one, assessing spend by a particular department or a business unit?
- Right, so here is where management groups are fundamental. So these let you manage and organize your subscriptions in a way that’s meaningful to your organization. Now, creating one is simple enough. And in the management group blade, you can create a new management group or add a subscription to an existing management group. I’ve also defined my management group structure here under my tenant and Trey Research represents my organization hierarchy. And there are three nested management groups underneath it that denote my different business units worldwide. And by the way, you can create as many management groups as you need and nest them as required to represent your organization. And each of these have their own associated subscriptions aligned to specific departments or cost centers. So there are five subscriptions under Trey US and one, Trey International UK, that’s under Trey Worldwide. And this hierarchy will allow me to view costs at the subscription level, which in my case represents various departments, as well as anywhere in the hierarchy of our management groups, which in my case represent business units. So with our management groups created, let’s take a look at how this now helps you to track costs using Azure Cost Management in the Azure portal. I’m in management groups under cost analysis, and this is my view of costs across the entire account. And I want to view spend against the management group we just set up. Under the scope picker, I’ll select my root management group option, and you’ll see my Trey research management group just popped up. And as I click in, you can see our associated management group nodes. So let’s look at Trey US, which if you recall had five subscriptions under it. This gives me a view of overall spend across this particular business unit. And it’s grouped by subscription so I can look at the cost breakdown for each of the represented departments.
- Why don’t we take a look at what you need to do then to configure cost tracking at that specific granular workload level?
- Right, and this is really where tagging becomes imperative. And as a rule, you’re best off tagging resources as they’re created to be able to associate cloud usage costs appropriately. You’ll want to spend some time establishing the right approach and naming of your tags for consistency. And in fact, I recommend looking at our cloud adoption framework guidance to get tips on how to go about this at aka.ms./CAFTagging. So let me show you how this works. I’m in the virtual machine blade. And as we click in, I can see my virtual machines and I’ll click into the first one. And I can view its tags. And you’ll see it’s been tagged by environment, and that environment is associated to different values. In this case, a StorSimple environment. Now you can think of a value as a qualifier for the tag. For example, if you create a tag named Country, then the associated values could equal country names. This is also where I can update the tags or add tags. And if I want to apply the same tag across a given subscription, for example, like our Trey International UK subscription, I can always set the tag at the subscription level. And because I’m using the same tag for my subscription, all the resources under this subscription will inherit the tag. Equally, I also have the option to set tags at the resource group level under my subscription. Now, the path for reviewing existing tags or adding or updating tags, is the same as I just showed you. And so because these tags have been established, here’s the payoff. When I go into Azure Cost Management in the portal under cost analysis, you have the option to filter costs by tags. Now, of course, we want to figure out the cost of a specific workload. And to do this, I can also group costs by specific tags. Under group, when I select tags, it brings up a list of all tags that have been defined to track costs. I’ll select costs by environment. And if I scroll down, I can see cost groups by the tags that have been created in this pie chart.
- And this is all really great guidance, but it does require that everyone gets into the habit of establishing management groups or tags. Now I saw there were about $300 in your case of untagged resources in the chart that you showed, probably where someone may have skipped the tagging step when creating resources like a virtual machine, but what can you do then retrospectively?
- You can, of course, set up management groups at any time. Equally, you can enforce tags on future or existing resources if you need to as well. If you want your subscription, resource groups, or resources to inherit a tag, in Azure policy all you need to do is create and assign a policy. Once you to assign policy, you can set the scope of the policy. In our case, we want it to run at the subscription level. I’ll select Trey International UK, next I’ll pick the policy I want to enforce, and we have a pre-configured inheritance policy available in the portal. So I’ll search for that. And you can see it brings up a few options at the subscription and the resource group level. The first policy adds a specified tag if it’s missing on a resource in the subscription. And the second one listed ensures that resources created inherit a tag from the subscription, which is what I want so I’ll select that one. And once I hit review and create, all new resources from this time onwards will inherit the tag from this subscription. Also something pretty cool: by selecting remediate, I can make sure that the tag is applied to existing resources as well as any new resources created on the subscription. And again, once that policy is enabled, filtering against these tags will work from that day forward.
- And that’s pretty reassuring, but that said, so far you’ve shown the benefits of Azure configuration as you use the portal. If you don’t live in the portal though, how does this effort then translate to things like Power BI?
- So once you’ve got everything configured as the way you want it, as I mentioned, there are two options for using Power BI. First, you can use our pre-built Power BI templates by downloading our cost management app at aka.ms/ACMApp. Now, once you’ve installed it, these templates, as I mentioned before, provide a great starting point. And they also help you to spot opportunities for cost optimization, such as reserved instances, which allow you to read cost savings compared to pay-as-you-go prices in return for a one or a three-year reservation. Our pre-built reserved instance coverage report gives you specific recommendations on reserved instance usage so that you can assess potential savings. And the Azure Hybrid Benefit report, for example, shows you how well you’re doing in terms of bringing over your on-premises licenses to the cloud, including how many cores this applies to, which you’ve taken advantage of properly can help you save up to 40%. And in fact, when you combine reserved instances and Azure Hybrid Benefit, your savings can be up to 80%. Now that said, the real power of Power BI is in the creation of custom views. And for that you’ll want to download the Azure Cost Management connector in Power BI desktop to make direct pulls against the data in Azure Cost Management. And we have specific guidance on how to do that here at aka.ms/ACMPowerBI. Now I started to build a customized report and here the data from our management groups are being pulled in on the right and I’m mapping the management group structure with my usage data. If I click in, I’m able to define the relationship between these two datasets, which are linked based on subscription. And now I can go into the report and create my own custom views. I can see the breakdown of management groups that I define in Azure and against all the subscriptions in the management group that align to specific departments in my organization. I’ve also created a view of costs by workload, which is pulling from the tags we created. And as you can see from the cost breakdown by tags, most of my costs are coming from the production tag for my production workloads. Now, this is just a taste of some of the custom views that you can create, and the sky really is the limit to tailor views and reports for your organizational needs. You could also, for example, set up historical reporting that goes back further than the three month views available in the portal.
- How do you then leverage APIs, maybe to integrate with your existing financial systems?
- So here you can integrate our APIs with your preferred finance and reporting systems. The export feature in Cost Management is an easy way to export your usage files into a storage blob. And you can automate regular exports to keep your reporting up to date. For example, here I’ve set up my data export to occur daily. I can decide to pull in data on my actual or amortized costs and enter other parameters such as the frequency of my export, and when I want the export to start. And if I’ve got a large file, I can split it up into multiple files by selecting file partitioning. I can also pick the subscription that the storage account will be associated with, as well as my container information and directory where I want to place the file. And once I save this, the data export will run based on the frequency I’ve specified, which in this case is daily. It saves by default as a CSV file. And you can also view it as a JSON file, as you can see here.
- So there are three great options then, depending on your preference and perspective then for tracking cloud spend in Azure. Of course, there’s probably more questions than we actually had time for to answer on the show today. So how can folks then watching learn more about all this?
- Well, to learn more about configuring resources in Azure, the Azure setup guide as part of the cloud adoption framework methodology is a great option for you, at aka.ms/AzureSetupGuide. Now at the end of the day, the data is there available to you in Azure. So with the right planning as an organization, you should be able to access what you need and map it to your reporting needs. And you can also find more tools, templates, and guidance at azure.com/cloudeconomics.
- Thanks so much for sharing these great tips today Matt, and also thank you for watching. And be sure to keep checking back to Microsoft Mechanics for all the latest updates. Subscribe to our channel if you haven’t already, and we’ll see you next time.
