Optimize Azure Log Costs: Split Tables and Use the Auxiliary Tier with DCR
This blog is continuation of my previous blog where I discussed about saving ingestion costs by splitting logs into multiple tables and opting for the basic tier! Now that the transformation feature for Auxiliary logs has entered Public Preview stage, I’ll take a deeper dive, showing how to implement transformations to split logs across tables and route some of them to the Auxiliary tier. A quick refresher: Azure Monitor offers several log plans which our customers can opt for depending on their use cases. These log plans include: Analytics Logs – This plan is designed for frequent, concurrent access and supports interactive usage by multiple users. This plan drives the features in Azure Monitor Insights and powers Microsoft Sentinel. It is designed to manage critical and frequently accessed logs optimized for dashboards, alerts, and business advanced queries. Basic Logs – Improved to support even richer troubleshooting and incident response with fast queries while saving costs. Now available with a longer retention period and the addition of KQL operators to aggregate and lookup. Auxiliary Logs – Our new, inexpensive log plan that enables ingestion and management of verbose logs needed for auditing and compliance scenarios. These may be queried with KQL on an infrequent basis and used to generate summaries. Following diagram provides detailed information about the log plans and their use cases: More details about Azure Monitor Logs can be found here: Azure Monitor Logs - Azure Monitor | Microsoft Learn **Note** This blog will be focussed on switching to Auxiliary logs only. I would recommend going through our public documentation for detailed insights about feature-wise comparison for the log plans which should help you in taking right decisions for choosing the correct log plans. At this stage, I assume you’re aware about different log tiers that Azure Monitor offers and you’ve decided to switch to Auxiliary logs for high volume, low-fidelity logs. Let’s look at the high-level approach we’re going to follow to achieve this: Review the relevant tables and figure out which portion of the log can be moved to Auxiliary tier Create a DCR-based custom table which same schema as of the original table. For Ex. If you wish to split Syslog table and ingest a portion of the table into Auxiliary tier, then create a DCR-based custom table with same schema as of the Syslog table. At this point, switching table plan via UI is not possible, so I’d recommend using PowerShell script to create the DCR-based custom table. Once DCR-based custom table is created, implement DCR transformation to split the table. Configure total retention period of the Auxiliary table (this configuration will be done while creating the table) Let’s get started Use Case: In this demo, I’ll split Syslog table and route “Informational” logs to the Auxiliary table. Creating a DCR-based custom table: Previously a complex task, creating custom tables is now easy, thanks to a PowerShell script by MarkoLauren. Simply input the name of an existing table, and the script creates a DCR-based custom table with the same schema. Let’s see it in action now: Download the script locally. Update the resourceID details in this script and save it. Upload the updated script in Azure Shell. Load the file and enter the table name from which you wish to copy the schema. In my case, it's going to be "Syslog" table. Enter new table name, table type and total retention period, shown below: **Note** We highly recommend you review the PowerShell script thoroughly and do proper testing before executing it in production. We don't take any responsibility for the script. As you can see, Aux_Syslog_CL table has been created. Let’s validate in log analytics workspace > table section. Since the Auxiliary table has been created now, next step is to implement transformation logic at data collection rule level. The next step is to update the Data Collection Rule template to split the logs Since we already created custom table, we should create a transformation logic to split the Syslog table and route the logs with SeverityLevel “info” to the Auxiliary table. Let’s see how it works: Browse to Data Collection Rule blade. Open the DCR for Syslog table, click on Export template > Deploy > Edit Template as shown below: In the dataFlows section, I’ve created 2 streams for splitting the logs. Details about the streams as follows: 1 st Stream: It’s going to drop the Syslog messages where SeverityLevel is “info” and send rest of the logs to Syslog table. 2 nd Stream: It’s going to capture all Syslog messages where SeverityLevel is “info” and send the logs to Aux_Syslog_CL table. Save and deploy the updated template. Let’s see if it works as expected Browse to Azure > Microsoft Sentinel > Logs; and query the Auxiliary table to confirm if data is being ingested into this table. As we can see, the logs where SeverityLevel is “info” is being ingested in the Aux_Syslog_CL table and rest of the logs are flowing into Syslog table. Some nice cost savings are coming your way, hope this helps!On-Demand Backups in Azure Database for PostgreSQL – Flexible Server Now Generally Available
We’re excited to announce the General Availability of On-Demand Backups in Azure Database for PostgreSQL – Flexible Server! In today’s dynamic data management landscape, ensuring the protection and recoverability of your databases is essential. Azure Database for PostgreSQL – Flexible Server streamlines this responsibility through comprehensive backup management, including automated, scheduled storage volume snapshots encompassing the entire database instance and all associated transaction logs. With the introduction of On-demand backups you now have the flexibility to initiate backups at any time, supplementing the existing scheduled backups. This capability is particularly valuable in scenarios involving high-risk operations, such as system upgrades or schema modifications, or when performing periodic data refreshes that do not align with the standard backup schedule. Benefits Instant Backup Creation: Trigger a full backup of your server on demand—no more waiting for the automated backup schedule. Cost Optimization: While Azure manages automated backups that cannot be deleted until the retention window is met, on-demand backups provide greater control over storage costs. Delete these backups once their purpose is served to avoid unnecessary storage expense. Enhanced Control & Safety: Take backups before schema changes, major deployments, or periodic refresh activities to meet your business requirements. Seamless Integration: Accessible via Azure Portal, Azure CLI, ARM templates, and REST APIs. Azure Database for PostgreSQL Flexible Server provides a comprehensive, user-friendly backup solution, giving you the confidence to manage your data effectively and securely. Let us explore how on-demand backups can elevate your database management strategy and provide peace of mind during high-stakes operations. Automated Backups vs On-Demand Backups Feature Automated Backups On-Demand Backups Creation Scheduled by Azure Manually initiated by the user Retention Based on the backup policy Based on the backup policy Deletion Managed by Azure User-controlled Use Cases Regular data protection High-risk operations, ad-hoc needs How to take On-Demand Backups using the portal. In the Azure portal, choose your Azure Database for PostgreSQL flexible server. Click Settings from the left panel and choose Backup and Restore. Click Backup and provide your backup name. Click Backup. A notification is shown that an On-demand backup trigger has been initiated. For more information: How to perform On-demand backups using Portal How to take On-Demand Backups using CLI. You can run the following command to perform an on-demand backup of a server. az postgres flexible-server backup create --resource-group <resource_group> --name <server> --backup-name <backup> Example: For more information: How to perform On-demand backups using CLI How to list all on-demand backups using CLI You can list currently available on-demand backups of a server via the az postgres flexible-server backup list command. az postgres flexible-server backup list --resource-group <resource_group> --name <server> --query "[?backupType=='Customer On-Demand']" --output table For more information: How to list all backups using Portal What's Next Once you have taken an on-demand backup based on your business needs, you can retain it until your high-risk operation is complete or use it to refresh your reporting or non-production environments. You can delete the backups to optimize storage costs when the backup is no longer needed. To restore or delete on-demand backups, you can use the Azure portal, CLI, or API for seamless management. Limitations & Considerations: SKU Support: On-demand backups are available for General Purpose and Memory-Optimized SKUs. Burstable SKUs are not supported. Storage Tier Compatibility: Currently, only the SSDv1 storage tier is supported. Support for SSDv2 is on our roadmap and will be introduced in a future update. You can take up to 7 on-demand backups per flexible server. This limit is intentional to help manage backup costs, as on-demand backups are meant for occasional use. The managed service already provides support for up to 35 backups in total, excluding on-demand backups. Take Control of Your Database Protection Today! The ability to create on-demand backups is critical for managing and safeguarding your data. Whether you're preparing for high-risk operations or refreshing non-production environments, this feature puts flexibility and control in your hands. Get started now: Create your first on-demand backup using the Azure Portal or CLI. Optimize your storage costs by deleting backups when no longer needed. Restore with ease to keep your database resilient and ready for any challenge. Protect your data effectively and ensure your database is always prepared for the unexpected. Learn more about Azure Database for PostgreSQL Flexible Server and explore the possibilities with on-demand backups today! You can always find the latest features added to Flexible server in this release notes page. We are eager to hear all the great scenarios this new feature helps you optimize, and look forward to receiving your feedback at https://aka.ms/PGfeedback.
Azure VMWare (AVS) Cost Optimization Using Azure Migrate Tool
What is AVS? Azure VMware Solution provides private clouds that contain VMware vSphere clusters built from dedicated bare-metal Azure infrastructure. Azure VMware Solution is available in Azure Commercial and Azure Government. The minimum initial deployment is three hosts, with the option to add more hosts, up to a maximum of 16 hosts per cluster. All provisioned private clouds have VMware vCenter Server, VMware vSAN, VMware vSphere, and VMware NSX. As a result, you can migrate workloads from your on-premises environments, deploy new virtual machines (VMs), and consume Azure services from your private clouds. Learn More: https://learn.microsoft.com/en-us/azure/azure-vmware/introduction What is Azure Migrate Tool? Azure Migrate is a comprehensive service designed to help you plan and execute your migration to Azure. It provides a unified platform to discover, assess, and migrate your on-premises resources, including servers, databases, web apps, and virtual desktops, to Azure. The tool offers features like dependency analysis, cost estimation, and readiness assessments to ensure a smooth and efficient migration process. Learn More: https://learn.microsoft.com/en-us/azure/migrate/migrate-services-overview How Azure Migrate can be used to Discover and Assess AVS? Azure Migrate enables the discovery and assessment of Azure VMware Solution (AVS) environments by collecting inventory and performance data from on-premises VMware environments, either through direct integration with vCenter (via Appliance) or by importing data from tools like RVTools. Using Azure Migrate, organizations can analyze the compatibility of their VMware workloads for migration to AVS, assess costs, and evaluate performance requirements. The process involves creating an Azure Migrate project, discovering VMware VMs, and generating assessments that provide insights into resource utilization, right-sizing recommendations, and estimated costs in AVS. This streamlined approach helps plan and execute migrations effectively while ensuring workloads are optimized for the target AVS environment. Note: We will be narrating the RVtools Import method in this article. What Is RVTools? RVTools is a lightweight, free utility designed for VMware administrators to collect, analyze, and export detailed inventory and performance data from VMware vSphere environments. Developed by Rob de Veij, RVTools connects to vCenter or ESXi hosts using VMware's vSphere Management SDK to retrieve comprehensive information about the virtual infrastructure. Key Features of RVTools: Inventory Management: Provides detailed information about virtual machines (VMs), hosts, clusters, datastores, networks, and snapshots. Includes details like VM names, operating systems, IP addresses, resource allocations (CPU, memory, storage), and more. Performance Insights: Offers visibility into resource utilization, including CPU and memory usage, disk space, and VM states (e.g., powered on/off). Snapshot Analysis: Identifies unused or orphaned snapshots, helping to optimize storage and reduce overhead. Export to Excel: Allows users to export all collected data into an Excel spreadsheet (.xlsx) for analysis, reporting, and integration with tools like Azure Migrate. Health Checks: Identifies configuration issues, such as disconnected hosts, orphaned VMs, or outdated VMware Tools versions. User-Friendly Interface: Displays information in tabular form across multiple tabs, making it easy to navigate and analyze specific components of the VMware environment. Hand-on LAB Disclaimer: The data used for this LAB has no relationship with real world scenarios. This sample data is self-created by the author and purely for understanding the concept. To discover and assess your Azure VMware Solution (AVS) environment using an RVTools extract report in the Azure Migrate tool, follow these steps: Prerequisites RVTools Setup: Download and install RVTools from the Official Website Ensure connectivity to your vCenter server. Extract the data by running RVTools and saving the output as an Excel (.xlsx) file Permissions: You need at least the Contributor role on the Azure Migrate project. Ensure that you have appropriate permissions in your vCenter environment to collect inventory and performance data. File Requirements: The RVTools file must be saved in .xlsx format without renaming or modifying the tabs or column headers. Note: Sample Sheet: Please check the attachment included with this article. Note that this is not the complete format; some tabs and columns have been removed for simplicity. During the actual discovery and assessment process, please do not modify the tabs or columns. Procedure Step 1: Export Data from RVTools Follow the steps provided in official website to get RVTools Extract Sample Sheet: Please check the attachment included with this article. Note that this is not the complete format; some tabs and columns have been removed for simplicity. During the actual discovery and assessment process, please do not modify the tabs or columns. Step 2: Discover Log in to the Azure portal. Navigate to Azure Migrate and select your project or create new project. Under Migration goals, select Servers, databases and web apps. On Azure Migrate | Servers, databases and web apps page, under Assessment tools, select Discover and then select Using import. In Discover page, in File type, select VMware inventory (RVTools XLSX). In the Step 1: Import the file section, select the RVTools XLSX file and then select Import. Wait for some time to Import Once import completed check for Error Messages if any and rectify those and re upload, otherwise wait 10-15 minutes to reflect imported VMs in the discovery. Post discovery Reference Link: https://learn.microsoft.com/en-us/azure/migrate/vmware/tutorial-import-vmware-using-rvtools-xlsx?context=%2Fazure%2Fmigrate%2Fcontext%2Fvmware-context Step 3: Assess After the upload is complete, navigate to the Servers tab. Click on Assess -->Azure VMware Solution to assess the discovered machines. Edit assessment settings based on your requirements and Save Target region: Select the Azure region for the migration. Node Type: Specify the Azure VMware Solution series (e.g., AV36, AV36P). Pricing model: Select pay-as-you-go or reserved instance pricing. Discount: Specify any available discounts. Note: We will be explaining all the parameters in optimize session. As of now just review and leave parameters as it is. In Assess Servers, select Next. In Select servers to assess > Assessment name > specify a name for the assessment. In Select or create a group > select Create New and specify a group name. Select the appliance and select the servers you want to add to the group. Then select Next. In Review + create assessment, review the assessment details, and select Create Assessment to create the group and run the assessment. Step 4: Review the Assessment View an assessment In Windows, Linux and SQL Server > Azure Migrate: Discovery and assessment, select the number next to Azure VMware Solution. In Assessments, select an assessment to open it. As an example (estimations and costs, for example, only): Review the assessment summary. You can select Sizing assumptions to understand the assumptions that went in node sizing and resource utilization calculations. You can also edit the assessment properties or recalculate the assessment. Step 5: Optimize We have received a report without any optimization in our previous steps. Now we can follow below steps to optimize the cost and node count even further High level steps: Find limiting factor Find which component in settings are mapped for optimization depending on limiting factor Try to adjust the mapped component according to Scenario and Comfort Find Limiting factor: First understand which component (CPU, memory and storage) is deciding your ESXI Node count. This will be highlighted in the report The limiting factor shown in assessments could be CPU or memory or storage resources based on the utilization on nodes. It is the resource, which is limiting or determining the number of hosts/nodes required to accommodate the resources. For example, in an assessment if it was found that after migrating 8 VMware VMs to Azure VMware Solution, 50% of CPU resources will be utilized, 14% of memory is utilized and 18% of storage will be utilized on the 3 Av36 nodes and thus CPU is the limiting factor. Find which option in the setting can be used to optimize: This is depending on the limiting factor. For eg: If Limiting factor is CPU, which means you have high CPU requirement and CPU oversubscription can be used to optimize ESXI Node. Likewise, if storage is the limiting factor editing FTT, RAID or introducing External storage like ANF will help you to reduce Node count. Even reducing one node count will create a huge impact in dollar value. Let's understand how over commitment or over subscription works with simple example. Let's suppose I have two VMs with below specification Name CPU Memory Storage VM1 9 vCPU 200 GB 500 GB VM2 4 vCPU 200 GB 500 GB Total 13 vCPU 400 GB 1000 GB We have EXSI Node which has below capacity: vCPU 10 Memory 500 GB storage 1024 GB Now without optimization I need two ESXI node to accommodate 13 vCPU of total requirement. But let's suppose VM1 and VM2 doesn't consume entire capacity all the time. The total capacity usage at a time will not go beyond 10. then I can accommodate both VM in same ESXI node, Hence I can reduce my node count and cost. Which means it is possible to share resources among both VMs. Without optimization With optimization Parameters effecting Sizing and Pricing CPU Oversubscription Specifies the ratio of number of virtual cores tied to one physical core in the Azure VMware Solution node. The default value in the calculations is 4 vCPU:1 physical core in Azure VMware Solution. API users can set this value as an integer. Note that vCPU Oversubscription > 4:1 may impact workloads depending on their CPU usage. Memory overcommit factor Specifies the ratio of memory overcommit on the cluster. A value of 1 represents 100% memory use, 0.5, for example is 50%, and 2 would be using 200% of available memory. You can only add values from 0.5 to 10 up to one decimal place. Deduplication and compression factor Specifies the anticipated deduplication and compression factor for your workloads. Actual value can be obtained from on-premises vSAN or storage configurations. These vary by workload. A value of 3 would mean 3x so for 300GB disk only 100GB storage would be used. A value of 1 would mean no deduplication or compression. You can only add values from 1 to 10 up to one decimal place. FTT : How many device failure can be tolerated for a VM RAID : RAID stands for Redundant Arrays of Independent Disks Explains how data should be stored for redundancy Mirroring : Data will be duplicated as it is to another disk E.g.: To protect a 100 GB VM object by using RAID-1 (Mirroring) with an FTT of 1, you consume 200 GB. Erasure Coding : Erasure coding divides data into chunks and calculates parity information (redundant data) across multiple storage devices. This allows data reconstruction even if some chunks are lost, similar to RAID, but typically more space-efficient E.g.: to protect a 100 GB VM object by using RAID-5 (Erasure Coding) with an FTT of 1, you consume 133.33 GB. Comfort Factor: Azure Migrate considers a buffer (comfort factor) during assessment. This buffer is applied on top of server utilization data for VMs (CPU, memory and disk). The comfort factor accounts for issues such as seasonal usage, short performance history, and likely increases in future usage. For example, a 10-core VM with 20% utilization normally results in a 2-core VM. However, with a comfort factor of 2.0x, the result is a 4-core VM instead. AVS SKU Sizes Optimization Result In this example we got to know that CPU is my limiting factor hence I have adjusted CPU over subscription value from 4:1 to 8:1 Reduced node count from 6 (3 AV36P+3 AV64) to 5 AV36P Reduced Cost by 31% Note: Over-provisioning or over-committing can put your VMs at risk. However, in Azure Cloud, you can create alarms to warn you of unexpected demand increases and add new ESXi nodes on demand. This is the beauty of the cloud: if your resources are under-provisioned, you can scale up or down at any time. Running your resources in an optimized environment not only saves your budget but also allows you to allocate funds for more innovative ideas.Enhance Cost Optimization in Azure Cosmos DB Without Compromising Service
Optimizing Azure cosmos DB involves using strategies and best practices to reduce the overall spending on the service while maintaining and improving performance and availability. In this blog, I’ll explore actionable strategies and tools for reducing your Azure Cosmos DB costs without sacrificing the speed, scalability, or reliability of your database. Whether you're managing large-scale applications or developing on a budget, these insights will help you make the most out of Azure Cosmos DB. The main sub-topics we shall dive into includes: Pricing model Free development Plan for OptimizationHow to use Defender for Cloud App with cost optimization for both environment ( Dev and Prod )
Hi All, I have two subscriptions 1. Development and 2. Production. In the Dev subscription, I have a lot of resources like about 20 storage accounts and 12 app service plans and 4 Azure SQL and etc. As you know, Defender for Cloud is subscription level, therefor If I enable it on a Dev subscription the cost should be more expensive. But in the Prod environment, I will enable Defender for Prod's resources. Now, I want to know how can I use Defender for Dev's resources with minimum cost or what's the best solution or best practice for this issue. My idea is to use Prod security recommendations for the same resources in the Dev environment. Is there another idea?How to reduce your Azure invoice?
I won't tell anyone that Cloud Computing's billing, is based on the consumption of your resources, which is called Pay As You Go (PAYG), which is one of its advantages. One of the best practices before deploying in the Cloud is obviously to estimate the costs of the resources you are going to deploy, simply to get an approximate idea of what they will cost to you. The problem is that the estimate is rarely made. Then, for some companies, costs are booming and it can be difficult to know where to start to try to reverse the trend. Some of them, after a bad financial experience, decide to stop their journey to the public Cloud, considering it too expensive. Others will define strong rules, and sometimes too strict, which go against innovation, which can punish the daily life of the Innovation and R&D teams. Fortunately, FinOps principles appeared to help companies to have a good knowledge about their Cloud costs. FinOps is the contraction of Finance and Operation. The https://www.finops.org/introduction/what-is-finops/ defines FinOps: as an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology and business teams to collaborate on data-driven spending decisions. Like any cloud provider, Azure offers several services and solutions to reduce your invoice, and ensure that the costs associated to your resources are not blockers to cloud adoption. To avoid this situation, let's see together the levers that will allow you to reduce your Azure bill. Azure Calculator As mentioned earlier, to avoid bad surprises and find out the cost of the resources deployed, one of the best practices is to estimate your costs before to deploy anything. So I see you coming, an estimate is still an estimate, but it can give you an idea of the budget for an application or a project. Let's imagine that the business wants to deploy a new service to order take-out meals. Obviously, once you have gathered the different elements, such as the needs, the technical, legal and financial constraints, you can move on to the phase of defining your architecture. Once defined, it will be time to estimate it, because it is pretty sure that the business doesn't have an infinite budget. It is at this precise moment that https://azure.microsoft.com/en-us/pricing/calculator/ takes all its interest. To estimate the cost of your services, simply select them from the list and enter their configuration that you are considering. Obviously several parameters come into consideration for the estimate, and some are not easy to establish such as the incoming / outgoing traffic at the level of your application, the bandwidth, or the storage required. But, remember, we do not want to have a precise amount, which is impossible to obtain, but an order of magnitude. Do not hesitate to be accompanied by a Cloud Architect, who will have defined the solution, to get you as close as possible to a coherent estimate. And as Leonardo da Vinci said: Not to anticipate is already to moan. Delete unused/orphaned resources Another way to simply lower your bill is to remove resources you don't use. Keep in mind that you are charged even when you are not using a service. It is therefore necessary to hunt for unused resources. So at first glance it seems simple, when you have a small team deploying resources, but imagine, when you are in a large organization with dozens, or maybe hundreds of people deploying resources, then it becomes a real brain teaser. Another good practice on Azure is to tag your resources. Tags allow you to add information in the form of keys/values, in order to identify, for example, the owner of a resource, the person who deployed it, or even the project to which this resource is attached. If you don't know if the resource is still in use, you can, at a minimum, contact the owner of the resource or the person who deployed it to get the information, and see if you can clean up. Remember to also delete the various orphan resources. By orphan resources, I mean those that are not attached to any service but for which you are still charged. This can be VM disks, public IP addresses, or even App Services Plans, which you are billed for even though these resources are no longer used. Here is for example the PowerShell command to list the unassociated hard disks: Get-AzDisk | where-object {$_.Diskstate -eq 'Unattached'} Or the PowerShell command to list the public IP addresses not associated with a resource: Get-AzPublicIpAddress | Where-Object {$_.IpConfiguration -eq $null} It may be interesting to create a script, which runs periodically to remove these unused resources. Stop and start your VMs For those who use VMs, one of the fairly simple solutions to reduce your bill is to stop and start your resources. If we draw a parallel with your electricity, especially in these more than complicated times, when you're not at home, you turn off your lights, well it's the same with Azure and the Cloud in general. So you are going to answer me, yes but how do I do with my Production VMs? Well precisely, our target is not necessarily the Production environment but the others, such as the Development, Acceptance, UAT, Staging, Pre-production environments, ... It is almost certain that the VMs of these environments do not have to be running all the time, so we can consider turning them off at night and even on weekends. And casually, these are significant costs that you can eliminate. Of course, it is possible to shut down and restart VMs automatically, either directly at the VM level for shutdown with the Auto-Shutdown option. Obviously, the start can also be automated with an Automation Account, through templats that are based on tags, considerably reducing your efforts. Right size your resources As you know, on many Azure services, the price is partly calculated, on the amount of allocated resources such as computing, network, and storage. Thus, more you have define resources, more the price will be high. You must therefore be vigilant to right size your resources, depending on the service, or the needs you have. It is useless to provision 8vCPU and 16GB of memory to host a Web server on a VM, or to provide a cache of 26GB of memory for an Azure Cache for Redis, if you only have 4GB of data to store! Let's take a new concrete example, I work at 10km from my home. I have the choice between a small electric car, and a large diesel 4x4, obviously I opt for the small car because it seems to be the most suitable for my needs. On Azure it's the same, size your resources according to your need. Of course, you can provide an additional margin that will be used during additional workload peaks. VM instance type versions Azure offers you different https://azure.microsoft.com/en-us/pricing/details/virtual-machines/series/ such as general use, optimized memory, high performance computing to name a few, with their different configurations depending of you needs. But that's not all. We are talking about Cloud Computing with Azure, the services offered are hosted on physical servers. These physical servers are arranged in racks, and these racks are stored in Microsoft Datacenters. And like any physical material, it must obviously be maintained, but also replaced in the event of failure. You have to know that when Microsoft teams add hardware or replace obsolete one or defective hardware, this new hardware is generally more powerful, and more affordable than that installed previously. (Obviously, this applies during stable economic periods, not like today with a context of war in Eastern Europe, but also with the shortage of electrical components, or the explosion of energy-related prices like it is the casenow). This is how new versions of instance types appear, suffixed in v2, v3, v4, v5, and indeed in normal times, the most recent versions offer more attractive prices than the old ones. And that's not all because you can also find instance types Promo for a limited period of time, which could be very interesting for their lowest prices. Azure VM Spot Microsoft also offers to use unused computing capacity at a given time to achieve savings of up to 90% of public prices on VMs, with the https://azure.microsoft.com/en-us/products/virtual-machines/spot/#overview feature. How does it work exactly? Sometimes, the Azure's infrastructure is not fully used, and rather than seeing its Infrastructure unused, Microsoft decided to offer this computing at a lower price. I can already see a question on your lips “What happens when Azure needs this computing power again? » It's simple, depending on the configuration you have defined, either your VM stops or it is deleted. Know that you can even set a floor price, below the public price, that you are ready to pay. Once this price is reached, your VM either shuts down or is deleted. If you plan to use this option, you can take help of Azure Resource Graph which will allow you to obtain the price history during the last 90 days and the eviction rates during the last 28 days and thus help you to set this price. So you will have understood that Spot VMs cannot be used for all types of scenarios, and even less for critical or production environments. On the other hand, they apply perfectly to the use cases of workloads capable of managing interruptions, such as batch processing jobs, development environments, tests, among others. A notification 30 seconds before the eviction of your VM, is sent to the system in order to warn it and anticipate the stop or the deletion of this one. The feature, Azure VM Spot, applies to Linux and Windows VMs, as well as the Virtual Machine Scale Set (VMSS) service. Price varies by region Another factor that affects the price of a service is the region in which it is deployed. But how is this explained? The price of a service is obviously calculated on the cost of the hardware on which it runs, but other parameters come into consideration such as: The labor cost of the country where the Datacenter is located The cost of energy that allows the operation of the Datacenter But other elements can influence the price, such as: The financial aid granted by certain countries to promote the establishment The national or global economic context But also the geopolitical stability of a region So obviously this factor should not be the one that defines the region in which you are going to deploy your application, but at least now you are aware that a service does not have the same cost if it is deployed in the US or in Brazil. Scaling mechanisms Scaling or autoscaling, is what allows a service or an application to adapt to a peak in traffic, requests, CPU/memory consumption, by provisioning new resources to absorb this surplus of workload. There are two scaling modes: Horizontal scaling or scale-out is when you provision new instances of a service or application. This can be manual or automatic scaling. Vertical scaling or scale-up is when you increase the capabilities of an existing instance. This action is generally manual, because it can lead to a service interruption, especially at the VM level. The choice of the scaling mode will depend of your application. Scale-out is generally used when the application is stateless, it means that no application data are stored at the level of the instance that executes the workload. Conversely, scale-up is used when data is stored at the instance level, then we speak of a statefull application. Here is a https://www.redhat.com/topics/cloud-native-apps/stateful-vs-statelesswhich explain furthr the difference between a stateless application and a statefull application. Of course, the horizontal scaling can be manual, but often automatic. The latter is based on criteria that you define, such as a rule that will deploy two new instances of your application, if the average CPU is greater than 70% during the last ten minutes. But the most interesting thing is that you can also define rules for de-provisioning, i.e. deleting instances on the same principle, but for stateless applications of course. So when the average CPU is less than 40% over the last fifteen minutes, an instance can be deleted. And since no data is stored there, there is no impact on your application. Other criteria are available, such as the hourly schedule to add or remove the number of instances. This criterion can be used if you are able to anticipate your traffic upstream, with for example video game servers, which are more popular in the evening and on weekends than during the day. In short, scaling mechanisms are valuable allies in the quest to reduce costs, provided that you define the right trigger thresholds, and all of this of course, without impacting the service level of your application. Reserved Instances One of the most used levers to control your costs in Azure is undoubtedly that of https://learn.microsoft.com/en-us/azure/cost-management-billing/reservations/save-compute-costs-reservations or RI. The very simple idea is to make a contractual commitment over a period of 1 to 3 years on the use of a service, in order to benefit from very significant discounts, which can sometimes reach more than 70% of the initial price on certain services. Obviously, the longer the commitment period, the more attractive the discounts. So by making a reservation, it will no longer be the public price that will be billed to you, but the new "reserved" price that will depend on your commitment. Several services are now available for reservations, such as VMs, databases (SQL, MySQL, PostgreSQL, Redis), computing resources but also storage. Be careful though when setting up reservations. As you commit to a period of 1 to 3 years, make sure that the reservation covers a service whose lifetime will be greater than your commitment, because once the reservation is made, you pay even if your resource is not not used. Azure Hybrid Benefit https://azure.microsoft.com/en-us/pricing/hybrid-benefit/ is a solution that allows you to use your existing licenses, such as Windows Server, Microsoft SQL, but also Red Hat Enterprise Linux (RHEL) or SUSE Linux Enterprise Server (SLES), to significantly reduce your licensing costs. your VMs hosted on Azure, but also on the SQL Database service. How does it actually work? On the Windows Server or Windows SQL part (on VM or on SQL Database), you must have one of the licenses mentioned below to be able to benefit from it on Azure: Windows Server Datacenter Edition with Software Assurance. Windows Server Standard Edition with Software Assurance. SQL Server Enterprise Edition Core licenses with Software Assurance or qualifying subscription licenses. SQL Server Standard Edition Core licenses with Software Assurance or qualifying subscription licenses. Microsoft describes Software Assurance as A comprehensive program that includes a unique set of technologies, services, and entitlements to enable you to effectively deploy, manage, and use Microsoft products. The idea is also to allow you to move some workloads to Azure, as Software Assurance provides 180 days of concurrent use between Azure and your On-Premise environment, which is great for a smooth migration. The principle is pretty much the same for RHEL and SUSE, coming to Azure with your own licenses. In addition, during https://youtu.be/_96cUyRvyMA which took place from October 12 to 14, 2022, Microsoft even extended Azure Hybrid Benefit by offering the possibility to deploy Azure Kubernetes Service on Windows Server and Azure Stack HCI at no additional cost in your environments. On-Premise if you have Software Assurance. And the top of the top is that it is also possible to combine Azure Hybrid Benefit and Reserved Instances, to further reduce your costs. Azure savings plan for compute During this same Microsoft Ignite, Microsoft announced a new way to optimize your costs, with the https://azure.microsoft.com/en-us/pricing/offers/savings-plan-compute/#benefits-and-features. The latter, which as its name suggests, only concerns computing resources, is a commitment to spend, a fixed hourly amount, over a period of 1 to 3 years as for reservations. Moreover, it is possible to combine it with Reserved Instances but also with Hybrid Benefit. According to Microsoft documentation with a 3-year commitment, Azure savings plan for compute could allow you to reduce your costs by up to 65% compared to public prices. Let's take a concrete example, in which you opt for an hourly amount of 5€ per hour. If you exceed this amount by €5, the additional costs will be calculated on the public prices. Conversely, if you do not use enough computing power to reach your €5, then you will still pay the €5. But what is interesting here is that this amount, which you have defined, does not apply to a single resource, but to all of your services running compute resources, with a small limitation all the same, supported services. At the time of writing these few lines, Azure savings plan for compute applies to all of your resources deployed worldwide for the services below: Azure VM Azure Container Instances Azure Functions for Premium plans Azure App Service And Azure Dedicated Host So if I take my previous example, my 5€ will be allocated for all of my resources using the services above. Azure DevTest Lab As you know, one of the advantages of the Cloud is also to be able to quickly test a service or a feature by deploying what you need. For example, it is very simple and quick to deploy identical environments, which can be intended for training sessions. So in these specific cases, we will not really be able to reduce the costs associated with the resources deployed, but above all we will be able to control them. https://azure.microsoft.com/en-us/products/devtest-lab/#overview allows you to deploy resources, from an existing repository, which can be Linux, Windows VMs, App Services, Oracle or SQL databases or even Dynamics instances, to name a few. Obviously you can if you wish, deploy your own images. I was talking earlier about cost control. If I go back to my training example, we will be able for example to define the type of instance to deploy, or their number. What's interesting is that you can schedule the stop and start, which we talked about a little earlier, but also an expiration date. This expiration date will automatically delete the resources, once the date is reached, and thus avoid having unused resources for which you will be charged if you forget to delete them. Azure Cost Management + Billing https://learn.microsoft.com/en-us/azure/cost-management-billing/cost-management-billing-overview is a suite of tools that help you easily analyze, manage, and optimize the costs of your workloads. We have previously mentioned the interest of tags, and here you can identify the cost of your resources by tag, by type of resource or by group resource for example. You have already received an Azure invoice. We agree, it's not always easy to decode it, well Azure Cost Management + Billing will help you hang up the wagons. It also offers you the possibility of creating monthly, quarterly or annual budgets and receiving alerts when a threshold that you have configured is reached. Obviously if the budget is exceeded, the resources will not be stopped or deleted, we are here on a notification system. It is also Azure Cost Management + Billing that will allow you to retrieve your invoices, and view information concerning the subjects of Reserved Instances, Hybrid Benefit and Azure savings plan for compute. In short, it is THE service that will allow you to track your Azure costs. Azure Advisor Another service that can help reduce your bill is https://azure.microsoft.com/en-us/products/advisor/#overview. It offers recommendations on good practices to implement on different aspects: The costs Security Reliability Operational excellence Performances Let's focus on what interests us today, the costs. You will find recommendations to help you optimize and reduce your costs by identifying inactive resources, oversized resources or those for which Reserved Instances could be considered. To do this, Azure Advisor relies on your resource usage history and associated metrics to offer you the best solution. It therefore requires a minimum of use before the recommendations are relevant. In addition, you can modify the basic criteria on which it is based in order to obtain advice that corresponds even more to your activity. A coherent architecture It can sometimes happen that proposed or defined architectures are disproportionate for a simple need. Let's take the example of a BU that wants to create a new micro-service that runs on the .Net 6 framework to host an API: After a quick analysis on his side, the application architect considers that the AKS service is ideal for his need. As a Cloud architect, the first thing is to understand the need and the constraints of the BU to challenge it. After various exchanges, it turns out that the critical points for the BU are the scalability of the platform, as well as the deployment of new versions of its API. The Cloud architect can thus challenge the AKS service, by proposing the use of an Azure App Service in container mode, which responds to scalability with native support for scale-out, but also for the delivery of new versions of the API, with support for deployment slots that will avoid service disruptions during deliveries. In short, the idea is always to do as simple as possible according to the client's inputs to design the architecture he needs, and not the one that makes you dream, or that is "sexy", and that will lead to costs that are ultimately not legitimate. Get accompanied by a partner And if you are not in a position to challenge your interlocutors, do not hesitate to have a partner accompany you, who will be able on the one hand to challenge, but on the other hand, carry out a complete analysis in order to help you reduce your bill. At Grow Una (external link removed by moderator), we support our customers on the Azure ecosystem, on different levels of expertise: Daily support The definition of architecture Deployment and MCO of your Cloud environments Migration of your application assets to Azure Azure training or increasing the skills of your teams While implementing DevOps methodologies and FinOps principles. So as you see, several solutions can allow you to decrease your Azure invoice, and now it's your turn to try them!
