Automatic scaling with Azure Virtual Machine Scale Sets flexible orchestration mode
At Ignite March 2021, we announced the Public Preview of Azure Virtual Machine Scale Sets with flexible orchestration mode, an evolution of Azure Virtual Machine Scale sets that makes it easier to run a variety of virtual machine workloads at high scale with high availability. We are excited to announce we are adding additional functionality to the VMSS Flexible Orchestration preview: Automatic scaling Flexible orchestration mode now allows you to scale your virtual machine application out or in manually, automatically based on metrics, or according to a schedule. Like the traditional VMSS in Uniform Orchestration Mode, you specify a virtual machine profile or template for virtual machine instances: VM size, networking configuration, data disks, etc, and the number of instances you would like. Once the profile is defined, the scale set will automatically create the number of instances you request or remove instances and associated NICs and disks. VMSS provides many options to help you scale out based on your application needs: Scale up to 1000 instances in the scale set Specify instances should be placed in a particular zone Spread across multiple fault domains Automatically scale based on metrics such as aggregate CPU load, disk throughput, memory usage, etc Use Spot or on demand priority Automatically remove NICs and Disks when deleting the VM instances When application demand goes down or you need fewer instances for your application, you can save cost by scaling and reducing the number of instances in your scale set. Faster, more reliable deployments VMSS Flexible Orchestration mode is built on our next generation datacenter deployment technologies, enabling more reliable deployment success, more consistent deployment times, and faster, more reliable scale out and scale in operations. Maintain application health with Application Monitoring and Automatic Instance Repair You can install the Application Health Extension on each instance to allow your application to report application specific health metrics to Azure. Azure can automatically remove and replace instances with unhealthy application state. Safely remove instances with Terminate Notification Your application can receive an instance termination notice and set a predefined delay to the terminate operation, allowing your application to perform any clean up activities or end of life workflow before the instance is deleted. Application aware In Guest Security Patching Orchestration Automatic VM guest patching for virtual machines helps ease update management by safely and automatically patching virtual machines to maintain security compliance. With automatic VM guest patching enabled, the VM is assessed periodically to determine the applicable patches for that VM. Updates classified as 'Critical' or 'Security' are automatically downloaded and applied on the VM during off-peak hours. Patch orchestration is managed by Azure and patches are applied following availability-first principles. Improve network security with explicit outbound connectivity Historically, Azure VMs are assigned a default outbound IP address, enabling outbound connectivity from the VM to the internet (default outbound access). There are several disadvantages of this default outbound access IP including inability to lock down access via network security groups, and SNAT port exhaustion. In order to support modern best practices based on the secure by default approach in zero trust network security, VM instances created with VMSS Flexible Orchestration will not have the default outbound access IP associated with it. VMSS Flexible Orchestration will require that you specify an explicit outbound connectivity method, for example: Associate a NAT Gateway to the subnet where the instances reside Associate a Standard Load balancer with Outbound Rules configured Associate a Public IP with the VM Network Interface Only VMs created implicitly by the VMSS scaling engine will be secure by default with no implicit IP. VMs associated with an Availability Set or VMSS Uniform Orchestration mode, or standalone VMs that are later added to a VMSS Flex will still have the default outbound access and implicit IP address enabled. If you are building new workloads for VMSS Flexible Orchestration, or migrating existing workloads to VMSS Flexible orchestration, you may need to review network configuration to ensure connectivity to external services, including: Windows Activation Key Management Service Establish Private Link to required Azure services like Storage accounts, Azure Key Vault, etc. Custom scripts that require access to external URIs, Azure Active Directory Domain jon, etc Windows Update service For more information, refer to Default Outbound Access Support for Azure Backup and Azure Site Recovery We have extended support for VM management service like Azure Backup and Azure Site Recovery to VMSS Flexible Orchestraton mode. Example: N-Tier Application with VMSS Flexible Orchestration Let’s look at a how you can use VMSS Flexible Orchestration mode to simplify a traditional N-Tier Application virtual machine architecture. Adapted from Azure Architecture Center: N-Tier application with Apache Cassandra Traditionally this application architecture requires that you managed each of the 14 VMs individually; you are responsible for monitoring each instance, performing all security patching and ensuring application update. Furthermore, if demand for your application grows or shrinks, you would have to manually create additional instances at the web and/or business tier to handle additional traffic. You can simplify deployment and management of this architecture by using a VMSS with Flexible Orchestration at each application tier, and rely on VMSS platform features to assist with monitoring and management tasks. Data-tier – As this database workload tends to be stateful and requiring that instances are spread across multiple racks or partitions, you can specify a VMSS Flexible Orchestration to spread virtual machines across fault domains Business-tier – Middle tier of the application is often stateless, so you may be able to specify VMSS Flexible with maximum spreading (allow Azure to manage spreading…no particular quorum requirement). You could take advantage of Automatic Instance Repair to monitor if application instances are reporting healthy, and automatically replacing unhealthy instances with new, healthy instances. Web tier – This also tends to be a stateless tier, and is most susceptible to dynamic changes in traffic. You can specify autoscaling rules to automatically increase or decrease the number of instances based on a schedule, or metrics based rules. You can help optimize costs by mixing demand types; adding 2-3 instance at full, on-demand pricing, and specifying auto scale rules to scale out with less-expensive Spot instances. Sample templates: vm-scale-sets/vmss-flex-n-tier-demo at master · Azure/vm-scale-sets (github.com) Looking toward General Availability and beyond We are excited to share this first step in our journey to combine Azure Virtual Machines, Availability Sets, and VMSS into a single, integrated offering in VMSS Flexible Orchestration. On the way to general availability, we expect to continue to improve the parity between VMSS Uniform and VMSS Flexible Orchestration. One feature we plan to add next is the ability to specify multi-zone deployments, so you can automatically spread instances across multiple availability zones. We also look forward to bring more API parity between VMSS Uniform and Flex for batch instance operations, support for all VM sizes, as well as VMSS orchestrations like Scale in Policy, and Instance Protection. We look forward to hearing your feedback and stories, so we can continue to help you build the applications and services for your organization. Resources to get you started Virtual Machine Scale Sets Learn how to deploy and manage VMSS FlexIncrease availability at scale with Azure Virtual Machine Scale Sets flexible orchestration mode
Today customers can deploy and manage their stateful and stateless applications on Azure using Availability Sets and/or Virtual Machine Scale Sets. Availability sets provide fault isolation guarantees and the ability to manage VMs individually, whereas Virtual Machine Scale Sets simplifies large-scale deployments and provide rich application runtime features. In various situations, large scale customers must choose between scale or high availability. However, availability sets only offer up to 200 VMs making it hard for customers to scale and meet customer demands, whereas VM scale sets can scale up to 1000 VMs but offer individual access to VMs or their resources for customization (if required). VMs created within a VM scale set also have a slightly different resource model compared to individual VMs provisioned individually outside of a scale set, additionally VMs cannot be attached or detached to the VM Scale Sets. As more workloads move to the cloud - like open-source databases, large-scale web services, stateful or stateless workloads, batch processing etc. - customers require high scale, autoscaling, agility, high availability, elasticity, in addition to greater ease of management and configuration, rich app management tools to successfully meet an ever-changing environment. Virtual Machine Scale Sets flexible orchestration mode We are thrilled to announce the evolution of Azure Virtual Machine Scale Sets with a new orchestration mode, now in public preview, that delivers a richer and more powerful platform for customers to easily run a variety of workloads at high scale and with high availability. The new Virtual Machine Scale Sets flexible orchestration mode or VMSS-flex offer new capabilities and can provide best in class experience for Quorum-based workloads Open-Source databases Stateful applications Services which require high availability and large scale Services which want to mix virtual machine types, or leverage Spot and on-demand VMs together. Existing Availability Set applications We recognize that a lot of customers have been using Virtual Machine Scale Sets to successfully run their applications. Customers will still be able to access/create/manage their existing scale sets using the existing APIs with the current orchestration mode now called “uniform orchestration mode” which is Generally Available and remains the default. Both uniform and flexible orchestration modes will move forward and will be supported. By selecting the Uniform Orchestration mode, customers can continue using VMSS without disruption. The preview announced today for Virtual Machine Scale Sets flexible orchestration mode (VMSS-flex) provides high availability guarantees and large scale up to 1000 VMs per AZ. You can chose to deploy your VMs across fault domains in a region or within an Availability Zone. This enables you to scale seamlessly along with fault isolation constraints to achieve High Availability, which are essential to run quorum-based or stateful workloads. In addition, scale sets also offer ‘sticky’ Fault Domains i.e. Fault Domains that do not change during a VM’s lifecycle until the VM is deleted. You can also specify a Fault Domain as part of a VM deployment, which makes it simpler to replace VMs without impacting workload configurations. This is specifically relevant for open-source databases like Cassandra or other quorum-based applications. E.g. az vm create –vmss “myVMSS” –-platform_fault_domain 1 We are also unifying the VM experience across our offerings. VMs which are created with VMSS-flex are Azure Resource Manager (ARM) based VMs. The VM and its resources are individually addressable like any other VM unlike the existing uniform orchestration mode. This enables far more flexibility for you to manage your infrastructure either as a set with VMSS-flex or individually. Here are more details about today’s preview: Supported VM series: General purpose: B, Dsv3, Dv3, Dasv4, Dav4, DSv2, Dv2, Av2, DC, DCv2, Dv4, Dsv4, Ddv4, Ddsv4 Compute Optimized: F, Fs, Fsv2 Memory Optimized: Esv3, Ev3, Easv4, Eav4, Ev4, Esv4, Edv4, Edsv4, DSv2, Dv2 Remaining series like memory optimized Mv2, M, storage optimized (Lsv2), GPU (NC, NCv2, NCv3, NCasT4_v3 (Preview), ND, NDv2 (Preview), NV, NVv3, NVv4) and high-performance compute (HB, HBv2, HC, H) are not supported and will be added to VMSS-flex in the future. Fault Domain availability: same as region provides. 1 when deployed in an Availability Zone Since VMSS-flex is unlocking significant value for you and providing high availability at large scale. To provide a more complete picture and transparency regarding future product enhancements, I’m happy to pre-announce some of the upcoming features that will be added to Virtual Machine Scale Sets flexible orchestration mode. In the near future VMSS-flex will also offer: Elasticity profile – ability to provide VM configuration and set instance count Ability to mix Spot and On-demand VMs in the same scale set Metrics based autoscaling – ability to automatically scale out or scale in based on aggregate CPU usage, disk IO performance, etc. Instance repair – automatically remove and replace instances with unhealthy application state Terminate notification – receive instance termination notifications and set a pre-defined delay timeout to the terminate operation Resources to get you started Virtual Machine Scale Sets Learn how to deploy and manage VMSS Flex
