Creating a Reliable Notification System for Azure Spot VM Evictions (preempt) events
Introduction Azure Spot VMs offer significant cost savings but come with a trade-off: they can be evicted with minimal notice when Azure needs the capacity back or price change. Building a reliable notification system for these evictions is critical for applications that need to respond gracefully to these events. What are Azure Spot VMs? Azure Spot VMs are virtual machines that use spare capacity in Azure data centers, available at significantly discounted prices compared to regular pay-as-you-go VMs. Microsoft offers this unused capacity at discounts of up to 90% off the standard prices, making Spot VMs an extremely cost-effective option for many workloads. However, there's an important caveat: when Azure needs this capacity back for regular pay-as-you-go customers, your Spot VMs can be evicted (reclaimed) with minimal notice - typically just 30 seconds. This eviction mechanism is what allows Microsoft to offer such deep discounts, as we maintain the flexibility to reclaim these resources when needed. https://azure.microsoft.com/en-gb/products/virtual-machines/spot Benefits of Spot VMs Significant cost savings: The most obvious benefit is the substantial discount, which can be up to 90% off standard VM prices. Same VM types and features: Spot VMs provide the same performance, features, and capabilities as regular VMs - the only difference is the eviction possibility. Ideal for interruptible workloads: For workloads that can handle interruptions, such as batch processing jobs, dev/test environments, or stateless applications, Spot VMs offer enormous value. Flexible sizing options: Spot VMs are available in most VM series and regions, giving you access to a wide range of computing options. Scaling opportunities: The cost savings enable you to run larger clusters or more powerful VMs than might be financially feasible with regular VMs. Effective for burst capacity: When you need additional capacity for temporary workloads, Spot VMs can provide it at minimal cost. Great for fault-tolerant applications: Modern cloud-native applications designed with redundancy and resilience can leverage Spot VMs excellently since they're built to handle node failures. Why Not Just Use Azure Resource Events? A common question is: "Why not simply listen for Azure Resource events like ResourceActionSuccess for VM evictions?" While Azure does emit platform events when resources change state through resource group as source for Azure Event Grid topic subscription, there are several critical limitations when relying on these for Spot VM evictions: Timing issues: By the time a ResourceActionSuccess event is generated for a VM eviction, it is possible that the VM is already being evicted. This gives you no time to perform graceful shutdown procedures. Reliability concerns: These events pass through multiple Azure systems before reaching your event handlers, adding potential points of failure and latency. Ambiguous events: Resource action events don't clearly distinguish between a normal VM shutdown and a Spot VM eviction, making it difficult to trigger the right response. For example: I initially attempted to capture Azure Spot VM eviction events by setting up event notifications on an Azure resource group and publishing them to Service Bus. While this configuration successfully captured some Azure Resource events, it ultimately proved unreliable for eviction monitoring. The solution missed several critical eviction events and, more problematically, could not reliably distinguish between intentional VM shutdowns and actual eviction events. This lack of differentiation made automated response handling impossible, as the system couldn't determine whether a VM was being evicted by Azure or simply stopped through normal administrative actions. Azure resource group as an Event Grid source - Azure Event Grid | Microsoft Learn For these reasons, the most reliable approach is to detect eviction events directly from within the VM using the Azure Instance Metadata Service (IMDS) Scheduled Events API, which is specifically designed to provide advance notice of impending VM state changes. This blog post will guide you through implementing a solution that: Detects Spot VM eviction events from within the VM Formats these events properly Sends them to an Azure Event Grid custom topic Sets up proper event handling downstream Understanding Spot VM Eviction Notices Spot VMs receive eviction notifications approximately 30 seconds before being reclaimed. These notifications are delivered through the Azure Instance Metadata Service (IMDS) Scheduled Events API - an endpoint available from within the VM at http://169.254.169.254/metadata/scheduledevents. When a Spot VM is about to be evicted, a "Preempt" event appears in the Scheduled Events data. Your application needs to poll this endpoint regularly to detect these events in time to take action. https://learn.microsoft.com/en-us/azure/virtual-machines/windows/scheduled-events Solution overview Our solution consists of below components: A custom Event Grid topic to receive and distribute the events - optional if you wish to handle on own from VM A monitoring script running inside the Spot VM - actual script to poll events running on VM Logic to format and send events from the VM to Event Grid Event subscribers that take action when evictions occur A) Setting Up the Event Grid Custom Topic First, create an Event Grid custom topic that will serve as the distribution mechanism for your eviction events - this can be optional if you plan to take actions from VM only like gracefully shutting down any existing processes. You can use below documentation to create custom event grid topic: Custom topics in Azure Event Grid - Azure Event Grid | Microsoft Learn B) Creating a Windows-Based Eviction Monitor For Windows Spot VMs, we'll use below PowerShell to poll preempt events & send it to custom event grid. Create a script file named SpotMonitor.ps1: Powershell script : SpotMonitor.ps1 # Configuration variables - replace with your values $EventGridTopicEndpoint = "https://<EG topic name>.westeurope-1.eventgrid.azure.net/api/events" $EventGridKey = "<EG key>" $CheckInterval = 3 # seconds between checks - feel free to modify as per your requirement $LogFile = "C:\Logs\spot-monitor.log" # Create log directory if it doesn't exist if (-not (Test-Path (Split-Path $LogFile))) { New-Item -ItemType Directory -Path (Split-Path $LogFile) -Force } function Write-Log { param ([string]$Message) $timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss" "$timestamp - $Message" | Out-File -FilePath $LogFile -Append } Write-Log "Starting Spot VM eviction monitor..." while ($true) { try { # Get the VM's metadata including scheduled events $headers = @{"Metadata" = "true"} $scheduledEvents = Invoke-RestMethod -Uri "http://169.254.169.254/metadata/scheduledevents?api-version=2020-07-01" -Headers $headers # Check if there are any events if ($scheduledEvents.Events -and $scheduledEvents.Events.Count -gt 0) { Write-Log "Found $($scheduledEvents.Events.Count) scheduled events" # Get VM metadata for context $vmName = Invoke-RestMethod -Uri "http://169.254.169.254/metadata/instance/compute/name?api-version=2020-09-01&format=text" -Headers $headers $resourceGroup = Invoke-RestMethod -Uri "http://169.254.169.254/metadata/instance/compute/resourceGroupName?api-version=2020-09-01&format=text" -Headers $headers $subscription = Invoke-RestMethod -Uri "http://169.254.169.254/metadata/instance/compute/subscriptionId?api-version=2020-09-01&format=text" -Headers $headers # Process each event foreach ($event in $scheduledEvents.Events) { if ($event.EventType -eq "Preempt") { Write-Log "ALERT: Spot VM preemption detected!" # Extract event details $eventId = $event.EventId $notBefore = $event.NotBefore Write-Log "VM $vmName will be preempted not before $notBefore" # Create Event Grid event as an array (critical for EventGrid schema) $eventGridEvent = @( @{ subject = "/subscriptions/$subscription/resourceGroups/$resourceGroup/providers/Microsoft.Compute/virtualMachines/$vmName" eventType = "SpotVM.Preemption" eventTime = (Get-Date).ToUniversalTime().ToString("o") id = [Guid]::NewGuid().ToString() data = @{ vmName = $vmName resourceGroup = $resourceGroup subscription = $subscription preemptionTime = $notBefore eventId = $eventId eventType = $event.EventType } dataVersion = "1.0" } ) # Convert to JSON - ensuring it stays as an array $eventGridPayload = ConvertTo-Json -InputObject $eventGridEvent -Depth 10 # Send to Event Grid $eventGridHeaders = @{ "Content-Type" = "application/json" "aeg-sas-key" = $EventGridKey } try { $response = Invoke-RestMethod -Uri $EventGridTopicEndpoint -Method Post -Body $eventGridPayload -Headers $eventGridHeaders Write-Log "Successfully sent event to Event Grid" # Take actions to prepare for shutdown Write-Log "Taking actions to prepare for shutdown..." # Example: Stop services gracefully # Stop-Service -Name "YourServiceName" -Force } catch { Write-Log "Failed to send to Event Grid: $_" } } } } } catch { Write-Log "Error checking for events: $_" } # Wait before checking again Start-Sleep -Seconds $CheckInterval } The script above checks for eviction events every 3 seconds by default. You can adjust this polling frequency by changing the "Check_Interval" variable in the script to better match your specific system requirements and performance considerations. More frequent polling provides faster detection but increases resource usage, while less frequent polling reduces overhead but might slightly delay event detection. B) Running monitor script as a scheduler or service For Windows Spot VMs, we'll use PowerShell to create a monitoring service. Run a script file named SpotMonitor.ps1 created in last step: You can use Windows Task Scheduler to run the script at startup or to run as a service and the logs will looks like this: Logs: 2025-03-19 18:48:27 - Starting Spot VM eviction monitor... 2025-03-19 20:04:33 - Found 1 scheduled events 2025-03-19 20:04:33 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:33 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:33 - Sending payload: [ { "eventTime": "2025-03-19T20:04:33.4655660Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "5d3e6430-dff5-45da-ae90-992e3e342d37", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:33 - Event Grid response: 2025-03-19 20:04:33 - Taking actions to prepare for shutdown... 2025-03-19 20:04:36 - Found 1 scheduled events 2025-03-19 20:04:36 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:36 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:36 - Sending payload: [ { "eventTime": "2025-03-19T20:04:36.6382480Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "b6152429-f4cb-43b9-8c53-b6ceb08946e5", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:36 - Event Grid response: 2025-03-19 20:04:36 - Taking actions to prepare for shutdown... 2025-03-19 20:04:39 - Found 1 scheduled events 2025-03-19 20:04:39 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:39 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:39 - Sending payload: [ { "eventTime": "2025-03-19T20:04:39.7567285Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "e0bde6d0-ae27-4c01-8e69-621e57d70f8d", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:39 - Event Grid response: 2025-03-19 20:04:39 - Taking actions to prepare for shutdown... 2025-03-19 20:04:42 - Found 1 scheduled events 2025-03-19 20:04:42 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:42 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:42 - Sending payload: [ { "eventTime": "2025-03-19T20:04:42.8339675Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "ab7a3b84-bcd8-4651-829e-c57043c54b92", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:42 - Event Grid response: 2025-03-19 20:04:42 - Taking actions to prepare for shutdown... 2025-03-19 20:04:45 - Found 1 scheduled events 2025-03-19 20:04:45 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:45 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:45 - Sending payload: [ { "eventTime": "2025-03-19T20:04:45.9317109Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "eacfae6b-4ea5-426d-8bc2-659320a7baf0", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:45 - Event Grid response: 2025-03-19 20:04:45 - Taking actions to prepare for shutdown... 2025-03-19 20:04:48 - Found 1 scheduled events 2025-03-19 20:04:49 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:49 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:49 - Sending payload: [ { "eventTime": "2025-03-19T20:04:49.0666732Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "b2142ee8-9ecf-441d-846e-c8ed663a949e", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:49 - Event Grid response: 2025-03-19 20:04:49 - Taking actions to prepare for shutdown... 2025-03-19 20:04:52 - Found 1 scheduled events 2025-03-19 20:04:52 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:52 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:52 - Sending payload: [ { "eventTime": "2025-03-19T20:04:52.1310990Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "d9eba318-9773-4e73-a694-dd1c1bf89c10", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:52 - Event Grid response: 2025-03-19 20:04:52 - Taking actions to prepare for shutdown... 2025-03-19 20:04:55 - Found 1 scheduled events 2025-03-19 20:04:55 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:55 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:55 - Sending payload: [ { "eventTime": "2025-03-19T20:04:55.2171546Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "c358c433-50f5-496d-8823-c2ffddd03390", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:55 - Event Grid response: 2025-03-19 20:04:55 - Taking actions to prepare for shutdown... 2025-03-19 20:04:58 - Found 1 scheduled events 2025-03-19 20:04:58 - ALERT: Spot VM preemption detected! 2025-03-19 20:04:58 - VM anivmnew will be preempted not before Wed, 19 Mar 2025 20:04:47 GMT 2025-03-19 20:04:58 - Sending payload: [ { "eventTime": "2025-03-19T20:04:58.3040422Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "Wed, 19 Mar 2025 20:04:47 GMT", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "3eacba95-e05f-41dc-b9e7-1593fe2a71e2", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:04:58 - Event Grid response: 2025-03-19 20:04:58 - Taking actions to prepare for shutdown... 2025-03-19 20:05:01 - Found 1 scheduled events 2025-03-19 20:05:01 - ALERT: Spot VM preemption detected! 2025-03-19 20:05:01 - VM anivmnew will be preempted not before 2025-03-19 20:05:01 - Sending payload: [ { "eventTime": "2025-03-19T20:05:01.3842973Z", "data": { "eventId": "DE2EC5FA-AF0A-4D59-85D2-677C66A6BC12", "preemptionTime": "", "eventType": "Preempt", "resourceGroup": "RG-TEST", "subscription": "azure-sub-id", "vmName": "anivmnew" }, "id": "85c058fc-4f2e-49ec-a027-6fcca60f7935", "subject": "/subscriptions/azure-sub-id/resourceGroups/RG-TEST/providers/Microsoft.Compute/virtualMachines/anivmnew", "eventType": "SpotVM.Preemption", "dataVersion": "1.0" } ] 2025-03-19 20:05:01 - Event Grid response: 2025-03-19 20:05:01 - Taking actions to prepare for shutdown... C) Configuring event subscribers Now that your Spot VMs are sending eviction events to Event Grid, set up subscribers to take action when these events occur. For example sending event to service bus queue: Conclusion By implementing this solution, you've created a reliable way to detect and respond to Spot VM evictions. This approach gives your applications precious time to react to evictions, significantly improving reliability while still benefiting from the cost savings of Spot VMs. While Azure does provide resource-level events through system topics, they simply don't provide the reliability, timing, and clarity needed for mission-critical workloads running on Spot VMs. The combination of the Azure Instance Metadata Service Scheduled Events API and custom Event Grid topics creates a powerful pattern for building resilient, event-driven architectures. This approach ensures you're getting the most accurate and timely notifications possible, giving your applications the best chance to gracefully handle Spot VM evictions while enjoying the substantial cost benefits that Spot VMs offer. Disclaimer The sample scripts provided in this article are provided AS IS without warranty of any kind. The author is not responsible for any issues, damages, or problems that may arise from using these scripts. Users should thoroughly test any implementation in their environment before deploying to production. Azure services and APIs may change over time, which could affect the functionality of the provided scripts. Always refer to the latest Azure documentation for the most up-to-date information. Thanks for reading this blog! I hope you've found this approach to handling Spot VM evictions helpfulDeploy and Install Windows Admin Center in an Azure VM
The great thing about Windows Admin Center (WAC) you manage every Windows Server doesn't matter where it is running. You can manage Windows Servers on-prem, in Azure or running at other cloud providers. Now if you want to use Windows Admin Center to manage your virtual machines running in Azure, you can use either an on-prem WAC installation and connecting it using a public IP address or a VPN connection, or you can deploy and install Windows Admin Center in Azure. This blog post will show you how you can deploy and install Windows Admin Center in an Azure virtual machine (VM). Check out my blog post to read more: https://www.thomasmaurer.ch/2019/10/deploy-and-install-windows-admin-center-in-an-azure-vm/Power Up Your Open WebUI with Azure AI Speech: Quick STT & TTS Integration
Introduction Ever found yourself wishing your web interface could really talk and listen back to you? With a few clicks (and a bit of code), you can turn your plain Open WebUI into a full-on voice assistant. In this post, you’ll see how to spin up an Azure Speech resource, hook it into your frontend, and watch as user speech transforms into text and your app’s responses leap off the screen in a human-like voice. By the end of this guide, you’ll have a voice-enabled web UI that actually converses with users, opening the door to hands-free controls, better accessibility, and a genuinely richer user experience. Ready to make your web app speak? Let’s dive in. Why Azure AI Speech? We use Azure AI Speech service in Open Web UI to enable voice interactions directly within web applications. This allows users to: Speak commands or input instead of typing, making the interface more accessible and user-friendly. Hear responses or information read aloud, which improves usability for people with visual impairments or those who prefer audio. Provide a more natural and hands-free experience especially on devices like smartphones or tablets. In short, integrating Azure AI Speech service into Open Web UI helps make web apps smarter, more interactive, and easier to use by adding speech recognition and voice output features. If you haven’t hosted Open WebUI already, follow my other step-by-step guide to host Ollama WebUI on Azure. Proceed to the next step if you have Open WebUI deployed already. Learn More about OpenWeb UI here. Deploy Azure AI Speech service in Azure. Navigate to the Azure Portal and search for Azure AI Speech on the Azure portal search bar. Create a new Speech Service by filling up the fields in the resource creation page. Click on “Create” to finalize the setup. After the resource has been deployed, click on “View resource” button and you should be redirected to the Azure AI Speech service page. The page should display the API Keys and Endpoints for Azure AI Speech services, which you can use in Open Web UI. Settings things up in Open Web UI Speech to Text settings (STT) Head to the Open Web UI Admin page > Settings > Audio. Paste the API Key obtained from the Azure AI Speech service page into the API key field below. Unless you use different Azure Region, or want to change the default configurations for the STT settings, leave all settings to blank. Text to Speech settings (TTS) Now, let's proceed with configuring the TTS Settings on OpenWeb UI by toggling the TTS Engine to Azure AI Speech option. Again, paste the API Key obtained from Azure AI Speech service page and leave all settings to blank. You can change the TTS Voice from the dropdown selection in the TTS settings as depicted in the image below: Click Save to reflect the change. Expected Result Now, let’s test if everything works well. Open a new chat / temporary chat on Open Web UI and click on the Call / Record button. The STT Engine (Azure AI Speech) should identify your voice and provide a response based on the voice input. To test the TTS feature, click on the Read Aloud (Speaker Icon) under any response from Open Web UI. The TTS Engine should reflect Azure AI Speech service! Conclusion And that’s a wrap! You’ve just given your Open WebUI the gift of capturing user speech, turning it into text, and then talking right back with Azure’s neural voices. Along the way you saw how easy it is to spin up a Speech resource in the Azure portal, wire up real-time transcription in the browser, and pipe responses through the TTS engine. From here, it’s all about experimentation. Try swapping in different neural voices or dialing in new languages. Tweak how you start and stop listening, play with silence detection, or add custom pronunciation tweaks for those tricky product names. Before you know it, your interface will feel less like a web page and more like a conversation partner.Deploy Open Web UI on Azure VM via Docker: A Step-by-Step Guide with Custom Domain Setup.
Introductions Open Web UI (often referred to as "Ollama Web UI" in the context of LLM frameworks like Ollama) is an open-source, self-hostable interface designed to simplify interactions with large language models (LLMs) such as GPT-4, Llama 3, Mistral, and others. It provides a user-friendly, browser-based environment for deploying, managing, and experimenting with AI models, making advanced language model capabilities accessible to developers, researchers, and enthusiasts without requiring deep technical expertise. This article will delve into the step-by-step configurations on hosting OpenWeb UI on Azure. Requirements: Azure Portal Account - For students you can claim $USD100 Azure Cloud credits from this URL. Azure Virtual Machine - with a Linux of any distributions installed. Domain Name and Domain Host Caddy Open WebUI Image Step One: Deploy a Linux – Ubuntu VM from Azure Portal Search and Click on “Virtual Machine” on the Azure portal search bar and create a new VM by clicking on the “+ Create” button > “Azure Virtual Machine”. Fill out the form and select any Linux Distribution image – In this demo, we will deploy Open WebUI on Ubuntu Pro 24.04. Click “Review + Create” > “Create” to create the Virtual Machine. Tips: If you plan to locally download and host open source AI models via Open on your VM, you could save time by increasing the size of the OS disk / attach a large disk to the VM. You may also need a higher performance VM specification since large resources are needed to run the Large Language Model (LLM) locally. Once the VM has been successfully created, click on the “Go to resource” button. You will be redirected to the VM’s overview page. Jot down the public IP Address and access the VM using the ssh credentials you have setup just now. Step Two: Deploy the Open WebUI on the VM via Docker Once you are logged into the VM via SSH, run the Docker Command below: docker run -d --name open-webui --network=host --add-host=host.docker.internal:host-gateway -e PORT=8080 -v open-webui:/app/backend/data --restart always ghcr.io/open-webui/open-webui:dev This Docker command will download the Open WebUI Image into the VM and will listen for Open Web UI traffic on port 8080. Wait for a few minutes and the Web UI should be up and running. If you had setup an inbound Network Security Group on Azure to allow port 8080 on your VM from the public Internet, you can access them by typing into the browser: [PUBLIC_IP_ADDRESS]:8080 Step Three: Setup custom domain using Caddy Now, we can setup a reverse proxy to map a custom domain to [PUBLIC_IP_ADDRESS]:8080 using Caddy. The reason why Caddy is useful here is because they provide automated HTTPS solutions – you don’t have to worry about expiring SSL certificate anymore, and it’s free! You must download all Caddy’s dependencies and set up the requirements to install it using this command: sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list sudo apt update && sudo apt install caddy Once Caddy is installed, edit Caddy’s configuration file at: /etc/caddy/Caddyfile , delete everything else in the file and add the following lines: yourdomainname.com { reverse_proxy localhost:8080 } Restart Caddy using this command: sudo systemctl restart caddy Next, create an A record on your DNS Host and point them to the public IP of the server. Step Four: Update the Network Security Group (NSG) To allow public access into the VM via HTTPS, you need to ensure the NSG/Firewall of the VM allow for port 80 and 443. Let’s add these rules into Azure by heading to the VM resources page you created for Open WebUI. Under the “Networking” Section > “Network Settings” > “+ Create port rule” > “Inbound port rule” On the “Destination port ranges” field, type in 443 and Click “Add”. Repeat these steps with port 80. Additionally, to enhance security, you should avoid external users from directly interacting with Open Web UI’s port - port 8080. You should add an inbound deny rule to that port. With that, you should be able to access the Open Web UI from the domain name you setup earlier. Conclusion And just like that, you’ve turned a blank Azure VM into a sleek, secure home for your Open Web UI, no magic required! By combining Docker’s simplicity with Caddy’s “set it and forget it” HTTPS magic, you’ve not only made your app accessible via a custom domain but also locked down security by closing off risky ports and keeping traffic encrypted. Azure’s cloud muscle handles the heavy lifting, while you get to enjoy the perks of a pro setup without the headache. If you are interested in using AI models deployed on Azure AI Foundry on OpenWeb UI via API, kindly read my other article: Step-by-step: Integrate Ollama Web UI to use Azure Open AI API with LiteLLM Proxy
