Transition from Alpine Linux to Debian for WordPress on App Service
We are excited to share with you that we will be transitioning from Alpine to Debian as the default Linux distribution for WordPress on App Service. While Alpine has served us well with its lightweight and performant nature, our evolving needs require a more feature-rich environment. Debian is the ideal candidate to support our next phase of development and growth. This transition will also align us with App Service Linux, which already utilizes Debian as the default Linux distribution.Build and Modernize Intelligent Java apps at Scale
Java on Microsoft Azure Java customers and developers are constantly exploring how they can bring their Java applications to the cloud. Some are looking to modernize existing applications, while others are building new cloud-native solutions from scratch. With these changes, they need a platform that lets them keep working the way they know, without sacrificing control or performance. That’s where Microsoft Azure comes in. As a company, Microsoft is committed to making Java developers as efficient and productive as possible, empowering them to use any tool, framework, and application server on any operating system. Microsoft Azure makes it easy to work with the tools and frameworks Java developers already know and love. Whether using IntelliJ, Eclipse, or VS Code, or managing dependencies with Maven or Gradle, developers can keep using their preferred setup. Azure supports trusted Java application servers and popular open-source tools like Spring Boot, JBoss EAP, and WebLogic, making the transition to the cloud seamless and natural. Scaling on Azure is designed with simplicity and security in mind. Developers can count on built-in tools for monitoring, automation, data support, and caching, along with robust security features. With Azure’s flexible services they can scale confidently, manage costs, and build resilient applications that meet business needs. Azure provides everything Java developers need to build and modernize their applications at scale, letting them do so on their own terms. Tooling for Java app migration and modernization priorities Moving your Java applications to the cloud is easier with the right tools. Azure offers a full set of solutions for every type of migration, whether you are rehosting, re-platforming, refactoring, or rearchitecting. These tools work together to help you transition smoothly, allowing you to work faster, more efficiently, and with greater insight. With Azure, you can achieve meaningful results for your business as you modernize your applications. Azure Migrate and Partner-built Solutions Azure Migrate is a key resource in this process. It provides a holistic view of your server and application estate and generates a cloud-readiness report. With app centricity, you can now assess applications at a portfolio level rather than server by server. This makes it easier for IT decision-makers to plan migrations on a larger scale while aligning with business priorities. In addition to Azure Migrate, you can leverage several partner-built solutions such as CAST, Unify, Dr. Migrate, and others to support additional use cases and scenarios. Azure Migrate application and code assessment For developers, Azure Migrate’s app and code assessment tool (AppCAT) offers in-depth code scanning for Java applications. With this tool, you can assess code changes needed to run your apps in the cloud right from within your preferred terminals, like Bash. GitHub Copilot Chat integration further simplifies the planning process, making it easy to explore modernization options through a conversational approach. AppCAT is especially useful for detailed assessments for refactoring and rearchitecting. GitHub Copilot upgrade assistant for Java A major advancement in this toolkit is the new GitHub Copilot upgrade assistant for Java. Upgrading Java code, runtimes, frameworks, and dependencies can be time-consuming, but with the upgrade assistant, you can streamline the process significantly. Start with your local Java project, receive an AI-powered upgrade strategy, and let Copilot handle the bulk of the work. This powerful tool helps you modernize faster, allowing you to focus on building intelligent applications at scale with confidence. Ready to save time upgrading Java? You can apply for the waitlist to the Technical Preview right here – aka.ms/GHCP-for-Java. This early access is open to a limited number of customers, so we encourage you to sign up soon and share your feedback! Deploy and Scale Java Apps on Azure The Java ecosystem is diverse, encompassing technologies like Java SE, Jakarta EE, Spring, and various application servers. Whatever your Java workload – whether building a monolithic app or a cloud-native microservice – Azure provides a comprehensive platform to support it. Azure offers multiple deployment paths to help meet your specific project goals. For those migrating existing Java applications, infrastructure-as-a-service (IaaS) options like Azure Virtual Machines allow you to lift and shift applications without significant re-architecture. Meanwhile, container options, such as Azure Kubernetes Service (AKS), Azure Container Apps and Azure Red Hat OpenShift, make it easier to manage Java applications in containers. Fully managed platform-as-a-service (PaaS) offerings, like Azure App Service, provide out-of-the-box scalability, DevOps integration, and automation for streamlined management. The following diagram shows recommended Azure services for every Java application type deployed as source or binaries: The following diagram shows the recommended Azure services for every Java application type deployed as containers: Building on Azure's reputation as a versatile platform for various applications, we now turn our focus to three specific offerings that demonstrate this flexibility. Today, we highlight JBoss EAP on Azure App Service, Java on Azure Container Apps, and WebSphere Liberty on Azure Kubernetes Service and how to quickly bring your apps to production with Landing Zone Accelerator. We will also walk you through how to build and modernize intelligent Java apps at scale with the latest AI tools and models. JBoss EAP on Azure App Service Azure App Service offers a fully managed platform with specific enhancements for Java, making it an excellent choice for running enterprise Java applications. Recently, several updates have been introduced to bring even greater value to Java developers using JBoss EAP on App Service: Reduced Licensing Costs: Licensing fees for JBoss EAP on App Service have been cut by over 60%, making it more accessible to a wider range of users. Free Tier Availability: A new free tier is available for those interested in testing the service without an upfront cost, providing an easy entry point for trials and evaluation. Affordable Paid Tiers: Lower-cost paid tiers of App Service Plan for JBoss EAP have been introduced, catering to businesses seeking a cost-effective, production-ready environment. Bring Your Own License Support: Soon, customers will be able to apply existing Red Hat volume licenses to further reduce operational costs, adding flexibility for organizations already invested in Red Hat JBoss EAP. These updates provide significant savings, making JBoss EAP on App Service a smart choice for those looking to optimize costs while running Java applications on a reliable, managed platform. Java on Azure Container Apps Azure Container Apps is a popular serverless platform for Java developers who want to deploy and scale containerized applications with minimal management overhead. Designed for microservices, APIs, and event-driven workloads, Azure Container Apps makes it simple to scale applications from zero up to millions of requests, adapting dynamically to meet real-time demand. Azure Container Apps includes several features tailored specifically for Java: Managed Components for Java: With built-in Spring Cloud services like Service Registry and Config Server, managing Java applications is straightforward. These components simplify service registration, discovery, and configuration management. Enhanced Java Monitoring: Azure Monitor provides Java-specific insights, giving developers visibility into their applications and enabling proactive management with detailed metrics. Effortless Scaling: Container Apps can scale down to zero during periods of low demand and scale out as traffic grows, helping optimize costs. The platform also supports GPU-enabled workloads, perfect for AI-powered Java applications. This fully managed platform supports a range of Java frameworks and runtimes, from Spring Boot to Quarkus to Open Liberty and beyond. With built-in DevOps, secure networking, role-based access, and pay-as-you-go pricing, Azure Container Apps offers a powerful and flexible foundation to build, deploy, and monitor any Java application type. WebSphere Liberty on Azure Kubernetes Service IBM's WebSphere is one of the most widely used middleware platforms globally, especially in large enterprises. Many organizations rely on WebSphere Traditional applications, which have strong market penetration in enterprise environments. As IBM focuses on cloud-native solutions, it is encouraging organizations to migrate from WebSphere Traditional to WebSphere Liberty - a more modern, cloud-native Java runtime. With Azure Kubernetes Service, this migration becomes straightforward and manageable, allowing organizations to bring existing WebSphere Traditional apps into a more flexible, scalable environment. Why Azure Kubernetes Service? AKS provides a powerful platform for running containerized Java applications without the complexity of setting up and maintaining Kubernetes yourself. It’s a fully managed Kubernetes service, integrated end-to-end with Azure’s foundational infrastructure, CI/CD, registry, monitoring, and managed services. Because AKS is based on vanilla Kubernetes, all Kubernetes tools work, and there’s no risk of lock-in. AKS offers global availability, enterprise-grade security, automated upgrades, and compliance, making it a reliable choice for organizations aiming to modernize WebSphere applications. Competitive pricing and cost optimization make AKS even more attractive. Why Transform to WebSphere Liberty? WebSphere Liberty, along with Open Liberty, offers compatibility with WebSphere Traditional, creating an easy migration path. Liberty is a lightweight, modular runtime that’s better suited for cloud-native applications. It reduces resource costs, requiring less memory and CPU than WebSphere Traditional and has quicker startup times. Liberty also embraces modern standards, like Jakarta EE Core Profile and MicroProfile, making it ideal for cloud-native applications. Organizations can even re-purpose existing WebSphere Traditional licenses, significantly reducing migration costs. Running WebSphere Liberty on Azure Kubernetes Service is simple and flexible. IBM and Microsoft have certified Liberty on AKS, providing a reliable path for enterprises to move their WebSphere applications to the cloud. With a solution template available in the Azure Marketplace, you can deploy WebSphere Liberty on AKS in a few clicks. This setup works with both new and existing AKS clusters, as well as any container registry, allowing you to deploy quickly and scale as needed. By combining WebSphere Liberty with AKS, you gain the agility of containers and Kubernetes, along with the robust features of a cloud-native runtime on a trusted enterprise platform. Build Right and Fast! Build Your Java or Spring Apps Environment: Development, Test, or Production in Just 15-30 Minutes with Landing Zone Accelerator! To ensure the scalability and quality of your cloud journey, we re-introduce Landing Zone Accelerators, specifically designed for Azure app destinations such as App Service, Azure Container Apps, and Azure Kubernetes Service. An accelerator allows you to establish secure, complaint, and scalable development, test, or production environments within 15-30 minutes. Adhering to Azure's best practices and embedding security by default, a Landing Zone Accelerator ensures that your cloud transition is not only swift but also robust and scalable. It paves the way for both application and platform teams to thrive in the cloud environment. From realizing cost efficiency to streamlining your migration and modernization journey to ensuring the scalability of your cloud operations, our goal is to demonstrate how your cloud transition can drive innovation, and efficiency, and accelerate business value. The Landing Zone Accelerators for App Service, Azure Container Apps, and Azure Kubernetes Service represent an authoritative, proven, and prescriptive infrastructure-as-code solution, designed to assist enterprise customers in establishing a robust environment for deploying Java, Spring, and polyglot apps. It not only expedites the deployment process but also provides a comprehensive design framework, allowing for the clear planning and designing of Azure environments based on established standards. Build Intelligent Java Apps at Scale Today, many enterprise applications are built with Java. As AI grows in popularity and delivers greater business outcomes, Java developers wonder how to integrate it with their apps. Python is popular for AI - particularly for model building, deploying and fine tuning LLMs, and data handling - but moving an app to a new language can be complex and costly. Instead, Java developers can use Azure to combine their Java apps with AI, building intelligent apps without needing to master Python. Azure makes it simple to bring AI into your existing Java applications. Many customers are already using the Azure platform to add intelligence to their Java apps, delivering more value to their businesses. Whether starting fresh or modernizing existing systems, Azure provides the tools needed to build powerful, intelligent applications that scale. Modernize and Build New Intelligent Apps with Azure. Wherever you are in your cloud journey, Azure helps you modernize and build intelligent apps. Azure offers app platform services, data handling at scale, and AI tools that make it easy to create applications that deliver meaningful business value. Intelligent apps can drive growth, amplify team capabilities, and improve productivity. With Azure, you can bring your Java apps into the future and stay ahead of the competition. The Right Tools for Intelligent Java Apps. Building intelligent applications requires a strong foundation. Azure provides essential services like a robust cloud platform, scalable data solutions, and AI tools, including pretrained models and responsible AI practices. These tools ensure your apps are efficient, scalable, and aligned with best practices. Azure offers several key services for this: Azure AI Studio: A one-stop platform for experimenting and deploying AI solutions. It provides tools for model benchmarking, solution testing, and monitoring, making it easy to develop use cases like customer segmentation and predictive maintenance. Azure OpenAI Service: With access to advanced AI models like GPT-4, this service is ideal for content generation, summarization, and semantic search. Build chatbots, create marketing content, or add AI-driven features to your Java apps. Azure Machine Learning: An end-to-end platform for building and deploying machine learning models. It supports various use cases such as recommendation systems, predictive analytics, and anomaly detection. MLOps capabilities ensure your models are continuously improved and managed. Azure AI Search: Uses retrieval-augmented generation (RAG) technology for powerful search capabilities. Enhance user experience with intelligent search options, helping users quickly find relevant information. Azure Cosmos DB: A globally distributed, multi-model database service ideal for high-performance, low-latency applications. It offers turnkey global distribution, automatic scalability, and integration with other Azure services, making it a strong choice for intelligent apps that handle large amounts of data. Azure Database for PostgreSQL with PGVector: This managed PostgreSQL service now includes the PGVector extension, designed for handling vector embeddings in AI applications. It’s a valuable tool for applications requiring fast, similarity-based searches and supports applications involving recommendation engines, semantic search, and personalization. Azure AI Infrastructure: Provides high-performance infrastructure for AI workloads. Whether training large models or performing real-time inference, Azure’s AI infrastructure meets demanding needs. Get Started with AI in Java. If you are a Java app developer, now is a great time to start integrating AI into your apps. Spring developers can use Spring AI for quick integration, and developers using Quarkus or Jakarta EE or any other app type can take advantage of LangChain4j. You can also use Microsoft Azure AI client libraries for Java. No matter what your framework is, Azure has the tools to help you add intelligence to your applications. Meet the Java team at the Microsoft Ignite 2024 Come meet the Java team at Microsoft Ignite 2024! Join our breakout session, "Java on Azure: Modernize and scale enterprise Java applications on Azure" BRK147, for a close look at the newest ways to build, scale, and modernize Java apps on Azure. In this session, our engineers and product experts will share the latest updates and tools for Java developers. You’ll learn about cost-saving options, new cloud tools, and how to add smart features to your apps. This is a session for all Java developers, whether you're moving apps to the cloud or building cloud-native apps from scratch. Everyone can join - either in person at Ignite or virtually from anywhere in the world. The virtual option is free, so you can attend without leaving your desk. Don’t miss the chance to connect with the Java team, ask questions, and get tips to make your Java apps succeed on Azure! Start Today! Join Us at Java + AI Events Worldwide. Sign Up for upcoming Java and AI events like JDConf 2025 and JavaOne 2025. You’ll also find our developer advocates sharing insights and tips at Java conferences and events around the world. Begin framing your app migration plans with resources to guide you through each step. Get started here – aka.ms/Start-Java. Explore the docs and deploy your first Java or Spring app in the cloud. Follow the quick steps here – aka.ms/Java-Hub. Use our tools and information to build a plan and show your leaders the benefits of Java app modernization. Get the details here – azure.com/Java. Start building, planning, and exploring Azure for Java today!Using TensorFlow on Azure Web App
TOC Introduction to OpenAI System Architecture Architecture Focus of This Tutorial Setup Azure Resources File and Directory Structure Bicep Template Running Locally Training Models and Training Data Predicting with the Model Publishing the Project to Azure Code Commit to Azure DevOps Publish to Azure Web App via Pipeline Running on Azure Web App Training the Model Using the Model for Prediction Troubleshooting docker.log freeze after deployment Others Conclusion References 1. Introduction to OpenAI TensorFlow is an open-source machine learning framework developed by Google. It provides tools for building and deploying machine learning models, with a focus on flexibility and scalability. TensorFlow supports deep learning, classical machine learning, and neural network models, enabling tasks like image recognition, natural language processing, and time series forecasting. At its core, TensorFlow uses computational graphs to model mathematical operations, allowing efficient computation on CPUs, GPUs, and TPUs. It features a high-level API, Keras, for easy model building, as well as lower-level APIs for advanced customization. TensorFlow also supports distributed training for large-scale datasets and diverse deployment options, including cloud services, mobile devices, and edge computing. TensorFlow’s ecosystem includes TensorFlow Hub (pre-trained models), TensorFlow Lite (for mobile and IoT), and TensorFlow.js (for JavaScript applications). Its integration with visualization tools like TensorBoard simplifies debugging and performance monitoring. TensorFlow excels in production environments, offering features like TensorFlow Extended (TFX) for end-to-end ML pipelines. With its versatile capabilities and large community, TensorFlow is widely used in industries like healthcare, finance, and technology, making it one of the most powerful tools for modern machine learning development. 2. System Architecture Architecture Development Environment OS: macOS Version: Sonoma 14.1.1 Python Version: 3.9.20 Azure Resources App Service Plan: SKU - Premium Plan 0 V3 App Service: Platform - Linux (Python 3.9, Version 3.9.19) Storage Account: SKU - General Purpose V2 File Share: No backup plan Focus of This Tutorial This tutorial walks you through the following stages: Setting up Azure resources Running the project locally Publishing the project to Azure Running the application on Azure Troubleshooting common issues Each of the mentioned aspects has numerous corresponding tools and solutions. The relevant information for this session is listed in the table below. Local OS Windows Linux Mac V How to setup Azure resources Portal (i.e., REST api) ARM Bicep Terraform V How to deploy project to Azure VSCode CLI Azure DevOps GitHub Action V 3. Setup Azure Resources File and Directory Structure Please open a terminal and enter the following commands: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai bash ./tensorflow/tools/add-venv.sh If you are using a Windows platform, use the following alternative PowerShell commands instead: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai .\tensorflow\tools\add-venv.cmd After completing the execution, you should see the following directory structure: File and Path Purpose tensorflow/tools/add-venv.* The script executed in the previous step (cmd for Windows, sh for Linux/Mac) to create all Python virtual environments required for this tutorial. .venv/tensorflow-webjob/ A virtual environment specifically used for training models (i.e., tokenize text, construct a neuron network for training). tensorflow/webjob/requirements.txt The list of packages (with exact versions) required for the tensorflow-webjob virtual environment. .venv/tensorflow/ A virtual environment specifically used for the Flask application, enabling API endpoint access for querying predictions (i.e., MBTI). tensorflow/requirements.txt The list of packages (with exact versions) required for the tensorflow virtual environment. tensorflow/ The main folder for this tutorial. tensorflow/tools/bicep-template.bicep The Bicep template to setup all the Azure resources related to this tutorial, including an App Service Plan, a Web App, and a Storage Account. tensorflow/tools/create-folder.* A script to create all directories required for this tutorial in the File Share, including train, model, and test. tensorflow/tools/download-sample-training-set.* A script to download a sample training set from MBTI text classifer trained on the Kaggle MBTI dataset, containing MBTI and post data from social media platforms, into the train directory of the File Share. tensorflow/webjob/train_mbti_model.py A script for tokenizing the posts from each record, training an LSTM-based model for MBTI classification, and saves the embedding vectors in the model directory of the File Share. tensorflow/App_Data/jobs/triggered/train-mbti-model/train_mbti_model.sh A shell script for Azure App Service web jobs. It activates the tensorflow-webjob virtual environment and starts the train_mbti_model.py script. tensorflow/api/app.py Code for the Flask application, including routes, port configuration, input parsing, vectors loading, predictions, and output generation. tensorflow/start.sh A script executed after deployment (as specified in the Bicep template startup command I will introduce it later). It sets up the virtual environment and starts the Flask application to handle web requests. tensorflow/pipeline.yml A process document for an Azure DevOps pipeline, detailing the steps to deploy code to an Azure Web App. Bicep Template We need to create the following resources or services: Manual Creation Required Resource/Service App Service Plan No Resource (plan) App Service Yes Resource (app) Storage Account Yes Resource (storageAccount) File Share Yes Service Let’s take a look at the tensorflow/tools/bicep-template.bicep file. Refer to the configuration section for all the resources. Since most of the configuration values don’t require changes, I’ve placed them in the variables section of the ARM template rather than the parameters section. This helps keep the configuration simpler. However, I’d still like to briefly explain some of the more critical settings. As you can see, I’ve adopted a camelCase naming convention, which combines the [Resource Type] with [Setting Name and Hierarchy]. This makes it easier to understand where each setting will be used. The configurations in the diagram are sorted by resource name, but the following list is categorized by functionality for better clarity. Configuration Name Value Purpose storageAccountFileShareName data-and-model [Purpose 1: Link File Share to Web App] Use this fixed name for File Share storageAccountFileShareShareQuota 5120 [Purpose 1: Link File Share to Web App] The value is in GB storageAccountFileShareEnabledProtocols SMB [Purpose 1: Link File Share to Web App] appSiteConfigAzureStorageAccountsType AzureFiles [Purpose 1: Link File Share to Web App] appSiteConfigAzureStorageAccountsProtocol Smb [Purpose 1: Link File Share to Web App] planKind linux [Purpose 2: Specify platform and stack runtime] Select Linux (default if Python stack is chosen) planSkuTier Premium0V3 [Purpose 2: Specify platform and stack runtime] Choose at least Premium Plan to ensure enough memory for your AI workloads planSkuName P0v3 [Purpose 2: Specify platform and stack runtime] Same as above appKind app,linux [Purpose 2: Specify platform and stack runtime] Same as above appSiteConfigLinuxFxVersion PYTHON|3.9 [Purpose 2: Specify platform and stack runtime] Select Python 3.9 to avoid dependency issues appSiteConfigAppSettingsWEBSITES_CONTAINER_START_TIME_LIMIT 1800 [Purpose 3: Deploying] The value is in seconds, ensuring the Startup Command can continue execution beyond the default timeout of 230 seconds. This tutorial’s Startup Command typically takes around 1200 seconds, so setting it to 1800 seconds (i.e., it is the max value) provides a safety margin and accommodates future project expansion (e.g., adding more packages) appSiteConfigAppCommandLine [ -f /home/site/wwwroot/start.sh ] && bash /home/site/wwwroot/start.sh || GUNICORN_CMD_ARGS=\"--timeout 600 --access-logfile '-' --error-logfile '-' -c /opt/startup/gunicorn.conf.py --chdir=/opt/defaultsite\" gunicorn application:app [Purpose 3: Deploying] This is the Startup Command, which can be break down into 3 parts: First (-f /home/site/wwwroot/start.sh): Checks whether start.sh exists. This is used to determine whether the app is in its initial state (just created) or has already been deployed. Second (bash /home/site/wwwroot/start.sh): If the file exists, it means the app has already been deployed. The start.sh script will be executed, which installs the necessary packages and starts the Flask application. Third (GUNICORN_CMD_ARGS=\"--timeout 600 --access-logfile '-' --error-logfile '-' -c /opt/startup/gunicorn.conf.py --chdir=/opt/defaultsite\" gunicorn application:app): If the file does not exist, the command falls back to the default HTTP server (gunicorn) to start the web app. Since the command is enclosed in double quotes within the ARM template, during actual execution, replace \" with " appSiteConfigAppSettingsSCM_DO_BUILD_DURING_DEPLOYMENT false [Purpose 3: Deploying] Since we have already defined the handling for different virtual environments in start.sh, we do not need to initiate the default build process of the Web App appSiteConfigAppSettingsWEBSITES_ENABLE_APP_SERVICE_STORAGE true [Purpose 4: Webjobs] This setting is required to enable the App Service storage feature, which is necessary for using web jobs (e.g., for model training) storageAccountPropertiesAllowSharedKeyAccess true [Purpose 5: Troubleshooting] This setting is enabled by default. The reason for highlighting it is that certain enterprise IT policies may enforce changes to this configuration after a period, potentially causing a series of issues. For more details, please refer to the Troubleshooting section below. Return to terminal and execute the following commands (their purpose has been described earlier). # Please change <ResourceGroupName> to your prefer name, for example: azure-appservice-ai # Please change <RegionName> to your prefer region, for example: eastus2 # Please change <ResourcesPrefixName> to your prefer naming pattern, for example: tensorflow-bicep (it will create tensorflow-bicep-asp as App Service Plan, tensorflow-bicep-app for web app, and tensorflowbicepsa for Storage Account) az group create --name <ResourceGroupName> --location <RegionName> az deployment group create --resource-group <ResourceGroupName> --template-file ./tensorflow/tools/bicep-template.bicep --parameters resourcePrefix=<ResourcesPrefixName> If you are using a Windows platform, use the following alternative PowerShell commands instead: # Please change <ResourceGroupName> to your prefer name, for example: azure-appservice-ai # Please change <RegionName> to your prefer region, for example: eastus2 # Please change <ResourcesPrefixName> to your prefer naming pattern, for example: tensorflow-bicep (it will create tensorflow-bicep-asp as App Service Plan, tensorflow-bicep-app for web app, and tensorflowbicepsa for Storage Account) az group create --name <ResourceGroupName> --location <RegionName> az deployment group create --resource-group <ResourceGroupName> --template-file .\tensorflow\tools\bicep-template.bicep --parameters resourcePrefix=<ResourcesPrefixName> After execution, please copy the output section containing 3 key-value pairs from the result like this. Return to terminal and execute the following commands: # Please setup 3 variables you've got from the previous step OUTPUT_STORAGE_NAME="<outputStorageName>" OUTPUT_STORAGE_KEY="<outputStorageKey>" OUTPUT_SHARE_NAME="<outputShareName>" # URL encode the storage key ENCODED_OUTPUT_STORAGE_KEY=$(python3 -c " import urllib.parse key = '''$OUTPUT_STORAGE_KEY''' encoded_key = urllib.parse.quote(key, safe='') # No safe characters, encode everything print(encoded_key) ") # Mount open smb://$OUTPUT_STORAGE_NAME:$ENCODED_OUTPUT_STORAGE_KEY@$OUTPUT_STORAGE_NAME.file.core.windows.net/$OUTPUT_SHARE_NAME Or you could simply go to Azure Portal, navigate to the File Share you just created, and refer to the diagram below to copy the required command. You can choose Linux or Windows if you are using such OS in your dev environment. After executing the command, the network drive will be successfully mounted. 4. Running Locally Training Models and Training Data Return to terminal and execute the following commands (their purpose has been described earlier). source .venv/tensorflow-webjob/bin/activate bash ./tensorflow/tools/create-folder.sh bash ./tensorflow/tools/download-sample-training-set.sh python ./tensorflow/webjob/train_mbti_model.py If you are using a Windows platform, use the following alternative PowerShell commands instead: .\.venv\tensorflow-webjob\Scripts\Activate.ps1 .\tensorflow\tools\create-folder.cmd .\tensorflow\tools\download-sample-training-set.cmd python .\tensorflow\webjob\train_mbti_model.py After execution, the File Share will now include the following directories and files. Let’s take a brief detour to examine the structure of the training data downloaded from the GitHub. The dataset used in this project focuses on MBTI (Myers-Briggs Type Indicator) personality types. Each record in the dataset contains a user’s MBTI type and a collection of their social media posts, separated by |||. This tutorial repurposes the dataset to classify personality types based on textual data. This image represents the raw data, where each line includes an MBTI type and its associated text. For training, the posts are tokenized and transformed into numerical sequences using TensorFlow's preprocessing tools. This step involves converting each word into a corresponding token based on a fixed vocabulary size. These sequences are then padded to a uniform length, ensuring consistency in the input data. During training, the performance is heavily influenced by factors like data balancing and hyperparameter tuning. The MBTI dataset is inherently imbalanced, with certain personality types appearing far more frequently than others. To address this, only 30 samples per type are used in training to ensure balance. However, this approach simplifies the task and may lead to suboptimal results. The inference step involves tokenizing a new input post and passing it through the trained model to predict the MBTI type. It is important to note that with the current setup, the inference results may often return the same prediction. This is due to the limited dataset size, imbalanced data handling, and the need for further tuning of training parameters such as the number of epochs, batch size, and learning rate. This tutorial introduces an approach to train and infer MBTI personality types using TensorFlow. While the process highlights key steps like data preprocessing, model training, and inference, it does not delve deeply into AI-specific topics like advanced model optimization or deployment. To achieve better results, we could setup the dataset can be expanded to include more samples per personality type. Or setup hyperparameters like the learning rate, number of epochs, and embedding dimensions should be fine-tuned. Predicting with the Model Return to terminal and execute the following commands. First, deactivate the virtual environment, then activate the virtual environment for the Flask application, and finally, start the Flask app. Commands for Linux or Mac: deactivate source .venv/tensorflow/bin/activate python ./tensorflow/api/app.py Commands for Windows: deactivate .\.venv\tensorflow\Scripts\Activate.ps1 python .\tensorflow\api\app.py When you see a screen similar to the following, it means the server has started successfully. Press Ctrl+C to stop the server if needed. Before conducting the actual test, let’s construct some sample query data: I am happy Next, open a terminal and use the following curl commands to send requests to the app: curl -X GET "http://0.0.0.0:8000/api/detect" -H "Content-Type: application/json" -d '{"post": "I am happy"}' You should see the prediction results. PS: Your results may differ from mine due to variations in the sampling of your training dataset compared to mine. 5. Publishing the Project to Azure Code Commit to Azure DevOps First, create a new and empty repository (referred to as repo) under your Azure DevOps project and get its URL. Open a terminal in the cloned azure-appservice-ai project directory and run the following commands to add the new repo as a push/pull target. Then, verify the associated git repos for the directory. git remote add azure https://<organization>@dev.azure.com/<organization>/<project>/_git/azure-appservice-ai git remote -v Next, run the following commands in the terminal to push the entire project to the Azure DevOps repo. git push azure --all The following steps need to be performed only once.These configurations ensure that the pipeline can automatically deploy the tensorflow portion of the azure-appservice-ai project to the newly created Azure Web App. Setup Service Connection: Go to Project Settings in Azure DevOps and perform the necessary operations. Specify the Service connection name as "azure-appservice-ai-tensorflow" (you can use any name for easy identification). Create Pipeline YAML File: Navigate to the tensorflow subdirectory and create a new file named azure-pipeline.yml Copy the contents of another file named pipeline.yml (in the same directory) into azure-pipeline.yml Modify the variables section as indicated by the comments, then save and commit the changes. Setup the Pipeline: Navigate to the Pipelines section and create a new pipeline. Follow the prompts to select the newly created azure-pipeline.yml as the pipeline script file. Save the configuration (do not run it yet). The above setup steps only need to be done once. Next, you can deploy the project to the Azure Web App using the pipeline in different ways. Publish to Azure Web App via Pipeline Manual Trigger: Navigate to the newly created pipeline. Click Run Pipeline to start the deployment process. Click on the deployment to monitor its progress. Below is an example of a successful deployment screen. Trigger on Push: Alternatively, you can configure the pipeline to run automatically whenever new code is pushed to the Azure DevOps repo: Open a terminal and run the following commands (after code updates): git push azure --all This will trigger a new pipeline deployment process. 6. Running on Azure Web App Training the Model Return to terminal and execute the following commands to invoke the WebJobs. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; curl -X POST -H "Authorization: Bearer $token" -H "Content-Type: application/json" -d '{}' "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/train-mbti-model/run?api-version=2024-04-01" Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/train-mbti-model/run?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"; "Content-type" = "application/json"} -Method POST -Body '{}' You could see the training status by execute the following commands. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; response=$(curl -s -H "Authorization: Bearer $token" "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01") ; echo "$response" | jq Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv); $response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"} -Method GET ; $response | ConvertTo-Json -Depth 10 Processing Complete And you can get the latest detail log by execute the following commands. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; history_id=$(az webapp webjob triggered log --resource-group <resourcegroup_name> --name <webapp_name> --webjob-name train-mbti-model --query "[0].id" -o tsv | sed 's|.*/history/||') ; response=$(curl -X GET -H "Authorization: Bearer $token" -H "Content-Type: application/json" "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/train-mbti-model/history/$history_id/?api-version=2024-04-01") ; log_url=$(echo "$response" | jq -r '.properties.output_url') ; curl -X GET -H "Authorization: Bearer $token" "$log_url" Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token = az account get-access-token --resource https://management.azure.com --query accessToken -o tsv ; $history_id = az webapp webjob triggered log --resource-group <resourcegroup_name> --name <webapp_name> --webjob-name train-mbti-model --query "[0].id" -o tsv | ForEach-Object { ($_ -split "/history/")[-1] } ; $response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/train-mbti-model/history/$history_id/?api-version=2024-04-01" -Headers @{ Authorization = "Bearer $token" } -Method GET ; $log_url = $response.properties.output_url ; Invoke-RestMethod -Uri $log_url -Headers @{ Authorization = "Bearer $token" } -Method GET Once you see the report in the Logs, it indicates that the training is complete, and the Flask app is ready for predictions. You can also find the newly trained models in the File Share mounted in your local environment. Using the Model for Prediction Just like in local testing, open a terminal and use the following curl commands to send requests to the app: # Note: Replace the instance of tensorflow-bicep-app with the name of your web app. curl -X GET "https://tensorflow-bicep-app.azurewebsites.net/api/detect" -H "Content-Type: application/json" -d '{"post": "I am happy"}' As with the local environment, you should see the expected results. 7. Troubleshooting docker.log freeze after deployment Symptom:I cannot get the latest deployment status after Azure DevOps publish the code to Web App via kudu site and frontpage getting error 504 Cause:This project includes two virtual environments, each containing a TensorFlow package. During the start.sh process of creating these environments, each environment takes approximately 10 minutes to set up. As a result, the docker.log or Log Stream might temporarily stall for about 20 minutes at a certain stage. Resolution: After roughly 20 minutes, once all the packages are downloaded, the logs will resume recording. Others Using Scikit-learn on Azure Web App Using OpenAI on Azure Web App 8. Conclusion TensorFlow, much like a Swiss Army knife, encompasses a wide range of training algorithms. While Azure Web App is not typically used for training models, it can still serve as a platform for inference. In the future, I plan to introduce how pre-trained models can be directly loaded using JavaScript. This approach allows the inference workload for non-sensitive models to be offloaded to the client side. 9. References TensorFlow MBTI text classifer trained on the Kaggle MBTI dataset (MBTI) Myers-Briggs Personality Type DatasetIntroducing Serverless GPUs on Azure Container Apps
We're excited to announce the public preview ofAzure Container Apps Serverless GPUs accelerated by NVIDIA. This feature provides customers with NVIDIA A100 GPUs and NVIDIA T4 GPUs in a serverless environment, enabling effortless scaling and flexibility for real-time custom model inferencing and other machine learning tasks. Serverless GPUs accelerate the speed of your AI development team by allowing you to focus on your core AI code and less on managing infrastructure when using NVIDIA accelerated computing. They provide an excellent middle layer option betweenAzure AI Model Catalog's serverless APIs and hosting models on managed compute. It provides full data governance as your data never leaves the boundaries of your container while still providing a managed, serverless platform from which to build your applications. Serverless GPUs are designed to meet the growing demands of modern applications by providing powerful NVIDIA accelerated computing resources without the need for dedicated infrastructure management. "Azure Container Apps' serverless GPU offering is a leap forward for AI workloads. Serverless NVIDIA GPUs are well suited for a wide array of AI workloads from real-time inferencing scenarios with custom models to fine-tuning. NVIDIA is also working with Microsoft to bring NVIDIA NIM microservices to Azure Container Apps to optimize AI inference performance.” - Dave Salvator, Director, Accelerated Computing Products, NVIDIA Key benefits of serverless GPUs Scale-to zero GPUs: Support for serverless scaling of NVIDIA A100 and T4 GPUs. Per-second billing: Pay only for the GPU compute you use. Built-in data governance: Your data never leaves the container boundary. Flexible compute options: Choose between NVIDIA A100 and T4 GPUs. Middle-layer for AI development: Bring your own model on a managed, serverless compute platform. Scenarios Whether you choose to use NVIDIA A100 or T4 GPUs will depend on the types of apps you're creating. The following are a couple example scenarios. For each scenario with serverless GPUs, you pay only for the compute you use with per-second billing, and your apps will automatically scale in and out from zero to meet the demand. NVIDIA T4 Real-time and batch inferencing: Using custom open-source models with fast startup times, automatic scaling, and a per-second billing model, serverless GPUs are ideal for dynamic applications that don't already have a serverless API in the model catalog. NVIDIA A100 Compute intensive machine learning scenarios: Significantly speed up applications that implement fine-tuned custom generative AI models, deep learning, or neural networks. High performance computing (HPC) and data analytics: Applications that require complex calculations or simulations, such as scientific computing and financial modeling as well as accelerated data processing and analysis among massive datasets. Get started with serverless GPUs Serverless GPUs are now available for workload profile environments in West US 3 and Australia East regions with more regions to come. You will need to have quota enabled on your subscription in order to use serverless GPUs. By default, all Microsoft Enterprise Agreement customers will have one quota. If additional quota is needed, please request it here. Note: In order to achieve the best performance with serverless GPUs, use an Azure Container Registry (ACR) with artifact streaming enabled for your image tag.Follow steps here to enable artifact streaming on your ACR. From the portal, you can select to enable GPUs for your Consumption app in the container tab when creating your Container App or your Container App Job. You can also add a new consumption GPU workload profile to your existing Container App environment through the workload profiles UX in portal or through the CLI commands for managing workload profiles. Deploy a sample Stable Diffusion app To try out serverless GPUs, you can use the stable diffusion image which is provided as a quickstart during the container app create experience: In the container tab select the Use quickstart image box. In the quickstart image dropdown, selectGPU hello world container. If you wish to pull the GPU container image into your own ACR to enable artifact streaming for improved performance, or if you wish to manually enter the image, you can find the image atmcr.microsoft.com/k8se/gpu-quickstart:latest. For full steps on using your own image with serverless GPUs, see the tutorial onusing serverless GPUs in Azure Container Apps. Learn more about serverless GPUs With serverless GPUs, Azure Container Apps now simplifies the development of your AI applications by providing scale-to-zero compute, pay-as you go pricing, reduced infrastructure management, and more. To learn more, visit: Using serverless GPUs in Azure Container Apps (preview) | Microsoft Learn Tutorial: Generate images using serverless GPUs in Azure Container Apps (preview) | Microsoft Learn
Connect Privately to Azure Front Door with Azure Container Apps
Azure Container Apps is a fully managed serverless container service that enables you to deploy and run containerized applications with per-second billing and autoscaling without having to manage infrastructure. The service also provides support for a number of enhanced networking capabilities to address security and compliance needs such as network security groups (NSGs), Azure Firewall, and more. Today, Azure Container Apps is excited to announce public preview for another key networking capability, private endpoints for workload profile environments. This feature allows customers to connect to their Container Apps environment using a private IP address in their Azure Virtual Network, thereby eliminating exposure to the public internet and securing access to their applications. With the introduction of private endpoints for workload profile environments, you can now also establish a direct connection from Azure Front Door to your Container Apps environment viaPrivate Link. By enabling Private Link for an Azure Container Apps origin, customers benefit from an extra layer of security that further isolates their traffic from the public internet. Currently, you can configure this connectivity through CLI (portal support coming soon). In this post, we will do a brief overview of private endpoints on Azure Container Apps and the process of privately connecting it to Azure Front Door. Getting started with private endpoints on Azure Container Apps Private endpoints can be enabled either during the creation of a new environment or within an existing one. For new environments, you simply navigate to theNetworking tab, disable public network access, and enable private endpoints. To manage the creation of private endpoints in an existing environment, you can use the newNetworking blade, which is also in public preview. Since private endpoints use a private IP address, the endpoint for a container app is inaccessible through the public internet. This can be confirmed by the lack of connectivity when opening the application URL. If you prefer using CLI, you can find further guidance in enabling private endpoints at Use a private endpoint with an Azure Container Apps environment (preview). Adding container apps as a private origin for Azure Front Door With private endpoints, you can securely connect them to Azure Front Door through Private Link as well. The current process involves CLI commands that guide you in enabling an origin for Private Link and approving the private endpoint connection. Once approved, Azure Front Door assigns a private IP address from a managed regional private network, and you can verify the connectivity between your container app and the Azure Front Door. For a detailed tutorial, please navigate toCreate a private link to an Azure Container App with Azure Front Door (preview). Troubleshooting Have trouble testing the private endpoints? After creating a private endpoint for a container app, you can build and deploy a virtual machine to test the private connection. With no public inbound ports, this virtual machine would be associated with the virtual network defined during creation of the private endpoint. After creating the virtual machine, you can connect via Bastion and verify the private connectivity. You may find outlined instructions atVerify the private endpoint connection. Conclusion The public preview of private endpoints and private connectivity to Azure Front Door for workload profile environments is a long-awaited feature in Azure Container Apps. We encourage you to implement private endpoints for enhanced security and look forward to your feedback on this experience at our GitHub page. Additional Resources To learn more, please visit the following links to official documentation: Networking in Azure Container Apps environment - Private Endpoints Use a private endpoint with an Azure Container Apps environment Create a private link to an Azure Container App with Azure Front Door (preview) What is a private endpoint? What is Azure Private Link?Using OpenAI on Azure Web App
TOC Introduction to OpenAI System Architecture Architecture Focus of This Tutorial Setup Azure Resources File and Directory Structure ARM Template ARM Template From Azure Portal Running Locally Training Models and Training Data Predicting with the Model Publishing the Project to Azure Running on Azure Web App Training the Model Using the Model for Prediction Troubleshooting Startup Command Issue App Becomes Unresponsive After a Period az cli command for Linux webjobs fail Others Conclusion References 1. Introduction to OpenAI OpenAI is a leading artificial intelligence research and deployment company founded in December 2015. Its mission is to ensure that artificial general intelligence (AGI)—highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. OpenAI focuses on developing safe and scalable AI technologies and ensuring equitable access to these innovations. Known for its groundbreaking advancements in natural language processing, OpenAI has developed models like GPT (Generative Pre-trained Transformer), which powers applications for text generation, summarization, translation, and more. GPT models have revolutionized fields like conversational AI, creative writing, and programming assistance. OpenAI has also released models like Codex, designed to understand and generate computer code, and DALL·E, which creates images from textual descriptions. OpenAI operates with a unique hybrid structure: a for-profit company governed by a nonprofit entity to balance the development of AI technology with ethical considerations. The organization emphasizes safety, research transparency, and alignment to human values. By providing access to its models through APIs and fostering partnerships, OpenAI empowers developers, businesses, and researchers to leverage AI for innovative solutions across diverse industries. Its long-term goal is to ensure AI advances benefit humanity as a whole. 2. System Architecture Architecture Development Environment OS: Ubuntu Version: Ubuntu 18.04 Bionic Beaver Python Version: 3.7.3 Azure Resources App Service Plan: SKU - Premium Plan 0 V3 App Service: Platform - Linux (Python 3.9, Version 3.9.19) Storage Account: SKU - General Purpose V2 File Share: No backup plan Focus of This Tutorial This tutorial walks you through the following stages: Setting up Azure resources Running the project locally Publishing the project to Azure Running the application on Azure Troubleshooting common issues Each of the mentioned aspects has numerous corresponding tools and solutions. The relevant information for this session is listed in the table below. Local OS Windows Linux Mac V How to setup Azure resources Portal (i.e., REST api) ARM Bicep Terraform V V How to deploy project to Azure VSCode CLI Azure DevOps GitHub Action V 3. Setup Azure Resources File and Directory Structure Please open a bash terminal and enter the following commands: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai bash ./openai/tools/add-venv.sh If you are using a Windows platform, use the following alternative PowerShell commands instead: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai .\openai\tools\add-venv.cmd After completing the execution, you should see the following directory structure: File and Path Purpose openai/tools/add-venv.* The script executed in the previous step (cmd for Windows, sh for Linux/Mac) to create all Python virtual environments required for this tutorial. .venv/openai-webjob/ A virtual environment specifically used for training models (i.e., calculating embedding vectors indeed). openai/webjob/requirements.txt The list of packages (with exact versions) required for the openai-webjob virtual environment. .venv/openai/ A virtual environment specifically used for the Flask application, enabling API endpoint access for querying predictions (i.e., suggestion). openai/requirements.txt The list of packages (with exact versions) required for the openai virtual environment. openai/ The main folder for this tutorial. openai/tools/arm-template.json The ARM template to setup all the Azure resources related to this tutorial, including an App Service Plan, a Web App, and a Storage Account. openai/tools/create-folder.* A script to create all directories required for this tutorial in the File Share, including train, model, and test. openai/tools/download-sample-training-set.* A script to download a sample training set from News-Headlines-Dataset-For-Sarcasm-Detection, containing headlines data from TheOnion and HuffPost, into the train directory of the File Share. openai/webjob/cal_embeddings.py A script for calculating embedding vectors from headlines. It loads the training set, applies the transformation on OpenAI API, and saves the embedding vectors in the model directory of the File Share. openai/App_Data/jobs/triggered/cal-embeddings/cal_embeddings.sh A shell script for Azure App Service web jobs. It activates the openai-webjob virtual environment and starts the cal_embeddings.py script. openai/api/app.py Code for the Flask application, including routes, port configuration, input parsing, vectors loading, predictions, and output generation. openai/start.sh A script executed after deployment (as specified in the ARM template startup command I will introduce it later). It sets up the virtual environment and starts the Flask application to handle web requests. ARM Template We need to create the following resources or services: Manual Creation Required Resource/Service App Service Plan No Resource (plan) App Service Yes Resource (app) Storage Account Yes Resource (storageAccount) File Share Yes Service Let’s take a look at the openai/tools/arm-template.json file. Refer to the configuration section for all the resources. Since most of the configuration values don’t require changes, I’ve placed them in the variables section of the ARM template rather than the parameters section. This helps keep the configuration simpler. However, I’d still like to briefly explain some of the more critical settings. As you can see, I’ve adopted a camelCase naming convention, which combines the [Resource Type] with [Setting Name and Hierarchy]. This makes it easier to understand where each setting will be used. The configurations in the diagram are sorted by resource name, but the following list is categorized by functionality for better clarity. Configuration Name Value Purpose storageAccountFileShareName data-and-model [Purpose 1: Link File Share to Web App] Use this fixed name for File Share storageAccountFileShareShareQuota 5120 [Purpose 1: Link File Share to Web App] The value is in GB storageAccountFileShareEnabledProtocols SMB [Purpose 1: Link File Share to Web App] appSiteConfigAzureStorageAccountsType AzureFiles [Purpose 1: Link File Share to Web App] appSiteConfigAzureStorageAccountsProtocol Smb [Purpose 1: Link File Share to Web App] planKind linux [Purpose 2: Specify platform and stack runtime] Select Linux (default if Python stack is chosen) planSkuTier Premium0V3 [Purpose 2: Specify platform and stack runtime] Choose at least Premium Plan to ensure enough memory for your AI workloads planSkuName P0v3 [Purpose 2: Specify platform and stack runtime] Same as above appKind app,linux [Purpose 2: Specify platform and stack runtime] Same as above appSiteConfigLinuxFxVersion PYTHON|3.9 [Purpose 2: Specify platform and stack runtime] Select Python 3.9 to avoid dependency issues appSiteConfigAppSettingsWEBSITES_CONTAINER_START_TIME_LIMIT 600 [Purpose 3: Deploying] The value is in seconds, ensuring the Startup Command can continue execution beyond the default timeout of 230 seconds. This tutorial’s Startup Command typically takes around 300 seconds, so setting it to 600 seconds provides a safety margin and accommodates future project expansion (e.g., adding more packages) appSiteConfigAppCommandLine [ -f /home/site/wwwroot/start.sh ] && bash /home/site/wwwroot/start.sh || GUNICORN_CMD_ARGS=\"--timeout 600 --access-logfile '-' --error-logfile '-' -c /opt/startup/gunicorn.conf.py --chdir=/opt/defaultsite\" gunicorn application:app [Purpose 3: Deploying] This is the Startup Command, which can be break down into 3 parts: First (-f /home/site/wwwroot/start.sh): Checks whether start.sh exists. This is used to determine whether the app is in its initial state (just created) or has already been deployed. Second (bash /home/site/wwwroot/start.sh): If the file exists, it means the app has already been deployed. The start.sh script will be executed, which installs the necessary packages and starts the Flask application. Third (GUNICORN_CMD_ARGS=\"--timeout 600 --access-logfile '-' --error-logfile '-' -c /opt/startup/gunicorn.conf.py --chdir=/opt/defaultsite\" gunicorn application:app): If the file does not exist, the command falls back to the default HTTP server (gunicorn) to start the web app. Since the command is enclosed in double quotes within the ARM template, during actual execution, replace \" with " appSiteConfigAppSettingsSCM_DO_BUILD_DURING_DEPLOYMENT false [Purpose 3: Deploying] Since we have already defined the handling for different virtual environments in start.sh, we do not need to initiate the default build process of the Web App appSiteConfigAppSettingsWEBSITES_ENABLE_APP_SERVICE_STORAGE true [Purpose 4: Webjobs] This setting is required to enable the App Service storage feature, which is necessary for using web jobs (e.g., for model training) storageAccountPropertiesAllowSharedKeyAccess true [Purpose 5: Troubleshooting] This setting is enabled by default. The reason for highlighting it is that certain enterprise IT policies may enforce changes to this configuration after a period, potentially causing a series of issues. For more details, please refer to the Troubleshooting section below. Return to bash terminal and execute the following commands (their purpose has been described earlier). # Please change <ResourceGroupName> to your prefer name, for example: azure-appservice-ai # Please change <RegionName> to your prefer region, for example: eastus2 # Please change <ResourcesPrefixName> to your prefer naming pattern, for example: openai-arm (it will create openai-arm-asp as App Service Plan, openai-arm-app for web app, and openaiarmsa for Storage Account) az group create --name <ResourceGroupName> --location <RegionName> az deployment group create --resource-group <ResourceGroupName> --template-file ./openai/tools/arm-template.json --parameters resourcePrefix=<ResourcesPrefixName> If you are using a Windows platform, use the following alternative PowerShell commands instead: # Please change <ResourceGroupName> to your prefer name, for example: azure-appservice-ai # Please change <RegionName> to your prefer region, for example: eastus2 # Please change <ResourcesPrefixName> to your prefer naming pattern, for example: openai-arm (it will create openai-arm-asp as App Service Plan, openai-arm-app for web app, and openaiarmsa for Storage Account) az group create --name <ResourceGroupName> --location <RegionName> az deployment group create --resource-group <ResourceGroupName> --template-file .\openai\tools\arm-template.json --parameters resourcePrefix=<ResourcesPrefixName> After execution, please copy the output section containing 3 key-value pairs from the result like this. Return to bash terminal and execute the following commands: # Please setup 3 variables you've got from the previous step OUTPUT_STORAGE_NAME="<outputStorageName>" OUTPUT_STORAGE_KEY="<outputStorageKey>" OUTPUT_SHARE_NAME="<outputShareName>" sudo mkdir -p /mnt/$OUTPUT_SHARE_NAME if [ ! -d "/etc/smbcredentials" ]; then sudo mkdir /etc/smbcredentials fi CREDENTIALS_FILE="/etc/smbcredentials/$OUTPUT_STORAGE_NAME.cred" if [ ! -f "$CREDENTIALS_FILE" ]; then sudo bash -c "echo \"username=$OUTPUT_STORAGE_NAME\" >> $CREDENTIALS_FILE" sudo bash -c "echo \"password=$OUTPUT_STORAGE_KEY\" >> $CREDENTIALS_FILE" fi sudo chmod 600 $CREDENTIALS_FILE sudo bash -c "echo \"//$OUTPUT_STORAGE_NAME.file.core.windows.net/$OUTPUT_SHARE_NAME /mnt/$OUTPUT_SHARE_NAME cifs nofail,credentials=$CREDENTIALS_FILE,dir_mode=0777,file_mode=0777,serverino,nosharesock,actimeo=30\" >> /etc/fstab" sudo mount -t cifs //$OUTPUT_STORAGE_NAME.file.core.windows.net/$OUTPUT_SHARE_NAME /mnt/$OUTPUT_SHARE_NAME -o credentials=$CREDENTIALS_FILE,dir_mode=0777,file_mode=0777,serverino,nosharesock,actimeo=30 Or you could simply go to Azure Portal, navigate to the File Share you just created, and refer to the diagram below to copy the required command. You can choose Windows or Mac if you are using such OS in your dev environment. After executing the command, the network drive will be successfully mounted. You can use df to verify, as illustrated in the diagram. ARM Template From Azure Portal In addition to using az cli to invoke ARM Templates, if the JSON file is hosted on a public network URL, you can also load its configuration directly into the Azure Portal by following the method described in the article [Deploy to Azure button - Azure Resource Manager]. This is my example. Click Me After filling in all the required information, click Create. Once the creation process is complete, click Outputs on the left menu to retrieve the connection information for the File Share. 4. Running Locally Training Models and Training Data In the next steps, you will need to use OpenAI services. Please ensure that you have registered as a member and added credits to your account (Billing overview - OpenAI API). For this example, adding $10 USD will be sufficient. Additionally, you will need to generate a new API key (API keys - OpenAI API), you may choose to create a project as well for future project organization, depending on your needs (Projects - OpenAI API). After getting the API key, create a text file named apikey.txt in the openai/tools/ folder. Paste the key you just copied into the file and save it. Return to bash terminal and execute the following commands (their purpose has been described earlier). source .venv/openai-webjob/bin/activate bash ./openai/tools/create-folder.sh bash ./openai/tools/download-sample-training-set.sh python ./openai/webjob/cal_embeddings.py --sampling_ratio 0.002 If you are using a Windows platform, use the following alternative PowerShell commands instead: .\.venv\openai-webjob\Scripts\Activate.ps1 .\openai\tools\create-folder.cmd .\openai\tools\download-sample-training-set.cmd python .\openai\webjob\cal_embeddings.py --sampling_ratio 0.002 After execution, the File Share will now include the following directories and files. Let’s take a brief detour to examine the structure of the training data downloaded from the GitHub. The right side of the image explains each field of the data. This dataset was originally used to detect whether news headlines contain sarcasm. However, I am repurposing it for another application. In this example, I will use the "headline" field to create embeddings. The left side displays the raw data, where each line is a standalone JSON string containing the necessary fields. In the code, I first extract the "headline" field from each record and send it to OpenAI to compute the embedding vector for the text. This embedding represents the position of the text in a semantic space (akin to coordinates in a multi-dimensional space). After the computation, I obtain an embedding vector for each headline. Moving forward, I will refer to these simply as embeddings. By the way, the sampling_ratio parameter in the command is something I configured to speed up the training process. The original dataset contains nearly 30,000 records, which would result in a training time of around 8 hours. To simplify the tutorial, you can specify a relatively low sampling_ratio value (ranging from 0 to 1, representing 0% to 100% sampling from the original records). For example, a value of 0.01 corresponds to a 1% sample, allowing you to accelerate the experiment. In this semantic space, vectors that are closer to each other often have similar values, which corresponds to similar meanings. In this context, the distance between vectors will serve as our metric to evaluate the semantic similarity between pieces of text. For this, we will use a method called cosine similarity. In the subsequent tutorial, we will construct some test texts. These test texts will also be converted into embeddings using the same method. Each test embedding will then be compared against the previously computed headline embeddings. The comparison will identify the nearest headline embeddings in the multi-dimensional vector space, and their original text will be returned. Additionally, we will leverage OpenAI's well-known generative AI capabilities to provide a textual explanation. This explanation will describe why the constructed test text is related to the recommended headline. Predicting with the Model Return to terminal and execute the following commands. First, deactivate the virtual environment used for calculating the embeddings, then activate the virtual environment for the Flask application, and finally, start the Flask app. Commands for Linux or Mac: deactivate source .venv/openai/bin/activate python ./openai/api/app.py Commands for Windows: deactivate .\.venv\openai\Scripts\Activate.ps1 python .\openai\api\app.py When you see a screen similar to the following, it means the server has started successfully. Press Ctrl+C to stop the server if needed. Before conducting the actual test, let’s construct some sample query data: education Next, open a terminal and use the following curl commands to send requests to the app: curl -X GET http://127.0.0.1:8000/api/detect?text=education You should see the calculation results, confirming that the embeddings and Gen AI is working as expected. PS: Your results may differ from mine due to variations in the sampling of your training dataset compared to mine. Additionally, OpenAI's generative content can produce different outputs depending on the timing and context. Please keep this in mind. 5. Publishing the Project to Azure Return to terminal and execute the following commands. Commands for Linux or Mac: # Please change <resourcegroup_name> and <webapp_name> to your own # Create the Zip file from project zip -r openai/app.zip openai/* # Deploy the App az webapp deploy --resource-group <resourcegroup_name> --name <webapp_name> --src-path openai/app.zip --type zip # Delete the Zip file rm openai/app.zip Commands for Windows: # Please change <resourcegroup_name> and <webapp_name> to your own # Create the Zip file from project Compress-Archive -Path openai\* -DestinationPath openai\app.zip # Deploy the App az webapp deploy --resource-group <resourcegroup_name> --name <webapp_name> --src-path openai\app.zip --type zip # Delete the Zip file del openai\app.zip PS: WebJobs follow the directory structure of App_Data/jobs/triggered/<webjob_name>/. As a result, once the Web App is deployed, the WebJob is automatically deployed along with it, requiring no additional configuration. 6. Running on Azure Web App Training the Model Return to terminal and execute the following commands to invoke the WebJobs. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; curl -X POST -H "Authorization: Bearer $token" -H "Content-Type: application/json" -d '{}' "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/cal-embeddings/run?api-version=2024-04-01" Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/cal-embeddings/run?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"; "Content-type" = "application/json"} -Method POST -Body '{}' You could see the training status by execute the following commands. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; response=$(curl -s -H "Authorization: Bearer $token" "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01") ; echo "$response" | jq Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv); $response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"} -Method GET ; $response | ConvertTo-Json -Depth 10 Processing Complete And you can get the latest detail log by execute the following commands. Commands for Linux or Mac: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; history_id=$(az webapp webjob triggered log --resource-group <resourcegroup_name> --name <webapp_name> --webjob-name cal-embeddings --query "[0].id" -o tsv | sed 's|.*/history/||') ; response=$(curl -X GET -H "Authorization: Bearer $token" -H "Content-Type: application/json" "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/cal-embeddings/history/$history_id/?api-version=2024-04-01") ; log_url=$(echo "$response" | jq -r '.properties.output_url') ; curl -X GET -H "Authorization: Bearer $token" "$log_url" Commands for Windows: # Please change <subscription_id> <resourcegroup_name> and <webapp_name> to your own $token = az account get-access-token --resource https://management.azure.com --query accessToken -o tsv ; $history_id = az webapp webjob triggered log --resource-group <resourcegroup_name> --name <webapp_name> --webjob-name cal-embeddings --query "[0].id" -o tsv | ForEach-Object { ($_ -split "/history/")[-1] } ; $response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/cal-embeddings/history/$history_id/?api-version=2024-04-01" -Headers @{ Authorization = "Bearer $token" } -Method GET ; $log_url = $response.properties.output_url ; Invoke-RestMethod -Uri $log_url -Headers @{ Authorization = "Bearer $token" } -Method GET Once you see the report in the Logs, it indicates that the embeddings calculation is complete, and the Flask app is ready for predictions. You can also find the newly calculated embeddings in the File Share mounted in your local environment. Using the Model for Prediction Just like in local testing, open a bash terminal and use the following curl commands to send requests to the app: # Please change <webapp_name> to your own curl -X GET https://<webapp_name>.azurewebsites.net/api/detect?text=education As with the local environment, you should see the expected results. 7. Troubleshooting Startup Command Issue Symptom: Without any code changes and when the app was previously functioning, updating the Startup Command causes the app to stop working. The related default_docker.log shows multiple attempts to run the container without errors in a short time, but the container does not respond on port 8000 as seen in docker.log. Cause:Since Linux Web Apps actually run in containers, the final command in the Startup Command must function similarly to the CMD instruction in a Dockerfile. CMD ["/usr/sbin/sshd", "-D", "-o", "ListenAddress=0.0.0.0"] This command must ensure it runs in the foreground (i.e., not in daemon mode) and cannot exit the process unless manually interrupted. Resolution: Check the final command in the Startup Command to ensure it does not include a daemon execution mode. Alternatively, use the Web SSH interface to execute and verify these commands directly. App Becomes Unresponsive After a Period Symptom: An app that runs normally becomes unresponsive after some time. Both the front-end webpage and the Kudu page display an "Application Error," and the deployment log shows "Too many requests." Additionally, the local environment cannot connect to the associated File Share. Cause: Clicking on "diagnostic resources" in the initial error screen provides more detailed error information. In this example, the issue is caused by internal enterprise Policies or Automations (e.g., enterprise applications) that periodically or randomly scan storage account settings created by employees. If the settings are deemed non-compliant with security standards, they are automatically adjusted. For instance, the allowSharedKeyAccess parameter may be forcibly set to false, preventing both the Web App and the local development environment from connecting to the File Share under the Storage Account. Modification history for such settings can be checked via the Activity Log of the Storage Account (note that only the last 90 days of data are retained). Resolution: The proper approach is to work offline with the enterprise IT team to coordinate and request the necessary permissions. As a temporary workaround, modify the affected settings to Enable during testing periods and revert them to Disabled afterward. You can find the setting for allowSharedKeyAccess here. Note: Azure Storage Mount currently does not support access via Managed Identity. az cli command for Linux webjobs fail Symptom: Got "Operation returned an invalid status 'Unauthorized'" message from different platforms even in Azure CloudShell with latest az version Cause: After using "--debug --verbose" from the command I can see the actual error occurred on which REST API, for example, I'm using this command (az webapp webjob triggered): az webapp webjob triggered list --resource-group azure-appservice-ai --name openai-arm-app --debug --verbose Which represent that the operation has invoked under this API: /Microsoft.Web/sites/{app_name}/triggeredwebjobs (Web Apps - List Triggered Web Jobs) After I directly test that API from the official doc, I still get such the error, which means this preview feature is still under construction, and we cannot use it currently. Resolution: I found a related API endpoint via Azure Portal: /Microsoft.Web/sites/{app_name}/webjobs (Web Apps - List Web Jobs) After I directly test that API from the official doc, I can get the trigger list now. So I have modified the original command: az webapp webjob triggered list --resource-group azure-appservice-ai --name openai-arm-app To the following command (please note the differences between Linux/Mac and Windows commands). Make sure to replace <subscription_id>, <resourcegroup_name>, and <webapp_name> with your specific values. Commands for Linux or Mac: token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; response=$(curl -s -H "Authorization: Bearer $token" "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01") ; echo "$response" | jq Commands for Windows: $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv); $response = Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/webjobs?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"} -Method GET ; $response | ConvertTo-Json -Depth 10 For "run" commands, due to the same issue when invoking the problematic API, so I also modify the operation. Commands for Linux or Mac: token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; curl -X POST -H "Authorization: Bearer $token" -H "Content-Type: application/json" -d '{}' "https://management.azure.com/subscriptions/<subscription_id>/resourceGroups/<resourcegroup_name>/providers/Microsoft.Web/sites/<webapp_name>/triggeredwebjobs/cal-embeddings/run?api-version=2024-04-01" Commands for Windows: $token=$(az account get-access-token --resource https://management.azure.com --query accessToken -o tsv) ; Invoke-RestMethod -Uri "https://management.azure.com/subscriptions/029b4739-1f55-4cab-bf84-a9393f8ac8fe/resourceGroups/azure-appservice-ai/providers/Microsoft.Web/sites/openai-arm-app/triggeredwebjobs/cal-embeddings/run?api-version=2024-04-01" -Headers @{Authorization = "Bearer $token"; "Content-type" = "application/json"} -Method POST -Body '{}' Others Using Scikit-learn on Azure Web App 8. Conclusion Beyond simple embedding vector calculations, OpenAI's most notable strength is generative AI. You can provide instructions to the GPT model through natural language (as a prompt), clearly specifying the format you need in the instruction. You can then parse the returned content easily. While PaaS products are not ideal for heavy vector calculations, they are well-suited for acting as intermediaries to forward commands to generative AI. These outputs can even be used for various applications, such as patent infringement detection, plagiarism detection in research papers, or trending news analysis. I believe that in the future, we will see more similar applications on Azure Web Apps. 9. References Overview - OpenAI API News-Headlines-Dataset-For-Sarcasm-Detection Quickstart: Deploy a Python (Django, Flask, or FastAPI) web app to Azure - Azure App Service Configure a custom startup file for Python apps on Azure App Service on Linux - Python on Azure Mount Azure Storage as a local share - Azure App Service Deploy to Azure button - Azure Resource Manager Using Scikit-learn on Azure Web AppUsing Scikit-learn on Azure Web App
TOC Introduction to Scikit-learn System Architecture Architecture Focus of This Tutorial Setup Azure Resources Web App Storage Running Locally File and Directory Structure Training Models and Training Data Predicting with the Model Publishing the Project to Azure Deployment Configuration Running on Azure Web App Training the Model Using the Model for Prediction Troubleshooting Missing Environment Variables After Deployment Virtual Environment Resource Lock Issues Package Version Dependency Issues Default Binding Missing System Commands in Restricted Environments Conclusion References 1. Introduction to Scikit-learn Scikit-learn is a popular open-source Python library for machine learning, built on NumPy, SciPy, and matplotlib. It offers an efficient and easy-to-use toolkit for data analysis, data mining, and predictive modeling. Scikit-learn supports a variety of machine learning algorithms, including classification, regression, clustering, and dimensionality reduction (e.g., SVM, Random Forest, K-means). Its preprocessing utilities handle tasks like scaling, encoding, and missing data imputation. It also provides tools for model evaluation (e.g., accuracy, precision, recall) and pipeline creation, enabling users to chain preprocessing and model training into seamless workflows. 2. System Architecture Architecture Development Environment OS: Windows 11 Version: 24H2 Python Version: 3.7.3 Azure Resources App Service Plan: SKU - Premium Plan 0 V3 App Service: Platform - Linux (Python 3.9, Version 3.9.19) Storage Account: SKU - General Purpose V2 File Share: No backup plan Focus of This Tutorial This tutorial walks you through the following stages: Setting up Azure resources Running the project locally Publishing the project to Azure Running the application on Azure Troubleshooting common issues Each of the mentioned aspects has numerous corresponding tools and solutions. The relevant information for this session is listed in the table below. Local OS Windows Linux Mac V How to setup Azure resources Portal (i.e., REST api) ARM Bicep Terraform V How to deploy project to Azure VSCode CLI Azure DevOps GitHub Action V 3. Setup Azure Resources Web App We need to create the following resources or services: Manual Creation Required Resource/Service App Service Plan No Resource App Service Yes Resource Storage Account Yes Resource File Share Yes Service Go to the Azure Portal and create an App Service. Important configuration: OS: Select Linux (default if Python stack is chosen). Stack: Select Python 3.9 to avoid dependency issues. SKU: Choose at least Premium Plan to ensure enough memory for your AI workloads. Storage Create a Storage Account in the Azure Portal. Create a file share named data-and-model in the Storage Account. Mount the File Share to the App Service: Use the name data-and-model for consistency with tutorial paths. At this point, all Azure resources and services have been successfully created. Let’s take a slight detour and mount the recently created File Share to your Windows development environment. Navigate to the File Share you just created, and refer to the diagram below to copy the required command. Before copying, please ensure that the drive letter remains set to the default "Z" as the sample code in this tutorial will rely on it. Return to your development environment. Open a PowerShell terminal (do not run it as Administrator) and input the command copied in the previous step, as shown in the diagram. After executing the command, the network drive will be successfully mounted. You can open File Explorer to verify, as illustrated in the diagram. 4. Running Locally File and Directory Structure Please use VSCode to open a PowerShell terminal and enter the following commands: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai .\scikit-learn\tools\add-venv.cmd If you are using a Linux or Mac platform, use the following alternative commands instead: git clone https://github.com/theringe/azure-appservice-ai.git cd azure-appservice-ai bash ./scikit-learn/tools/add-venv.sh After completing the execution, you should see the following directory structure: File and Path Purpose scikit-learn/tools/add-venv.* The script executed in the previous step (cmd for Windows, sh for Linux/Mac) to create all Python virtual environments required for this tutorial. .venv/scikit-learn-webjob/ A virtual environment specifically used for training models. scikit-learn/webjob/requirements.txt The list of packages (with exact versions) required for the scikit-learn-webjob virtual environment. .venv/scikit-learn/ A virtual environment specifically used for the Flask application, enabling API endpoint access for querying predictions. scikit-learn/requirements.txt The list of packages (with exact versions) required for the scikit-learn virtual environment. scikit-learn/ The main folder for this tutorial. scikit-learn/tools/create-folder.* A script to create all directories required for this tutorial in the File Share, including train, model, and test. scikit-learn/tools/download-sample-training-set.* A script to download a sample training set from the UCI Machine Learning Repository, containing heart disease data, into the train directory of the File Share. scikit-learn/webjob/train_heart_disease_model.py A script for training the model. It loads the training set, applies a machine learning algorithm (Logistic Regression), and saves the trained model in the model directory of the File Share. scikit-learn/webjob/train_heart_disease_model.sh A shell script for Azure App Service web jobs. It activates the scikit-learn-webjob virtual environment and starts the train_heart_disease_model.py script. scikit-learn/webjob/train_heart_disease_model.zip A ZIP file containing the shell script for Azure web jobs. It must be recreated manually whenever train_heart_disease_model.sh is modified. Ensure it does not include any directory structure. scikit-learn/api/app.py Code for the Flask application, including routes, port configuration, input parsing, model loading, predictions, and output generation. scikit-learn/.deployment A configuration file for deploying the project to Azure using VSCode. It disables the default Oryx build process in favor of custom scripts. scikit-learn/start.sh A script executed after deployment (as specified in the Portal's startup command). It sets up the virtual environment and starts the Flask application to handle web requests. Training Models and Training Data Return to VSCode and execute the following commands (their purpose has been described earlier). .\.venv\scikit-learn-webjob\Scripts\Activate.ps1 .\scikit-learn\tools\create-folder.cmd .\scikit-learn\tools\download-sample-training-set.cmd python .\scikit-learn\webjob\train_heart_disease_model.py If you are using a Linux or Mac platform, use the following alternative commands instead: source .venv/scikit-learn-webjob/bin/activate bash ./scikit-learn/tools/create-folder.sh bash ./scikit-learn/tools/download-sample-training-set.sh python ./scikit-learn/webjob/train_heart_disease_model.py After execution, the File Share will now include the following directories and files. Let’s take a brief detour to examine the structure of the training data downloaded from the public dataset website. The right side of the figure describes the meaning of each column in the dataset, while the left side shows the actual training data (after preprocessing). This is a predictive model that uses an individual’s physiological characteristics to determine the likelihood of having heart disease. Columns 1-13 represent various physiological features and background information of the patients, while Column 14 (originally Column 58) is the label indicating whether the individual has heart disease. The supervised learning process involves using a large dataset containing both features and labels. Machine learning algorithms (such as neural networks, SVMs, or in this case, logistic regression) identify the key features and their ranges that differentiate between labels. The trained model is then saved and can be used in services to predict outcomes in real time by simply providing the necessary features. Predicting with the Model Return to VSCode and execute the following commands. First, deactivate the virtual environment used for training the model, then activate the virtual environment for the Flask application, and finally, start the Flask app. Commands for Windows: deactivate .\.venv\scikit-learn\Scripts\Activate.ps1 python .\scikit-learn\api\app.py Commands for Linux or Mac: deactivate source .venv/scikit-learn/bin/activate python ./scikit-learn/api/app.py When you see a screen similar to the following, it means the server has started successfully. Press Ctrl+C to stop the server if needed. Before conducting the actual test, let’s construct some sample human feature data: [63, 1, 3, 145, 233, 1, 0, 150, 0, 2.3, 0, 0, 1] [63, 1, 3, 305, 233, 1, 0, 150, 0, 2.3, 0, 0, 1] Referring to the feature description table from earlier, we can see that the only modified field is Column 4 ("Resting Blood Pressure"), with the second sample having an abnormally high value. (Note: Normal resting blood pressure ranges are typically 90–139 mmHg.) Next, open a PowerShell terminal and use the following curl commands to send requests to the app: curl -X GET http://127.0.0.1:8000/api/detect -H "Content-Type: application/json" -d '{"info": [63, 1, 3, 145, 233, 1, 0, 150, 0, 2.3, 0, 0, 1]}' curl -X GET http://127.0.0.1:8000/api/detect -H "Content-Type: application/json" -d '{"info": [63, 1, 3, 305, 233, 1, 0, 150, 0, 2.3, 0, 0, 1]}' You should see the prediction results, confirming that the trained model is working as expected. 5. Publishing the Project to Azure Deployment In the VSCode interface, right-click on the target App Service where you plan to deploy your project. Manually select the local project folder named scikit-learn as the deployment source, as shown in the image below. Configuration After deployment, the App Service will not be functional yet and will still display the default welcome page. This is because the App Service has not been configured to build the virtual environment and start the Flask application. To complete the setup, go to the Azure Portal and navigate to the App Service. The following steps are critical, and their execution order must be correct. To avoid delays, it’s recommended to open two browser tabs beforehand, complete the settings in each, and apply them in sequence. Refer to the following two images for guidance. You need to do the following: Set the Startup Command: Specify the path to the script you deployed bash /home/site/wwwroot/start.sh Set Two App Settings: WEBSITES_CONTAINER_START_TIME_LIMIT=600 The value is in seconds, ensuring the Startup Command can continue execution beyond the default timeout of 230 seconds. This tutorial’s Startup Command typically takes around 300 seconds, so setting it to 600 seconds provides a safety margin and accommodates future project expansion (e.g., adding more packages). WEBSITES_ENABLE_APP_SERVICE_STORAGE=1 This setting is required to enable the App Service storage feature, which is necessary for using web jobs (e.g., for model training). Step-by-Step Process: Before clicking Continue, switch to the next browser tab and set up all the app settings. In the second tab, apply all app settings, then switch back to the first tab. Click Continue in the first tab and wait for several seconds for the operation to complete. Once completed, switch to the second tab and click Continue within 5 seconds. Ensure to click Continue promptly within 5 seconds after the previous step to finish all settings. After completing the configuration, wait for about 10 minutes for the settings to take effect. Then, navigate to theWebJobs section in the Azure Portal and upload the ZIP file mentioned in the earlier sections. Set its trigger type to Manual. At this point, the entire deployment process is complete. For future code updates, you only need to redeploy from VSCode; there is no need to reconfigure settings in the Azure Portal. 6. Running on Azure Web App Training the Model Go to the Azure Portal, locate your App Service, and navigate to the WebJobs section. Click on Start to initiate the job and wait for the results. During this process, you may need to manually refresh the page to check the status of the job execution. Refer to the image below for guidance. Once you see the model report in the Logs, it indicates that the model training is complete, and the Flask app is ready for predictions. You can also find the newly trained model in the File Share mounted in your local environment. Using the Model for Prediction Just like in local testing, open a PowerShell terminal and use the following curl commands to send requests to the app: # Note: Replace both instances of scikit-learn-portal-app with the name of your web app. curl -X GET https://scikit-learn-portal-app.azurewebsites.net/api/detect -H "Content-Type: application/json" -d '{"info": [63, 1, 3, 145, 233, 1, 0, 150, 0, 2.3, 0, 0, 1]}' curl -X GET https://scikit-learn-portal-app.azurewebsites.net/api/detect -H "Content-Type: application/json" -d '{"info": [63, 1, 3, 305, 233, 1, 0, 150, 0, 2.3, 0, 0, 1]}' As with the local environment, you should see the expected results. 7. Troubleshooting Missing Environment Variables After Deployment Symptom: Even after setting values in App Settings (e.g., WEBSITES_CONTAINER_START_TIME_LIMIT), they do not take effect. Cause: App Settings (e.g., WEBSITES_CONTAINER_START_TIME_LIMIT, WEBSITES_ENABLE_APP_SERVICE_STORAGE) are reset after updating the startup command. Resolution: Use Azure CLI or the Azure Portal to reapply the App Settings after deployment. Alternatively, set the startup command first, and then apply app settings. Virtual Environment Resource Lock Issues Symptom: The app fails to redeploy, even though no configuration or code changes were made. Cause: The virtual environment folder cannot be deleted due to active resource locks from the previous process. Files or processes from the previous virtual environment session remain locked. Resolution: Deactivate processes before deletion and use unique epoch-based folder names to avoid conflicts. Refer to scikit-learn/start.sh in this tutorial for implementation. Package Version Dependency Issues Symptom: Conflicts occur between package versions specified in requirements.txt and the versions required by the Python environment. This results in errors during installation or runtime. Cause: Azure deployment environments enforce specific versions of Python and pre-installed packages, leading to mismatches when older or newer versions are explicitly defined. Additionally, the read-only file system in Azure App Service prevents modifying global packages like typing-extensions. Resolution: Pin compatible dependency versions. For example, follow the instructions for installing scikit-learn from the scikit-learn 1.5.2 documentation. Refer to scikit-learn/requirements.txtin this tutorial. Default Binding Symptom: Despite setting the WEBSITES_PORT parameter in App Settings to match the port Flask listens on (e.g., Flask's default 5000), the deployment still fails. Cause: The Flask framework's default settings are not overridden to bind to 0.0.0.0 or the required port. Resolution: Explicitly bind Flask to 0.0.0.0:8000 in app.py . To avoid additional issues, it’s recommended to use the Azure Python Linux Web App's default port (8000), as this minimizes the need for extra configuration. Missing System Commands in Restricted Environments Symptom: In the WebJobs log, an error is logged stating that the ls command is missing. Cause: This typically occurs in minimal environments, such as Azure App Services, containers, or highly restricted shells. Resolution: Use predefined paths or variables in the script instead of relying on system commands. Refer to scikit-learn/webjob/train_heart_disease_model.shin this tutorial for an example of handling such cases. 8. Conclusion Azure App Service, while being a PaaS product with less flexibility compared to a VM, still offers several powerful features that allow us to fully leverage the benefits of AI frameworks. For example, the resource-intensive model training phase can be offloaded to a high-performance local machine. This approach enables the App Service to focus solely on loading models and serving predictions. Additionally, if the training dataset is frequently updated, we can configure WebJobs with scheduled triggers to retrain the model periodically, ensuring the prediction service always uses the latest version. These capabilities make Azure App Service well-suited for most business scenarios. 9. References Scikit-learn Documentation UCI Machine Learning Repository Azure App Service DocumentationBuild AI faster and run with confidence
Intelligence is the new baseline for modern apps. We’re seeing the energy and excitement from AI coming to life in apps being built and modernized today. AI is now the critical component of nearly every application and business strategy. And most importantly, we’ve reached an inflection point where organizations are moving from widespread experimentation to full production. In fact, we just completed a global survey of more than 1,500 developers and IT influencers: 60% of respondents have AI apps under development that will go to production over the next year. What’s even more exciting is the evolution of what organizations are looking to achieve. Agents and intelligent assistants continue to shape customer service, make customer experiences more personal and help employees make faster, more accurate decisions. And agents that unlock organizational knowledge are increasingly vital. But more than ever, AI is reinventing core business processes. From long-established business to startups (where our business has increased 200%), this holds incredible promise to drive revenue and growth as well as competitive differentiation. AI is both an inspiration for how the next generation of apps will be defined and an accelerator for how we will build and deliver on that promise. This is the tension that we see with every business; intense pressure to move fast, to take advantage of incredible innovation with speed, while delivering every app securely with the performance and scale that meets the unique needs of AI apps. Developers are on the front lines, responsible for moving quickly from idea to code to cloud while delivering secure, private and trusted AI services that meet ROI and cost requirements. This combination of building fast and running with confidence is core to Microsoft’s strategy. So, as we kick off Microsoft Ignite 2024 today, we’re bringing a slate of new and enhanced innovation that makes this transition to production faster and easier for all. With new innovations across the Copilot and AI stack, new integrations that bring together services across our AI platform and developer tools and an expanding set of partnerships across the AI toolchain, there’s a ton of great innovation at Ignite. Let’s look at just a few… GitHub Copilot GitHub Copilot is the most widely used AI pair-programming tool in the world, with millions of developers using it daily. We’re seeing developers code up to 55% faster through real-time code suggestions and solutions, and elevating quality, generating cleaner, more resilient, code that is easier to maintain. This week we’re showing how Copilot is evolving to drive efficiency across the entire application lifecycle. Developers in VS Code now have the flexibility to select from an array of industry-leading models, including OpenAI’s GPT-4o and Anthropic Claude Sonnet 3.5. And they have the power to use natural language chat to implement complex code changes across multiple files. GitHub Copilot upgrade assistant for Java Now the power of Copilot can help with existing apps as well. Keeping Java apps up to date can be a time-consuming task. GitHub Copilot upgrade assistant for Java uses AI to simplify Java applications upgrades with autonomous agents. The process is transparent, keeping a human in the loop, while your environment actively learns from your context and adjustments, improving accuracy for future upgrades. GitHub Copilot for Azure GitHub Copilot for Azure streamlines path from code to production on Azure for every developer, even those new to Azure. Through Copilot, you can use natural language to learn about your Azure services and resources, find sample applications and templates and quickly deploy to Azure while supporting your enterprise standards and guidelines. Once in production, GitHub Copilot for Azure helps you troubleshoot and resolve application issues stay on top of costs. Copilot knows the full context of you as a developer and your systems to make every recommendation tailored to your unique needs. Available now in public preview, it does it all from the tools you already use helping you minimize interruptions and stay focused. Azure AI Foundry New at Ignite, Azure AI Foundry brings together an end-to-end AI platform across models, tooling, safety, and monitoring to help you efficiently and cost-effectively design and scale your AI applications. By integrating with popular developer tools like GitHub, Visual Studio, and Copilot Studio, Azure AI Foundry opens up this full portfolio of services for developers, giving them access to the best, most advanced models in the world along with tools for building agents on Azure and a unified toolchain to access AI services through one interface. Azure AI Foundry is a key offering enabling easy integration of Azure AI capabilities into your applications. AI Template Gallery AI App Template Gallery is a new resource designed to help you build and deploy AI applications in a matter of minutes, with the flexibility to use the programming language, framework and architecture of your choice. The gallery offers more than 25 curated, ready-to-use application templates, creating a clear path to kickstart your AI projects with confidence and efficiency. And developers can easily discover and access each of them through GitHub Copilot for Azure, further simplifying access through your preferred developer tool. Azure Native Integrations Azure Native Integrations gives developers access to a curated set of ISV services available directly in the Azure portal, SDK, and CLI. This means that developers have the flexibility to work with their preferred vendors across the AI toolchain and other common solution areas, with simplified single sign-on and management, while staying in Azure. Joining our portfolio of integrated services arePinecone,Weights & Biases,Arize, and LambdaTest all now available in private preview. Neon,Pure Storage Cloud for Azure VMware Solution (AVS), and Dell APEX File Storage will also be available soon as part of Azure Native Integrations. Azure Container Apps with Serverless GPUs Azure Container Apps now supports Serverless GPUs in public preview, enabling effortless scaling and flexibility for real-time custom model inferencing and other machine learning tasks. Serverless GPUs enable you to seamlessly run your AI workloads on-demand, accessing powerful NVIDIA accelerated computing resources, with automatic scaling, optimized cold start and per-second billing without the need for dedicated infrastructure management. Azure Essentialsfor AI Adoption We also recognize great technology is only part of your success. Microsoft has published design patterns, baseline reference architectures, application landing zones, and a variety of Azure service guides for Azure OpenAI workloads along with FinOps guidance for AI. This week, we are excited to announce new AI specific guidance in the Cloud Adoption Framework and the Azure Well-Architected Framework to help adopt AI at scale while meeting requirements for reliability, security, operations and cost.
