Create and Deploy Linux WebJobs on Azure App Service

Introduction

WebJobs are a feature of Azure App Service that allows you to run background tasks or scripts in your App Service. They are particularly useful for executing jobs such as scheduled tasks, data processing, or background maintenance tasks. WebJobs can run continuously, on-demand, or on a scheduled basis, and they can be written in various programming languages.

The goal of this blog post is to guide you through the process of setting up and deploying a WebJob on an Azure App Service running Linux. We will walk you through the necessary steps to create and deploy a WebJob, configure its settings, and manage it effectively. By the end of this blog, you will have a working WebJob integrated into your Azure App Service, capable of performing background tasks or processing jobs as needed.

Prerequisites

• Azure Subscription: An active Azure subscription is required to create and manage Azure resources.
• Linux App Service: An App Service is necessary for deploying your WebJob. For your WebJob to run, the App Service stack must match the WebJob. For example, a Python-based WebJob requires a Python App Service stack.

• In the Environment variables tab of the App Service, add 'WEBSITES_ENABLE_APP_SERVICE_STORAGE' app setting and set its value to 'true'.

• Integrated Development Environment (IDE): An IDE or text editor of your choice for developing your WebJob.
• File Compression Tool: For packaging WebJob files into a ZIP archive needed for deployment.

Types of WebJobs

When selecting the type of WebJob to use in Azure App Service, it is important to consider the nature of the task you need to perform. Continuous WebJobs are designed to run continuously in the background, making them suitable for tasks that need to be perpetually active. These WebJobs are ideal for ongoing operations such as background processing, monitoring, or maintaining a steady stream of work. They stay active as long as the App Service is operational and will automatically restart if they encounter issues or stop unexpectedly.

In contrast, Triggered WebJobs are executed based on specific triggers or schedules, which makes them well-suited for tasks that need to run at predefined times or in response to events. For example, you might use a Triggered WebJob for scheduled data updates, batch processing, or responding to specific actions like the arrival of new data. The choice between a Continuous or Triggered WebJob will depend on whether your task requires constant operation or is more event-driven, allowing you to tailor the WebJob type to your specific needs.

Creating a WebJob

In this section, we will create a Python WebJob that reads data from a CSV file and update. This example will help you understand the basic structure of a WebJob. Below is redacted version of my webjob.py

import pandas as pd
import os, datetime, logging

# Configure logging
logging.basicConfig(
    filename='/home/LogFiles/webjob.log',
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)


def insert_record(file_path):
    print('--- Inserting Record ---')

    try:
        # Determine the next value
        value = get_next_value(file_path)
        
        # Create a new DataFrame with the new record
        new_record = pd.DataFrame({'Timestamp': [datetime.now()], 'Value': [value]})
        
        if os.path.exists(file_path):
            # Load the existing CSV file
            df = pd.read_csv(file_path)
            logging.info('Loaded existing CSV file successfully.')
            # Append the new record
            df = pd.concat([df, new_record], ignore_index=True)
        else:
            # If the file does not exist, create a new DataFrame
            df = new_record
            logging.info('CSV  file not found. Created new file with columns: Timestamp, Value.')
        
        # Save the updated DataFrame back to Excel
        df.to_csv(file_path, index=False)
        logging.info('Saved updated CSV file successfully with value: %d', value)
    
    except Exception as e:
        logging.error('Error processing record: %s', e)

if __name__ == "__main__":
    print('--- WebJob Execution Started ---')
    file_path = '/home/site/wwwroot/data.csv'
    insert_record(file_path)
    print('--- WebJob Execution Ended ---')

Deploying a WebJob

Deploying a WebJob on App Service Linux involves creating a shell script, packaging it, and then deploying it via the Azure Portal.

Create a shell script (script.sh) with the command to start the WebJob script. For Python WebJob, I used below script to install dependencies and execute the script:

#!/bin/bash
/opt/python/3.10.14/bin/pip3 install pandas
/opt/python/3.10.14/bin/python3.10 webjob.py

Zip the webjob.py and script.sh file together. Ensure that the files are present at the root of the zipped archive.

Webjob Deployment requires Basic Authentication. Enable SCM Basic Auth Publishing Credentials from the Configuration Blade of the App Service.

Now Navigate to the WebJob Balde of the App Service to Add and Configure the WebJob. Click the “Add” button to create a new WebJob. In the configuration panel:

• Upload the WebJob File: Choose the webjob.zip file you prepared.
• Set WebJob Type: Select the WebJob type—either “Continuous” for a constantly running job or “Triggered” for a job that runs on-demand or on a schedule.
• Configure Schedule (if using a Triggered WebJob): Specify the cron expression for scheduling the job. For example, use 0 */5 * * * to run every 5 minutes.
• Click “OK” or “Save” to upload the WebJob and apply the configuration.

Testing & Logging

For a Triggered WebJob, you can manually run it by selecting the WebJob and clicking “Run.”

On App Service Linux, WebJobs are deployed under /tmp/jobs/. To check the logs, you can either view it by clicking the Logs button. You can navigate to /home/data/jobs/triggered/<webjob_name>. Here you will find a directory corresponding to each execution of the WebJob. Within each directory, you will find the ‘status’ and ‘output_log.txt. files.

The status file indicates the schedule set for the WebJob along with the result of the WebJob execution. You can also find the start and end time of the execution.

The output_log.txt file contains general information and status updates during execution. It will also contain execution logs.

By reviewing these logs, you can confirm the WebJob's operational status, diagnose issues, and understand its execution behaviour.

Conclusion

WebJobs on Azure App Service Linux offers a powerful way to run background tasks and automate processes within your applications. By following the outlined steps you can seamlessly integrate WebJobs into your Linux-based App Service. With a robust understanding of WebJobs, you can leverage this feature to enhance your application's functionality and maintain high performance across your services.

Additional Links

Run background tasks with WebJobs - Azure App Service | Microsoft Learn

Github Repository: App Service Linux WebJob (github.com)