A few weeks ago, we announced the Public Preview Refresh release of Logic Apps hybrid deployment model that allows customers to run Logic Apps workloads on a customer managed infrastructure.

This model provides the flexibility to execute workflows, either on-premises or in any cloud environment, thereby offering enhanced control over the operation of logic apps. By utilizing customer-managed infrastructure, organizations can adhere to regulatory compliance requirements and optimize performance according to their specific needs.

As customers consider leveraging hybrid environments, understanding the performance of logic apps under various configurations and scenarios becomes critical.

This document offers an in-depth performance evaluation of Azure Logic Apps within a hybrid deployment framework. It examines, several key factors such as CPU and memory allocation and scaling mechanisms, providing valuable insights aimed at maximizing the application’s efficiency and performance.

Achieving Optimal Logic Apps Performance in Hybrid Deployments

In this section, we will explore the key aspects that affect Logic Apps performance when deployed in a hybrid environment. Factors such as the underlying infrastructure of the Kubernetes environment, SQL configuration and scaling configuration can significantly impact the efficiency of workflows and the overall performance of the applications.

The following blog entry provides details of the scaling mechanism of Hybrid deployment model - Scaling mechanism in hybrid deployment model for Azure Logic Apps Standard | Microsoft Community Hub

Configure Container Resource allocation:

When you create a Logic App, a default value of 0.5 vCPU and 1GiB of memory would be allocated. From the Azure Portal, you can modify  this allocation from the Container blade. - Create Standard logic app workflows for hybrid deployment - Azure Logic Apps | Microsoft Learn

Currently, the maximum allocation is set to 2vCPU and 4 GiB memory per app. In the future, there would be a provision made to choose higher allocations.

For CPU intense/memory intense processing like custom code executions, select a higher value for these parameters. In the next section, we will be comparing the performance with different values of the CPU and memory allocation.

This allocation would impact the billing calculation of the Logic App resource. Refer vCPU calculation for more details on the billing impact.

Optimize the node count and size in the Kubernetes cluster.

Kubernetes runs application workloads by placing containers into Pods to run on Nodes. A node may be a virtual or physical machine, depending on the cluster.

A node pool is a group of nodes that share the same configuration (CPU, Memory, Networking, OS, maximum number of pods, etc.). You can choose the capacity (cores and memory), minimum node count and maximum node count for each node pool of the Kubernetes cluster. We recommend allocating a higher capacity for processing CPU intense, or memory intense applications

Configure Scale rule settings: 

For a Logic App resource, we recommend you configure the maximum and minimum replicas which could be scaled out when a scale event occurs. A higher value for the max replicas helps in sudden spikes in the number of application requests.

The interval with which the scaler checks for the scaling event and the cooldown period for the scaling event can also be configured from the Scale blade of Logic Apps resource. These parameters impact the scaling pattern.

Optimize the SQL server configuration:

The hybrid deployment model uses Microsoft SQL for runtime storage. As such, there are lot of SQL operations performed throughout the execution of the workflow and SQL capacity has a significant impact on the performance of the app.

Microsoft SQL server could either be a SQL server on Windows, or an Azure SQL database.

Few recommendations on the SQL configuration for better performance:

1. If you are using, Azure SQL database, run it on a SQL elastic pool.
2. If you are using SQL server on Windows, run with at least 4vCPU configuration.
3. Scale out the SQL server once the CPU usage of the SQL server hits 60-70% of the total available CPU.

Performance analysis:

 For this performance analysis exercise, we have used a typical enterprise integration scenario which includes the below components.

1. Data transformation: XSLT transformation, validation, and XML parsing actions
2. Data routing: File system connector for storing the transformed content in a file share.
3. Message queuing: RabbitMQ connector for sending the transformation result to Rabbit MQ queue endpoint.
4. Control operations: For-each loop for looping through multiple records, condition execution, Scope, and error handling blocks.
5. Request response: The XML data transmitted via HTTP request, and the status returned as a response.

Summary:

For these tests, we used the following environment settings:

Kubernetes cluster: AKS cluster with Standard D2sV3 (2vCPU, 8GiBmemory)

Max replicas: 20

Cooldown period: 300 seconds

Polling interval: 30

With the above environment and settings, we have performed multiple application tests with different configuration of SQL server, resource allocation and test durations using Azure load testing tool.

In the following table, we have summarized the response time, throughput, and the total vCPU consumption for each of these configurations. You can check each scenario for detailed information.

 

Scenario 1: SQL general purpose V2 with 1vCPU and 2 GiB Memory – 10 minutes test with 50 users

In this scenario, we conducted a load test for 10 minutes with 50 users with the Logic App configuration of: 1 vCPU and 2 GiB Memory and Azure SQL database running on General purpose V2 plan.

There were 503 requests with multiple records in each payload and it achieved the 68.62 seconds as the 90^th percentile response time and a throughput of 0.84 requests per second.

 

 

Scaling:

The Kubernetes nodes scaled out to 12 nodes and in total 3.42 vCPUs used by the app for the test duration.

 

SQL Metrics:

The CPU usage of the SQL server reached 90% of CPU usage quite early and stayed above 90% for the remaining duration of the test. From our backend telemetry as well, we observed that the actions executions were faster, but there was latency between the actions, which indicates SQL bottlenecks.

Scenario 2: SQL elastic pool, with 1vCPU and 2 GiB memory- 10 minutes test with 50 users

In this scenario, we conducted a load test for 10 minutes with 50 users with the Logic App configuration of: 1 vCPU and 2 GiB Memory and Azure SQL database running on a SQL elastic pool with 4000 DTU.

There were 1004 requests with multiple records in each payload and it achieved the 40.74 seconds as the 90^th percentile response time and a throughput of 1.65 requests per second.

 Scaling:

The Kubernetes nodes scaled out to 15 nodes and in total 3 vCPUs used by the app for the test duration.

SQL Metrics:

The SQL server’s CPU utilization peaked to 2% of the elastic pool.

Scenario 3: SQL elastic pool, with 2vCPU and 4 GiB memory- 10 minutes test with 50 users

In this scenario, we conducted a load test for 10 minutes with 50 users with the Logic App configuration of 2 vCPU and 4 GiB Memory and Azure SQL database running on a SQL elastic pool with 4000 DTU.

There were 997 requests with multiple records in each payload and it achieved the 40.63 seconds as the 90^th percentile response time and a throughput of 1.66 requests per second.

Scaling:

The Kubernetes nodes scaled out to 21 nodes and in total 4 vCPUs used by the app for the test duration.

 

SQL Metrics:

The SQL server’s CPU utilization peaked to 5% of the elastic pool.

 

Scenario 4: SQL elastic pool, with 2vCPU and 4 GiB memory- 30 minutes test with 50 users

In this scenario, we conducted a load test for 30 minutes with 50 users with the Logic App configuration of: 2 vCPU and 4 GiB Memory and Azure SQL database running on a SQL elastic pool with 4000 DTU.

There were 3421 requests with multiple records in each payload and it achieved the 26.67 seconds as the 90^th percentile response time and a throughput of 1.90 requests per second.

 

Scaling:

The Kubernetes nodes scaled out to 20 nodes and in total 18.6 vCPUs used by the app for the test duration.

 

 

SQL Metrics:

The SQL server’s CPU utilization peaked to 4.7% of the elastic pool.

Scenario 5: SQL Elastic pool, with 0.5vCPU and 1 GiB memory- 30 minutes test with 50 users

In this scenario, we have conducted a load test for 30 minutes with 50 users with the Logic App configuration of 0.5 vCPU and 1 GiB Memory and Azure SQL database running on a SQL elastic pool with 4000 DTU.

There were 3055 requests with multiple records in each payload and it achieved the 31.38 seconds as the 90^th percentile response time and a throughput of 1.70 requests per second.

 

Scaling:

The Kubernetes nodes scaled out to 18 nodes and in total 12.4 vCPUs used by the app for the test duration.

 

SQL Metrics:

The SQL server’s CPU utilization peaked to 8.6% of the elastic pool CPU. 

Scenario 6: SQL 2022 Enterprise Gen2 on Windows 2022 on Standard D4s v3 image, with 0.5vCPU and 1 GiB memory- 30 minutes test with 50 users

In this scenario, we conducted a load test for 30 minutes with 50 users with the Logic App configuration of: 0.5 vCPU and 1 GiB Memory and Azure SQL database running on an on-premises SQL 2022 Enterprise Gen2 version running on a Windows 2022 OS with Standard D4s v3 image (4 vCPU and 16GIB memory)

There were 4105 requests with multiple records in each payload and it achieved the 27.15 seconds as the 90^th percentile response time and a throughput of 2.28 requests per second.

Scaling:

The Kubernetes nodes scaled out to 8 nodes and in total 10 vCPUs used by the app for the test duration. 

SQL metrics:

The CPU usage of the SQL server went above 90% after few minutes and there was latency on few runs.

Findings and recommendations:

The following are the findings and recommendations for this performance exercise. Consider that this load test was conducted using unique conditions. If you conduct a similar test, the results and findings might vary, depending on factors such as workflow complexity, configuration, resource allocation and network configuration.

1. The KEDA scaler performs the scale-out and scale-in operations faster, as such, while the total vCPU usage remains quite low, though the nodes scaled out in the range of 1-20 nodes.
2. The SQL configuration plays a crucial role in reducing the latency between the action executions. For a satisfactory load test, we recommend starting with at least 4vCPU configuration on SQL server and scale out once CPU usage of the SQL server hits 60-70% of the total available CPU.
3. For critical applications, we recommend having a dedicated SQL database for better performance.
4. Increasing the dedicated vCPU allocation of the Logic App resource is helpful for the SAP connector, Rules Engine, .NET Framework based custom code operations and for the applications with many complex workflows.
5. As a general recommendation, regularly monitor performance metrics and adjust configurations to meet evolving requirements and follow the coding best practices of Logic Apps standard. Consider reviewing the following article, for recommendations to optimize your Azure Logic Apps workloads: https://techcommunity.microsoft.com/blog/integrationsonazureblog/logic-apps-standard-hosting--performance-tips/3956971