Function App - Service Bus triggered throughput decrease with message session
Published Mar 01 2023 07:31 PM 3,399 Views
Microsoft

Background:

There is a symptom with low throughput in Service Bus triggered function due to Service Bus's message session enabled. This blog we're going to do a lab to demonstrate the cause and effect of this situation.

 

Common causes:

According to session features description (Azure Service Bus message sessions - Azure Service Bus | Microsoft Learn), since a session receiver is created by a client accepting a session. When the session is accepted and held by a client, the client holds an exclusive lock on all messages with that session's session ID in the queue or subscription. It will also hold exclusive locks on all messages with the session ID that will arrive later.

James_Yu320_0-1677724758434.gif

Hence, if all the messages are with same session id, there is only one receiver can consume these messages one by one in ordered, so although the Function App has been scaled to multiple instances but just only 1 instance can serve all invocations and cause the low throughput symptom of Service Bus triggered function.

 

Prerequisites:

Before lab starting, please read following documents for basic concept.

1.Azure Service Bus trigger for Azure Functions | Microsoft Learn

2.Azure Service Bus message sessions - Azure Service Bus | Microsoft Learn

 

Above documents, we will learn that the Service Bus triggered function is to react/respond to messages from Service Bus queue or topic. Service Bus includes Message sessions feature that create a first-in, first-out (FIFO) guarantee in Service Bus. In other words, Message sessions enable ordered handling of unbounded sequences of related messages and guaranteeing ordered delivery.

James_Yu320_1-1677230294975.png

 

Setup environment:

1.Create Service Bus namespace and a queue with sessions enabled.

Use the Azure portal to create a Service Bus queue - Azure Service Bus | Microsoft Learn

James_Yu320_2-1677230294979.png

 

After queue created, we will see the Sessions filed is Enabled.

James_Yu320_3-1677230294981.png

 

 

2.Create a Service Bus sender to send message session into queue.

Please refer to this document for C# sample code. (azure-sdk-for-net/Sample01_SendReceive.md at main · Azure/azure-sdk-for-net (github.com)).

Code snippet sample:

James_Yu320_4-1677230294982.png

 

3.Create Service Bus triggered function as a consumer. (C# Script as example)

Azure Service Bus trigger for Azure Functions | Microsoft Learn

- run.csx

James_Yu320_5-1677230294983.png

 

- function.json (with isSessionsEnabled = true)

James_Yu320_6-1677230294984.png

 

4.Set Function App scale out to 5 instances

James_Yu320_7-1677230294986.png

 

Lab - Testing Result:

We will observe the different throughput and active instance count of Function App between Test case1 and Test case2 below.

1.Test case1: Sending messages with same Session ID to Service Bus queue, then Service Bus triggered function consumes the messages.

(1 Service Bus sender ; 1 service bus triggered function)

 

2.Test case2: Sending messages with different Session ID to Service Bus queue, then Service Bus triggered function consumes the messages.

(1 Service Bus sender ; 1 service bus triggered function)

 

From Service Bus aspect, we observed the following metrics of Service Bus. The volume of incoming message was same between test case1 and test case2. For the count of completed messages on queue, the test case1 was with high-volume throughput than test case2.

  • Metric: Incoming Messages.

    • Test case1/Test case2: Both throughput volume were 290 messages per min.

James_Yu320_8-1677230294987.png

 

  • Metric: Completed Messages.

    • Test case1: the throughput volume was 150 messages per min.
    • Test case2: the throughput volume was 290 messages per min.

James_Yu320_9-1677230294988.png

 

 

From Function App aspect, we observed the following Application Insight metrics. For the count of Service Bus triggered function - Successes, the test case1 was with high-volume throughput than test case2.

  • Metric: Service Bus triggered function - Successes by Function App host:

    • Test case1: the throughput volume was 150 messages per min.
    • Test case2: the throughput volume was 290 messages per min.

James_Yu320_10-1677230294989.png

 

  • Metric: Service Bus triggered function - Successes by Function App each instance:
    • Test case1 only had 1 instance to process all messages with same session id.
    • Test case2 had maximum 5 instances (based on scale out setting) to evenly process all messages with different session id.

James_Yu320_11-1677230294990.png

  • Application Insight log query to observe the executed invocation count of each instance.
    • traces
      | where message contains 'Executed'
      | summarize count() by cloud_RoleInstance
    • Test case1:
      There was only 1 instance to process messages.
      James_Yu320_12-1677230294992.png
    • Test case2:
      There were 5 instances evenly to process messages.
      James_Yu320_13-1677230294994.png
  • Application Insight log query to observe the message session of each invocation.
    • traces
      | extend invocationid = tostring(customDimensions.InvocationId)
      | where message contains 'Executed'
      | join (traces| extend invocationid = tostring(customDimensions.InvocationId)
      | extend sessionid = customDimensions.prop__SessionId
      | where isnotnull(sessionid) ) on invocationid
      | order by timestamp asc
    • Test case1: The message of each invocation was with same session id.
      James_Yu320_2-1677249040132.png
    • Test case2: The message of each invocation was with unique session id.
      James_Yu320_3-1677249040134.png

 

According to result above, since a session receiver is created by a client accepting a session. When the session is accepted and held by a client, the client holds an exclusive lock on all messages with that session's session ID in the queue or subscription. It will also hold exclusive locks on all messages with the session ID that will arrive later. (Azure Service Bus message sessions - Azure Service Bus | Microsoft Learn)

James_Yu320_14-1677230294994.gif

So, if all the messages are with same session id, there is only one receiver can consume these messages one by one in ordered. In other words, although this Function App has been scaled to 5 instances but just only 1 instance can serve all invocations.

 

Conclusion:

If we use the Service Bus triggered function to consume the message session that would affect several dimensions such as below:

1. The throughput of Function App.

2. The active instances of Function App.

3. A distribution of invocations across multiple instances.

 

Meanwhile, the following situations that you could consider optimizing the performance such as throughput of Service Bus triggered function.

Based on your business requirement,

1.If same session id on each message is required, you could scale up to get more resources in Function App to increase the throughput.

2.If there is no need for #1 situation, you could scale out to multiple instances in Function App to increase the throughput.

 

Hope this article is helpful for you and thank you for reading.

 

Reference:

1.Azure Service Bus trigger for Azure Functions. (Azure Service Bus trigger for Azure Functions | Microsoft Learn)

2.Azure Service Bus message sessions. (Azure Service Bus message sessions - Azure Service Bus | Microsoft Learn)

3.How to create Service Bus queue in Portal. (Use the Azure portal to create a Service Bus queue - Azure Service Bus | Microsoft Learn)

4.Sending and receiving messages from Service Bus queue (.NET SDK sample code). (azure-sdk-for-net/Sample01_SendReceive.md at main · Azure/azure-sdk-for-net (github.com))

1 Comment
Co-Authors
Version history
Last update:
‎Mar 02 2023 09:33 AM
Updated by: