Are you not getting the performance levels that you want when using Foreach Loops in Logic Apps?
The foreach loop with variables is slow by design, because each time the array variable is updated there is a round trip to storage and there is contention as well. “Foreach” scope is intended for running scenarios that require durable pan-out and pan-in and not intended for data transformation.
The foreach loop is not intended to be used for data transformation or heavy loads. Assigning variables in foreach loops can become a heavy scenario if the iterations are high, this is because the foreach persists the data information in storage and the round trip adds up to the total time of the process. A better solution is to use inline code for this purpose which will give you much better results.
Before we start with our simple solution, let's explore some other options that we have to accomplish the same:
- Select Action:
- Using Select Action prior to the foreach loop to eliminate assigning variables inside the loop.
- Perform operations on data - Azure Logic Apps | Microsoft Docs
- Filter Action:
- Using Filter Array to reduce the number of loops prior to the foreach.
- Perform operations on data - Azure Logic Apps | Microsoft Docs
- Use Logic Apps Standard in Stateless mode
- Will give you better performance of around 15-30% in foreach loops
- Single-tenant versus multi-tenant Azure Logic Apps - Azure Logic Apps | Microsoft Docs
- De-batching to another Logic App
- For example: You will have your first Logic App that generates the array that you want to loop through, then create another Logic App that accepts an HTTP request as a trigger. On the second Logic App, under settings enable the Split-On function. Now put your 'work' in the second Logic App, then go back to the first and make the last step to send the array to the second Logic App. When using this pattern the first Logic App will initialize a run for each item in the array, they will all run concurrently, rather than limited to the concurrency of the for each loop. Here is some more detail in the documentation about this function. Schema reference for trigger and action types - Azure Logic Apps | Microsoft Docs
Now the inline script solution:
Instead of using Append to array variable, you can remove the loop and replace it with inline code.
Reference: Add and run code snippets by using inline code - Azure Logic Apps | Microsoft Docs
Using inline code in Logic Apps Consumption requires an Integration Account, while in Logic App Standard it doesn't.
Single-tenant versus multi-tenant Azure Logic Apps - Azure Logic Apps | Microsoft Docs
The script below, very simple, and you can alter it based on your scenario, like returning CSV, concatenate data, or parse JSON in different ways, etc.
I am using body.body because my testing JSON has it twice. In your scenario reference your JSON data as it is.
var myinput = workflowContext.actions.Parse_JSON.outputs.body.body;
var myarray = [];
for(var i=0;i<myinput.Table1.length;i++)
{
myarray.push(myinput.Table1[i].Office);
}
return myarray;
Please Note: That the size limit that the script can handle is max 50MB, and can only run for 5 seconds for Consumption, and 15 seconds for Standard. Limits and configuration reference guide - Azure Logic Apps | Microsoft Docs
You can use/reference Node.js functions in inline code as well. Add and run code snippets by using inline code - Azure Logic Apps | Microsoft Docs
If your scenario requires more running time for the script, you can move the inline code to a function app and use it.