Blog Post

Azure Integration Services Blog
2 MIN READ

Get iteration index in Foreach action

Drac_Zhang's avatar
Drac_Zhang
Icon for Microsoft rankMicrosoft
Feb 01, 2023

Sometimes we need to get iteration index in Foreach loop for data processing, one resolution is to set the "Concurrency Control" to 1 and use a variable as index which always be added by 1 for each iteration.

This approach can achieve what we would like to have, but it will impact the performance since the foreach is in sequential mode.

 

In order to keep the "Foreach" action running in parallel, there's a new approach as following:

 

Steps

1.  Generate a new array which save the all the index of the original array which need to be processed. For example, if we have a 10 items array, then the index array need to be generated as [ 0, 1, 2 ... 9 ] (start from 0, end with [array length - 1] ).

Here I'm using Azure Monitor KQL expression "range" (reference : range operator - Azure Data Explorer | Microsoft Learn) to generate the index array as per the original array length (in screenshot, the variable "TestData" is my original array).

The query result is following:

 

Meanwhile, if you don't have Log Analytics Workspace, the index array also can be generated by various ways (e.g. Azure Function, Azure Automation, Logic App Foreach action, etc.)

 

2. In the Foreach action, we need to iterate for the newly generated array

 

3. Use expression to pickup the original items directly refer to "index".

Since we are directly get items by index which has been generated outside the foreach loop, so it will not be impacted by the parallel iterations.

 

Performance comparison 

In my local environment, for processing 1000 elements array, the new approach can save around 90% time.

 

For more testing runs, result as following:

RunID

Parallel_Action

Parallel_Duration

Sequential_Action

Sequential_Duration

08585263805686940847269633553CU04

Scope_-_Parallel

26292

Scope_-_Sequential

382392

08585263805721763244014366564CU05

Scope_-_Parallel

28928

Scope_-_Sequential

386164

08585263806421048241938496487CU18

Scope_-_Parallel

33225

Scope_-_Sequential

418940

08585263806457138684407646963CU03

Scope_-_Parallel

29956

Scope_-_Sequential

390212

08585263806629391869208870544CU12

Scope_-_Parallel

30782

Scope_-_Sequential

403036

 

 

Updated Feb 01, 2023
Version 1.0
No CommentsBe the first to comment