The parallel shape in BizTalk Server (from v2004) is widely misinterpreted to be multi-threaded.
 |
What the parallel shape does well is that it ensures that all branches complete their execution before the next process takes place. The parallel shape does a context switch on a thread wait (busy waiting, such as waiting for a message to come in from a receive port) between branches. |
|
The reason why the parallel shape is single-threaded, is because the orchestration workflow itself, is single-threaded. However, it is still possible to conduct “multi-threaded” execution within BizTalk Server.
|
|
However – suppose that the number of branches of executions are not known at design time, then the parallel shape would not be suitable to use. A different approach will be required.
A classic example of parallel execution is de-batching a large message file into smaller files, and then aggregating all the results into a single large result. Each subset (small file) represents an independent unit of work that does not rely or relate to other subsets, except that it belongs to the same batch (large message). Eventually each child process (the independent unit of work) will return a result message that will be collected with other results, into a collated response message to be leveraged for further processing.
 |
0. The initial step consists of de-batching the large message in the receive pipeline into smaller child messages. 1. The receive pipeline publishes the child messages to the message box database. 2. Child business process orchestrations (or 2-way send ports) subscribe against the child messages to accommodate business requirements. Note that individual business processing does not explicitly require orchestrations. 3. Responses from Line-of-Business (LOB) applications are submitted back to the child orchestrations (or to 2-way send ports). 4. These responses are aggregated back by an aggregator (can be accommodated by an orchestration as depicted, or by a SQL stored procedure). 5. The aggregator will submit the final aggregated result. 6. The next process (which may or may not use the final aggregated result) will proceed. |
De-batching is conducted within the receive pipeline. To learn how to split a large file into smaller files, please refer to the BizTalk Server 2006 samples – Split File Pipeline example. Even though the sample is based on BizTalk Server 2006 (R1) – it remains relevant for all BizTalk Server versions, include v2020
.
• Note: There are several examples on how to de-batch messages, such as de-batching from SQL Server, as communicated by Rahul Garg’s blog on de-batching.
The next step is a bit trickier. The smaller messages need to be correlated somehow, so that their results can be aggregated into a singular large response message (for steps 4 and 5 to be successful). Correlation is typically done in orchestration, but the proposed method here avoids the use of orchestration altogether. In order to have correlation in messaging, please refer to Paolo Salvatori’s blog on Synchronous To Asynchronous Flows Without An Orchestration.
In it, he describes a method that requires a 2-way receive port (such as a web service) , because he leverages a couple of message context properties that are unique to a 2-way receive. These are
• ReqRespTransmitPipelineID
• IsRequestResponse
In addition to
• EpmRRCorrelationToken
• Correlation Token
It is strongly suggested to learn how correlation can be achieved in messaging (without orchestration) as described by Paolo Salvatori before proceeding to the next step.
Once that method has been mastered, a slight adjustment is needed. Since the de-batching method is leveraged in the pipeline, correlation in messaging can be achieved with a 1-way receive port (such as picking up the large message from a file directory), by using message context properties that are unique to the de-batch method.
These are
• InterchangeID
• InterchangeSequenceNumber
• LastInterchangeMessage
Each child message (which is now in the messagebox database) should instantiate a business process (either with an orchestration workflow or a 2-way send port) that will eventually receive a result message.
Finally an aggregator is required to collect and collate the results from the child business processes. This aggregator can be an orchestration, or the preferred way would be from a SQL stored procedure.
*Image courtesy of Paolo Salvatori
• A message is received by a FILE Receive Port
• The message is routed to a SQL Send Port where the original message is wrapped into a sort of envelope by a map which uses a custom XSLT. The outbound message is an Updategram which calls a stored procedure to insert the message into the table.
• A SQL receive port calls a stored procedure to retrieve the records of a certain type from the custom table. This operation generates a single document.
• This document is routed to a File Send Port where the message is eventually transformed into another format.
This method was also communicated in http://geekswithblogs.net/asmith/archive/2005/06/06/42281.aspx
Note that the XSLT can be eventually replaced by a custom pipeline component or directly by an envelope (in this case the send pipeline needs to contain the Xml Assembler component).
For more information concerning the parallel shape, see “How to Configure the Parallel Actions Shape” topic in the BizTalk Server documentation.
Here is a related blog regarding the parallel shape:
• http://www.codeproject.com/KB/biztalk/BizTalkParallelBranching.aspx