Introduction
This blog post is the second part of a series that started with Custom Splitting of SAP BAPIs in BizTalk 2020 - Part 1 (which will be referred to simply as "Part 1"), on the topic of the new support for custom grouping of BAPI transactions in the WCF-SAP adapter. This recent feature lets BizTalk application implementations specify how to group BAPI transactions, by exposing the new writable built-in property BAPIContextId. Previously, the only supported grouping was based on a common interchange id, which meant for example that messages in the same batch had to be mapped to the same LUW on the SAP adapter.
In the first post, we looked at the "custom" aspect in custom grouping: by assigning a unique BAPIContextId value to each BAPI transaction message within the same batch, it was possible to commit each transaction independently. In this post, we look at the "grouping" aspect of the improvement by extending the initial scenario of part 1 to a situation where message batches contain groups of BAPI transactions within. The idea of the scenario is to assume that batches could contain the same BAPI operation (like creating an order) hundreds of times but for different customers, some having more orders than others. Without BAPIContextId, it would have been difficult to implement. With the new property, the simplified implementation of Part 1 can be leveraged with minor changes to schemas, custom pipeline component and orchestration, which remain the same for the most part.
Note: Familiarity with Part 1 is recommended.
Updated Scenario
The bulk of the scenario implementation being the same as the one presented in detail in Part 1, implementing support for groups within batches requires the following modifications:
1. Messages within a batch are grouped based on a common field, BatchId.
2. Grouping is done in the custom pipeline component by assigning the same BAPIContextId to messages in the same group.
3. Messages within a group are processed by the same orchestration instance, which corresponds to the same LUW on the SAP server.
4. Each orchestration instance processes all messages in the same group until the last message is received.
These differences are shown in the following message flow diagram:
Change #1: Introduce a property schema for promoted properties
Messages within a batch are grouped based on a common field, BatchId (i.e., group id within a batch). An additional field, IsLastInBatch, is used as a sentinel value to indicate the last message of the group, which is not necessarily the last message in the batch. Both properties are in a property schema, type documented in Property Schemas.
The property schema base of BatchId is the default, MessageDataPropertyBase. As we will see later, this property needs to be explicitly promoted in the custom receive pipeline as it is used as correlation in the orchestration.
The property schema base of IsLastInBatch is set to MessageContextPropertyBase instead of the default, which gives the following advantages to IsLastInBatch (cf. BizTalk Property promotion in receive pipeline for an exhaustive list):
- It is carried over to the context of CREATEFROMDAT2 messages, which do not (nor need to) contain such a field in their data.
- It can attach to every message, for instance when constructing and copying CREATEFROMDAT2 messages inside the orchestration.
- It is promoted in the receive pipeline.
- It will be copied to the new mapped message in the receive port mapping.
The latter point was already mentioned in Part 1 for the BAPIContextId: receive port mapping copies all the context property values of the original message to the new mapped message.
In the Order schema, BatchId and IsLastInBatch are promoted as property fields according to the process explained in How to Copy Data to the Message Context as Property Fields and summarized below.
Change #2: Update the custom pipeline component
Inside a received batch (Orders message), each disassembled Order message can be one of the following:
- If BatchId is null or empty, the message is not in any group.
- If BatchId is not empty and IsLastInGroup is not provided or is set to false, the message is in a group but is not the last.
- If BatchId has a value and IsLastInGroup is true, the message is the last one in the group.
The value of BAPIContextId is the same for each Order within the same group. Messages which are not in any group are assigned a unique BAPIContextId in the same way as in Part 1.
The corresponding pipeline component code is shown below. Differences with the original code from Part 1 are highlighted.
|
const string SystemPropertiesSchema = "http://schemas.microsoft.com/BizTalk/2003/system-properties"; string interchangeId = (string)mBaseMessage.Context.Read("InterchangeID", SystemPropertiesSchema); int interchangeSequenceNumber = (int)mBaseMessage.Context.Read("InterchangeSequenceNumber", SystemPropertiesSchema);
string batchId = (string)mBaseMessage.Context.Read("BatchId", "https://SapBAPITxClientDebatching.PropertySchema1");
string bapiContextId = string.IsNullOrEmpty(batchId) ? interchangeId + interchangeSequenceNumber // If there is no batch id, just set BAPIContextId unique per message : interchangeId + batchId; // If batchId is provided, let's make it unique per group
// Promote BAPIContextId context property in order to use it as correlation in the orchestration
bool isLastInBatch = true; // By default first assume that each message is by itself
// Write the IsLastInBatch property to the message context. No need to promote it explicitly here.
// Set recoverability to true for the mapping phase in case the receiver wants to transform from client to BAPI in the RL. |
BAPIContextId set to interchange id + batchId is unique per group within the same batch. Note that it would be easy to support groups potentially spanning multiple batches by simply setting BAPIContextId to something unique to the group regardless of the batch in which it is received.
Change #3: Assign the same orchestration instance to each group
Once the CREATEFROMDAT2 messages produced by the receive location are published to the message box, we need some way to make sure that all messages in the same group are processed by the same orchestration instance. In the same way as in Walkthrough: Correlations in BizTalk Orchestration and SAP BAPI Transactions Walkthrough, this is achieved by setting the orchestration correlation set to the promoted context property used to group messages, namely BAPIContextId.
Change #4: Process sequential convoys in the orchestration
The following changes were made to the orchestration of Part 1 for convoy processing:
1. Add ordered delivery to the receive port (see Ordered Delivery of Messages in BizTalk Server for more info) to make sure the last message in a group is processed last.
2. Listen shape to process subsequent messages in the same group.
3. Delay shape to suspend the orchestration instance as resumable if the last message in a group has not been received within a pre-determined period of time. It is a simplistic way to set some boundary on the processing for demo purposes and there are alternatives beyond the scope of this scenario (for instance, if the LastInterchangeMessage property is found true in the receive pipeline, assume that each active orchestration instance has seen all messages in the same group).
Details on the orchestration implementation are presented in the following diagram.
Concluding Remarks
This concludes the two-part series on BAPIContextId. The new property was announced in Support for custom grouping of BAPI transactions in the WCF-SAP adapter and the goal here was to show that there is more to it than meets the eye.
In our scenarios, BAPIContextId is used "upstream" of the processing to bundle BAPI transactions ahead of time, but other scenarios come to mind, such as resubmitting suspended messages and simpler error handling in scenarios such as the one presented in Recovering SAP BAPI Transactions with Custom Pipelines.
Code and implementation artifacts are provided in the attached archive.
References
Custom Splitting of SAP BAPIs in BizTalk 2020 - Part 1
Improvement: Support for custom grouping of BAPI transactions in the WCF-SAP adapter
SAP BAPI Transactions Walkthrough
Walkthrough: Correlations in BizTalk Orchestration
BizTalk Property promotion in receive pipeline
How to Copy Data to the Message Context as Property Fields
Ordered Delivery of Messages in BizTalk Server