Handling Errors in SAP BAPI Transactions

 

Introduction

 

BizTalk artifacts are typically deployed in production once they have been thoroughly tested for functionality, stress, performance, integration, to name a few. Regardless of the amount of testing though, a system is vulnerable to the external data being received. Here, we consider what happens when BAPI transactions requests contain invalid data, such as fields that are not of the right formats or do not have expected values. The reader should be familiar with the orchestration presented in Debatching SAP BAPI Transaction Requests, which is augmented here for better error handling and fault-tolerant batch processing.

 

Generally speaking, after the BAPI transaction requests are created, validation can happen:

(1) locally, in the BizTalk host instance, when the BAPI transactions are published and sent, and

(2) remotely, on the SAP server, when the transactions are received and processed.

Local failures result in the SAP adapter throwing such exceptions as:

"A message sent to adapter "WCF-SAP" on send port "…" with URI "…" is suspended. Error details: System.InvalidOperationException: The Inner Channel to use was not found for the key."

or other exceptions documented in Exceptions and Error Handling with the SAP adapter.

Remote data failures correspond to the SAP server returning error messages in the RETURN elements of the BAPI responses.

 

The distinction local/remote is important because error handling is implemented differently depending on whether errors happen on outbound or inbound data flows. For instance, as we shall see, BAPI_TRANSACTION_COMMIT and BAPI_TRANSACTION_ROLLBACK are not always applicable depending on how validation fails. From a customer perspective though, errors from invalid data need to be surfaced consistently regardless of where the errors happen. Some may expect "fail fast", where any invalid data should abort further processing. Others would rather handle invalid data with graceful error handling. This is especially relevant in the context of BAPI transactions where a Logical Unit of Work (LUW) may comprise a large number of transactions and the "one-fails/all-fail" behavior would be cost-prohibitive; In such cases, it would be preferable to keep the processing going rather than to suspend an orchestration instance for some invalid field in one out of many transactions (e.g., a vendor no longer exists but a request uses the vendor's code).

 

Note: At this point, it is noteworthy to mention that the Old BAPI Transaction Model (with Commit) to execute a commit after every single RFC call has been phased out a long time ago in favor of the new(er) BAPI Transaction Model (Without Commit), i.e., multiple BAPI calls in one LUW. In the latter model, BAPIs can be used as a way to do mass data transfer effectively as documented in BAPIs for Mass Data Transfer thereby taking advantage of the improved performance through common updates. The "Without Commit" model has also been enhanced to a "Transaction Model with Buffering" (Buffering with Write BAPIs), which allows creates and updates to happen in the same LUWs.

 

1. Scenario

 

The first stage of the main orchestration is the debatching pipeline explained in detail in Debatching SAP BAPI Transactions and summarized below.

 

 

Debatching Pipeline

 

A few changes were made to the original pipeline of Debatching SAP BAPI Transaction Requests.

 

First, we introduced a Boolean element ProcessErrors to indicate the desired error behavior when BAPI transaction requests fail ("failure" being defined in the next section).

- If set to true, the orchestration will log the errors and continue processing the debatched transaction requests.

- If false, the orchestration will record the errors, roll back existing transactions, and exit. Orchestration instances are not suspended.

The idea of using ProcessErrors came out of being consistent with the BAPI parameter BEHAVE_WHEN_ERROR, which is documented by SAP as a way to control order creation when some sale items within a transaction (BAPISDITM field) cannot be created. A value of "P" (Process Errors) means that the order will be saved when errors occur and problematic items will not be saved. 

 

The second modification was as follows: the method AddFromXMLString of helper class BAPIOrdersList, used for caching debatched CREATEFROMDAT2 documents,  now allocates a BAPIRET2[] array inside each CREATEDFROMDAT2, which is used on the SAP side as a return parameter (explained in the next section).

[Serializable]
public class BAPIOrdersList : List<CREATEFROMDAT2>
{
    public BAPIOrdersList() { }

    public void AddFromXMLString(XmlDocument document)
    {
        MemoryStream stream = new MemoryStream();
        document.Save(stream);
        stream.Flush();
        stream.Position = 0;

        XmlSerializer reader = new System.Xml.Serialization.XmlSerializer(typeof(CREATEFROMDAT2));
        StreamReader st = new StreamReader(stream);

        CREATEFROMDAT2 transact = (CREATEFROMDAT2)reader.Deserialize(st);
        transact.RETURN = new BAPIRET2[1]; // Allocate this structure to get the return values in the response.
        this.Add(transact);
        st.Close();
    }

    public CREATEFROMDAT2 Get(int index){ return this[index]; }
    public int OrdersCount() { return this.Count; }
    public void Insert(BAPIOrdersList orders) { this.AddRange(orders); }
}

 

Last, exception processing was introduced in the debatching pipeline in order to catch errors that could happen during transforms from bad data. The details are covered in a separate article as they are not specific to BAPIs. 

 

2. Invalid Data

 

We classified "invalid" data into two types based on where the error happen and the corresponding impact: value errors and format errors.

 

2.1 Invalid Data Values

 

Data value errors correspond to values that cause transaction errors on the SAP side. For instance, using a vendor or material that do not exist. In these cases, the CREATEFROMDAT2Response contains information in the BAPIRET2 structure parameter provided in the CREATEFROMDAT2 request. Example of data value error:

<RETURN>
  <BAPIRET2 xmlns="http://Microsoft.LobServices.Sap/2007/03/Types/Rfc/">
    <TYPE>E</TYPE>
    <ID>CZ</ID>
    <NUMBER>95</NUMBER>
    <MESSAGE>Sales organization abcd does not exist</MESSAGE>
    <LOG_NO></LOG_NO>
    <LOG_MSG_NO>0</LOG_MSG_NO>
    <MESSAGE_V1>abcd</MESSAGE_V1>
    <MESSAGE_V2></MESSAGE_V2>
    <MESSAGE_V3></MESSAGE_V3>
    <MESSAGE_V4></MESSAGE_V4>
    <PARAMETER>SALES_HEADER_IN</PARAMETER>
    <ROW>0</ROW>
    <FIELD></FIELD>
    <SYSTEM>T90CLNT090</SYSTEM>
  </BAPIRET2>
  <BAPIRET2 xmlns="http://Microsoft.LobServices.Sap/2007/03/Types/Rfc/">
    <TYPE>E</TYPE>
    <ID>V4</ID>
    <NUMBER>219</NUMBER>
    <MESSAGE>Sales document was not changed</MESSAGE>
    ...
  </BAPIRET2>
</RETURN>

As explained in Using BAPIs in Distributed Systems (ALE), a value of "E" in any of the TYPE fields of BAPIRET2 indicates transaction failure and therefore, nothing to commit.

 

2.2 Invalid Data Formats

 

Data Format errors correspond to field formats that do not abide by the metadata schemas provided by the SAP server, when generated in Visual Studio (cf. SAP BAPI Transactions Walkthrough). Such errors cause exceptions in the SAP adapter before a BAPI transaction is even sent, which then results in the SAP channel being broken. For example, if we used a string of length greater than 4 in the SALES_ORG field, we would get the following error in the event log:

A message sent to adapter "WCF-SAP" on send port "BAPI2032SalesOrdersSP" with URI "sap://CLIENT=800;LANG=EN;@a/..." is suspended.  Error details: Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: An error occurred when trying to convert the XML string thisistoolong of RFCTYPE RFCTYPE_CHAR with length 4 and decimals 0 to a .NET type. Parameter/field name: SALES_ORG   Error message: The length of the value for the field exceeds the allowed value. Value: 13  Field: SALES_ORG Allowed value: 4. ---> Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: The length of the value for the field exceeds the allowed value. Value: 13  Field: SALES_ORG Allowed value: 4    at Microsoft.Adapters.SAP.SapMetadataUtilities.ConvertXmlStringToRfcStringNCo(String data, RfcTypes type, Int32 singleByteLength, Int32 decimals, String fieldname, Encoding encoding, Encoding realPartnerEncoding, Boolean padToExactLength, SAPConnection sapconnection)

 

Any further attempt to submit BAPI transactions in the same LUW would result in the dreaded "Inner Channel" exception which indicates that the connection to the server was already closed:

A message sent to adapter "WCF-SAP" on send port "BAPI2032SalesOrdersSP" with URI "sap://CLIENT=800;LANG=EN;@a/..." is suspended.  Error details: System.InvalidOperationException: The Inner Channel to use was not found for the key {0A73F66F-31FA-4E48-BAC5-14EAED9571D4}_{27D07099-1874-47C3-9E69-2AD1FA42DE8D};URI=sap://CLIENT=800;LANG=EN;@a/.... Specify a valid key.

 

We would see the "Inner Channel" error if a BAPI_TRANSACTION_COMMIT or BAPI_TRANSACTION_ROLLBACK was sent after the data format error. To make things worse, if we were to send a BAPI_TRANSACTION_ROLLBACK in an exception handler, we would then get:

A message sent to adapter "WCF-SAP" on send port "BAPI2032SalesOrdersSP" with URI "sap://CLIENT=800;LANG=EN;@a/..." is suspended.  Error details: System.ServiceModel.CommunicationException: The server did not provide a meaningful reply; this might be caused by a contract mismatch, a premature session shutdown or an internal server error.

 

The latter happens because the ABORT message of BAPI_TRANSACTION_ROLLBACK is sent as first message on a new connection, which produces undesirable results as documented in Run BAPI Transactions in SAP using BizTalk Server. As a side-note, this illustrates the point that BAPI_TRANSACTION_ROLLBACK is not always suited to an orchestration exception block depending on the error.

 

Exceptions in the SAP adapter, caused by data format errors as well as from other errors such as using the wrong URL, result in suspended messages as shown below:

 

Suspended Messages

In the orchestration design presented next, such exceptions are handled gracefully and do not cause suspended orchestrations.

 

3. Orchestration Overview

 

3.1 Logical Flow

 

The logical flow through the orchestration can be divided into four "tracks":

a. Process errors = true, data format errors,

b. Process errors = false, data format errors,

c. Process errors = true, data value errors,

d. Process errors = false, data value errors.

 

Orchestration Tracks

Legends correspond to the flows presented below. Stages 1 and 2 are the same as in Debatching SAP BAPI Transaction Requests. The differences start at stage 3, CREATEFROMDAT2 and CREATEFROMDAT2Response processing:

- Stage 3: Per-BAPI-transaction error processing;

- Stage 4: Per-BAPI-transaction exception handling;

- Stage 5: Commit and Rollback logic.

 

Orchestration Overview

3.2 Logging

 

For the sake of keeping the implementation as generic as possible, processing is geared towards logging everything that happens in order to show the different places of interest that could be adapted to "real-world" applications. 

 

We look at the messages being constructed and the data being logged. What gets saved:

1. Original CREATEFROMDAT2 documents if there are any errors (value or format);

2. CREATEFROMDAT2Response documents received from SAP; 

3. Errors contained in CREATEFROMDAT2Response documents, formatted as strings;

4. Exception strings.

 

Logging error strings:

Value errors are contained in the CREATEFROMDAT2Response messages received from the SAP server. They are extracted by the orchestration into a variable of type the following helper class (complete code in attached archive):

[Serializable]
public class BAPIReturnValues : List<BAPIRET2>
{
  ...
  public bool IsError()
  {
    for (int i = 0; i < this.Count; i++)
    {
      if (this[i].TYPE == "E")
      {
        return true;
      }
    }
    return false;
  }

  public string GetErrorString()
  {
    string errorString = string.Empty;
    for (int i = 0; i < this.Count; i++)
    {
      if (this[i].TYPE == "E")
      {
        errorString += string.Format("Index {0}: {1}; ", i, this[i].MESSAGE);
      }
    }
    return errorString;
    }
  }
  ...

which is used as in the expression:

BAPIResponseRetVal.Set(BAPIResponse);

where BAPIResponse is of type BUS2032.CREATEFROMDAT2Response. The method BAPIReturnValues.GetErrorString() produces the error string logged to a file by using the RawString class documented in sending string to file location in BizTalk included in the BizTalk code attached to this blog post.

DataValueErrorMessage = new Microsoft.Samples.BizTalk.XlangCustomFormatters.RawString(
System.String.Format("Message {0}: {1}", BAPIOrdersCount, BAPIResponseRetVal.GetErrorString()));

 

Logging exception strings:

Exceptions are logged in XML with the schema:

XML messages are created with the expression:

DataFormatErrorMessageVar = new System.Xml.XmlDocument();
DataFormatErrorMessageVar.LoadXml(System.String.Format(
"<ns0:ExceptionInfo xmlns:ns0=\"http://SapBAPITxClientDebatching.ExceptionInfo\"><Type>{0}</Type><Message><![CDATA[{1}]]></Message></ns0:ExceptionInfo>",
SystemException.GetType(), 
SystemException.ToString()));

Note the use of CDATA to prevent parsing of characters that would otherwise invalidate the xml format. Example of output for the same data value error:

<ns0:ExceptionInfo xmlns:ns0="http://SapBAPITxClientDebatching.ExceptionInfo">   <Type>Microsoft.XLANGs.Core.XlangSoapException</Type>   <Message><![CDATA[Microsoft.XLANGs.Core.XlangSoapException: An error occurred while processing the message, refer to the details section for more information Message ID: {1EC13267-2C5A-4AA0-A8B0-BD5F2632162D} Instance ID: {9556809F-27B8-49C5-AB2E-6814CB9EC6FA} Error Description: Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: An error occurred when trying to convert the XML string... </ns0:ExceptionInfo>

 

Each of the stages is presented in detail in the next sections. We will look at value errors, then format errors, and finally commit/rollback. 

 

4. Orchestration Details: Processing Value Errors

 

Detailed steps are shown below:

Processing Value Errors

 

In case of success:

CREATEFROMDAT2Response documents are saved and CREATEFROMDAT2 are added to a list of BAPI orders to commit (highlighted in yellow). 

 

In case of errors:

1. Errors get logged. Example of error output:

Message 1: Index 0: For sales org. abcd distribution channel 12 is not allowed; Index 1: Sales document  was not changed;

2. When ProcessError is set to false: the flow is interrupted with a System.Exception created by:

BAPIDataValueExceptionVar = new System.Exception(BAPIResponseRetVal.GetErrorString());

The exception will be caught later on, in the rollback stage of the orchestration. Example of logged exception:

<ns0:ExceptionInfo xmlns:ns0="http://SapBAPITxClientDebatching.ExceptionInfo">
<Type>System.Exception</Type>
<Message><![CDATA[System.Exception: Index 0: For sales org. abcd distribution channel 12 is not allowed; Index 1: Sales document was not changed; 
at SapBAPITxClientDebatching.BAPIOrchestration.segment12(StopConditions stopOn)
at Microsoft.XLANGs.Core.SegmentScheduler.RunASegment(Segment s, StopConditions stopCond, Exception& exp)]]></Message>
</ns0:ExceptionInfo>

 

5. Orchestration Details: Processing Format Errors and Adapter Exceptions

 

Format errors are handled in a per-request catch block for System.Exception, which encompasses the System.Web.Services.Protocols.SoapException and Microsoft.XLANGs.Core.XlangSoapException. 

 

At this stage, regardless of ProcessErrors, exceptions are logged.

 

Example of exception message:

<ns0:ExceptionInfo xmlns:ns0="http://SapBAPITxClientDebatching.ExceptionInfo">
<Type>Microsoft.XLANGs.Core.XlangSoapException</Type>
<Message><![CDATA[Microsoft.XLANGs.Core.XlangSoapException: An error occurred while processing the message, refer to the details section for more information 
Message ID: {1EC13267-2C5A-4AA0-A8B0-BD5F2632162D}
Instance ID: {9556809F-27B8-49C5-AB2E-6814CB9EC6FA}
Error Description: Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: An error occurred when trying to convert the XML string thisistoolong of RFCTYPE RFCTYPE_CHAR with length 4 and decimals 0 to a .NET type. Parameter/field name: SALES_ORG Error message: The length of the value for the field exceeds the allowed value. Value: 13 Field: SALES_ORG Allowed value: 4. ---> Microsoft.ServiceModel.Channels.Common.XmlReaderParsingException: The length of the value for the field exceeds the allowed value. Value: 13 Field: SALES_ORG Allowed value: 4
at Microsoft.Adapters.SAP.SapMetadataUtilities.ConvertXmlStringToRfcStringNCo(String data, RfcTypes type, Int32 singleByteLength, Int32 decimals, String fieldname, Encoding encoding, Encoding realPartnerEncoding, Boolean padToExactLength, SAPConnection sapconnection)
--- End of inner exception stack trace ---
Server stack trace: 
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)...
</Message>
</ns0:ExceptionInfo>

 

When ProcessErrors is set to true: Transactions that have been successful are re-enqueued at the back of the BAPIOrders list to be resent. The assumption that in this particular scenario configuration, BAPIs can be submitted in any order, which is reasonable if one is willing to ignore errors and let the processing continue. The BAPIConnectionState is reset to "OPEN" so that a new connection to the SAP server may be open to replace the connection broken from the SAP adapter exception. The re-enqueueing expression is: 

BAPIOrders.Insert(BAPIOrdersToCommit);
BAPIOrdersToCommit.Clear();
BAPIConnectionState = "OPEN";
SuccessCount = 0;

 

Note: As mentioned in Run BAPI Transactions in SAP using BizTalk Server, the send port binding property EnableConnectionPooling has to be set to false to prevent the same connection to be reused when an LUW is unexpectedly interrupted.

 

As can be seen below, iterations will continue until BAPIOrders.Count() has been reached, which will include re-enqueued requests.

 

BAPI Processing Loop

 

6. Orchestration Details: BAPI_TRANSACTION_COMMIT/ROLLBACK

 

Commit and Rollback stages are almost identical. The former happens when there is no error or if any error has been processed; the latter when execution is interrupted by exceptions. In both cases, BAPI commit/rollback requests are sent only if there has been any successful transaction.

BAPI requests/responses are processed in their own scope for exception handling and logging as previously described.

 

Commit stage:

Orchestration Commit Stage

 

Rollback stage:

Orchestration Rollback Stage

7. Deploying the Solution

 

All artifacts used in this article can be created with the bindings file provided in the attached archive. There is one file location per logical send port used for logging.

 

BizTalk Artifacts

 

The SAP Static Solicit-Response port is configured with RetryCount = 0 and "propagate fault message" checked. 

The archive also contains some sample input files and output illustrating the various logical flows through the orchestration with variations on batch size, error type, ProcessErrors flag, etc.

 

8. Concluding Remarks

 

There is little BizTalk material on how to use error handling and rollback in BAPI transactions. Our aim here was to explore what can be done based on existing orchestrations such as the ones presented in this series of blog articles. Basically, what it would look like if we wanted to do more than "putting a BAPI_TRANSACTION_ROLLBACK in a catch exception shape", while keeping things as simple as possible. This meant leaving some items intentionally out-of-scope for this article, and it is worth mentioning some of them here:

 

- Re-enqueuing may be expensive if  channel-ending errors tend to happen at the end of a large batch of error-free transactions. The ideal workload would be one with either many errors at the beginning of a batch, or where errors happen randomly only in some batches. Clearly, there are some "pathological" cases, which could be mitigated by tuning the batch sizes, or by simply saving the transactions for later processing by a different orchestration instance. 

 

- BAPI requests that could not be sent due to exceptions in the SAP adapter are suspended with resumable status. In the case of errors caused by the data in the messages, as considered in this article, resuming is of limited use. But there are ways to flow these messages back so they don't linger for too long in the message box.

 

- Last, as hinted at in Scatter-Gather Pattern with SAP BAPI Transactions, an interesting way to extend the topic presented here could be to combine error handling with asynchronous processing in multiple orchestrations.

 

The next article in the BAPI series, Recovering SAP BAPI Transactions with Custom Pipelines, combines the ideas presented here with debatching done in the receive pipeline.

 

References

 

SAP BAPI Transactions Walkthrough

Debatching SAP BAPI Transaction Requests

Exceptions and Error Handling with the SAP adapter

Old BAPI Transaction Model (with Commit)

BAPI Transaction Model (Without Commit)

BAPIs for Mass Data Transfer

Buffering with Write BAPIs

Using BAPIs in Distributed Systems (ALE)

Run BAPI Transactions in SAP using BizTalk Server

sending string to file location in BizTalk

Run BAPI Transactions in SAP using BizTalk Server

Scatter-Gather Pattern with SAP BAPI Transactions