Introduction
According to SAP, 77% of the world’s transaction revenue touches an SAP system. To understand these systems and extracting the data is a challenging job. To gain insights, it is always helpful to integrate the data source systems seamlessly with analytics services.
During a hack session, we tried to accomplish the challenge of extracting the data from SAP source and integrating it with analytical services using the new Database Templates/Lake Database feature that is available in Synapse workspaces.
The scope for our Hack
We integrated SAP S/4 HANA system data with Synapse Lake database templates.
Requirements:
1.Extract data from SAP source system
The first step is to create a linked service to the SAP source system.
To establish a connection, we set up a self-hosted integration runtime as a compute engine and a gateway for data connection.
Create a self-hosted integration runtime - Azure Data Factory & Azure Synapse | Microsoft Learn
We used SAP Table connector to create linked service for our source system (the latest CDC connector was still not available within Synapse for some regions).
Provide the client id and login details to establish a connection to SAP source system.
Next, created the datasets for the source and destination.
Source Destination
A simple copy activity in a pipeline to extract the data from source and land in the destination data lake
Source and sink settings
Once the data lands in ADLS, it should look similar to the below:
(The depth of the table folders should be: <container>/<foldername>/<inputfiles> this is important for the data mapping step later)
2.Create an empty Lake database for our source data
We created our lake database using the Manufacturing industry template. It is important to identify the industry template relevant for the data.
We then added the entities to the database canvas as per our needs and created the database definition.
As needed, edit the table definitions with the column names, data types, nullability, relationships etc.
No data is ingested yet, so the database will be empty.
We then published to save the changes. Now, we have an empty lake database based off the manufacturing template with the customizations that we needed.
Note that the database will show under the "Lake Database" in Synapse but yet the corresponding folders in ADLS would still not be created. These folders will be created while the data gets ingested which brings us to the next step of ingesting data into the Lake Database that is continued in second part of this blog below.
Integrate Synapse Database Templates with SAP Data - Part 2 - Microsoft Community Hub
Additional reading:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.