Customers use SAP systems for their business-critical operations. Today, customers want to be able to combine their SAP data with non-SAP data for their analytics needs. Azure Data Factory (ADF) is an industry-leading data integration service which enables customers to ingest data from diverse data sources (e.g., multi-cloud, SaaS, on-premises), transform data at scale, and more.
Azure Data Factory (ADF) works seamlessly to combine data and prepare it at cloud-scale. Customers are using ADF to ingest data from different SAP data sources (e.g., SAP ECC, SAP Hana, SAP Table, SAP BW Open Hub, SAP BW via MDX, SAP Cloud for Customers), and combining them with data from other operational stores (e.g., Cosmos DB, Azure SQL family, and more). This enables customers to gain deep insights from both SAP and non-SAP data.
Today, we are excited to announce the General Availability of SAP CDC support in Azure Data Factory and Azure Synapse Analytics.
Using Azure Data Factory and Azure Synapse Analytics mapping data flows, customers will now be able to extract changes from different SAP systems using the new SAP CDC connector, transform the data at scale, and deliver the data into a data lake or data warehouse for analytics. The ability to work with change data significantly reduces resource consumption and provides a tremendous efficiency boost in high data volume scenarios.
What does the new connectivity offer?
Wide variety of sources: With the SAP CDC connector, customers get access to the most widely adopted SAP sources, including
- SAP ECC extractors
- SAP S/4HANA CDS views
- SAP BW and SAP BW/4HANA InfoProviders, InfoObjects and Queries
- SAP tables (in combination with SAP Landscape Transformation Replication Server SLT)
For each of these options, SAP delivers thousands of objects out-of-the box, providing data integration interfaces across all SAP components. Of course, custom-built objects are accessible with the new connector as well.
Scalable, CDC enabled data integration: Via SAP’s Operational Data Provisioning framework (ODP), these sources provide a robust and scalable interface for data extraction, including full and delta (change data capture) capabilities with low latency.
Multi-cloud enablement: The architecture can be applied to virtually any SAP system wherever it is deployed: using a self-hosted integration runtime as a proxy between Microsoft Azure and the SAP system, you can connect to SAP whether it is running on-premises, in Microsoft Azure cloud, or on any other hyperscaler’s infrastructure.
How does it work?
Starting point are mapping data flows in Azure Data Factory or Azure Synapse Analytics which provide an easy-to-use, integrated end-to-end experience to extract the data from SAP, transform it and apply the changes to any supported sink.
Below, you see a simple mapping data flow extracting raw change data from SAP, applying a simple transformation (consider the derived column as a placeholder for your actual technical or business transformations), and updating the resulting data to a sink (in this case a Delta table in Azure Data Lake Gen2).
The beauty of the solution is that the source transformation does all the hard work for you: it reads the raw change data from SAP and interprets the associated metadata to handle inserts, updates and deletes correctly. That way, developers can concentrate on the required technical or business transformations without having to worry about the complex internal logic of the SAP change feeds.
Unleashing the power of the Microsoft Azure Data portfolio on your SAP data has never been easier. With the new SAP CDC connector, Azure Data Factory and Azure Synapse Analytics now provide you with an easy-to-use tool to extract SAP data in a robust, scalable and high performant way.
Click here to get started!