General availability of SAP CDC capabilities for Azure Data Factory and Azure Synapse Analytics
Customers use SAP systems for their business-critical operations. Today, customers want to be able to combine their SAP data with non-SAP data for their analytics needs. Azure Data Factory (ADF) is an industry-leading data integration service which enables customers to ingest data from diverse data sources (e.g., multi-cloud, SaaS, on-premises), transform data at scale, and more. Azure Data Factory (ADF) works seamlessly to combine data and prepare it at cloud-scale. Customers are using ADF to ingest data from different SAP data sources (e.g., SAP ECC, SAP Hana, SAP Table, SAP BW Open Hub, SAP BW via MDX, SAP Cloud for Customers), and combining them with data from other operational stores (e.g., Cosmos DB, Azure SQL family, and more). This enables customers to gain deep insights from both SAP and non-SAP data. Today, we are excited to announce the General Availability of SAP CDC support in Azure Data Factory and Azure Synapse Analytics.Storage Event Trigger - Permission and RBAC setting
Storage Event Trigger in Azure Data Factory is the building block to build an event driven ETL/ELT solution. In this blog, we will deep dive into permissions settings for Storage Event Trigger, and properly understand how to enforce access control in this architecture.Announcing Public Preview of the SAP CDC solution in Azure Data Factory and Azure Synapse Analytics
UPDATE! We've launched the new SAP connector in ADF today, June 30, 2022, and updated the blog post below accordingly. For decades, companies have relied on Microsoft and SAP software to run their most mission-critical operations. Today, we’re excited to launch public preview of SAP Change Data Capture (CDC) in Azure Data Factory (ADF). Combining a new data connector with predefined data flow templates, this solution streamlines the integration of SAP data within core Azure services like Azure Synapse Analytics and Azure Machine Learning. The new SAP ODP connector leverages SAP Operational Data Provisioning (ODP) framework, which is an established best practice for data integration within SAP landscapes. ODP provides access to a wide range of sources across all major SAP applications and comes with built-in CDC capabilities. In combination with the predefined data flow templates to process and update the changed records to any sink, this makes SAP data integration into Azure very much straight forward.Orchestrate and operationalize Synapse Notebooks and Spark Job Definitions from Azure Data Factory
Today, we are introducing support for orchestrating Synapse notebooks and Synapse spark job definitions (SJD) natively from Azure Data Factory pipelines. These new native activities make it easy to run Synapse notebooks and SJD from an ADF pipeline!Implement Fill Down in ADF and Synapse Data Flows
"Fill Down" is an operation common in data prep and data cleansing meant to solve the problem with data sets when you want to replace NULL values with the value from the previous non-NULL value in the sequence. Here is how to implement this in ADF and Synapse data flows.
