Improve Spark pool utilization with Synapse Genie
Synapse Genie Framework improves Spark pool utilization by executing multiple Synapse notebooks on the same Spark pool instance. It considers the sequence and dependencies between notebook activities in an ETL pipeline, which results in higher usage of a full cluster for resources available in a Spark pool.
Data mesh: A perspective on using Azure Synapse Analytics to build data products
This is a multi-part blog series, and it discusses various aspects of implementing data mesh architecture on Azure. This part focuses on data as a product principle and presents a perspective on using Azure Synapse Analytics as a data product. We discuss (at a high-level) data product functions & capabilities and apply that lens to Synapse Analytics. We discuss how workspaces can be partitioned to give domains scale and agility to build data products.
Unleashing the capabilities of Azure Synapse Analytics for a healthcare customer
This blog post aims to provide you with insights into the best practices for optimizing performance in data ingestion and transformation. You will learn how to efficiently use available resources, compute capacity, and workload management in Azure Synapse Analytics workspace. While the solution discussed in this blog pertains to a healthcare industry customer, the optimization techniques presented here are applicable to other industries as well.
Ingest and Transform Data with Azure Synapse Analytics With Ease
The Integrate Hub within Azure Synapse Analytics workspace helps to manage, creating data integration pipelines for data movement and transformation. In this article, we will use the knowledge center inside the Synapse Studio to ingest sample data and use a data flow to filter and transform the data to create a final CSV file in Azure Data Lake Storage Gen2.How to – Advanced Properties of Linked Services
Azure Data Factory and Synapse Pipelines have a wealth of linked service connection types that allow them to connect and interact with many services and data stores. The Workspace UI provides the most important properties that are needed for the connection. However, at times we need more control that the UI doesn’t offer. In this tutorial, we will focus on customer scenario to see how you can find all the available properties for the Linked Service you are working with, then how to incorporate that property into your Linked Service directly from within the UI.How to use CI/CD integration to automate the deploy of a Synapse Workspace to multiple environments
As an integrated analytics service that accelerates time to insight across data warehouses and big data systems, Azure Synapse Analytics delivers a unified experience through Synapse Studio, promoting collaborative work between Data Engineers, Data Scientists and Business Analysts. By enabling this collaborative environment, Azure Synapse Analytics facilitates the integration of its Big Data and Analytics capabilities with the enterprise CI/CD process. In this article we are going to demonstrate how Azure Synapse Analytics can easily integrate with one of the most adopted software development methodologies: DevOps.
