As data grows exponentially and becomes more complex, data preparation and transformation are increasingly important for accurate analysis and reduced time to insights.
In October, we announced the general availability of Mapping Data Flows for code-free data transformation in the cloud. Today we’re adding new capabilities for data preparation, with the public preview of Wrangling Data Flows in Azure Data Factory (ADF), our productive and trusted hybrid integration service. Data Factory now empowers users with a code-free, serverless environment that simplifies data preparation in the cloud and scales to any data size, no infrastructure management required.
Built to handle all the complexities and scale challenges of big data integration, Wrangling Data Flows leverages the industry leading Power Query data preparation technology (also used in Power Platform dataflows, Excel, and Power BI) to seamlessly prepare and shape data at scale. Build resilient data pipelines in an accessible visual environment with our browser-based designer and let ADF handle the complexities of Spark execution.
Wrangling Data Flows allow data engineers to enrich, shape, and publish data in a scalable manner that dramatically improves productivity. Data Engineers can now fix errors quickly, ensure data standardization, and surface high quality data to inform business decisions. With Wrangling Data Flows, customers like OMERS (Ontario Municipal Employees Retirement System) are empowering their employees to prepare data seamlessly, regardless of data complexity or the coding skills of their teams.
Wrangling Data Flows will be instrumental in enhancing OMERS data & analytics capabilities by enabling data preparation in a user-friendly and code-free environment and allowing us to streamline the deployment of high-quality data assets to support business decision makers.
Simon Nuss, Director of Data & Analytics, OMERS
Accelerate time to insights by focusing on building your business logic without worrying about managing and maintaining server clusters or writing code to build pipelines. Quickly design, build, and manage data flows that execute at scale with the performance you need to prepare your data for downstream analytics. Use built-in wrangling functions including column management, row filters, merging and joining tables, group by, sorting, and reducing rows, among many others.
Our code-free, browser-based designer provides an accessible visual environment to build complex data preparation workflows using a familiar excel-based grid environment. As you build your logical graphs, validate in real-time using live data preview capability which shows the data after each data prep step.
Finally, build pipelines and debug your new ETL process end-to-end using the drag and drop pipeline builder with interactive debugging.
Build schedules for your pipelines and monitor your data flow executions from the ADF monitoring portal. Easily manage data availability SLAs with ADF’s rich availability monitoring and alerts and leverage built-in CI/CD capabilities to save and manage your flows in a managed Data-ops environment. And establish alerts and view execution plans to validate that your logic is performing as planned as you tune your data flows.
Wrangling Data Flows now enables organizations to use proven, industry leading Power Query data preparation technology to seamlessly enrich and shape their data, and make data integration and preparation faster, easier, and accessible to everyone.
Learn more and get started today using ADF with Wrangling Data Flows.
Microsoft