In August and September of 2023, I published two blogs in this Tech Community about Metadata Driven Pipelines in Fabric, Metadata Driven Pipelines for Microsoft Fabric and Metadata Driven Pipelines for Microsoft Fabric Part 2, Data Warehouse Style. The response from the community has been amazing and I have received numerous requests for the source code so people can deploy within their own Fabric tenants. The difficulty is that Git support for Fabric at this time is limited, and there are no ARM templates to deploy in like there is in Azure.
To meet these requests, I have created step-by-step instructions for these 2 patterns in our Analytics in a Box GitHub Repo. While you will be able to upload the Spark Notebooks as well as upload and run the SQL scripts used in this solution, you will follow the instructions for creating the pipelines and pipeline activities. I have tried to facilitate this by providing you with all the configurations for each pipeline and activity that you can cut and paste into each pipeline or activity setting. While this may be a bit of a laborious effort, I believe you will actually learn more about building Metadata Driven Fabric Pipelines then if you truly could just deploy from out of the box.
If you're not quite ready for SaaS for your data analytics, check out the available patterns for Synapse in the same GitHub repo!
