Forum Discussion
Best solution to improve performance of enterprise scale data
Hi Team,
Calling Microsoft Fabric Experts! Need Your Guidance!
Before the introduction of Microsoft Fabric, we relied on Dataflows Gen2 to connect to SQL sources, pull data, and transform it. For context, Dataflows sit inside workspaces and not within any Lakehouse/Data Warehouse. We then used these Dataflows as the source for our Power BI reports. However, with enterprise-scale data, this approach sometimes slowed down report performance.
Enter Fabric's OneLake
With OneLake storing data in Parquet and Delta table formats, performance can be significantly improved. I’m exploring the best way to leverage this, and I’d love your insights!
Here are my two thoughts:
Option 1:
- Create a Lakehouse.
- Bring in the existing Dataflows to create Delta tables.
- Use these Delta tables as the source for Power BI reports.
Will this improve Power BI report performance?
Option 2:
- Create a Lakehouse.
- Build a Pipeline within the Lakehouse to pull data directly from SQL.
- Use this transformed data as the source for BI reports.
What’s your take?
Your suggestions aren’t just helpful for me but could benefit many in the community. I truly appreciate the time and effort from those who love sharing their knowledge.
Looking forward to hearing from the amazing experts out there!
Thank you!
1 Reply
How about Using Existing Dataflows to Create Delta Tables
- Create a Lakehouse.
- Import Existing Dataflows to Create Delta Tables.
- Use These Delta Tables as the Source for Power BI Reports.
This option can improve report performance because:
- Delta Tables: Delta tables in OneLake can optimize read and write operations, reducing latency and improving query performance.
- Centralized Data: Consolidating dataflows into a Lakehouse ensures a more streamlined data management process.