This post was authored by Kyle Hale, a Solutions Architect at Databricks.
If you’re a BI practitioner, this diagram (and others like it) may be intimately familiar to you:
The “BI stack” is a conceptual framework for describing the paths and processes data takes from its creation to its consumption for analytical insights. In this view, each layer in the stack provides a specific set of capabilities to manage, enrich, or consume data as it moves its way up through each layer:
Over the years, numerous analytics platforms and products have come, rearranging, optimizing, simplifying, and in some cases even eliminating the various layers of this stack.
This blog is here to offer at least one more new and compelling entrant to candidates for a reference architecture to meet these capabilities - a model I’m calling the semantic lakehouse. It’s a surprisingly simple one, … no points for guessing if you read the blog title :)
Let’s look at how these two best-in-class tools (check the receipts) combine to form a great modern BI stack!
And in addition to the benefits of these powerful and valuable capabilities in and of themselves, the lakehouse vision also proposes to replace the granddaddy of all analytics products: the data warehouse.
After all, the lakehouse:
… and after
When you throw in:
The lakehouse is an exciting place to be!
So we’ve seen the lakehouse vision is powerful, and when combined with popular BI tools, it becomes even more powerful. At the serving and presentation layers, capabilities such as semantic modeling for shared business logic and relevant metrics; in-memory caching for extreme performance around your most important insights; a powerful visualization designer; and collaboration and sharing of analytics assets to build a data-driven culture can still provide a ton of value to data analysts and end users.
Luckily, today Power BI not only provides all of these capabilities out of the box, it has strong native integration with Azure Databricks and even has a few “killer features” that can really take your lakehouse all the way to the top of the stack.
Taking a step back, the lakehouse is more of an “idea whose time has come” than a particularly surprising solution. The continued evolution of cheap object storage, data-friendly storage formats, easily scalable on-demand compute, distributed analytics processing engines, and - most importantly - consumption-based cloud economics meant it was only a matter of time before the “BI stack” was simplified.
Best-in-class BI tools extend Azure Databricks’ lakehouse strengths up to the users at the “top of the stack”, and Power BI’s integration with Azure Databricks and complementary feature set make the two a perfect combination for delivering the modern BI stack in your organization in the form of a semantic lakehouse.
Watch this Microsoft DevRadio episode to learn more about using Power BI with Azure Databricks and to see these concepts in action.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.