Forum Discussion
How to build lake data warehouse
Hi,
We need to build new data warehouse. I am hearing multiple comments on this.
We have very little budgets on Azure.
1. We have Azure elastic pool with 1 TB storage
2. D365 is our main application whose data is pushed to ADLS Gen2 using synapse link using dataverse
3. We have many other external applications(third party) which we are importing into elastic pool using SSIS running on Azure VM.
Options being MS suggests to use Modern DWH where we don't need to store the data in sql databases. We are able to query Datalake using Synapse serverless pool but the performance is really bad.
Moreover, we need to transform(apply various business logics) data after querying from datalake and also need to join data from external applications and then sql views are created to feed thru PBI reports.
In serverless pool, how can we apply business logic? I mean stored procedures/complex logic...
Is it possible to use Serverless pool to build our data warehouse?
We are really struck here and not able to argue with many commenters.
Please suggest an idea