Data Architecture Blog

Options
961
kegorman on 07-29-2020 11:42 AM
859
Darwin Schweitzer on 07-28-2020 09:20 PM
621
Buck Woody on 07-21-2020 06:26 AM
1,466
MichelleWallig on 07-14-2020 09:19 AM
1,365
Rodrigo Souza on 07-10-2020 10:05 AM
2,316
Buck Woody on 06-09-2020 07:09 AM
1,668
kegorman on 05-29-2020 10:06 AM
1,561
George Walters on 05-17-2020 08:40 AM
1,079
Rodrigo Souza on 04-30-2020 11:51 AM
944
Buck Woody on 04-14-2020 05:14 AM
1,056
kegorman on 04-10-2020 01:05 PM
776
Bradley Ball (MICROSOFT) on 03-24-2020 06:19 AM
593
Buck Woody on 03-24-2020 04:57 AM
1,982
Rodrigo Souza on 03-16-2020 09:49 AM
595
PhilHarvey on 03-10-2020 08:54 AM
923
Buck Woody on 03-10-2020 04:56 AM
1,697
James Herring on 03-09-2020 10:26 AM
1,655
George Walters on 02-22-2020 01:58 PM
736
James Herring on 02-21-2020 10:07 AM
4,037
kegorman on 02-20-2020 12:20 PM
3,082
Rodrigo Souza on 02-19-2020 09:05 AM
1,318
Buck Woody on 02-11-2020 09:53 AM
1,005
Buck Woody on 01-27-2020 04:30 AM
1,571
Buck Woody on 01-23-2020 10:50 AM

Latest Comments

All good points. I think when it comes to cost it is not a simple equation. For example if it is a really active OLTP application running extracts from it directly are probably off the table. An active readable secondary, replicated copy, or Change Data Capture of the tables needed for analysis are ...
0 Likes
@Darwin Schweitzer I don't think that a RDBMS is a real viable solution together with the SPARK. The reason is that the join you talk about is taking place in SPARK not in the database which means that you will have to load the full tables from the RDBMS into SPARK and most RDBMS are not designed to...
0 Likes
Hello @Darwin Schweitzer ! Thank you for you answer and feedback. Going to your question, I think that there are some decision points: How big is the data? Does it fit in your RDBMS? How much is it to scale your RDBMS? How big it can go? Data Lake is storage only, no processing costs to ingest data....
0 Likes
Rodrigo, seeing this late. Great blog post. I have always wondered about the best way to organize the data lake files. Also what are your thoughts on putting RDBMS data in csv format in the Data lake vs just landing it in a landing zone RDBMS. Then just join the RDBMS data and file-based data lake d...
0 Likes
Thanks for the comment Dennes. It is amazing what can be done with Power BI and an on-premise data gateway but by adding even just cloud storage like ADLS you can open up so much more (Data Lake, Data Mart, Data Warehouse, AI and Machine Learning, and Applications). Glad you found the references hel...
0 Likes