Read data from ADLS Gen2 and write data to Azure Analysis services

%3CLINGO-SUB%20id%3D%22lingo-sub-2279671%22%20slang%3D%22en-US%22%3ERead%20data%20from%20ADLS%20Gen2%20and%20write%20data%20to%20Azure%20Analysis%20services%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2279671%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Experts%2C%3C%2FP%3E%3CP%3EGood%20to%20meet%20you%20all%20!%3C%2FP%3E%3CP%3EThere%20are%20two%20requirement%20here%26nbsp%3B%3C%2FP%3E%3CP%3E1.Need%20to%20read%20data%20from%20ADLS%20Gen2%20and%20write%20data%20to%20Azure%20Analysis%20services.%3C%2FP%3E%3CP%3E2.Need%20to%20read%20data%20from%20Azure%20DW%20and%20write%20data%20to%20Azure%20Analysis%20services.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1.%20There%20are%20facts%20and%20dimension%20data%20available%20in%20ADLS%20Gen2%20which%20needs%20to%20be%20written%20to%20Azure%20Analysis%20services.%20Data%20is%20in%20Parqute%20format%20or%20with%20CSV%20format.%3CBR%20%2F%3EData%20would%20be%20available%20on%20daily%20base%20in%20ADLS%20Gen2.%3CBR%20%2F%3EHow%20to%20push%20data%20from%20ADLS%20Gen2%20to%20AAS%3F%3CBR%20%2F%3EIs%20it%20is%20possible%20only%20by%20using%20Visual%20studio%20%3F%20Is%20there%20any%20alternate%20way%20%2C%20if%20so%20could%20you%20please%20suggest%20the%20approach%3F%3CBR%20%2F%3EHere%20I%20can%20say%20that%20the%20data%20in%20facts%20and%20dimensions%20are%20being%20generated%20daily%20based%20on%20the%20input%20from%20another%20source%20system%2C%20from%20ADLS%20Gen2%20need%20to%20write%20data%20directly%20to%20AAS%2C%20so%20insert%20operation%20or%20Truncate%2Finsert%20operation%20here.%3CBR%20%2F%3EHow%20much%20data%20can%20be%20pushed%20at%20a%20time%20from%20Gen2%20to%20AAS%3F%3CBR%20%2F%3EThere%20are%20around%2025million%20rows%20in%20ADLS%20Gen2%20how%20to%20push%20those%20data%20in%20the%20100K%20interval%20to%20AAS(Azure%20Analysis%20services).%20Should%20we%20use%20ADF%20here%20%3F%26nbsp%3B%3C%2FP%3E%3CP%3E%3CBR%20%2F%3ECan%20Powershell%20scripts%20be%20used%20to%20read%20data%20from%20ADLS%20Gen2%20to%20push%20data%20to%20AAS(Azure%20Analysis%20services)%3F%3C%2FP%3E%3CP%3EAlso%20could%20you%20please%20suggest%20what%20ways%2F%20approaches%20we%20can%20achieve%20this%20%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E2.%20One%20more%20thing%20is%20the%20Data%20is%20in%20Azure%20DW%20%2C%20how%20to%20push%20those%20data%20into%20AAS%3F%20What%20is%20the%20best%20way%20to%20do%20%3F%3C%2FP%3E%3CP%3E%26nbsp%3BIs%20ADF%20only%20option%20for%20me%20to%20push%20data%20from%20Azure%20DW%20to%20AAS%3F%20or%20can%20i%20use%20ADB%20or%20Powershell%20script%20or%20any%20other%20approach%20here%20to%20achieve%20my%20goal%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThank%20you%20in%20advance%20for%20your%20valuable%20time%20and%20suggestions.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks!%3C%2FP%3E%3CP%3EYuvi%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Regular Visitor

Hi Experts,

Good to meet you all !

There are two requirement here 

1.Need to read data from ADLS Gen2 and write data to Azure Analysis services.

2.Need to read data from Azure DW and write data to Azure Analysis services.

 

1. There are facts and dimension data available in ADLS Gen2 which needs to be written to Azure Analysis services. Data is in Parqute format or with CSV format.
Data would be available on daily base in ADLS Gen2.
How to push data from ADLS Gen2 to AAS?
Is it is possible only by using Visual studio ? Is there any alternate way , if so could you please suggest the approach?
Here I can say that the data in facts and dimensions are being generated daily based on the input from another source system, from ADLS Gen2 need to write data directly to AAS, so insert operation or Truncate/insert operation here.
How much data can be pushed at a time from Gen2 to AAS?
There are around 25million rows in ADLS Gen2 how to push those data in the 100K interval to AAS(Azure Analysis services). Should we use ADF here ? 


Can Powershell scripts be used to read data from ADLS Gen2 to push data to AAS(Azure Analysis services)?

Also could you please suggest what ways/ approaches we can achieve this ?

 

2. One more thing is the Data is in Azure DW , how to push those data into AAS? What is the best way to do ?

 Is ADF only option for me to push data from Azure DW to AAS? or can i use ADB or Powershell script or any other approach here to achieve my goal

 

Thank you in advance for your valuable time and suggestions. 

 

Thanks!

Yuvi

 

0 Replies