Forum Discussion

aditya8003's avatar
aditya8003
Copper Contributor
Jan 07, 2022

Designing system to enable Adhoc queries

Hi, we are designing a data processing system in which the data goes through three different stages as shown below. What azure platforms or technologies do you recommend for a dynamic scenario like the one below where the input file format can change all the time, the transformations applied are not standard and the reports generated vary every time?  

Extract

  • Data size can be around 1 GB. Can be of various formats and from various sources like FTP, API etc.

Transform

  • Transformations are applied on the data.

Results

  • After the transformations, results are exported to a final report table from which reports are generated.

Resources