I am currently working on a data analytics project and have the challenge of handling time series data within azure.
We have several thousand sensors, which report their values to a control unit, which writes this data in form of json files into an Azue Data Lake.
From what I've learnt about Data Factory so far, there are different ways to do that: 1) Use HD Insights I see this as pretty much work to setup and get familiar with it. Therefore looking for an easier option
2) Use a U-SQL Query with C#/Python or R As far as I understand, this is only possible via Data Lake analytics, thus only a Data Lake is a possible input source, correct?
3) Create custom activity This is only in Data Factory v2 available, which is currently in preview, not available in our favorite location and the integration into Visual Studio 2017 is almost 0. Moreover it is a pretty complex setup alltogehter with the batch processing.
My question now is: Is there any other possible setup you would suggest I did not see? And if not, which of the solutions you would suggest?