how to configure process of continuous export of data from Azure Data Explorer to Azure Data Lake v1

[alexander tikhomirov]

Hi

Is is possible to configure continuous export of data from Azure Data Explorer to Azure Data Lake v1 in background?

For example 

I have created table

.create external table ExternalTableADL01 (name:string, age:int, [date]:datetime)

kind=adl

dataformat=csv

(

h@'adl://xxxxx.azuredatalakestore.net/folder;impersonate'

)

.create-or-alter continuous-export ContinuousExportDemo01

to table ExternalTableADL01

with

(intervalBetweenRuns=5m

)

<| TestCursor

But I have error

An admin command cannot be executed due to an invalid state: State='External table 'ExternalTableADL01' cannot be used for continuous export as it uses impersonate authentication type'

based on official docs there is only one way 

https://docs.microsoft.com/en-us/azure/data-explorer/kusto/api/connection-strings/storage

Append ;token=AadToken to the URI, with AadToken being a base-64 encoded AAD access token (make sure the token is for the resource https://management.azure.com/)

but it is kind of interactive, because token is expiring.

[#2 · Accepted solution]

I have created ticket with MS support and having answer:

I have reached our engineering team and you are correct - this scenario is currently not yet supported. Very few customers use ADLS gen1, so there has been very little demand for feature this so far. ADX does not support refresh tokens, so we cannot renew the token OBO of the service principal. I have sent your feedback on the need of this feature to the engineering team – and for long term (unfortunately no ETA right now), we plan to support using managed identities in continuous export, and then this will be supported.

In the meanwhile, our suggestion is to move to ADLS gen2, where you can configure the export with an account key (or a very long living SAS key).

[#3]

Thanks for updating the forum Alexander. This is accurate.