SOLVED

how to configure process of continuous export of data from Azure Data Explorer to Azure Data Lake v1

Contributor

Hi

Is is possible to configure continuous export of data from Azure Data Explorer to Azure Data Lake v1 in background?

 

For example 

 

I have created table

 

.create external table ExternalTableADL01 (name:string, age:int, [date]:datetime)
kind=adl
dataformat=csv
(
h@'adl://xxxxx.azuredatalakestore.net/folder;impersonate'
)

 

.create-or-alter continuous-export ContinuousExportDemo01
to table ExternalTableADL01
with
(intervalBetweenRuns=5m
)
<| TestCursor
 
But I have error
An admin command cannot be executed due to an invalid state: State='External table 'ExternalTableADL01' cannot be used for continuous export as it uses impersonate authentication type'
 
based on official docs there is only one way 
 
Append ;token=AadToken to the URI, with AadToken being a base-64 encoded AAD access token (make sure the token is for the resource https://management.azure.com/)
 
but it is kind of interactive, because token is expiring.
 
 
 
2 Replies
best response confirmed by alexander tikhomirov (Contributor)
Solution
I have created ticket with MS support and having answer:

I have reached our engineering team and you are correct - this scenario is currently not yet supported. Very few customers use ADLS gen1, so there has been very little demand for feature this so far. ADX does not support refresh tokens, so we cannot renew the token OBO of the service principal. I have sent your feedback on the need of this feature to the engineering team – and for long term (unfortunately no ETA right now), we plan to support using managed identities in continuous export, and then this will be supported.

In the meanwhile, our suggestion is to move to ADLS gen2, where you can configure the export with an account key (or a very long living SAS key).
Thanks for updating the forum Alexander. This is accurate.