SOLVED

-- Microsoft Azure Storage Explorer || Private Endpoints on ADLS Gen2 --

Brass Contributor

I have provided access to my ADLS Gen2 through ACL.

My users have at least the ACL r-x on the filesystem and on the subsfolders or files when need access to.

From Home Office (through VPN) and using the client (MASE) "Microsoft Azure Storage Explorer"

  1. When the Public Ip of the users is whitelisted the client MASE (Microsoft Azure Storage Explorer) can access the ADLS Storage Account.
  2. When using Private Endpoints (tried 'dfs' and 'blob') I got the following error :
     
    image.png

     

15 Replies
As a last chance I have created a private endpoint for each Id Group of Storage Account resources : dfs, web, blob, table, queue and file but the result is the same, the Authentication step goes through Internet and ask for the user to have the list containers or list account keys privilege. In my ADLS ACL context I just can't just give those privileges.

@Jamesdld   I checked on this and while it appears that Storage Explorer through the Azure portal works as expected with private endpoints, the client tool didn't.  I suspect that Azure Storage Explorer uses legacy Blob APIs to enumerate / list contents of storage accounts.  I was able to get Storage Explorer client working (from a machine within the VNET) by creating a private endpoint and private DNS zone for Blob access.

Hi Roy

 

I confirm that connecting to blob Storage Accounts works fine.

 

My issue is connecting to a Storage Account that is enabled for ADLS Gen 2, the target Subresources are blob and dfs.

 

Can you confirm the issue? It occurs with Private Endpoints when you try to connect with a user that has ACL read and execute ACL.

 

Thank you,

James

@Jamesdld It may be useful to test by connecting with an account with storage account owner permissions

Hi

please find below the test results.

image.png

----

----image.png

----

----

 

image.png

@Jamesdld Did you validate whether access through the Azure Portal Storage Explorer works in either case? (browsing from a machine on the same VNET as the private endpoint).  Also notice that your errors on Test 1 and Test 2 are different - check the last part of this article (although it is written for Databricks/Spark) as well around reviewing ADLS ACLs : https://deep.data.blog/category/azure-data-lake-storage-gen-2/

 

Hi back and thank you for checking everything but concerning the ACL I am sure it works, as revealed in the following screenshot the connection with same user with the upper mentioned privileges works when connecting through Internet. 

I can test from a VM which is the same vnet than my private endpoint, I will do that tomorrow.

 

image.png

Hi Roy,

 

I just did the test from a VM located in the same vnet than my private endpoints, the result is exactly the same.

 

James

@Jamesdld Did you try it from Storage Explorer WITHIN the account blade of the Azure portal to rule out issues with the Azure Storage Explorer tool? If this still is problematic try raising a call through support @https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/

Having an issue from the portal too with private endpoint, I did raise the support ticket number 120041722000372.

Just for info, I did overwrite the main dns record, not only the one recommended with ".private." and it worked for both users: the one that is owner and the one that has ACL privileges. This is definitely not a good an option but the result is interesting, I will share that with the support team.

I just ended a call with Garrett Curtin from Microsoft Storage support team (ticket id 120041722000372) and unfortunately "Storage Explorer" doesn’t support using privatelink dns zones as recommended here. The workaround is to overwrite the public dns record like "mysa.blob.core.windows.net" and not the privatelink one like: "mysa.privatelink.blob.core.windows.net"

 

A feature request was made here to make Storage Explorer support this.

I am still investigating here for info, my privatelink dns zone is hosted on On-Premises dns servers because my client are in my On-Premises network, I will dig this area, maybe its a ttl misconfiguration or something like that...

 

Using a DNS forwarder VM could be a solution but I would have preferred a solution with my current infra services. url: https://github.com/Azure/azure-quickstart-templates/tree/master/301-dns-forwarder/


There is here an interesting feature request : "Simplify Private Endpoint DNS resolution from on-premises". url: https://feedback.azure.com/forums/34192--general-feedback/suggestions/39697135-simplify-private-endp...

best response confirmed by Jamesdld (Brass Contributor)
Solution

I just understood and solved my issue, was all about DNS resolution, adding below some explanation.

 

The need:

End users need to connect to PaaS services from home through VPN or from On-Premises private networks through their Private Endpoints IPs.

 

Overview of the solution:

Forward DNS request to a DNS VM proxy located on Azure.

The DNS VM proxy is in a vnet that has a link with your Azure private DNS zones hosting the "privatelink" recommended DNS zones.

 

Detail of the solution:

 

  1. Create the Azure private dns zone “privatelink.blob.core.windows.net” with the DNS A record “mystoragename.privatelink.blob.core.windows.net” that returns the IP of my Storage Account private endpoint’s IP.
  2. Use a VM DNS proxy, this VM’s vNet is linked to the upper mentioned Azure private dns zone. This DNS VM forwards DNS request to Azure DNS IP 168.63.129.16.
    • Here, a solution could be to use Azure AD DS as the DNS proxy, it’s natively configured to forward dns request to 168.63.129.16.
  3. Use a DNS conditional forwarder on the On-Premises DNS servers to forward dns requests for the DNS zone “mystoragename.blob.core.windows.net” to the DNS proxy on Azure (in my context, to the 2 private Ips of my Azure AD DS service).

 

Feature request:

A feature request has been published here to simplify Private Endpoint DNS resolution from On-Premises.

1 best response

Accepted Solutions
best response confirmed by Jamesdld (Brass Contributor)
Solution

I just understood and solved my issue, was all about DNS resolution, adding below some explanation.

 

The need:

End users need to connect to PaaS services from home through VPN or from On-Premises private networks through their Private Endpoints IPs.

 

Overview of the solution:

Forward DNS request to a DNS VM proxy located on Azure.

The DNS VM proxy is in a vnet that has a link with your Azure private DNS zones hosting the "privatelink" recommended DNS zones.

 

Detail of the solution:

 

  1. Create the Azure private dns zone “privatelink.blob.core.windows.net” with the DNS A record “mystoragename.privatelink.blob.core.windows.net” that returns the IP of my Storage Account private endpoint’s IP.
  2. Use a VM DNS proxy, this VM’s vNet is linked to the upper mentioned Azure private dns zone. This DNS VM forwards DNS request to Azure DNS IP 168.63.129.16.
    • Here, a solution could be to use Azure AD DS as the DNS proxy, it’s natively configured to forward dns request to 168.63.129.16.
  3. Use a DNS conditional forwarder on the On-Premises DNS servers to forward dns requests for the DNS zone “mystoragename.blob.core.windows.net” to the DNS proxy on Azure (in my context, to the 2 private Ips of my Azure AD DS service).

 

Feature request:

A feature request has been published here to simplify Private Endpoint DNS resolution from On-Premises.

View solution in original post