Private Endpoint to Azure Blob Storage from On-Premise

[sc2317]

Hi,

I am working on solution to implement private endpoint for traffic from on-premise(DNS solution already in place) to Azure Blob Storage solution. I have referred below link(On-premises workloads using a DNS forwarder) from Microsoft which gives an overview of how we can configure. However, I have some queries so can someone please suggest.

https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-dns#azure-services-dns-zone-con... 

1) In this article, private endpoint is created in hub VNET with VPN/Expressroute connection, but is it mandatory or can we create private endpoint in other Spoke VNET with peering to hub VNET ?

2) Is it mandatory to setup Conditional DNS forwarder in Azure or can we achieve the required solution by just creating new DNS zone(blob.core.windows.net) in on-premise DNS server and create A-record pointing to private endpoint IP ?

3) If its possible(As mentioned in 2nd point) to create required solution using DNS zone in on-premise DNS server, then do we still need Private DNS Zore linked to VNET ?

4) If conditional forwarder(blob.core.windows.net) is mandatory(for DNS solution already in place), then would there be any issues with users already using other storage accounts(With no Private endpoint) on public endpoint because those storage accounts will also have name "storageaccountname.blob.core.windows.net" ? I believe there will be no issues, because when query will go to internal DNS server(If users are connected on office VPN) then DNS forwarder in Azure will forward query to Microsoft IP 168.63.129.16 which will resolve it to storage accounts public IP.

[#2]

Hi @sc2317 ,

ad 1) you can create Private Endpoints in Spoke VNets (creating them in a Hub is a pattern one could consider only for shared resources that are "consumed" from several Spokes. Azure Landing Zone methodology recommends creating them in spokes (landing zones). As long as the routing is configured correctly (prefixes from your spokes are advertised to on-prem), you are good to go. There is however a complex topic that needs to be addressed - hybrid DNS. This article explains it well.

ad 2) you could in theory create such zone in your on-prem DNS, but the maintenance of records will become a manual (tedious) process, which doesn't scale. Imagine creating new records in this way every time someone enables a Private endpoint in your Azure tenant. With Conditional forwarding, you configure it once and it works regardless of how many private endpoints will be created.

ad 3) Private DNS zones have two key benefits: it is a managed service (you don't need to host any infrastructure) and the lifecycle management of records (record sets) is done via ARM, so you can reuse the same tool you have for Infrastructure as Code (JSON ARM, Bicep, Terraform, PowerShell) also for your DNS configuration. Also, your Azure resources (e.g., VMs in spokes) wouldn't be able to resolve those private endpoints correctly (unless you extend your on-prem DNS infrastructure with that "new zone" to Azure VNets = configure custom DNS to use your servers instead of Azure DNS).

ad 4) The combination of conditional forwarding for "blob.core.windows.net" (in your on-prem DNS) + DNS resolvers in Azure Hub VNet + Private DNS zone for Azure Blob (linked to the Hub VNet) + Spoke VNets configured to use 'custom DNS' (pointing to those Hub Resolvers) will ensure that you can resolve correctly both storage accounts with private endpoint enabled as well as those with a public endpoint only. 

As a final note, I believe that the most scalable design for Hybrid DNS is described in these two articles: DNS for on-premises and Azure - Cloud Adoption Framework | Microsoft Docs and Private Link and DNS integration at scale - Cloud Adoption Framework | Microsoft Docs.

I hope it helps.

[#3]

@David Pazdera 

Many thanks for your response, it has clarified many points along with best practices to be used for this kind of solution. However, I have few more further queries after reading the relevant documents.
1) This link says that, If the DNS forwarders in Azure are not authoritative for customer's corporate domains (for example, Active Directory domain names), they should have conditional forwarders for the customer's corporate domains, pointing to the on-premises DNS Servers. Can you please confirm if it is mandatory to make the solution work ?

2) Is it required to put, DNS forwarders in Azure in AD domain ? or can we just keep it in workgroup ?

3) Documentation says to create private DNS zones in central subscription where hub vnet will be deployed ? Is it best practice or is it good to use separate private DNS zone linked to spoke vnet in which private endpoint will be deployed ?

[#4 · Accepted solution]

Hi @sc2317 ,

I was on holidays from 20th December, so I couldn't respond earlier.

ad 1) in a scenario, where your on-prem DNS zone is not AD-integrated DNS (hosted on domain controllers), then it is correct you need to setup a conditional forwarding rule on your DNS Forwarders in Azure for those zones, otherwise your Azure VMs and other resources won't be able to resolve those names. Of course, you also need to make sure that DNS traffic from the forwarders to your on-prem is not blocked by a firewall.

ad 2) again, if your on-prem DNS is AD-integrated, the simplest solution is to extend your on-prem AD to Azure, where you can setup one or several replica domain controllers on Azure VMs. Those DCs can become your DNS Forwarders, able to resolve on-prem zones as well as Azure Private DNS zones. If your on-prem DNS is not hosted on Windows Servers (let's say you use BIND), your DNS Forwarders on Azure could be anything. I've seen a minimalistic Linux distro configured with a DNS forwarding for that purpose. You might be able to use a standalone Windows Server with DNS server role as well (I haven't seen this scenario in practice).

ad 3) It is definitely a recommended practice to centralize this configuration and both provision all Private Link DNS zones in a Hub subscription as well as link those zones with Hub VNets. Since your Spoke VNets need to be configured with Custom DNS pointing to your DNS Forwarders in the Hub, any links between Spoke VNets and DNS zones will be ignored!

[#5]

Hi David,
Many thanks for your inputs, Once again, thanks for your time to explain this and making me understand this whole configuration in best possible way.

[#6]

Thanks for your feedback, Sumit. I am happy I could help. Have a great day going forward :)

[#7]

Hi David,
While implementing the configuration, I came across one more setting that raises doubt.
Lets say, If I implement the solution with DNS (Not AD Integrated) forwarders in Azure VNET and private endpoint in same VNET linked to private dns zone "privatelink.blob.core.windows.net". I assume everything will work with respect to private endpoint. However, I am not sure about what should I do for following:
1) What DNS suffix should I use for DNS forwarders in Azure. Should I leave it to default ?
2) What DNS servers should I use on VNET ? Should I leave it to default or use on-premise DNS as application in on-premise will use this private endpoint.

[#8]

Hello,

If apart from enabling Private Endpoints you also want to enable cross-premises DNS resolution (being able to resolve Private DNS zones from on-prem / retrieve private endpoint IP addresses as well as resolving on-prem hosted DNS zones from Azure), this is what you should do:

• Your "Spoke VNets" (peered to the Hub, where you host VPN Gateway, DNS Forwarders and Private DNS zones) should be configured with 'Custom DNS' option (and not the 'Azure-provided' default) and point to those two DNS Forwarders VMs you host in the Hub. In this way, your Azure resources in Spokes can resolve both Azure Private zones and on-prem DNS zones.
• Your DNS forwarders should have a DNS "role", where you configure Conditional forwarding for your on-premises zones, pointing to your on-prem DNS servers, authoritative for those zones. You need to make sure that those Forwarders can reach your DNS servers (e.g., firwall restrictions, routing).
• In order to resolve Azure Private DNS zones from on-prem, your DNS servers needs to be configured with Conditional forwarding, where for 'blob.core.windows.net' zone queries you point to those DNS Forwarders in the Hub (using IP addresses). Obviously, when you want to leverage other Private Endpoints, e.g., Azure SQL Database, you will add respective zone to conditional forwarding and create a new private DNS zone in the Hub for that service.
• I don't think that the configuration of DNS suffixes on the Forwarders matters. You will be referring to them using IP addresses in the configuration.

Let me know, if you have more questions.

[#9]

Hi David,
That's again perfect explanation. Your first and last point explains what I was looking for. Now, one last question how do I allow my on-premise resources to access storage account(No Public Access) over private endpoint. Do I only have to allow VNET (With VPN Gateway Connection to On-Premise) on storage account or do I need to allow public IP of my VPN device ?

[#10]

Great question :)
So, the beauty of Private Endpoints is that they are enabled inside your VNets, meaning IP ranges you choose and control. If you build a cross-premises connectivity - either a S2S VPN or Express Route - those Azure VNet prefixes will be advertised to your on-prem network. If you configure name resolution correctly, it should work automatically:

• any DNS query to blob.core.windows.net domain (the one from your example, let's assume you configured forwarding for this domain in your on-prem DNS servers) would resolve to a private IP address that was assigned in Azure VNet to that private endpoint.
• your on-prem router / VPN gateway will route this traffic to Azure, where it should reach the correct VNet and IP (network card) and through Private Link it would get to the storage account.
• the response would follow the same path back

Private Endpoints allow you to build this connectivity to Azure PaaS services without a need to use public IP addresses for those services. You can't remove those public endpoints but you could configure IP filtering in the storage account configuration to block any traffic from the Internet, effectively disabling that public endpoint. Eventually, you could enable only some source IP prefixes.

You don't need to allow VNet(s) on storage account, since this configuration is related to Service Endpoints, which is a different capability (works only on Azure, not from on-prem).

[#11]

Perfect, many thanks :)