Blog Post

Apps on Azure Blog
9 MIN READ

Hide Your Public Connection From PaaS To Storage Account

theringe's avatar
theringe
Icon for Microsoft rankMicrosoft
May 03, 2022

Using PaaS resources in Azure brings convenience to developers such as saving times, concentrating on coding, and etc. However, there is a paradox which convenience and secure might never on the same direction.

 

Your customers might challenging you by exposing the inter-connection among resources under public networks. For example, You builded a Function App for tracking customers' information and storing them in to a Blob container (from a Storage Account). Meanwhile, connections invoked by that function uses public domain name to the container, which means those requests might have a great opportunity routing outside.

 

For security purpose such as ISO/IEC 27001, you may want your resources to co-op privately, however, detail network settings among PaaS services are extreamly hard to find (at least for me, a pure newbie). So this article is my research for connecting PaaS devices such as Azure Function App to a Blob container under private network (and cross region).

 

Here is the outline, I will inreoduce 2 methods: Service Endpoints and Private Endpoints, help you to protect your connection.

 

Outline

 

  • Service Endpoints
  • Private Endpoints
  • Additional Information
  • Conclusion
  • References

 

Service Endpoints

A quick conclusion: We cannot perform a cross-region private connection under a Service Endpoints settings.

 

Service Endpoints is a combination of settings related to "route for the inter-resource connections by avoiding expose them under public networks". For example, a function app invoking an outbound request to a blob container under a domain named "someblobs.blob.core.windows.net", then, a DNS server resolve that name to an IP address of that container, and you all know the rest of the things.

 

The point is, "someblobs.blob.core.windows.net" via resolvable in Global DNS server and you will retrieve a public IP address from that, even thought you can directly reach to that resource through private network. In some special cases under the function app and blob container are located in the different regions, you may not have a chance to achieve such a purpose.

 

If you don't mind, studying from this part is a goot way to start. Let's go to the Azure portal and create a Resource Group.

 

 

Column Name Value
Resource group <you specified>
Region <you specified> e.g., Japan East

 

Create a Log Analytics Workspaces inside that Resource Group.

 

 

Column Name Value
Name <you specified>
Region <you specified> e.g., Japan East

 

 

Create 2 Storage Accounts, one of them in the region is the same with the Resource Group, the other is different from it.

 

 

Column Name Value
Storage Account Name <you specified>
Region <you specified> e.g., Japan East / West US

 

Inside 2 Storage Accounts, create a Blob container from each of them.

 

 

Column Name Value
Name <you specified>
Public Access Level Blob

 

Inside 2 Blobs, upload a test file to each of them, you can create a file named "test" and write some message in it, and upload through Azure portal.

 

 

Last step of the Storage Account, go to the Diagnostic Settings and add a setting on it. We use it to determine the behavior to the blob such as information (source IP) from the requests.
Left side manu > Monitoring > Diagnostic Settings > Blob (under your Storage Account Name) > Add diagnostic setting

 

 

Column Name Value
Diagnostic Settings Name <you specified>
Categories StorageRead
Destination Details Send to Log Analytics workspace
Log Analytics Workspaces <you specified>

 

Now we start creating our Function App. We are using Functions Premium plan because it supports Virtual Network Integration, thus, please ensure to remove all the resources once you're finished testing.

 

 

Column Name Value
Resource Group <you specified>
Function App Name <you specified>
Publish Code
Runtime Stack Node.js
Version 14 LTS
Region Japan Esat
Operating System Windows
Plan Type Functions Premium
Windows Plan <you specified>
Sku and Size EP1
Storage Account <you specified> please do not use your previous created ones
Enable network injection Off
Enable Application Insights No

 

Please download the sample codes from my Github, consider the readme file and make some changes to fit your experiment. Finally deploy it on your Function App.

 

 

 

 

 

 

 

 

 

git clone https://github.com/theringe/AzureSupportLabs.git --branch lab01
cd AzureSupportLabs
# consider the readme file and make some changes
zip -r --symlinks AzureSupportLabs.zip .
az functionapp deploy --resource-group <you specified> --name <you specified> --src-path AzureSupportLabs.zip

 

 

 

 

 

 

 

 

Now by accessing the function triggers (there are 2) via the following URLs, they may invoke a GET request to the blob file from the container:

https://0502lsecpriendplfuncapp.azurewebsites.net/track_a  (Function App and Blob are in the same region)
https://0502lsecpriendplfuncapp.azurewebsites.net/track_b  (Function App and Blob are in different regions)

 

You will get an OK message once you click it, and can close it after that.

 

Now we're going to the Log Analytics workspace.
Left side manu > General > Logs
Please input the following KQL in the Query Zone

 

 

 

 

 

 

 

 

StorageBlobLogs
| where AccountName contains "0502lsecpriendplstorage"
| where OperationName == 'GetBlob'
| where Uri contains "test"
| sort by TimeGenerated desc
| take 10
| project TimeGenerated, AccountName, CallerIpAddress, OperationName

 

 

 

 

 

 

 

 

 

You can see the line with AccountName 0502lsecpriendplstoragea which Function App and Blob are in the same region, the Caller IP Address is 100.79.52.132, which is an interal IP address[5].
And the line with AccountName 0502lsecpriendplstorageb which Function App and Blob are in different regions, the Caller IP Address is 20.210.7.31, whioh can be found on outbound IP section from your Function App.

 

It means you are exposing the connection in public when a cross-region inter-connection be invoked, and might get the potencial risk from your system.

 

 

Now we're going to create a Virtual Network for our Function App, to verify whether a Function App inside a VNet can reach the containers under private network or not.

 

 

Column Name Value
Name <you specified>
Region <you specified> e.g., Japan East

 

And then we immediatly specify some subnet range for our Function APP
Left side manu > Settings > Subnets > Add subnet

 

 

Column Name Value
Name <you specified>
Subnet Access Range 10.1.1.0/24 (i.e., it could be different on yours based on your default suubnet ranges)
All the rest use the default value

 

Now we're going back to our Function App and bind it on the VNet
Left side manu > Settings > Networking > Outbound Traffic > VNet integration > Add VNet > Provice the information you've just created

 

 

In the rest of this part, we gonna make a service endpoint by specifying a VNet from both of your Storage Accounts.
Left side manu > Security + Networking > Networking > Firewalls and Virtual Networks

 

Column Name Value
Public network access Enabled from selected virtual networks and IP addresses
Virtual networks Add Existing Virtual Network
Virtual Netework <you specified>
subnet default / <you specified> (i.e., yes, add them separately)
All the rest use the default value

 

And please wait for nearly 15 minutes for populating network settings inside Azure.

 

 

Maybe you are realizing what's going on, you cannot specify the VNet from a Storage Account if they are on different regions, so we temporally stop here.

 

Since you've just specify the Function App (Japan East) inside a VNet (Japan East) corresponding to a Storage Account (Japan East), we're going to check the result by re-invoke a connection

 

PS: You can restart the Function App before invoking, and then click this link:

 

https://0502lsecpriendplfuncapp.azurewebsites.net/track_a

 

And see the result

 

 

You can see the line with AccountName 0502lsecpriendplstoragea which Function App and Blob are in the same region, the Caller IP Address is change to the VNet internal IP.

 

Maybe you're thinking: why don't we put a Function App (Japan East) inside a VNet (West US) corresponding to a Storage Account (West US)?

 

Well, the "Firewalls and Virtual Networks" setting is a passive restriction, use to detect the source information and apply some extra work (e.g., accept, drop) due to rules. So it has no idea how to interrupt the resolving rule from the Function App of a VNet. You still get the public IP address from the Logs when changing to new VNet (West US) and accessing Storage Account (West US). Meanwhile, you cannot even choose the VNet on the binding list from Function App under Azure portal if they are not in the same region[4].

 

PS: Although there do have some extra setting would be launch once the "Firewalls and Virtual Networks" are finished setting, we cannot apply this on our situation because we're on the PaaS service. The extra setting is: Adding a "Effective security rules" to the NIC if there are VMs inside the target VNet. Which used to tell the VM taking internal resolving rules to the "someblobs.blob.core.windows.net " domain.

 

Finally, the bad news is we now understand why using Service Endpoints between PaaS resources and Storage Accounts under cross-region connection is currently unhidable. And the good news is there are some ways to compromise this kind of issue.

 

Thus, I am going to introduce the next tool.

 

Private Endpoints

Private Endpoints shares part of the same purpose from Service Endpoints but more flexible (i.e., which means there are more details settings behind each resources). Now we use the previous resources created from last chapter and keep going.

 

Firstly we clean up some of the previous resources:

  1. In the first Storage Account, in the "Firewalls and virtual networks", remove all the VNet, and switch back to the "Enabled from all networks".
  2. In the Function App, disconnect the (previously binded) VNet from it.
  3. In the Virtual Network, delete the VNet you've created on last chapter.

Then, we create a VNet in same regions matching your Function App:

 

We skip the similar snapshots.

 

Column Name Value
Name <you specified>
Region <you specified> e.g., Japan East

 

Inside this VNet, we keep creating a subnet form in it

 

Column Name Value
Name <you specified>
Subnet Access Range 10.1.1.0/24
All the rest use the default value

 

We have to link VNet which under the different region from Function App to the related Storage Account, so go to Storage Account:
Left side manu > Security + Networking > Networking > Firewalls and Virtual Networks > Disable "Public network access"
Left side manu > Security + Networking > Networking > Private Endpoint Connections > Add Private Endpoint

 

 

Column Name Value
Name <you specified>
Region <you specified> select the VNet under the same region from Function App
Target sub-resource Blob
Virtual Network <you specified>
subnet default

 

You might find the tricky part: you can specify the region of your VNet different from the private endpoint of your Storage Account, that means we could logically invoke an internal connection from our Function App to the blob.

 

Now go to the Private DNS zones:
Left side manu > Settings > Virtual network links > Add

 

 

Column Name Value
Link Name <you specified>
Virtual Network <you specified>
Auto-Registration check

 

PS: You may already found one on the list, if so, please delete it first

 

This step grants the ability to the target VNet which could use the internal name resolving rules form my Private DNS zones, and is the key step to override the VNet's resolving rule. You have to add each of the VNets from that interface when having a complex VNet chaining topology.

 

Back to the Function App, and bind it on the VNet
Left side manu > Settings > Networking > Outbound Traffic > VNet integration > Add VNet > Provice the information you've just created

 

Since you've just specify the Function App (Japan East) inside a VNet (Japan East) corresponding to a Storage Account (West US) using Private Endpoints (Japan East), we're going to check the result by re-invoke a connection

 

PS: You can restart the Function App before invoking, and then click this link:

 

https://0502lsecpriendplfuncapp.azurewebsites.net/track_b

 

And see the result

 

 

Voila! You can see the line with AccountName 0502lsecpriendplstorageb which Function App and Blob are in the different regions, and the Caller IP Address is change to the VNet internal IP.

 

Finally, we secure the inter-connectionn among our cross-region PaaS resources with some additional settings and get the information about how the Vnet works. It's cool isn't it?

 

Additional Information

  • You cannot create a new Function App using an existing Blob you've create for test, or you will no longer acccessing the file system from the Function App once you disable the Public network access from that Storage Account.
  • Although you disable the Public network access from that Storage Account, the accessing log still occurs on the Log Analytics workspace when there is an outside invoking attempt.
  • For some Function Apps upgraded from consumption plan, you might add a confuguration "WEBSITE_CONTENTOVERVNET=1" to resolve the Storage Accounts' restriction[1][2].

Conclusion

To achieve cross-region inter-connection, we have to choose a right network resources equipping a "region choosable" feature, such as Private Endpoints in this case.
To resolve the public domain name correctly, we have to add a Virtual Network Link[3] on each of the related VNets.

 

References

  1. Tutorial: Integrate Azure Functions with an Azure virtual network by using private endpoints 
  2. App settings reference for Azure Functions 
  3. What is a virtual network link? 
  4. (MS internal) App Service to Storage Account Connection Condition Summary 
  5. (MS internal) Pseudo IP does not SNAT ICMP requests 

Special Thanks (alphabetical order)

  • boqian.wang
  • dylan.ho
  • po.chen
  • wanjing
  • yawei.hu
Updated May 03, 2022
Version 7.0
No CommentsBe the first to comment