Azure Storage is excited to announce public preview of 5000 account limit features, which allows customers to create additional 5000 storage accounts per subscription per region. This is 20x increase from the current limit of 250. Many of our customers wish need several hundreds or thousands of storage accounts to address their storage needs. This helps customers build scenarios which need hundreds or thousands of storage accounts within a single subscription instead of creating more subscriptions. (Internal name PartitionedDns Feature)
This blog will help you to
- understand the feature in-depth, provides detailed steps on how to use &
- faq+known issues are at the end.
Feature details :
With this feature, Azure Storage is introducing two new concepts,
- A new storage account limit called 'Azure DNS Zone' which allows customers to create upto 5000 additional storage accounts. To leverage this new 5000 account limit, customers will need to create storage account with dnsEndpointType as 'AzureDNSZone'.
- dnsEndpointType is a new attribute added to storage account and can have values 'standard' or 'AzureDNSZone'.
When dnsEndpointType is 'AzureDNSZone', the endpoint of the storage account will be created on one of the zones of the new DNS++ architecture of Azure and endpoint url will have a zone in it.
E.g. myAccountname.[dnszone].[service type].storage.azure.net
where dnszone can be z00 to z99 and [service type] is blob, queue, table etc
Here are the difference the two dnsEndpointType - standard and AzureDNSZone
|
|
dnsEndpointType = Standard |
dnsEndpointType = AzureDNSZone |
|
1 |
Quota is 250. Can create upto maximum of 250 accounts |
Quota is 5000. Can create upto maximum of 5000 accounts |
|
2 |
You could derive the Endpoint url follow the standard scheme of : myAccountname.[service type].core.windows.net
E.g: "primaryEndpoints": { "dfs": "https://myaccountname.dfs.core.windows.net /", "web": "https://myaccountname.web.core.windows.net /", "blob": "https://myaccountname.blob.core.windows.net /", "queue": "https://myaccountname.queue.core.windows.net /", "table": "https://myaccountname.table.core.windows.net /", "file": "https://myaccountname.file.core.windows.net /" }
|
You cannot derive the endpoint url and your application or 3rd party tool will need to be updated to get the new endpoints & use it . At the time of creation of the storage account the endpoint will be assigned a specific dns zone. The endpoint url will be myAccountname.[dnszone].[service type].storage.azure.net where dnszone can be z00 to z99.
Below is an example of account where it was assigned dns zone “z18” "primaryEndpoints": { "dfs": "https://myaccountname.z18.dfs.storage.azure.net/", "web": "https://myaccountname.z18.web.storage.azure.net/", "blob": "https://myaccountname.z18.blob.storage.azure.net/", "queue": "https://myaccountname.z18.queue.storage.azure.net/", "table": "https://myaccountname.z18.table.storage.azure.net/", "file": "https://myaccountname.z18.file.storage.azure.net/" }
Your application or 3rd party tool will need to be updated to get and use the new endpoints. To get your endpoints either,
|
Exclusion :
- Azure Storage Classic accounts are not supported for Azure DNS Zone account limit i.e 5000 accounts per region per subscription.
A. Regions where this public preview is available :
Only available in the all public regions and not sovereign or government regions. If you try in other regions, you will get an error message (Error "code": "FeatureNotSupportedForAccount","message": "Partitioned Dns is not supported for the account.”)
- It most likely will be available in government regions around March 2023.
B. How to leverage this feature :
0. Pre-requisite - Register your subscription for the preview feature. Specify "PartitionedDnsPublicPreview" as the feature name and Microsoft.Storage as the provider namespace. More info - Set up preview features in your Azure subscription.
You could do this easily in Azure Portal
- Login to Azure Portal and Goto your subscription
- On the left pane click on "Preview features" , search for "PartitionedDnsPublicPreview"
- Select Register button
Here are the steps to register via Powershell. Goto powershell via Azure Portal. More info - Azure Cloud Shell Quickstart - PowerShell | Microsoft Docs
- Command: Register-AzProviderFeature -FeatureName "PartitionedDnsPublicPreview" -ProviderNamespace "Microsoft.Storage"
- If you participated in private preview, please unregister feature "PartitionedDns", Command: Unregister-AzProviderFeature -FeatureName "PartitionedDns" -ProviderNamespace "Microsoft.Storage"
- To check the status Command: Get-AzProviderFeature -FeatureName "PartitionedDnsPublicPreview" -ProviderNamespace "Microsoft.Storage"
Please reach azurestorage5klimit@microsoft.com if you face any problems or have any questions.
1. Step 1 – Create Storage Account with AzureDnsZone
- Arm Template, Powershell / CLI: Add dnsEndpointType = AzureDnsZone (see detailed steps in F. Steps for Common Tasks/Jobs )
- Azure Portal: In Create a Storage account workflow, under Networking tab, for Endpoint Type, choose azurednszone.
2. Step 2 - Use the new endpoints in your connection strings in your application or Storage client SDKs / Tools
Here is sample connection string format
DefaultEndpointsProtocol=https;AccountName=<storageaccount>;AccountKey=***;BlobEndpoint=https://<storageaccount>.<zone>.blob.storage.azure.net ;FileEndpoint=https://<storageaccount>.<zone>.file.storage.azure.net;QueueEndpoint=https://<storageaccount>.<zone>.queue.storage.azure.net; TableEndpoint=https://<storageaccount>.<zone>.table.storage.azure.net;
You can get your endpoints / connection string via Azure Portal , powershell, cli or rest api.
- You could use api GetProperties from storage resource provider API. Storage Accounts - Get Properties - REST API (Azure Storage Resource Provider) | Microsoft Docs
- You could use Azure Portal- Goto your storage account. On left goto ‘Access Keys’ blade, you will see the complete connection string and if you goto endpoints you will see the endpoints.
C. SDK/Tools supporting 5000 Account Limit (partitioneddns) feature
|
Azure Portal |
|
|
Storage Management API |
2021-09-01 or later. |
|
.Net |
Azure.Storage.Common 12.8.0 or later Azure.Storage.Files.DataLake 12.7.0 or later |
|
Java |
azure-storage-blob v12.10.0 or later azure-storage-common v12.10.0 or later azure-storage-file-datalake v12.4.0 or later |
|
JS |
No support yet Estimated date to add support is middle of June 2022. |
|
Python |
- azure-storage-blob v12.7.0 (including 12.7.0b1) - azure-storage-file-datalake v12.2.0 |
| Powershell | Install Az.Storage 4.4.2 and later |
| CLI | Azure CLI 2.37.0 or above |
| AzCopy | v10.9.0 or later |
| Storage Explorer | 1.19.0 or later |
| Management plane SDK | 24.0.0 or later |
Reference links for SDK / Tools
.Net - https://www.nuget.org/packages/Azure.Storage.Blobs/
Java - https://mvnrepository.com/artifact/com.azure/azure-storage-blob
Python - https://pypi.org/project/azure-storage-blob/
AzCopy - https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10
Storage Explorer - https://azure.microsoft.com/en-us/features/storage-explorer/
Azure Powershell - https://github.com/Azure/azure-powershell/
Azure CLI - https://github.com/Azure/azure-cli
D. Your applications or tools will need an update
Your applications and tools will need to use the supported sdks and be updated to get the new endpoint by calling GetProperties api & then use the endpoint to connect to the storage account.
New endpoint is of this format myAccountname.[dnszone].[service type].storage.azure.net
where dnszone can be z00 to z99 and [service type] is blob, queue, table etc
E. For ISV(Independent Software Vendor) or Services vendor
This feature may affect you. Please review our recommendation for you appropriate scenario.
|
Scenario |
What is the impact? |
Recommendation |
|
|
1. Your service/product connects to your Customer created Azure Storage account |
a. Collects only storage account name and No endpoint url, and figures endpoint url |
Failure / Error to connect to Storage Account with new of endpointType =AzureDNSZone, which gets new endpoint schema assigned while creation of storage account. |
Add ability for Customers to specify endpoint url and use it if customer provides endpoint url. |
|
b. Collects endpoint url from Customer/user |
No impact |
||
|
2. Your service/product creates Storage Account for your / your customers’ needs and you don’t intend to use the 5000 Account Limit feature. |
No Impact |
N/A |
|
|
3. Your service/product wishes to leverage this feature to create upto 5000 accounts with your subscription |
If you wish to participate in public preview, please follow B. How to leverage this feature section |
|
|
F. Steps for Common Tasks/Jobs
1. Powershell:
Releases –
- Release Az.Storage v4.4.2-preview · Azure/azure-powershell (github.com)
- PowerShell Gallery | Az.Storage 4.4.2-preview
a. Install the correct version (with admin privileges) :
i. Install the latest PowershellGet
Install-Module PowerShellGet –Repository PSGallery –Force
ii. Close and reopen the Powershell console
iii. Install Az.Storage preview module
Install-Module Az.Storage -Repository PsGallery -RequiredVersion 4.4.2-preview -AllowClobber -AllowPrerelease -Force
d. Create an account with dnsEndpointType == AzureDnsZone
New-AzStorageAccount -ResourceGroupName $rgname -Name $accountName -SkuName Standard_LRS -Location eastus2euap -Kind StorageV2 -DnsEndpointType AzureDnsZone
e. Get the connection string
i. Create storage Context
# only have dataplane access
$ctx = New-AzStorageContext -BlobEndpoint "https://accountname.z10.blob.core.windows.net/" -TableEndpoint "https://accountname.z10.table.core.windows.net/" -QueueEndpoint "https://accountname.z10.queue.core.windows.net/" -FileEndpoint "https://accountname.z10.file.core.windows.net/" -StorageAccountName “accountname” -StorageAccountKey $key
# if have management plane access:
$ctx = (Get-AzStorageAccount -ResourceGroupName $resourceGroupNam -Name $accountName).Context
ii.Get Connection string
$connectionString = $ctx.ConnectionString
f. Upload a file(myfile.txt) – no change
Set-AzStorageBlobContent -File $localSrcFile -Container $containerName -Blob $blobName -Force -Context $ctx
g. Download a file(myfile.txt) – no change
Get-AzStorageBlobContent -Container $containerName -Blob $blobName -Destination $localDestFile -Force -Context $ctx
h. Delete the previous created account – no change
Remove-AzStorageAccount -ResourceGroupName $rgname -Name $accountName
2. With CLI:
Releases –
- CLI extension storage-preview 0.8.2 released to support `--dns-endpoint-type` for creating storage account
- CLI 2.36.0 will be released to support specifying endpoint for each service
a. Install the correct version
az extension add -n storage-preview
b. Create an account with dnsEndpointType == AzureDnsZone
az storage account create -n $accountName -g $rgName -l eastus2euap --dns-endpoint-type AzureDnsZone
c. Get the service endpoint
az storage account show –n $accountName --query “primaryEndpoints”
az storage account show -g $resourcegroupName -n $accountName --query “primaryEndpoints”
d. Get Connection String
i. Use show-connection-string
az storage account show-connection-string -g $resourcegroupName -n $accountName
ii. Alternate approach is to add the endpoint
az storage account show-connection-string –blob-endpoint “https://myaccountname.z18.blob.storage.azure.net/” -g $resourcegroupName -n $accountName
c. Upload a file
$blobEndpoint = az storage account show –n $accountName –query “primaryEndpoints.blob”
az storage blob upload --file $localFile --name $blobName --container $containerName --blob-endpoint $blobEndpoint
d. Download a file
$blobEndpoint = az storage account show –n $accountName –query “primaryEndpoints.blob”
az storage blob download --file $localFile --name $blobName --container $containerName --blob-endpoint $blobEndpoint
e. Delete the previous created account
az storage account delete –n $accountName –g $rgName
3. With .net management plane sdk:
a. Install the correct version
Install-Package Microsoft.Azure.Management.Storage -Version 24.0.0
b. Create an account with dnsEndpointType == AzureDnsZone
See test case: https://github.com/Azure/azure-sdk-for-net/blob/30cc1b933378780cccf22c00fcb68ccd7c09e6e6/sdk/storage...
c. Get the connection string
N/A (Management plane SDK doesn’t support create dataplane connection string directly)
d. Upload a file(myfile.txt)
N/A (Management plan SDK doesn’t support dataplane operation)
e. Download a file(myfile.txt)
N/A (Management plan SDK doesn’t support dataplane operation)
f. Delete the previous created account
No change, see sample in https://github.com/Azure/azure-sdk-for-net/blob/30cc1b933378780cccf22c00fcb68ccd7c09e6e6/sdk/storage...
G. Frequently asked questions
- What if my subscription is registered for private preview and public preview feature?
- Please email us azurestorage5klimit@microsoft.com, we need to unregister your subscription from the private preview of this feature.
- Or you could
- unregister from private preview feature by following these steps - Set up preview features in Azure subscription to unregister from feature name "PartitionedDns" and provider namespace "Microsoft.Storage"
- and register for public preview, follow the instructions provided in Set up preview features in Azure subscription. Specify PartitionedDnsPublicPreview as the feature name and Microsoft.Storage as the provider namespace
- I don’t have a need to add more than 250 storage account, should I participate in public preview.
- If you foresee that in future you may need to add more than 250 storage account then, please participate.
- Will we be impacted with the launch of this feature?
- This feature is available only for subscriptions registering for the public preview feature(PartitionedDnsPublicPreview). If your subscription is not registered then, there is no impact.
- Once your subscription is registered you can leverage additional 5000 Account limit to create storage accounts with dnsEndpointType as AzureDNSZone.
- Will enabling this feature on my subscription, impact my existing storage accounts ?
- No, this will have no impact to your existing storage accounts.
- Am waiting more than 1 business day for my subscription registration request for "PartitionedDnsPublicPreview" feature. What shoud i do?
- Please email azurestorage5klimit@microsoft.com
- Will enabling this feature on my subscription, impact my current application or process interacting with Azure Storage?
- If you create new storage accounts with dnsEndpointType = AzureDnsZone then, the endpoint url of will be of new format
- myAccountname.[dnszone].[service type].storage.azure.net
- where dnszone can be z00 to z99
- Your application or process will need to get endpoint url from GetProperties api and use it to connect to your storage account. Your application can cache or save the endpoint url as it does not change for a storage account
- If you create new storage accounts with dnsEndpointType = AzureDnsZone then, the endpoint url of will be of new format
- Can I change the dnsEndpointType after creation of storage account?
- No, this wont be possible once dnsEndpointType is either standard or AzureDNSZone, it cannot be changed.
- Will the endpoint url of the storage account change?
- No, once the endpoint url is assigned while creation of storage account it will not change for the life of the storage account.
- How does this feature affect secondary endpoints?
- Secondary endpoint will be created in the same way. For dnsEndpointType is AzureDNSZone, the secondary endpoint will be as follows,
- myAccountname-secondary.[dnszone].[service type].storage.azure.net
- Secondary endpoint will be created in the same way. For dnsEndpointType is AzureDNSZone, the secondary endpoint will be as follows,
- Will Firewall rules need to change ?
- Customers will need to add *.storage.azure.net to the firewall exception rules
- How to asks questions ?
- Email us at azurestorage5klimit@microsoft.com with your questions
- How to give feedback or report issues ?
- We would appreciate your feedback and reporting issues. Please email us at azurestorage5klimit@microsoft.com
H. Known Issues:
The following services are not compatible with this feature. We recommend not to use these services / features with 5000 Account limit feature in preview period. As these teams add compatibility, we will update dates on this blog. If you have any question, please email us at azurestorage5klimit@microsoft.com
- Azure Blob Filesystem driver (ABFS)
- ADX - Azure Data Explorer (Kusto)
Azure Data Factory -(August 12) ADF is now compatibile with this feature- Azure Databricks / Databricks
- Azure Devops Terraform Plugin
- Azure Event Grid
- Azure Event hub
- Azure File Sync
- Azure Files AD
- Azure Function
- Azure Service Bus
- Azure Monitor - Diagnostics
- Azure Storage - Blob Inventory
- Azure Storage - Changefeed
- Azure Storage - Object Replication Rules
- Power Apps - Platform Connectors
- SQL DW
Thank you for taking time to read this blog. We hope this feature helps you and your business.
