While trying to upload multiple blobs to the azure blob storage, there could be a scenario wherein you may have faced the below errors:
Invalid Block/Blob 'The specified blob or block content is invalid'
The uncommitted block count cannot exceed the maximum limit of 100,000 blocks ErrorCode:BlockCountExceedsLimit
Invalid Block 'The specified block list is invalid'
For the Error 1
This can occur if the application / client specify a block size that is not supported.
Reference link for the Put Block REST API:
For the Error 2
This can occur if the application doing the copy is exceeding this limit during the copy or if there are some uncommitted blocks left from a previous copy that was canceled.
A blob can have a maximum of 100,000 uncommitted blocks at any given time.
Reference Link : https://docs.microsoft.com/en-us/azure/storage/common/storage-scalability-targets#azure-blob-storage...
For the Error 3
This happens when you upload blocks with different applications and there are uncommitted blocks for which we don’t find same length as different application use different length to upload the data. This could have happened due to a previous upload that failed.
The issue that has been observed on both the above errors can be resolved by following either of the 3 methods:
Perform a GetBlockList (with blocklisttype=uncommitted) to retrieve the uncommitted block list, commit the block list, then delete the blob.
Create a dummy blob (can be of length zero) with the same blob name in the same container and transfer it using a non-blocked transfer
Wait 7 days for the uncommitted block list to garbage collect.
You can also consider using Azure Blob Events.
The event is only triggered when a Put Blob or Put Block List is completed so you will know that the blob is fully committed before trying to take action on it.
This has the benefit of being notified of when a new blob is written and you can reference this blob directly without having to blobs in the container to find blobs that need action.
- Azure Storage GPV2 / Blob Storage Account/ ADLSGEN2 account used with blob endpoint.
- Windows PowerShell
The below is a PowerShell script would help with the resolution mentioned on Method 1
By using the following materials or sample code you agree to be bound by the license terms below and the Microsoft Partner Program Agreement the terms of which are incorporated herein by this reference.
These license terms are an agreement between Microsoft Corporation (or, if applicable based on where you are located, one of its affiliates) and you. Any materials (other than sample code) we provide to you are for your internal use only. Any sample code is provided for the purpose of illustration only and is not intended to be used in a production environment. We grant you a nonexclusive, royalty-free right to use and modify the sample code and to reproduce and distribute the object code form of the sample code, provided that you agree:
- to not use Microsoft’s name, logo, or trademarks to market your software product in which the sample code is embedded;
- to include a valid copyright notice on your software product in which the sample code is embedded;
- to provide on behalf of and for the benefit of your subcontractors a disclaimer of warranties, exclusion of liability for indirect and consequential damages and a reasonable limitation of liability
- to indemnify, hold harmless, and defend Microsoft, its affiliates and suppliers from and against any third party claims or lawsuits, including attorneys’ fees, that arise or result from the use or distribution of the sample code."
Perform a GetBlockList (with blocklisttype=uncommitted) to retrieve the uncommitted block list and then delete the blob.
#Please enter storage account name
[Parameter(Mandatory=$true,Position=1)] [string] $StorageAccountName,
#Please enter Shared Access Signature <SAS token > generated with the permissions <ss=b;srt=sco;sp=rwldc>
[Parameter(Mandatory=$True,Position=1)] [string] $SharedAccessSignature,
#Please enter storage Container name
[Parameter(Mandatory=$True,Position=1)] [string] $ContainerName ,
#Please enter Blob name
[Parameter(Mandatory=$True,Position=1)] [string] $Blobname
# below rest API helps in getting the uncommitted block List
$Blob = "https://$StorageAccountName.blob.core.windows.net/"+$ContainerName+"/"+$Blobname+"$SharedAccessSignature&comp=blocklist&blocklisttype=uncommitted"
$listfileURI = Invoke-WebRequest -Method Get -Uri $Blob
$FilesystemName = $listfileURI.Content
$String=$FilesystemName -replace 'ï»¿' , ''
$String | Select-Xml –Xpath “/BlockList/UncommittedBlocks/Block”|Select-Object -Expand Node
#deletion of the blob & uncommitted block
if($Count.Count -gt 0)
$listfileURI1 = Invoke-WebRequest -Method Delete -Uri $delete
$FilesystemName1 = $listfileURI1.StatusCode
Write-Host "Deletion has been successfully , API returned status code " $FilesystemName1
Write-Host "Check if the uncommitted block are still present"
$Blobcheck = "https://$StorageAccountName.blob.core.windows.net/"+$ContainerName+"/"+$Blobname+"$SharedAccessSignature&comp=blocklist&blocklisttype=uncommitted"
$listfileURI2 = Invoke-WebRequest -Method Get -Uri $Blobcheck
Write-Host "StatusCode:" $_.Exception.Response.StatusCode.value__
Write-Host "StatusDescription:" $_.Exception.Response.StatusDescription
Write-Host "With the above error message we can confirm that the uncommitted blocks and their respective blob has been deleted"
Write-Host "Name and size of uncommitted block that has been deleted are as below"
Hope this helps