cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Azure Storage - Undelete Soft-delete Objects
By
jasoncao
Published Oct 06 2022 06:20 PM 2,358 Views
jasoncao
Microsoft
‎Oct 06 2022 06:20 PM

Azure Storage - Undelete Soft-delete Objects

‎Oct 06 2022 06:20 PM

Overview

 

Azure Storage supports soft delete to protect your data for Blob Storage and Azure Data Lake Storage Gen2. Soft delete allows you to recover data that has been deleted within a retention period. With public documentation Manage and restore soft-deleted blobs - Azure Storage | Microsoft Learn, we can recover a particular soft deleted file via portal, code and Powershell script. The documentation provides some Powershell examples to recover all soft-deleted objects. In some scenarios, you may want to undelete with Powershell scripts to undelete many blobs with certain conditions.

 

This article introduces 2 scripts that are used to undelete blobs with condition of path prefix and deletion time for both blob storage and Azure Data Lake Storage Gen2 respectively.

 

The scripts in this article are all tested with Powershell 7.2.6, Az.Storage 4.9.0. From documentation PowerShell Gallery | Az.Storage 4.9.0  ,if the Az.Storage module is less than 4.9.0, Az.Storage preview module is required. The Az CLI scripts are all tested with CLI version 2.40.0. For the version is less than 2.39.0, the extension storage-preview is required.

 

The Powershell version could be checked with $PSVersionTable.

jasoncao_0-1664530899978.png

Az.Storage module version could be checked with Get-InstalledModule.

jasoncao_1-1664530953498.png

Az CLI version could be checked with az version.

jasoncao_2-1664530988244.png

 

Azure Data Lake Storage Gen2

 

Powershell

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxx" 
$StorageAccountKey = "xxxx" 
#get context  
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
$filesystemName= "xxx"  
$dirName="xxx"
$MaxReturn = 100    
$Token = $Null   
$Year = ''
$Month=''
$Day=''
$Hour='0'
$Minute='0'
$Second = '0'
If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
   $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
}else{
   $sinceDate = (Get-Date -AsUTC).AddDays(-366)
}
write-host $sinceDate
do   
{   
   # list the deleted blobs in the path  
   $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token #get all deleted items
   write-host "========================================"   
   write-host "Soft deleted items: "   
   $DeletedItems.Path
   $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken
   $DeletedItems| Where-Object {$_.DeletedOn -ge [DateTime] $sinceDate} |Restore-AzDataLakeGen2DeletedItem #restore the items that meet the criteria
}while ($Token -ne '') 
​

  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.

    $storageAccountName = "xxx" 
$StorageAccountKey = "xxx" 
#get context  
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
$filesystemName= "xxxx"  
$dirName="xxx"
$pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
$MaxReturn = 100    
$Token = $Null    
if($pathexpression -eq ''){
   $pathexpression = '*'
}
write-host $pathexpression
do   
{   
   # list the deleted blobs in the path  
   $DeletedItems= Get-AzDataLakeGen2DeletedItem -Context $ctx -FileSystem $filesystemName -Path $dirName -MaxCount $MaxReturn -ContinuationToken $Token
   write-host "========================================"   
   write-host "Soft deleted items: "   
   $DeletedItems.Path
   $Token = $DeletedItems[$DeletedItems.Count-1].ContinuationToken
   $DeletedItems| Where-Object {$_.Path -like $pathexpression} |Restore-AzDataLakeGen2DeletedItem   
}while ($Token -ne '')

 

Az CLI

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxxx" 
$StorageAccountKey = "xxxx" 
$filesystemName= "xxx"  
$dirName="xxx"
$c=0
$Year = ''
$Month=''
$Day=''
$Hour='0'
$Minute='0'
$Second = '0'
If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
   $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
}else{
   $sinceDate = (Get-Date -AsUTC).AddDays(-366)
}
write-host $sinceDate
If($dirName.Length -eq 0){
   $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName
}else{
   $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName 
}
$DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json
write-host "========================================"   
write-host "Soft deleted items: "  
Foreach($item in $DeletedItemsJson) {
   if($item.deletedTime.ToUniversalTime() -ge [DateTime] $sinceDate) {
      $item.name + " - " +  $item.deletionId + " - " + $item.deletedTime.ToUniversalTime()
      az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName
      $c++
   }
} 
write-host "A total of " $c " blobs were recovered"​
  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.
    $storageAccountName = "xxxx" 
$StorageAccountKey = "xxxx" 
$filesystemName= "xxxx"  
$dirName="xxxx"
$pathexpression= '' #pathexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
$c=0
if($pathexpression -eq ''){
   $pathexpression = '*'
}
If($dirName.Length -eq 0){
   $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName
}else{
   $DeletedItems= az storage fs list-deleted-path -f $filesystemName --account-key $storageAccountKey --account-name $storageAccountName --path-prefix $dirName 
}
$DeletedItemsJson = "$DeletedItems" | ConvertFrom-Json
write-host "========================================"   
write-host "Soft deleted items: "  
Foreach($item in $DeletedItemsJson) {
   if($item.name -like $pathexpression) {
      $item.name + " - " +  $item.deletionId + " - " + $item.deletedTime.ToUniversalTime()
      az storage fs undelete-path -f $filesystemName --deleted-path-name $item.name --deletion-id $item.deletionId --account-key $storageAccountKey --account-name $storageAccountName
      $c++
   }
} 
write-host "A total of " $c " blobs were soft deleted and recovered" ​

 

Blob Storage (Below scripts are only work for no versioning enabled accounts, if you would like to restore soft-deleted items with versioning enabled, please check related documentation)

 

  1. Below script is used to restore soft-deleted items that were deleted after a specific time. The sincedate could be specified with parameters $Year, $Month, $Day, $Hour, $Minute, $Second. Please note, the date relevant parameters are using current system environment time zone. If sincedate is not set, the script will restore all soft-deleted items.
    $storageAccountName = "xxx" 
$StorageAccountKey = "xxx" 
$storageContainer = "xxx"
#get context
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
$MaxReturn = 100    
$Token = $Null    
$Year = ''
$Month=''
$Day=''
$Hour='0'
$Minute='0'
$Second = '0'
If($Year -ne '' -and $Month -ne '' -and $Day -ne ''){
    $sinceDate = Get-Date -Year $Year -Month $Month -Day $Day -Hour $Hour -Minute $Minute -Second $Second -AsUTC -ErrorAction Stop
}else{
    $sinceDate = (Get-Date -AsUTC).AddDays(-366)
}
write-host $sinceDate
do   
{   
      # get a list of all of the blobs in the container  
    $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn  -ContinuationToken $Token 
    write-host "========================================"   
    write-host "All Blobs including soft deleted ones: "   
    $Blobs.Name   
    $DeletedBlobs = $Blobs | Where-Object { ($_.ICloudBlob.Properties.DeletedTime -ge $sinceDate) -and ($_.IsDeleted -eq $true)}
    if($DeletedBlobs.Count -gt 0){
        $DeletedBlobs.BlobBaseClient.Undelete()
    }
    $Token = $Blobs[$Blobs.Count -1].ContinuationToken;   
}while ($Token -ne $Null) 
​
  2. Below script is used to restore soft-deleted items that item path is in a specific pattern. $pathexpression is used to specify the pattern. It supports wildcard expression(*,?,[]). If the pathexpression is not set, it will restore all soft delete items.
    $storageAccountName = "xxx" 
$StorageAccountKey = "xxxx" 
$storageContainer = "xxx"
#get context
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey  
$nameexpression= '' #nameexpression, a wildcard expression (containing *, ?, and [ ])like 'xxx*', if not setting, recover all soft delete blobs
$MaxReturn = 100    
$Token = $Null    
if($nameexpression -eq ''){
    $nameexpression = '*'
}
do   
{   
      # get a list of all of the blobs in the container  
    $Blobs = Get-AzStorageBlob -Container $storageContainer -Context $ctx -IncludeDeleted -MaxCount $MaxReturn  -ContinuationToken $Token 
    write-host "========================================"   
    write-host "All Blobs including soft deleted ones: "   
    $Blobs.Name   
    $DeletedBlobs = $Blobs | Where-Object { ($_.Name -Like $nameexpression) -and ($_.IsDeleted -eq $true)}
    if($DeletedBlobs.Count -gt 0){
        $DeletedBlobs.BlobBaseClient.Undelete()
    }
    $Token = $Blobs[$Blobs.Count -1].ContinuationToken;   
}while ($Token -ne $Null) 
​

 

Related documentation

 

https://learn.microsoft.com/en-us/azure/storage/blobs/soft-delete-blob-overview

Azure Storage Explorer soft delete guide | Microsoft Learn

https://learn.microsoft.com/en-us/azure/storage/blobs/blob-powershell#restore-a-deleted-blob

Manage and restore soft-deleted blobs - Azure Storage | Microsoft Learn

Co-Authors
Version history
Last update:
‎Oct 06 2022 06:20 PM
Updated by:
Labels