Forum Discussion

Arslan11's avatar
Arslan11
Brass Contributor
Mar 17, 2022

Azure File sync: Script to delete the files, which has been modified for certain days

 

I would like to have a script, where we can delete the files from the azure file share, if it is has not been modified for certain days.

 

I know we can use logic app , but it deletes the files after certain days without looking at, when it was modified last time.

 

Can somebody please guide me on this, as this a good automation, to save azure storage cost.

azure blob storage

3 Replies

Sort By
  • sanketghorpade's avatar
    sanketghorpade
    Copper Contributor
    Apr 06, 2022
    Hi Arslan11,

    Azure Lifecycle Management for storage is generally used for this purpose but I see you are talking about the File Shares and so far there is no support for lifecycle management from Azure for File Shares.
    I do not see a reason why you cannot use Logic App for the same purpose. You can use the logic app to check for the last modified date and use the date function in the condition action to add the logic which will help you to find if this is something "expired" and has to be deleted.

    We are doing something similar using PowerShell script to find the last modified date of each blob file (as in our case Life Cycle Management didn't work for blob storage as we had different retention period for each file). If you can share the script where you are facing issue then I can help you further with it.

    Thanks,
    Sanket
    • Arslan11's avatar
      Arslan11
      Brass Contributor
      Apr 06, 2022
      Thanks for recommending to use logic apps, and go with last modified date. I am afraid haven't got a script.

      As you have got a script, are you able to share with me. And I can have a view of it.
      • sanketghorpade's avatar
        sanketghorpade
        Copper Contributor
        Apr 21, 2022

        Arslan11 sorry for the delay in reply. You can use the following code snippet to understand how you can find the last modified date of the blob file. Do not use this code as it is as I have copied it from our script and I have only copied the main logic but some parameters might be missing which you will be able to figure out easily.

         do
{
    Write-Output "---> Batch: $currentBatch"
    $currentBatchItems = Get-AzDataLakeGen2ChildItem -Context $storageContext -FileSystem $storageContainerName -Recurse -MaxCount $maxItemsPerBatch -ContinuationToken $continuationToken -FetchProperty
    if ($null -eq $currentBatchItems -or $currentBatchItems.Length -le 0) {
        break;
    }

    $allItems += $currentBatchItems
    $continuationToken = $currentBatchItems[$currentBatchItems.Length-1].ContinuationToken;
    $currentBatch++
}
while ($null -ne $continuationToken)

$allDirectoryItems = $allItems | Where-Object { $_.IsDirectory -eq $true }
Write-Output "--> Done - Folders and items found: $($allItems.Count)"

# Filter directories for correct conditions
Write-Output "--> Filtering directories for selected path"

## Filter for correct path level
$filteredDirectoryItems = @()
if ($null -ne $baseFolderPath -and $baseFolderPath -ne "") {
    $filteredDirectoryItems = $allDirectoryItems | Where-Object {
        $null -ne $_.Path -and
        $_.Path -ne "" -and
        $_.Path.StartsWith($baseFolderPath) -and
        -not $_.Path.Replace($baseFolderPath, "").Contains("/")
    }
} else {
    $filteredDirectoryItems = $allDirectoryItems | Where-Object {
        $null -ne $_.Path -and
        $_.Path -ne "" -and
        -not $_.Path.Contains("/")
    }
}

## Filter for expired folder based on last modified
$expiredDirectoryItems = New-Object System.Collections.Generic.List[System.Object]

$filteredDirectoryItems | ForEach-Object {
    $currentFolderPath = $_.Path

    Write-Output "---> Checking folderpath '$currentFolderPath'"
    
    $folderItemsAndFolders = $allItems | Where-Object {
        $null -ne $_.Path -and
        $_.Path -ne "" -and
        $_.Path.StartsWith($currentFolderPath) }

    $lastModifiedItem = $folderItemsAndFolders | Sort-Object -Property LastModified -Descending | Select-Object -Property Path, LastModified -First 1

    if ($null -ne $lastModifiedItem) {
        Write-Output "---> Last modified item - Folderpath: '$($lastModifiedItem.Path)', LastModified: $($lastModifiedItem.LastModified)"

        if ($lastModifiedItem.LastModified.AddDays($folderRetentionInDays) -lt (Get-Date)) {
            Write-Output "----> Folder marked for deletion"
            $expiredDirectoryItems.Add($_)
        }
    } else {
        Write-Output "---> No last modified item found.."
    }
}

        I hope this helps you. Feel free to mark this as answer if you think this has helped you.

Resources