Upload a large list of folders in to AzureStorage area using AZCopy/Powershell

Brass Contributor

Hi Guys, 


I posted this in Azure space but probably its more relevant to PowerShell hence re-posted here :)


Newbie to Azure/PowerShell and trying to figure how to script this task. I have a large list of folders that are named after username of people in the orgnisation (see below). I'd like to upload selected set of these folder at a given time (say 100) and that 100 will be based on a CSV file which contains the folder name (username). 


Root folder path "C:\SourceFolders\"




Inside each folder there are files but no 2nd level folders. I'd like to use a CSV as the source reference to select the folders to upload  (say User2FirstName.LastName, User3FirstName.LastName). Basically, although the folder root location has a large number of folders, only the ones defined in the CSV will be uploaded (serial or parellel) 




.\azcopy copy "C:\SourceFolders\User1FirstName.LastName" "https://4980835d12c748f391037a5.blob.core.windows.net/ingestiondata?sv=2015-04-05&sr=c&si=IngestionS..." --recursive=true


Using AZCopy tool and PowerShell, I'd be grateful if someone please can help me to create a script to run this as a batch and potentially set a single output log file for the whole batch. :)



Thanks very much in advance !


1 Reply

Here you go: https://github.com/Raindrops-dev/RAIN-TechCommunityContributions/blob/main/Start-AzCopyBackup.ps1


The script is kept on purpose very simple (so you can learn and develop it further yourself :) ) but I integrated the log to a file in the same folder which will include all the output and the output of azcopy. There is a lot more you can do but this would be a good start.


Small consideration: ensure that you keep the SAS URL protected, ideally in an Azure Keyvault ;)