SOLVED

Automating the changing of blob tiers

%3CLINGO-SUB%20id%3D%22lingo-sub-152763%22%20slang%3D%22en-US%22%3EAutomating%20the%20changing%20of%20blob%20tiers%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-152763%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20all%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EI'm%20in%20the%20process%20of%20migrating%20data%20to%20Azure%20Blob%20Storage.%20This%20is%20data%20which%20will%20be%20rarely%20required%20or%20accessed%2C%20so%20we've%20decided%20to%20store%26nbsp%3Bthe%20files%2Fblobs%20in%20the%20Archive%20Tier.%20Currently%20blobs%20cannot%20be%20directly%20added%20to%20Archive%20tier%2C%20so%20I'll%20need%20to%20copy%20them%20from%20on-premise%20to%26nbsp%3Bmy%20Hot%20Tier%20container%20%26amp%3B%20then%20change%20each%20individual%20blob%20tier%20to%20Archive.%20There%20will%20be%20a%20large%20amount%20of%20data%20copied%20up%20initially%2C%20and%20then%20a%26nbsp%3Bincremenatl%20nightly%20copy%20job%20will%20need%20to%20be%20done.%20I'll%20will%20be%20using%20a%20scheduled%20task%26nbsp%3B(with%20Azcopy%20script)%20for%20both%20of%20these%20jobs.%3CBR%20%2F%3E%3CBR%20%2F%3EMy%20question%20is%2C%20once%20the%20files%2Fblobs%20have%20been%20uploaded%2C%20whats%20the%20best%20way%20of%20automating%20the%20process%20of%20changing%20the%20the%20blob%20tier%20from%20'Hot%20to%20'Archive'%3F%3F%20I've%20read%20a%20bit%20on%20Azure%20Automation%20and%20Runbooks%2C%20is%20this%20the%20route%20I%20should%20be%20going%20down.%3CBR%20%2F%3E%3CBR%20%2F%3EAppreciate%20any%20help%20or%20suggestions.%3CBR%20%2F%3E%3CBR%20%2F%3EThanks%3CBR%20%2F%3EBrendan%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-152763%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EData%20%26amp%3B%20Storage%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EStorage%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-168523%22%20slang%3D%22en-US%22%3ERe%3A%20Automating%20the%20changing%20of%20blob%20tiers%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-168523%22%20slang%3D%22en-US%22%3E%3CP%3EHello%2C%3C%2FP%3E%0A%3CP%3EWhile%20looking%20for%20a%20way%20to%20do%20this%20directly%20using%20AzCopy%20command%20line%20tool%2C%20I%20fall%20on%20your%20question%20and%20might%20have%20a%20first%20answer%20for%20you.%3C%2FP%3E%0A%3CP%3ERight%20now%2C%20I'm%20automating%20some%20backup%20in%20Azure%20Blob%20Archive%20Storage%20as%20well%20and%20end%20up%20using%20AzCopy%20(because%20Azure%20PS%20module%20AzureRM%20doesn't%20seem%20to%20support%20uploading%20large%20files%2C%20command%20Set-AzureStorageBlobContent).%3C%2FP%3E%0A%3CP%3EAnyhow%2C%20to%20change%20the%20storage%20tier%2C%20I%20am%20using%20a%20command%20from%20the%20same%20PS%20module%20called%26nbsp%3BSetStandardBlobTier%20and%20used%20as%20follow%3A%3C%2FP%3E%0A%3CP%3E%24blob.ICloudBlob.SetStandardBlobTier(%22Archive%22)%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHope%20it%20helps%2C%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-159144%22%20slang%3D%22en-US%22%3ERe%3A%20Automating%20the%20changing%20of%20blob%20tiers%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-159144%22%20slang%3D%22en-US%22%3EIts%20the%20solution%20i%20personaly%20would%20use%2C%20specialy%20if%20you%20have%20may%20files%20you%20could%20execute%20the%20achieve%20command%20on%20multiple%20files%20in%20parallel%20if%20the%20Storage%20account%20supports%20this%2C%20i%20haven't%20tested%20it.%20you%20could%20use%20a%20AAD%20service%20prinicpal%20or%20store%20the%20storage%20account%20keys%20as%20a%20encrypted%20value%20in%20your%20automation%20account.%3C%2FLINGO-BODY%3E
Highlighted
Deleted
Not applicable

Hi all,

 

I'm in the process of migrating data to Azure Blob Storage. This is data which will be rarely required or accessed, so we've decided to store the files/blobs in the Archive Tier. Currently blobs cannot be directly added to Archive tier, so I'll need to copy them from on-premise to my Hot Tier container & then change each individual blob tier to Archive. There will be a large amount of data copied up initially, and then a incremenatl nightly copy job will need to be done. I'll will be using a scheduled task (with Azcopy script) for both of these jobs.

My question is, once the files/blobs have been uploaded, whats the best way of automating the process of changing the the blob tier from 'Hot to 'Archive'?? I've read a bit on Azure Automation and Runbooks, is this the route I should be going down.

Appreciate any help or suggestions.

Thanks
Brendan

2 Replies
Highlighted
Its the solution i personaly would use, specialy if you have may files you could execute the achieve command on multiple files in parallel if the Storage account supports this, i haven't tested it. you could use a AAD service prinicpal or store the storage account keys as a encrypted value in your automation account.
Highlighted
Best Response
Solution

Hello,

While looking for a way to do this directly using AzCopy command line tool, I fall on your question and might have a first answer for you.

Right now, I'm automating some backup in Azure Blob Archive Storage as well and end up using AzCopy (because Azure PS module AzureRM doesn't seem to support uploading large files, command Set-AzureStorageBlobContent).

Anyhow, to change the storage tier, I am using a command from the same PS module called SetStandardBlobTier and used as follow:

$blob.ICloudBlob.SetStandardBlobTier("Archive");

 

Hope it helps,