Azure Storage Data Management Solutions/Services:

1. Movement​: An end-to-end experience to discover, plan and move data into Azure in a performant, cost effective, secure, and reliable way​

2. Insights: Store and manage metrics and metadata enabling deep insights into the data estate

3. Actions​: Flexible, scalable, serverless platform to effortlessly process data for data management, protection, security and governance​

Big Data Challenges​:

Organizations must manage ever-increasing data volumes​

• Data Management​
• Data Movement​
• Tagging & Classification​
• Security and Access Control​
• Data Protection​
• Orchestration

Customer challenges​:

• Customers have increasingly large volumes of data in hundreds of storage accounts with billions of objects​
• Challenging to process millions of objects for bulk operations​
• Lifecycle management, data protection, object tagging and security operations require increasing complexity​
• Out of box policies in storage can be constricted and extensibility is limited

Introducing Azure Storage Actions

A fully managed platform that helps you automate data management tasks.​Process billions of blobs in your storage account effortlessly​. Supports Azure Blob Storage and Azure Data Lake Storage​

How Storage Actions Works?

Event-Condition-Action framework​

• Schedule-based and on-demand execution​
• Conditional processing of blobs based on blob properties​
• Use native blob operations as actions on the blob​

Serverless

• Fully managed infrastructure​
• Deploy in minutes – eliminates need for any complex software or infrastructure​
• Auto-scales with your storage

No-code composition & simplified management

• Use clicks to compose tasks​
• Easily apply tasks to multiple storage accounts​
• Monitor task execution across your storage execution with aggregate metrics and drilldowns

Storage Actions Overview​

• Data Protection – blob immutability, legal holds and blob expiry​
• Cost optimization – tiering or deleting blobs​
• Managing blob tags​
• Undelete blobs​
• Copy blobs, folder operations​

Key Concepts​

How to start with Storage Actions:

Login to Portal: Search for Azure Storage Actions

Create a task à Define Conditions:

[[and(endsWith(Name, 'pdf'), equals(BlobType, 'BlockBlob'))]]

The query [[and(endsWith(Name, 'pdf'), equals(BlobType, 'BlockBlob'))]] is a logical expression used to filter and retrieve specific items from a dataset. Here's a breakdown of its components:

1. endsWith(Name, 'pdf'): This part of the query checks if the Name attribute of an item ends with the string 'pdf'. Essentially, it filters items whose names end with .pdf, indicating that they are PDF files.
2. equals(BlobType, 'BlockBlob'): This part of the query checks if the BlobType attribute of an item is equal to 'BlockBlob'. This is used to filter items that are of the type BlockBlob, which is a type of storage blob in cloud storage systems.
3. and(...): The and operator combines the two conditions above. It ensures that only items meeting both criteria are retrieved. In other words, the query will return items that are PDF files (Name ends with .pdf) and are of the type BlockBlob.

In summary, this query is used to find items that are PDF files stored as BlockBlob in a dataset.

If above query verified, it would set tags process = true, and set blob immutability policy to locked

Create Assignment: 

Once task will run it will create report which will get added in Storage account container

Example Use Cases

• Retention Management: Automatically manage the retention and expiry durations of audio files using a combination of index tags and creation times 4.
• Version History Management: Manage the retention and lifecycle of datasets using metadata and tags for optimal protection and cost 4.
• One-off Processing: Define tasks to rehydrate large datasets from the archive tier, reset tags on part of a dataset, or clean-up redundant and outdated datasets