Forum Discussion
How to archive diagnostic logs sent to storage account
I need help understanding storage append blobs created by diagnostic settings. When a diagnostic setting is configured to log to a storage account, the logs are created as append blobs. I have compliance requirements that mean I need to retain these blobs in immutable storage for 6 years, however, it seems I cannot use the blob lifecycle management feature to change the access tier of the append blobs to "archive tier". It is only supported for block blobs.
This page states "Setting the access tier is only allowed on Block Blobs. They are not supported for Append and Page Blobs."
https://learn.microsoft.com/en-au/azure/storage/blobs/access-tiers-overview
I feel like the lifecycle management feature is often touted as the answer to how to change the access tier for long-term storage scenarios, but it seems that it does not even work with diagnostic logs, which is pretty baffling.
How does Microsoft recommend changing diagnostic logs in a storage account to archive tier storage? The only answer I can see would be to implement some an azure function or logic app to read each blob as it's written and write it back to another storage account as a block blob. But the, how do you when the new file has finished being written to. Nevermind the fact that this violates my immutability requirement.
3 Replies
How about Azure Function or Logic App:
- Read the append blob once it’s “complete.”
- Write it to a new block blob in a separate container or storage account.
- Apply immutability policies and lifecycle rules to the new blob.
Yeah I'm going with the function approach. I just find it hard to believe that there isn't a better solution for this. This whole area seems pretty broken and underdiscussed. DCR filtering is a joke too. It doesn't work on the most key tables.
Hi,
Configuring the diagnostic settings is one way to get the logs, but there is one more setting which by default dumps the logs into your blob or azure storage file share. Maybe this setting might allow you to define the lifecycle. This setting is only for App services in the App services > Monitoring > App Service Logs.
If this works for your use case.
