How To Determine What Devices Are Connecting To a Storage Account

[AndrewCoughlin]

Introduction

Have you ever wondered how to determine if any devices are still using a storage account blob, file, table, or queues? In this blog post I will talk about the process of setting up monitoring to understand if/what devices are still communicating to a storage account.  What we will not cover is how to determine what the purpose of the storage account is or the impact of removing the storage account.  This is an exercise that would need to be done by yourself and may include others within your organization.

Before we start setting the solution up, we first need to understand our options for collecting the data.   To collect the data for a storage account we need to enable diagnostics logs, to configure diagnostic logs we have the following options:

• Send to Log Analytics workspace – Sending alerts to help integrate into queries, alerts, and visualization.
• Archive to a storage account – Useful for audit, static analysis, or backup.
• Stream to an event hub – Streaming data to an external solution such as a 3^rd party solution (SIEM)
• Send to a partner solution – 3^rd party integration solutions:
  • Apache Kafka for Confluent Cloud - Fully managed event streaming platform powered by Apache Kafka
  • Datadog - Monitor your servers, clouds, metrics, and apps in one place.
  • Elastic - Monitor the health and performance of your Azure environment.
  • Logz.io - Monitor the health and performance of your Azure environment.
  • Dynatrace for Azure (preview) - Use Dynatrace for Azure (preview) for monitoring your workflows using the Azure portal.
  • NGINX for Azure (preview) - Use NGINX for Azure (preview) as a reverse proxy within your Azure environment.

I will be focusing on determining if the storage account is being accessed using log analytics, but I will show you an easy way to configure:

• Log Analytics workspace
• Archive to a storage account
• Stream to an event hub

Prerequisites

If you will be using Log analytics workspace, then the only thing you will need to pre-create would be the log analytics workspace.

• Review Azure Monitor pricing: Pricing - Azure Monitor | Microsoft Azure
• Creating Log Analytics Workspace: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/quick-create-workspace?tabs=azure-portal if required.
• Creating storage account: https://docs.microsoft.com/en-us/azure/storage/common/storage-account-create?tabs=azure-portal if required.
• Creating EventHub: Azure Quickstart - Create an event hub using the Azure portal - Azure Event Hubs | Microsoft Docs if required.

 

Reference Articles

• https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings?tabs=portal
• https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/tutorial?pivots=azuredataexplorer
• https://docs.microsoft.com/en-us/learn/modules/write-first-query-kusto-query-language/

Configure Diagnostic Logs for Storage Account

Let’s review first what we will be doing and what this will look like once we have completed this exercise.  When we are completed with this exercise, blob, queue, table, and files diagnostic status will be Enabled and the storage account status will be Disabled still.  This is to be expected as the only thing we can collect on the storage account itself is metric data.

1. Login to the Azure portal.
2. Click on the storage account you are looking for to monitor for connections.
3. Click on Diagnostics settings.

4. Click on the resource you’re looking for to find out what is still being used.

5. Click on Add diagnostic settings.

6. Provide the settings name, select the log categories, in this example we are selecting all.
7. Select the destination details, in this example we will select log analytics.
8. Click Save.

9. You may need to repeat steps 4 – 8 if you want to understand what is using each service.
10. Once completed it should look like the below:

11. Once completed it can take up to 15 minutes before diagnostic data starts showing up in log analytics.

 

Query Log Analytics workspace

1. Login to the Azure portal if you haven’t done so already.
2. Select the log analytics workspace we configured the diagnostic settings to export to.

3. Click on Logs and select the table you’re wanting to query.
4. Type the query that represents the services you are wanting to see what devices are talking to the storage service:

Storage Service	Example Query
Blob	StorageBlobLogs | where AccountName == "<storageaccount>" | project TimeGenerated, Protocol, OperationName, AuthenticationType, Uri, CallerIpAddress, Category | order by TimeGenerated desc
Files	StorageFileLogs | where AccountName == "<storageaccount>" | project TimeGenerated, Protocol, OperationName, AuthenticationType, Uri, CallerIpAddress, Category | order by TimeGenerated desc
Queues	StorageQueueLogs | where AccountName == "<storageaccount>" | project TimeGenerated, Protocol, OperationName, AuthenticationType, Uri, CallerIpAddress, Category | order by TimeGenerated desc
Table	StorageTableLogs | where AccountName == "<storageaccount>" | project TimeGenerated, Protocol, OperationName, AuthenticationType, Uri, CallerIpAddress, Category | order by TimeGenerated desc

NOTE: Replace "<storageaccount>" with the storage account name.

5. Ensure the Time range is set then click Run.

6. The results will provide you with the list of IP addresses that have connected in the time specified.

Conclusion

There you have it; with this list you can now start to understand what devices by ip address are connecting to the storage account. Thank you for taking the time to read this blog, I hope this helps you and see you next time.

Disclaimer

The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without a warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.