Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
Moving Azure Activity Connector to an improved method
Published Jun 24 2021 03:32 AM 148K Views
Microsoft

The Activity log is a platform log in Azure that provides insight into subscription-level events. This includes such information as when a resource is modified or when a virtual machine is started. You can view the Activity log in the Azure portal or retrieve entries with PowerShell and CLI. For additional functionality, you should create a diagnostic setting to send the Activity log to your Azure Sentinel.

 

 

What changed?

The Azure Activity connector used a legacy method for collecting Activity log events, prior to its adoption of the diagnostic settings pipeline. If you're using this legacy method, you are strongly encouraged to upgrade to the new pipeline, which provides better functionality and consistency with resource logs.

Diagnostic settings send the same data as the legacy method used to send the Activity log with some changes to the structure of the AzureActivity table.

The columns in the following table have been deprecated in the updated schema. They still exist in AzureActivity but they will have no data. The replacement for these columns are not new, but they contain the same data as the deprecated column. They are in a different format, so in the event, you have any private or internal content (such as hunting queries, analytics rules, workbooks, etc.) based on the deprecated columns, you may need to modify it and make sure that it points to the right columns.

 

 

ShaharAviv_0-1624519414672.png

 

Here are some of the key improvements resulting from the move to the diagnostic settings pipeline:

  • Improved ingestion latency (event ingestion within 2-3 minutes of occurrence instead of 15-20 minutes).
  • Improved reliability.
  • Improved performance.
  • Support for all categories of events logged by the Activity log service (the legacy mechanism supports only a subset - for example, no support for Service Health events).
  • Management at scale with Azure policy.
  • Support for MG-level activity logs (coming in preview now).

 

Set up the (new) Azure Activity connector

The new Azure Activity connector includes two main steps- Disconnect the existing subscriptions from the legacy method, and then Connect all the relevant subscriptions to the new diagnostics settings pipeline via azure policy.

 

 

ShaharAviv_2-1624519414696.png

 

 

 

ShaharAviv_3-1624519414706.png

 

 

Please go to Connect Azure Activity log data to Azure Sentinel to learn more about the new connector experience.

 

11 Comments
Brass Contributor

@ShaharAviv , @Tiander Turpijn  Thanks for sharing this article. I have 30+ analytic rules based on azure activity logs. Is there any way to update the new column names in all the analytic rules at once? I would also like to know by when the deprecated columns will be removed permantely. 

Microsoft

Hi @Pavan_Gelli1910 , thanks for reaching out!

Unfortunately, I'm not familiar with a good and effective way to update all analytic rules at once. The deprecated columns will not be removed permanently in the near future (a few years) but we do not recommend using them. 

Silver Contributor

How can we tell if the old connector is still being used so that it can be upgraded?

Microsoft

Hi @Dean Gross,

In step #1 on the connector page, we check which of the subscriptions you own are connected through the old pipeline (you will see them in the scrolling list below). In case there are a few- just click on the "Disconnect All" button below the list.

 

Make sure that you connecting them all back using the Policy in step #2 or manually enable diagnostic settings logs for each subscription.

Silver Contributor

@ShaharAviv thanks, we use Lighthouse and have many instances of Sentinel. Do you have any recommendations about how to this at scale?

Microsoft

@Dean Gross The best way is to assign a Policy for each Tenant. Feel free to contact me for more details/guidance.

Silver Contributor

@ShaharAviv thanks for the offer. How should I contact you to get more details?

Microsoft

@Dean Gross You can find me at: t-shaviv@microsoft.com

Brass Contributor

Will analytic templates created from the earlier version of the data connector be dynamically updated when the new connector is set up, or do we have to delete the old rules and replace them with new analytic templates?

Microsoft

Hi @Julian, thanks for reaching out.

All the content created by us has been updated.

As for your own analytic rule templates- You just need to make sure that they are pointing to the right columns and expecting the right value format.

Iron Contributor

hi @ShaharAviv , I have some questions about the 'new' way to collect Azure Activity logs using Azure Policy.

 

The 'new' way:

Sentinel > Azure Activity Log Connector > create a policy to pull the logs.

 

Please confirm my assumptions below:


- It is NOT recommended to assign a policy at the root tenant level - this will fail unless you apply additional roles to the global admin. You should use subscription groups and not the root subscription - correct?

- It IS recommended to assign this policy at either a subscription level or a subscription group level - correct?

- For any existing subscriptions you may need to apply a remediation as the policy will only apply to NEW resources - correct?

 

In some cases I've created an Azure Policy and it shows it 'failed compliance' because there are no matching resources in the subscription - why does this happen? I expect this policy to simply log all azure activity under the scope of 1 or more subscriptions.

 

Thanks!

 

 

Co-Authors
Version history
Last update:
‎Jun 24 2021 04:53 AM
Updated by: