Blog Post

ITOps Talk Blog
3 MIN READ

Azure Monitor agent and Data Collection Rules

Pierre_Roman's avatar
Pierre_Roman
Icon for Microsoft rankMicrosoft
Oct 05, 2021

Last week Sonia Cuff wrote about finding your Azure Log Analytics agent deployments in your environment in preparation for the Azure Monitor agent migration.

 

This was in response to an email you might have received if you are currently using the Log Analytics Agent.

 

We previously covered the reasons behind the need for a new agent.

 

 

But the reason for today’s article is to explore a bit the DCR or Data Collection Rules.

 

 

DCRs are a way to define data coming into Azure Monitor and specify where that data should be sent or stored. And it opens possibilities.

 

Log Analytics Workspace Access

 

I’ve been asked before (many times) if it was possible for building hub-&-Spoke type Log Analytics workspace.

 

The reason that was given for the request was to allow a way to have a hierarchy in the access of logs and monitoring data.   Something like branch office admins would only have access to view, report and configure alerts & actions for data from their own branch.  Regional admins would have rights to all the branches data in their regions and finally Corp Admin would see all the data.

 

The way to achieve that used to be complicated, expensive (you would be charged for data ingestion in all workspaces) and not supported since it could be achieved in a much more elegant way by using the Resource-context access model in a single workspace instead of duplicating the data ingestion in multiple workspaces and using the Workspace-context access model.

 

Log Analytics workspace are containers that includes data and configuration information used by Azure Monitor.  And they serve as an administrative boundary.  There are 2 access control mode:

 

  • Workspace-context: You can view all logs in the workspace you have permission to
  • Resource-context: You can view logs for only resources in all tables that you have access to.

While you can deploy one or more workspaces in your Azure subscription, there are several considerations you should understand.  We covered this during our ITOpsTalks: All Things Hybrid event earlier this year.

 

 

 

How DCR can help.

If you have a real requirement for multiple workspaces based on one or more of the following requirements:

 

  • You are a global company, and you need log data stored in specific regions for data sovereignty or compliance reasons.
  • You are using Azure and you want to avoid outbound data transfer charges by having a workspace in the same region as the Azure resources it manages.
  • You manage multiple departments or business groups, and you want each to see their own data, but not data from others. Also, there is no business requirement for a consolidated cross department or business group view.

DCR could now be your key to having the data in multiple workspaces.

 

NOTE:  DCRs only collect data from virtual machines using the Azure Monitor agent

 

For example, a virtual machine may have an association to multiple DCRs.  This allows you to define a set of DCRs, each matching a particular requirement, and apply them to only the virtual machines where they apply.

 

For example, If you have sets of VMs running Line of business application (LOB) and other running SQL Server...

 

You could have one DCR that applies to all virtual machines and separate DCR that collect data specifically from the LOB VMs and for SQL Server.

 

 

 

That way you could define in each DCRs where the data is being sent to, in effect sending the same data to multiple workspaces.  Keep in mind that your cost will increase since you are ingesting the same data in multiple workspaces.

 

Now,  if this is something you think can help your enterprise.  the first step is to migrate to the new Azure Monitor Agent.

Migration considerations

The Azure Monitor agent is generally available and fully supported, however it doesn’t yet have full feature parity with the Log Analytics agent. At the time of writing:

 

  • Not all Log Analytics solutions are supported today. Learn what's supported .
  • No support for Azure Private Links.
  • No support for collecting file based logs or IIS logs.

 

Defining the data collection rules is also handled differently in the Azure Monitor agent, allowing for more unique, scoped configurations for subsets of machines. Learn more at Changes in data collection.

 

Finally, see more migration considerations at Should I switch to the Azure Monitor agent?

 

I hope this helps.

 

Cheers!

 

Pierre

Updated Oct 05, 2021
Version 1.0
  • brink668's avatar
    brink668
    Brass Contributor

    How do you go about creating a Azure Policy so the DCR Rule applies dynamically to newly provisioned VMs? 

     

    Instead I have to assign Azure Monitoring Agent DCR to each VM Manually and I do not see a way to apply dynamically.

     

    How do you create an Azure Policy to dynamically add new VMs?  

  • AkosB's avatar
    AkosB
    Copper Contributor

    GregorySuvalian Yes it's GA, and they've even announced that the old agent is getting retired, but I'm holding off for now; The old agent is used for the Azure Updates solution in the automation account, and if you have AMA, then you don't have updating (yet, new solution is coming). Next, if you have both old and new agent, you'll end up with ingesting logs twice. Also Security Center afaik is still in private preview. Then there's an annoyance in the Linux version of the new agent that if you monitor disks, you can't monitor "*", because it then just monitors total disk space of all disks combined (_Total), instead of each disk individually.

     

    I'm sure all of this will be ironed out in the future, and the potential is enormous. Azure Policy based setting of monitoring (which can make it granular), and possibility to send metric data to the Azure Metrics instead of Log Analytics and granular filtering of what you want to ingest in Eventlogs are awesome!

     

    Pierre_Roman I would still try to keep all my logs on the same workspace, even if I have VMs in multiple regions. Azure Security Center just handles a single workspace that you can select. You *can* have multiple workspaces, but then you get ugly automatic naming, or you can get something with your own naming convention via an API call to set the workspace, but then the gui in ASC just tells you with a nice warning that you can only manage the continuous export using API now (I set ASC to continous export so I can have the protectionstatus table, which is where the Antimalware agent writes its data to, so I can get info on whether the VM is updating its antimalware definitions).

     

  • So it his GA? They mentioned April/June of 2022 or this was recorded earlier and it's already GA?