Anatomy of an Event Collection Rule for Azure Operational Insights (Advanced targeting when using OpsMgr attach)

First published on TECHNET on Aug 27, 2014

[NOTE -  Operational Insights is now a part of Operations Management Suite. Learn more at microsoft.com/OMS ]

[Edited February 21st 2015– with support for Security Events]

So we have heard loud and clear that you want more granular targeting options for Log Management – both to control which machines collect which logs , as well as to avoid the noisy errors in the Operations Manager event log that rightfully complains it can't find the log (it’s not a big deal anyhow – if it doesn’t find it, it won’t collect it; this becomes cosmetic in ‘Direct Agent’ scenario.

If you use Operations Manager,  even synchronizing and exposing all the current ON-Prem-defined groups and classes is something we don't do at the moment - and it would be difficult to allow this type of granular selection/targeting from the cloud. Furthermore we need to finalize the story for *other* type of attach scenarios, not just OpsMgr - i.e. for the 'directly attached agents' or the data pulled from Azure - so we need to rethink what 'targeting' really looks like in the HYBRID world, before we tackle that problem in the UI/Portal properly. It’s in our backlog.

But there is a workaround (if you use System Center Operations Manager)!

For those customers who have OpsMgr today, the great news is that all of the collection policy is controlled via MANAGEMENT PACKs . Which means you don't HAVE to rely on the Portal-configured collection policy... like already shown in this other article, you CAN inject other data in the system in the same way . Now, we do advise to leave AT LEAST ONE collection rule in the portal (i.e. for 'Application' or 'System' or some log that is broadly available and you do wish to collect for all machines) - otherwise our Portal will revert the 'log management' tile to its 'day 0' experience, which takes you to configuration screen, not to the data/search-powered drill-down page... but once you have a basic rule in place, you can do the rest more granularly in OpsMgr if you aren't scared to copy/paste a couple of XML snippet. In fact, you don’t even need to do that because I am giving you an Authoring Template ready to download import and use – but please read this theory first to understand what happens under the hoods!

So how does an OpInsights event collection rule look like?

Actually, you can simply export the Log Management Unsealed MP you see in Operations Manager and take a look - and I am sure some of you did - and see how it's put together. You'd find out it is actually quite simple. The event collection rules look EXACTLY like 'regular' rules you could have built (and probably have built) in OpsMgr to collect events to OpsDB or DW databases - but there's one secret sauce to it: the new WRITE ACTION modules used! There are two flavors:

1. the ones for ‘Log Management’ – these have a write action ( IPTypes!Microsoft.SystemCenter.CollectCloudGenericEvent ) that sends data from agents to the management server and then from the management server to the cloud. These events will be indexed and will appear in search as Type=Event , and have a very generic shape which is meant for operational investigations and troubleshooting scenarios. There is a very good reasons why we disallow collection ‘Security’ event logs in the portal under Log Management : Security logs are VERY CHATTY. Very LARGE VOLUMEs of data will flow thru your OpsMgr infrastructure, way more than from any other log in any normal (or even exceptional) circumstance. This will probably affect performance of your Management Servers . So please don’t create this type of rules for ‘Security’ events. Events sent by rules of this first kind will be reported as part of the ‘Log Management’ Intelligence Pack in the ‘Usage’ breakdown of the amount of data sent.
2. The second type of rules are coming with the ‘Security and Audit’ Intelligence pack . By default they collect ALL Security events. These are very voluminous, hence these rules use a write action ( IPTypes!Microsoft.SystemCenter.CollectHighVolumeDirectChannelCloudEvent ) configured to send events directly FROM each agent to the cloud , bypassing the management server entirely . Even this way, you might not want to collect ALL security events, but only light up some scenarios and be selective about which events you collect. In this case you can set overrides to disable the original rules we have in sealed MP’s, and we allow you to create more granular rules that work for ‘Security’ logs and will give them the specialized Type=SecurityEvent shape in search, which is designed for security investigations. Events sent by rules of this second kind will be reported as part of the ‘Security and Audit’ Intelligence Pack in the ‘Usage’ breakdown of the amount of data sent. This is the original unfiltered rule

In order to use these write actions in your MP’s, you first need to add a reference to the OpInsights 'Types' library (this library contains most modules used by Intelligence Packs to send data) in your management pack:

<Reference Alias="IPTypes">
<ID>Microsoft.IntelligencePacks.Types</ID>
<Version>7.0.9014.0</Version>
<PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
</Reference>

And the following write actions are  ALL the magic incantation you need at the end of your rules.

This Write action (to be used with any 'generic' event log - excluding the 'Security' event log) will:
- send from agent to MS and from MS to cloud service
- make the data in search indexed/searchable as Type="Event"

<WriteActions>
<!-- Send data thru the management server, and MS will send to cloud, Type=Event shape -->
<WriteAction ID="HttpWA" TypeID="IPTypes!Microsoft.SystemCenter.CollectCloudGenericEvent" />
</WriteActions>

This Write action (to be used with SECURITY Event Log only) will:
- send directly from the agent
- make the data in search indexed/searchable as Type="SecurityEvent"

<WriteActions>
<!-- Send data directly from agent to cloud, Type=SecurityEvent shape -->
<WriteAction ID="HttpWA" TypeID="IPTypes!Microsoft.SystemCenter.CollectHighVolumeDirectChannelCloudEvent" />
</WriteActions>

So you can simply take a rule (you can create it 'disabled' in OpsConsole), use all the fancy EventDS criteria and go wild with targeting/scoping/overriding , and then simply swap these write actions (I.e. collect only onprem/only to cloud/both as you wish).

Now that we have explained the hard way, the EASY way is to download and import THIS MANAGEMENT PACK , which will enable a new authoring rule template in OpsConsole:

The wizard behaves exactly like the one above it (which has been in Operations Manager since the first 2007 version), and allows all the fancy targeting and filtering, but adds the right references and the right write actions for you.