Parse CEF logging and a map field within message

[skarol1337]

Hi all,

I have successfully configured the oms-agent and the Microsoft CEF python log forward script on a Ubuntu 20.04 x64 VM to forward Fortinet Analyzer logging to Sentinel. I receive the CEF logging in Sentinel. Most fields are mapped correctly. However, not all fields within the <message> are mapped, such as "ad.srccountry". This field is now in the AdditionalExtensions variable along with other data.

I want to have this variable in a separate field for log alerting purposes.

Problem:

Within CEF -> message -> AdditionalExtensions there is the ad.srccountry variable. I want the "ad.srccountry" variable mapped to 'cs1' (

DeviceCustomString1) in Sentinel. I got this idea from a list of all supported CEF message mapping field.

However, I cannot figure out how to configure this additional parse within the message field.

My configuration:

vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/security_events.conf 

<source>
  type syslog
  port 25226
  bind 127.0.0.1
  protocol_type tcp
  tag oms.security
  format /(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.[\w\-\:\+]{3,12}):?\s*(?:(?<host>[^: ]+) ?:?)?\s*(?<ident>.*CEF.+?(?=0\|)|%ASA[0-9\-]{8,10})\s*:?(?<message>0\|.*|.*)/
  <parse>
     message_format auto
  </parse>
</source>

<filter oms.security.**>
  type filter_syslog_security
</filter>

Works (raw message and primary CEF filter): https://regex101.com/r/rdf4iP/1

vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/syslog.conf

<source>
  @type syslog
  port 25224
  bind 127.0.0.1
  protocol_type tcp
  tag oms.syslog
</source>

<filter oms.syslog.**>
  @type parser
  key_name message
  format /.*ad.srccountry=(?<cs1>.*?)\s.*$/
</filter>

Raw message and secondary ad.srccountry filter: https://regex101.com/r/v6iZHs/1

systemctl restart omsagent-<TenantID>.service && systemctl restart syslog

But the DeviceCustomString1 (cs1) field in the CommonSecurityLog table from Sentinel stays empty:

Any tips or tricks how I can get this to work?

Thanks!

[#2]

You maybe able to parse this from the AdditionalExtensions column in your query, see these examples:

https://github.com/Azure/Azure-Sentinel/search?q=AdditionalExtensions+extract

[#3]

Thank you @Clive_Watson, but ideally I want to place the ad.srccountry variable in the cs1.

Configuring this at the syslog forwarder server side is much cleaner and easier to work with. Would you happen to know how to do this?

[#4]

@skarol1337  Sorry I don't, hence my alternative method, hopefully someone else will know

[#5]

Did you make any progress? I have the same issue.

thanks
Ian

[#6]

The conf file you're editing is for non-cef syslog messages. The conf that applies to CEF messages is security_events.conf.