Forum Discussion
Parse CEF logging and a map field within message
Hi all,
I have successfully configured the oms-agent and the Microsoft CEF python log forward script on a Ubuntu 20.04 x64 VM to forward Fortinet Analyzer logging to Sentinel. I receive the CEF logging in Sentinel. Most fields are mapped correctly. However, not all fields within the <message> are mapped, such as "ad.srccountry". This field is now in the AdditionalExtensions variable along with other data.
I want to have this variable in a separate field for log alerting purposes.
Problem:
Within CEF -> message -> AdditionalExtensions there is the ad.srccountry variable. I want the "ad.srccountry" variable mapped to 'cs1' (
DeviceCustomString1) in Sentinel. I got this idea from a list of all supported CEF message mapping field.
However, I cannot figure out how to configure this additional parse within the message field.
My configuration:
vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/security_events.conf
<source>
type syslog
port 25226
bind 127.0.0.1
protocol_type tcp
tag oms.security
format /(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+|\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.[\w\-\:\+]{3,12}):?\s*(?:(?<host>[^: ]+) ?:?)?\s*(?<ident>.*CEF.+?(?=0\|)|%ASA[0-9\-]{8,10})\s*:?(?<message>0\|.*|.*)/
<parse>
message_format auto
</parse>
</source>
<filter oms.security.**>
type filter_syslog_security
</filter>
Works (raw message and primary CEF filter): https://regex101.com/r/rdf4iP/1
vi /etc/opt/microsoft/omsagent/<tenantID>/conf/omsagent.d/syslog.conf
<source>
@type syslog
port 25224
bind 127.0.0.1
protocol_type tcp
tag oms.syslog
</source>
<filter oms.syslog.**>
@type parser
key_name message
format /.*ad.srccountry=(?<cs1>.*?)\s.*$/
</filter>
Raw message and secondary ad.srccountry filter: https://regex101.com/r/v6iZHs/1
systemctl restart omsagent-<TenantID>.service && systemctl restart syslog
But the DeviceCustomString1 (cs1) field in the CommonSecurityLog table from Sentinel stays empty:
Any tips or tricks how I can get this to work?
Thanks!
5 Replies
- Jordan MillsCopper ContributorThe conf file you're editing is for non-cef syslog messages. The conf that applies to CEF messages is security_events.conf.
- Ian-W2215Copper ContributorDid you make any progress? I have the same issue.
thanks
Ian - Clive_WatsonBronze ContributorYou maybe able to parse this from the AdditionalExtensions column in your query, see these examples:
https://github.com/Azure/Azure-Sentinel/search?q=AdditionalExtensions+extract- skarol1337Copper Contributor
Thank you Clive_Watson, but ideally I want to place the ad.srccountry variable in the cs1.
Configuring this at the syslog forwarder server side is much cleaner and easier to work with. Would you happen to know how to do this?
- Clive_WatsonBronze Contributor
skarol1337 Sorry I don't, hence my alternative method, hopefully someone else will know