Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community
SOLVED

Cannot get CommonSecurityLog Events to show in Sentinel "pattern not match"

Copper Contributor

There is a thread similar to this question but the other thread is specific to Fortinet.

 

I am building an integration with Sentinel and we have a product that generates Syslog messages under the kernel facility.  I am able to get the messages to the Syslog section of LogManagement, but I would like to see them in the CommonSecurityLog section so that my messages arebroken down for better querying.

 

In my omsagent.log I see them heartbeat messages going through and I see my CEF messages in the log but it says:

2019-04-30 09:49:37 -0700 [warn]: pattern not match: "Apr 30 16:48:34 WIN-J93TGI1PPFC Cyber-Ark[Vault] CEF:0|Cyber-Ark|Vault|10.8.0000|99|Open File|5|act=....

 

Currently I am sending messages from my solution to an rsyslog server and I installed the agent following the CEF connector in the Sentinel workspace.  I modified the security-config-omsagent.conf file to listen for:

kernel.notice @127.0.0.1:25226

 

made a modification to the security_events.conf to go through tcp instead of udp and the pattern not match error went away but now I don't see anything but heartbeat messages going through.

 

Please help, I have exhausted all of the documents I have, the one thing I have noticed is the rsyslog daemon is not interpreting my CEF headers correctly and I don;t know why, this is the only thing left that I think would be an issue.

 

Thank you ,

 -James

 

5 Replies

@Valon_Kolica 

@Ofer_Shezaf

@Chris Boehm 

Follow on question, where does the omsagent pull the data from, If I tail the logs on my rsyslog solution the structure is entirely different than what I am seeing in the omsagent logs.

If I modify the structure on the Syslog source I see it reflected on the rsyslog solution but not in the omsagent.  Is there a template for structuring data going to Sentinel?

best response confirmed by James_Stutes (Copper Contributor)
Solution

SOLUTION:

The pattern that it is matching to is defined in the security_events.conf under:

/etc/opt/microsoft/omsagent/f1886a55-033b-4c04-9198-50e9ddf678bb/conf/omsagent.d

 

<source>
    type syslog
    port 25226
    bind 127.0.0.1
    protocol_type udp
    tag oms.security
    format /^(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+):? ?(?:(?<host>[^: ]+) ?:?)? (?<ident>[a-zA-Z0-9_%\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?: *(?<message>.*)$/
    message_length_limit 4096
</source>

 

<filter oms.security.**>
    type filter_syslog_security
</filter>

 

This regex string needs to match the message that you are seeing in the log.  In my case the pid grouping did not match and that was causing the error.  Once you get the string that I have bolded above to match your message then that will clear the error.

NOTE: The '/' bookend characters need to be in the config file for the omsagent or when you restart the service you will get an error that the template is not recognized.

 

Now I am onto the next error:
[warn]: Failed to find data type for record with ident: 'Cyber-Ark'

 

What this is identifying is that the grouping of that regex string that is labeled ident does not have a known data type.... not sure what that means yet, more to follow

 

@James_Stutes I was struggling with the same error. I went through the code and there are only two idents it will take (for two of the appliances). I got rid of this error by changing the type from "filter_syslog_security" to "filter_syslog". 

are there any implications of changing "filter_syslog_security" to "filter_syslog", specially doing it to remove warning messages for Cisco ASA??
1 best response

Accepted Solutions
best response confirmed by James_Stutes (Copper Contributor)
Solution

SOLUTION:

The pattern that it is matching to is defined in the security_events.conf under:

/etc/opt/microsoft/omsagent/f1886a55-033b-4c04-9198-50e9ddf678bb/conf/omsagent.d

 

<source>
    type syslog
    port 25226
    bind 127.0.0.1
    protocol_type udp
    tag oms.security
    format /^(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+):? ?(?:(?<host>[^: ]+) ?:?)? (?<ident>[a-zA-Z0-9_%\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?: *(?<message>.*)$/
    message_length_limit 4096
</source>

 

<filter oms.security.**>
    type filter_syslog_security
</filter>

 

This regex string needs to match the message that you are seeing in the log.  In my case the pid grouping did not match and that was causing the error.  Once you get the string that I have bolded above to match your message then that will clear the error.

NOTE: The '/' bookend characters need to be in the config file for the omsagent or when you restart the service you will get an error that the template is not recognized.

 

Now I am onto the next error:
[warn]: Failed to find data type for record with ident: 'Cyber-Ark'

 

What this is identifying is that the grouping of that regex string that is labeled ident does not have a known data type.... not sure what that means yet, more to follow

 

View solution in original post