Failed to configure/use CEF syslog facility

[Etienne Noiret]

Hello,

I'm stuck with the configuration of CEF connector as described at https://docs.microsoft.com/en-us/azure/sentinel/connect-common-event-format. I have chosen to go for an automatic deployment, so I'd expect everything works out of the box, but still I have issues:

- There seems to be no process listening on UDP (or TCP) port 514 in the created VM. I had to modify several lines in the config to enable this.

- When sending messages to this port, nothing gets forwarded to port 25226

- When sending messages directly to port 25226, I encounter the following warning message in omsagent.log:  pattern not match: "blabla"

Is there a command line or utility I can use to send a simple CEF text message and have it been acquired by the VM and sent to Sentinel? The CommonSecurityLogs table remains desperately empty...

Thanks for the help!

Etienne

[#2]

@Eliav Levi: Is this something you can speak to?  

[#3]

@ seem to have a similar problem trying to configure this:

https://docs.microsoft.com/en-gb/azure/sentinel/connect-fortinet#step-1-connect-your-fortinet-applia...

[#4]

@Rohan Cragg 

@Chris Boehm : Is this something you could help with?

@Ofer_Shezaf 

[#5]

@Rohan Cragg 

@Etienne Noiret 

Whenever you're configuring the rsyslog service we're pushing a security_events.conf file as part of the integration process ( creating a listener for port 25226 )

Example in the steps:

Download and install the security_events config file that configures the Syslog agent to listen on port 25226.

sudo wget -O /etc/opt/microsoft/omsagent/{0}/conf/omsagent.d/security_events.conf "https://aka.ms/syslog-config-file-linux"

Where {0} should be replaced with your workspace GUID.

Security_events.conf

<source>
  type syslog
  port 25226
  bind 127.0.0.1
  protocol_type udp
  tag oms.security
  format /^(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+):? ?(?:(?<host>[^: ]+) ?:?)? (?<ident>[a-zA-Z0-9_%\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?: *(?<message>.*)$/
  message_length_limit 4096
</source>

<filter oms.security.**>
  type filter_syslog_security
</filter>

You'll note the config file has UDP, please move it to TCP if you're using TCP and restart the service.

This should fix the communication issues

I'd advise following this doc for testing your communication: https://docs.microsoft.com/en-gb/azure/sentinel/connect-fortinet#step-3-validate-connectivity 

[#6]

Thanks @Chris Boehm 

We eventually got this working, and we think (thought not sure) that the problem was more that the logs being sent from Fortinet applicance were not yet in the CEF format - which we went and fixed.

Admittedly I did go a try a few other things so not 100% what fixed this for us but we're all good now.

[#7]

@Chris Boehm 

We're having a similar issue.  I've verified that I have the security events.conf file on my server ,and that the Fortinet logs I'm pulling are in CEF format. 

The logs are screaming in on 514 - but never go to Sentinel on 25266.  Anything else I can check?  Would setting the syslog port to 25266 on the firewall help at all?

CEF:0|Fortinet|Fortigate|v6.0.2

-rw-r--r-- 1 root root 359 Feb 12 01:58 /etc/opt/microsoft/omsagent/6e7121df-eea3-4365-b775-410954a08c7d/conf/omsagent.d/security_events.conf

[#8]

@LauraPen 

If you're seeing everything on 514 but not 25226 it seems like the forwarding isn't occurring correctly.

Confirm the configuration file

Make sure you've restarted the service, after adding the configuration change it requires a reboot to start up the new security file configuration.

I would confirm the following:

For rsyslog, ensure the following is in the configuration file for CEF

:msg, contains, "CEF"       @127.0.0.1:25226

if the configuration file seems to be configured correctly, I would personally advise collecting the following information:

• Files:
  • /etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/security_events.conf
  • /var/opt/microsoft/omsagent/<workspace id>/log/omsagent.log
  • /etc/rsyslog.conf
  • The content of folder: /etc/rsyslog.d/

• Output of the following commands:
  1. CEF logs is the correct format in the var/messages file ("tac /var/log/syslog | grep CEF -m 10" or "tac /var/log/messages | grep CEF -m 10")
  2. TCP dump on ports 514 and 25226  ("tcpdump -i any dst port 514 -vv" or "tcpdump -i any dst port 25226 -vv")

Then create a support ticket if my comment above didn't help.

[#9]

@Chris Boehm 

thanks for replying!

it actually turned out to be a setting in the rsyslog.conf file.  I was setting up logging from 3 diff points, and 2 out of the 3 had the right settings - but the host I wrote about didn't have this uncommented:

the moral of the story is to never assume that all the files are the same :) 

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514