Azure Sentinel CEF Logs

[Robbie Wallis]

We are having a problem with our log collector setup whereby CEF logs are not being parsed to the right columns. They also repeat in syslog. Firstly can you see anything wrong in the format? Secondly could this be adjusted through a regex? I've seen syslog-ng can do adjustments I think?

Syslog entry

Aug 5 21:26:26 GREMLIN EventsFeederImporter.Host.exe: CEF:1|Panda Security|paps|02.54.00.0000|socket|socket|1|Date=2021-08-05 20:22:48.051462 MachineName=xxxxxx MachineIP=xxxxxxxxxxx User=NT AUTHORITY\\SYSTEM MUID=xxxxxxxxxxxxxxxxxxxxx LocalDateTime=2021-08-05T20:23:43.051+01:00 PandaTimeStatus=2 Protocol=TCP LocalPort=58823 Direction=Up LocalIp=172.16.11.27 Hash=xxxxxxxxxxxxxxxx DriveType=Fixed Path=PROGRAM_FILES_COMMON|\\Microsoft Shared\\ClickToRun\\officesvcmgr.exe Hostname=clients.config.office.net IP=51.11.16.254 Port=443 Times=1 Pid=45490 ValidSig=true Company=Microsoft Corporation Broken=true ImageType=EXE 64 ExeType=Unknown Prevalence=High PrevLastDay=Low Cat=Goodware MWName=

[#2]

You may need a parser, examples for various products are here that you can get ideas from: https://github.com/Azure/Azure-Sentinel/tree/master/Parsers that wont help with duplication however.

Also see: ASIM https://techcommunity.microsoft.com/t5/azure-sentinel/what-s-new-asim-file-activity-schema/ba-p/2609... there is a link to this upcoming webcast and other info: Deep Dive into Azure Sentinel Normalizing Parsers and Normalized Content: next week, register here.

[#3]

@Robbie Wallis 

So, basically CEF format should be as bellow - 
CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|Name|Severity|Extension 

In your case im not sure whether Signature ID  & Name value are correct or not. because they cant be same .

And yes you can do this adjustment with regex .

Thanks,

@Rabi_Sahu 

[#4]

I am wondering if the "EventsFeederImporter.Host.exe:" bit is a problem. There is a colon directly preceding CEF. I wonder if that is why the columns are being messed up? Does someone know how to remove it using syslog-ng?

[#5]

I managed to replace the program part and it wasnt that causing the issue. What I have noticed is that with the Panda Siemfeeder product there is a different amount of columns in logs depending on the type of event. There is always more than the base CEF format of CEF:Version|Device Vendor|Device Product|Device Version|Device Event Class ID|Name|Severity|[Extension]

It is almost as though Azure starts from the right hand side and works to the left as everything is offset. How does Azure no what column is what when you exceed the 8 default columns? I've tried to remove the additional columns by replacing the additional breaks | using subst in syslog-ng but so far no luck in doing so.

[#6]

I found the issue was two fold. One CEF version needed to be 0 and not 1. Also some reports had additional | characters and needed rewriting, I used syslog-ng to do this