Forum Discussion

omrip's avatar
omrip
Copper Contributor
Oct 17, 2019

Parsing syslog

1. i am ingesting firewall logs as syslog and trying to parse out the fields accordingly using the split command, i have a problem that the beginig of the logs is not piped and i have made the split ...
data collection
Log Data
CliveWatson's avatar
CliveWatson
Former Employee
Oct 17, 2019

omrip  

Options like:

print SyslogMessage = "Security F180 Block: FWD|UDP|p4|192.168.x,x|67|00:15:5d:0f:c4:01|255.255.255.255|68|bootpc||LAN-2-INTERNET|4017|0.0.0.0|0.0.0.0|0|1|"
| project SyslogMessage 
| extend device       = extract("Security (.*?)Block:", 1, SyslogMessage) 
| extend deviceaction = extract("USER=(.*?)COMMAND", 1, SyslogMessage)

 

or

print SyslogMessage = "Security F180 Block: FWD|UDP|p4|192.168.x,x|67|00:15:5d:0f:c4:01|255.255.255.255|68|bootpc||LAN-2-INTERNET|4017|0.0.0.0|0.0.0.0|0|1|"
| extend p = split(SyslogMessage, "|") 
//| extend cleanp  = trim(@"[^\w]+",tostring(p))
| extend pos1 = split(p.[0], " ")
| extend FWactivity   = trim(@"[^\w]+",tostring(pos1.[0]))
| extend Device       = trim(@"[^\w]+",tostring(pos1.[1]))
| extend DeviceAction = trim(@"[^\w]+",tostring(pos1.[2]))
| extend srcMAC       = trim(@"[^\w]+",tostring(p.[5]))
| extend DestPort     = trim(@"[^\w]+",tostring(p.[4]))
// etc...
| project-away SyslogMessage , p, pos1

 

KQL is good at doing parsing like this at execution time.

omrip's avatar
omrip
Copper Contributor
Oct 20, 2019

CliveWatson  that is very helpful, tnx

when ingesting the logs to the syslog instead of CEF  connector i am very limited due to the small amount of fileds that exists on the syslog table in comparison with the CEF

how can i overcome it?

 

  • CliveWatson's avatar
    CliveWatson
    Former Employee
    Oct 20, 2019

    omrip 

     

    CommonSecurityLog
| getschema 
| summarize count(ColumnName) 

Syslog
| getschema 
| summarize count(ColumnName)

     

    I make the difference 152 vs 15 columns of data,  are there certain columns you are missing in Syslog?  Is the data you require in the Syslog but needs extracting / parsing which is I believe one of the things CEF does for you?   BTW, I'm no expert on CEF or Syslog, but keen to understand your use case.

     

    Thanks 

     

     

    • Villagerchan8829's avatar
      Villagerchan8829
      Copper Contributor
      Jan 12, 2025

      *Jan 11 23:43:30: %SYS-5-CONFIG_I: Configured from console by admin0001 on vty0 (192.168.10.1)

      *Jan 11 23:43:23: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/2, changed state to up

      Hello CliveWatson May get for advise, I would like to parse syslog message above for example logs. I'm still looking for the document to parse the syslog messages. please help me to guide and how can i parse for like above messages. 

       

       

Resources