Forum Discussion

Jamie_Seddon's avatar
Jamie_Seddon
Copper Contributor
Oct 24, 2019

Palo Alto Data Connector - pattern not match

Hello, hoping someone can help me with what appears to be incorrect Regex syntax in a configuration file.

 

I'm trying to connect our Palo Alto logs to Sentinel and i've followed all of the instructions here:

https://docs.microsoft.com/en-us/azure/sentinel/connect-paloalto#step-2-forward-palo-alto-logs-to-the-syslog-agent

 

I am receiving syslogs thought rsyslog, the OMS Agent is also receiving logs, however the OMS agent log file shows this:

 

2019-10-24 15:55:45 -0700 [warn]: pattern not match: "Oct 24 15:55:45  1,2019/10/24 15:55:45,013201006249,TRAFFIC,start,2049,2019/10/24 15:55:45
 
From what I have determined the match problem stems from this file:
 
/etc/opt/microsoft/omsagent/<workspace ID>/conf/omsagent.d/security_events.conf
 
and specifically this line:
 
format /^(?<time>(?:\w+ +){2,3}(?:\d+:){2}\d+):? ?(?:(?<host>[^: ]+) ?:?)? (?<ident>[a-zA-Z0-9_%\/\.\-]*)(?:\[(?<pid>[0-9]+)\])?: *(?<message>.*)$/ 
 
The problem is, this .conf file containing this regex came from Microsoft as part of the Palo Alto data collector setup instructions so i'm not entirely sure where to begin formatting the regex to provide what Sentinel expects?
 
Any ideas?
 
Thanks in advance,
Jamie

4 Replies

  • Roger_Fleming's avatar
    Roger_Fleming
    Former Employee
    Nov 13, 2019

    Jamie_Seddon 

     

    The CEF format we support follows this format:

    CEF:Version|Device Vendor|Device Product|Device Version|Signature ID|Name|Severity|Extension

    Its import to use the right template from the Palo Alto PDF listed in the Palo Alto Connector page

    here is an example Palo Alto version 8.0

    Palo Alto Traffic format for version 8
    CEF:0|Palo Alto Networks|PAN-OS|$sender_sw_version|$subtype|$type|1|rt=$cefformatted-receive_time deviceExternalId=$serial src=$src dst=$dst sourceTranslatedAddress=$natsrc destinationTranslatedAddress=$natdst cs1Label=Rule cs1=$rule suser=$srcuser duser=$dstuser app=$app cs3Label=Virtual System cs3=$vsys cs4Label=Source Zone cs4=$from cs5Label=Destination Zone cs5=$to deviceInboundInterface=$inbound_if deviceOutboundInterface=$outbound_if cs6Label=LogProfile cs6=$logset cn1Label=SessionID cn1=$sessionid cnt=$repeatcnt spt=$sport dpt=$dport sourceTranslatedPort=$natsport destinationTranslatedPort=$natdport flexString1Label=Flags flexString1=$flags proto=$proto act=$action flexNumber1Label=Total bytes flexNumber1=$bytes in=$bytes_sent out=$bytes_received cn2Label=Packets cn2=$packets PanOSPacketsReceived=$pkts_received PanOSPacketsSent=$pkts_sent start=$cefformatted-time_generated cn3Label=Elapsed time in seconds cn3=$elapsed cs2Label=URL Category cs2=$category externalId=$seqno reason=$session_end_reason PanOSDGl1=$dg_hier_level_1 PanOSDGl2=$dg_hier_level_2 PanOSDGl3=$dg_hier_level_3 PanOSDGl4=$dg_hier_level_4 PanOSVsysName=$vsys_name dvchost=$device_name cat=$action_source PanOSActionFlags=$actionflags PanOSSrcUUID=$src_uuid PanOSDstUUID=$dst_uuid PanOSTunnelID=$tunnelid PanOSMonitorTag=$monitortag PanOSParentSessionID=$parent_session_id PanOSParentStartTime=$parent_start_time PanOSTunnelType=$tunnel Threat
    n
    • ChrisRussell's avatar
      ChrisRussell
      Copper Contributor
      Nov 14, 2019

      Roger_Fleming I'm using the CEF format from the PDF, and I've fixed the issues with copying and pasting into a text editor, but I'm only getting maybe 10% of the log into Sentinel, almost none of the pertinent key value pairs make it. Looking at the rsyslog server, they are hitting that box with maybe one line and almost nothing else. Can't figure out what step might have been missed but it seemed all pretty straight forward in the documentation. 

      • Roger_Fleming's avatar
        Roger_Fleming
        Former Employee
        Nov 14, 2019

        ChrisRussell 

         

        Make sure you ae using the correct version of the product. If you could run the following command it will provide the data being received by the syslog and the omsagent

        tcpdump -A -ni any port 514 -vvv -s 0

        should look like this in return 
        TCEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Log|tcp-high-ports|Unknown|act=Drop deviceDirection=0 rt=1573748935000 spt=46783 dpt=42627 cs2Label=Rule Name layer_name=Network layer_uuid=edf46e83-f10b-4fbc-93e9-fab40887b8d1 match_id=3 parent_rule=0 rule_action=Drop rule_uid=3f994325-9c52-4b18-ba44-307ad4929fb2 ifname=eth0 logid=0 loguid={0x5dcd80c9,0x1,0x501a8c0,0x1737aca9} origin=192.168.1.5 originsicname=cn\=cp_mgmt,o\=FlemingGW..y76ath sequencenum=2 version=5 dst=192.168.1.5 inzone=External outzone=Local product=VPN-1 & FireWall-1 proto=6 service_id=tcp-high-ports src=89.248.168.222

        Then run

        tcpdump -A -ni any port 25226 -vvv -s 0

        should look the same