Forum Discussion

TarunV12's avatar
TarunV12
Copper Contributor
Dec 15, 2020

Azure sentinel log data fields missing

HI Team,

WE have observed that some of the fields are missing after we normalize the data in Azure Sentinel. Where customer want some other fields also need to show in Normalized data.

If we ingest payload data into the log Analytics the cost is going double.

Could we have  solution for this.

Regards,

Tv19

6 Replies

  • AndrewX's avatar
    AndrewX
    Iron Contributor
    Jun 21, 2021
    Hey any updates on this?

    My use case is similar. I'm using the standard CEF log format coming from the PaloAlto to the Azure Log Agent server. It is successfully sending logs into my Azure Log Analytics Workspace and connected successfully to Azure Sentinel.

    The issue I am having is that when I search the CommonSecurityLog table, I don't see certain columns like "source zone" or "destination zone" and many others, but I do see columns like source and target IP..
    • pemontto's avatar
      pemontto
      Brass Contributor
      Jun 21, 2021

      AndrewX this should come through as a device custom field for Palo Alto, if not does the field appear in the `AdditionalExtensions` field?

       

      FieldValue
      DeviceCustomString4UNTRUSTED
      DeviceCustomString4LabelSource Zone
      DeviceCustomString5
      TRUSTED
      DeviceCustomString5Label
      Destination Zone

       

      E.g. for some of our logs we want the `ThreatCategory` but it's not a top level field so we need to extract it from `AdditionalExtensions`.

       

       

      CommonSecurityLog
| extend ThreatCategory = extract(@"PanOSThreatCategory=(.*?);", 1, AdditionalExtensions)

       

       

      • AndrewX's avatar
        AndrewX
        Iron Contributor
        Jun 23, 2021

        pemontto hello and thank you for your reply.  I do have those columns it appears, but they seem to be blank?

         

         

        This is the output of a syslog message come off the PA, being received at the Azure log agent server using the following command "tcpdump port 514 -v"

         

        cs4Label=Source Zone cs4=UNTRUSTED cs5Label=Destination Zone cs5=TRUSTED

         Is my custom format wrong? Should i amend the formatting output to be the full "DeviceCustomString4" instead of "cs4"

  • CliveWatson's avatar
    CliveWatson
    Former Employee
    Dec 16, 2020
    Hi, which data sources and fields? Which Tables are you using i.e CommonSecurityLog and the missing data is 'a column called, "my data"'. A screenshot and samples would be helpful? How have you normalized the data, is this your own function or one or one (or all) of the provided ones?

    Thanks

Resources