Blog Post

Microsoft Sentinel Blog
3 MIN READ

Ingest Sample CEF data into Azure Sentinel

Alp Babayigit's avatar
Alp Babayigit
Icon for Microsoft rankMicrosoft
Dec 13, 2019

An Azure Sentinel Proof of Concept (PoC) is a great opportunity to effectively evaluate technical and business benefits. The onboarding of Microsoft cloud services is mostly a one-click experience; and thus, the ingestion of Syslog/CEF events presents the most notable challenge.

 

In this blog, I will go through the required steps showing how to ingest CEF events into Azure Sentinel for evaluation purposes.

 

Preparation & Use

The following tasks describe the necessary preparation steps.

  • Install an Ubuntu server - it does not matter whether in Azure or On-Prem. This server will be used as a connector server.
  • Deploy and configure the CEF agent on the connector server.
  • Validate the connectivity between connector server and Azure Sentinel.
  • Ingest data into Azure Sentinel via connector server.
  • How to use the ingested data in Azure Sentinel.

 

Installation of connector server

The installation of the connector server is quite straight forward. After successful installation, make sure that the connector server is deployed with latest updates and includes Python.

 

The connector server does not require an incoming connection, it should only have an outside connection to the Azure Sentinel instance.

 

Deploy and validate the CEF agent on the connector server

The connector CEF connector page describes the required steps for installation and validation – Link.

 

 

Sample events for evaluation

For this blog, I decided to use the sample CEF events generated by Advanced Threat Analytics - Link. For evaluation purposes, any vendor's CEF events can be used.

 

Import CEF events into Azure Sentinel via connector server

I created the following command lines to ingest four sample Advanced Threat Analytics CEF events into Azure Sentinel.

 

logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|AbnormalSensitiveGroupMembershipChangeSuspiciousActivity|Abnormal modification of sensitive groups|5|start=2018-12-12T18:52:58.0000000Z app=GroupMembershipChangeEvent suser=krbtgt msg=krbtgt has uncharacteristically modified sensitive group memberships. externalId=2024 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c113d028ca1ec1250ca0491"

logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|LdapBruteForceSuspiciousActivity|Brute force attack using LDAP simple bind|5|start=2018-12-12T17:52:10.2350665Z app=Ldap msg=10000 password guess attempts were made on 100 accounts from W2012R2-000000-Server. One account password was successfully guessed. externalId=2004 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c114acb8ca1ec1250cacdcb"

logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|EncryptionDowngradeSuspiciousActivity|Encryption downgrade activity|5|start=2018-12-12T18:10:35.0334169Z app=Kerberos msg=The encryption method of the TGT field of TGS_REQ message from W2012R2-000000-Server has been downgraded based on previously learned behavior. This may be a result of a Golden Ticket in-use on W2012R2-000000-Server. externalId=2009 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c114f938ca1ec1250cafcfa"

logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|EncryptionDowngradeSuspiciousActivity|Encryption downgrade activity|5|start=2018-12-12T17:00:31.2975188Z app=Kerberos msg=The encryption method of the Encrypted_Timestamp field of AS_REQ message from W2012R2-000000-Server has been downgraded based on previously learned behavior. This may be a result of a credential theft using Overpass-the-Hash from W2012R2-000000-Server. externalId=2010 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c113eaf8ca1ec1250ca0883"

 

 

To validate whether the CEF events are received by Syslog server, use the following command line.

 

sudo tac /var/log/syslog | grep CEF -m 10

 

 

How to use the ingested data in Azure Sentinel

Once the ingestion is processed, you can query the data. The CEF logs will reside within the data in the CommonSecurityLog table.

 

 

Updated Nov 03, 2021
Version 2.0
  • mclaes's avatar
    mclaes
    Brass Contributor
    Hey, Question: Has anyone got the CEF collector to work on CentOS (7)? I tried on three fresh Azure CentOS VM instances and they never worked. I was sniffing the TCP/25226 to see whether CEF logs were forwarded from the rsyslog deamon but no luck.. Yesterday, I decided to 'change religion 🙂 ' and went with a Ubuntu image and it worked in 5'.. thanks! Maarten.
  • mangatbitc's avatar
    mangatbitc
    Copper Contributor

    Same issue with CentOS. it sends the cef logs to syslog table instead of CommonSecurityLog in Azure Sentinel. Works for Ubuntu Server though.

  • pavankemi's avatar
    pavankemi
    Brass Contributor

    HI Alp,

     

    Where can we find the logs that are being ingested using the logger command? I can see after ingesting these events are being sent to Azure sentinel but could not find the directory var/log/syslog on the Centos server where CEF connector has been installled./.