Ingest Sample CEF data into Azure Sentinel
An Azure Sentinel Proof of Concept (PoC) is a great opportunity to effectively evaluate technical and business benefits. The onboarding of Microsoft cloud services is mostly a one-click experience; and thus, the ingestion of Syslog/CEF events presents the most notable challenge.
In this blog, I will go through the required steps showing how to ingest CEF events into Azure Sentinel for evaluation purposes.
Preparation & Use
The following tasks describe the necessary preparation steps.
- Install an Ubuntu server - it does not matter whether in Azure or On-Prem. This server will be used as a connector server.
- Deploy and configure the CEF agent on the connector server.
- Validate the connectivity between connector server and Azure Sentinel.
- Ingest data into Azure Sentinel via connector server.
- How to use the ingested data in Azure Sentinel.
Installation of connector server
The installation of the connector server is quite straight forward. After successful installation, make sure that the connector server is deployed with latest updates and includes Python.
The connector server does not require an incoming connection, it should only have an outside connection to the Azure Sentinel instance.
Deploy and validate the CEF agent on the connector server
The connector CEF connector page describes the required steps for installation and validation – Link.
Sample events for evaluation
For this blog, I decided to use the sample CEF events generated by Advanced Threat Analytics - Link. For evaluation purposes, any vendor's CEF events can be used.
Import CEF events into Azure Sentinel via connector server
I created the following command lines to ingest four sample Advanced Threat Analytics CEF events into Azure Sentinel.
logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|AbnormalSensitiveGroupMembershipChangeSuspiciousActivity|Abnormal modification of sensitive groups|5|start=2018-12-12T18:52:58.0000000Z app=GroupMembershipChangeEvent suser=krbtgt msg=krbtgt has uncharacteristically modified sensitive group memberships. externalId=2024 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c113d028ca1ec1250ca0491"
logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|LdapBruteForceSuspiciousActivity|Brute force attack using LDAP simple bind|5|start=2018-12-12T17:52:10.2350665Z app=Ldap msg=10000 password guess attempts were made on 100 accounts from W2012R2-000000-Server. One account password was successfully guessed. externalId=2004 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c114acb8ca1ec1250cacdcb"
logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|EncryptionDowngradeSuspiciousActivity|Encryption downgrade activity|5|start=2018-12-12T18:10:35.0334169Z app=Kerberos msg=The encryption method of the TGT field of TGS_REQ message from W2012R2-000000-Server has been downgraded based on previously learned behavior. This may be a result of a Golden Ticket in-use on W2012R2-000000-Server. externalId=2009 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c114f938ca1ec1250cafcfa"
logger -p local4.warn -t CEF "CEF:0|Microsoft|ATA|1.9.0.0|EncryptionDowngradeSuspiciousActivity|Encryption downgrade activity|5|start=2018-12-12T17:00:31.2975188Z app=Kerberos msg=The encryption method of the Encrypted_Timestamp field of AS_REQ message from W2012R2-000000-Server has been downgraded based on previously learned behavior. This may be a result of a credential theft using Overpass-the-Hash from W2012R2-000000-Server. externalId=2010 cs1Label=url cs1=https://192.168.0.220/suspiciousActivity/5c113eaf8ca1ec1250ca0883"
To validate whether the CEF events are received by Syslog server, use the following command line.
sudo tac /var/log/syslog | grep CEF -m 10
How to use the ingested data in Azure Sentinel
Once the ingestion is processed, you can query the data. The CEF logs will reside within the data in the CommonSecurityLog table.