All sorts of activity and security data can be collected by Azure Sentinel for storage and mining. The Syslog data collector is good for collecting data from Linux platforms but needs a helping hand to access information produced by the Linux kernel’s audit subsystem, kaudit, and the optional user-space daemon, auditd. These components can be configured to generate event data when syscalls are invoked, such as process creations, file access, and other telemetry that could be used to identify malicious activity.
While it is possible to use the audisp daemon to redirect auditd events to syslog, there are a couple of potential problems with this approach. The first issue is that while kaudit is a standard component on most Linux distributions, the user-space daemon auditd is not, and audisp relies on auditd to work. The second issue is that audisp simply forwards the auditd event data without any filtering or processing.
Filtering events is essential to reduce the noise generated by known system tools that run regularly; these include cron jobs to rotate logs and system tools that ensure software is kept up to date. There is usually little need to see this data in your SIEM and filtering it at the source reduces bandwidth and storage requirements.
Similarly, event processing is important to enrich the data so that it makes more sense when it is mined. The Linux audit sub-system uses numerical values for a range of identifiers, and these need to be converted into corresponding names for them to make sense. It’s possible to do this in the SIEM but it is easier if this happens before the events leave the machine that generated them.
The Microsoft audit collection tool, AUOMS, includes configurable filtering and processing steps, collects events from either kaudit or auditd/audisp, and outputs them in a range of formats to specified locations and pipes. The MSTIC (Microsoft Threat Intelligence Center) research branch of AUOMS can be used to forward events to the syslog, from where they can be collected by Azure Sentinel. This blog post will describe how to install this branch of AUOMS and configure Azure Sentinel to collect the events.
In this blog post, we will cover how to:
Azure Sentinel also supports the use of Jupyter Notebooks and Ian Hellen has already written a great blog post Jupyter Notebooks in Sentinel which covers their use.
If you don’t already have an Azure Sentinel workspace, then you’ll need to create one. The Quickstart guide provides details on the prerequisites and steps to create an Azure Sentinel workspace.
The Operations Management Suite agent is used by Azure Sentinel to collect the syslog. Installing it is straight forward and is covered in this related blog post.
Installing our research branch of AUOMS is a three step process: the first step is to install the required mainline packages; the second step is to download and compile the source; and the third step is to copy the programs and configuration files to the correct locations.
AUOMS requires the following packages, which can be installed as follows on Ubuntu Linux:
sudo apt install rapidjson-dev libmsgpack-dev libxml2-dev libboost-all-dev libaudit-dev libauparse-dev build-essential cmake
Other distributions may require different commands and some of the packages may have slightly different names.
The source code of AUOMS is available on Github. Enter the following commands to download and build the project:
git clone https://github.com/microsoft/OMS-Auditd-Plugin.git cd OMS-Auditd-Plugin git checkout MSTIC-Research cmake . make
Once the source code has been compiled, the program and its configuration files need to be copied to the correct locations. You shouldn’t need to edit the auoms.conf file; you may edit the syslog.conf file to modify the event filtering and processing, and you may edit the mstic-research.rules file to change which events are generated.
Enter the following commands to install the AUOMS agent:
sudo cp auoms auomscollect auomsctl /opt/microsoft/auoms/bin sudo cp conf/auoms.conf /etc/opt/microsoft/auoms sudo cp conf/outconf.d/testout.conf /etc/opt/microsoft/auoms/outconf.d/syslog.conf sudo cp rules/mstic-research.rules /etc/opt/microsoft/auoms/rules.d sudo service auoms start sudo update-rc.d auoms defaults
If you are installing OMS Agent and AUOMS on a computer that already has auditd (including audisp) installed, or you later install auditd on a computer after installing OMS Agent and AUOMS, then you will need the following additional configuration steps to use AUOMS in this scenario:
sudo service auoms stop sudo service auditd stop sudo sed -i -e 's/active = no/active = yes/' /etc/audisp/plugins.d/auoms.conf sudo update-rc.d auditd defaults sudo service auditd start sudo service auoms start
Note, auditd can be installed with:
sudo apt install auditd
The mstic-research.rules file (installed into /etc/opt/microsoft/auoms/rules.d) contains the kaudit/auditd rules that generate event records. These rules initially report on:
You may edit this file to change these rules to suit your environment. The AUOMS agent will automatically notice the change when the file is written and will load the new rules. You can check which rules are in use with:
sudo /opt/microsoft/auoms/bin/auomsctl -l
The syslog.conf file (installed into /etc/opt/microsoft/auoms/outconf.d) contains the filter descriptions that specify which processes and syscalls are excluded from being reported upon. The initial configuration filters events from the following:
Note that the provided rules aren’t configured to generate events from the connect syscall and that the filters to block it are provided as examples only.
Now that AUOMS is generating audit events and sending them to the syslog, we need to configure Azure Sentinel to collect them.
The events will start to be collected and may take fifteen minutes to arrive. The raw events can be viewed as follows:
Syslog events will appear in the results window. Note that the audit event data is contained in the SyslogMessage column. This column can be parsed to extract the EventType. Example parsers have been uploaded to the Azure Sentinel GitHub to demonstrate how to parse process creations (execve), syscalls, and user errors such as logon failures. These can be saved as functions for use in hunting queries.
As an example of threat hunting, you can search for some popular crypto currency miners being downloaded using the CryptoCurrencyMiners.yaml hunting query:
Hopefully, this article has helped you understand how easy it is to use Azure Sentinel to obtain audit event data from Linux machines and to hunt for threats within it.
We will continue to develop the MSTIC-Research branch of the AUOMS agent, adding additional functionality and improving throughput. We will also update our mstic-research.rules file with extra event types to report, and our syslog.conf file with extra processes and syscalls to filter out.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.