By Nicholas DiCola & Cristhofer Romeo Munoz
What is CEF collection?
Most network and security systems support either Syslog or CEF (which stands for Common Event Format) over Syslog as means for sending data to a SIEM. Azure Sentinel provides the ability to ingest data from an external solution. If your appliance or system enables you to send logs over Syslog using the Common Event Format (CEF), the integration with Azure Sentinel enables you to easily run analytics, and queries across the data. This makes Syslog or CEF the most straight forward ways to stream security and networking events to Azure Sentinel.
The advantage of CEF over Syslog is that it ensures the data is normalized making it more immediately useful for analysis using Sentinel. However, unlike many other SIEM products, Sentinel allows ingesting unparsed Syslog events and performing analytics on them using query time parsing.
The number of systems supporting Syslog or CEF is in the hundreds, please make sure to check out the Azure Sentinel grand list for a comprehensive list of sources supporting CEF.
In this blogpost, we will provide details on how CEF collection works and the best practices you should consider when configuring common event format collection in Azure Sentinel.
How does it work?
CEF Collection in Azure Sentinel uses a Linux machine that is used as a log forwarder between your security solution and Azure Sentinel. The Linux machine can be inyour on-prem environment, in Azure or in other clouds. As part of the deployment process, the Log Analytics agent is installed on the Linux machine and serves to relay the events securely to your Azure Sentinel workspace.
The following flow chart details the high-level steps to configure CEF collection in Azure Sentinel:
For detailed information on deploying CEF and how CEF collection works, please visit the Azure Setninel CEF documentation.
Tip: Want to ingest test CEF data? here is how to do that.
Securing your CEF collection
Secure your machine
Make sure to configure the machine's security according to your organization's security policy. For example, you can configure your network to align with your corporate network security policy and change the ports and protocols in the daemon to align with your requirements. You can use the following instructions to improve your machine security configuration: Secure VM in azure, Best practices for Network security.
Note: To use TLS communication between the security solution and the Syslog machine, you will need to configure the Syslog daemon (rsyslog or syslog-ng) to communicate in TLS: Encrypting Syslog Traffic with TLS - rsyslog, Encrypting log messages with TLS – syslog-ng.
Properly Secure an Azure VM with Public IP
Common Event Format uses UDP 514 therefore messages will be sent in clear text. If we use an unencrypted connection and a third party intercepts the connection, they can see all the information exchanged in plain text. TLS can protect this communication traffic. Additionally, a bad actor can potentially intercept and send messages to your Azure VMs to create false records. Make sure to protect the Azure VM with either a Network Security Group or NVA. On the NSG, you should configure a list of source IP addresses you want to allow to communicate to the VM.
Deploy CEF collector machine close to source
Let’s review the common scenarios:
On-prem source – We recommend you deploy the CEF collector on-prem. Syslog CEF is sent over UDP port 514 in plain text. Once it reaches the CEF collector, it is sent to Azure Sentinel using HTTPs. Deploying on-prem ensure your data is not sent in the clear outside your network.
If on-prem deployment is not an option at all, we recommend you deploy in a cloud VM and use TCP over TLS or ensure there is a secure path (VPN) between the source and the collector VM.
Cloud source – We recommend you deploy the CEF collector in the same virtual network as the source. You can also use VNet peering to have multiple VNets connected to the CEF collector. This again prevents your traffic from being sent over the internet in the clear.
Ensure the quality of your CEF collection
Do not configure the source to send to 2 CEF Collectors
To avoid duplicate records/messages in Azure Sentinel do not configure your third-party security appliance to send data to 2 Common Event Format collectors. Additionally, sending duplicate data will increase your consumption bill therefore ensure that your security appliances are set to send data to a single CEF collector. You may consider using a load balancer or rsyslog cluster (https://www.rsyslog.com/load-balancing-for-rsyslog/) as an aleternative.
TCP is the default protocol, always use
Syslog CEF can support UDP or TCP. However, TCP is now the default protocol and should always be used unless the source device does not support TCP. The advantage is the reliability of the TCP protocol.
Scaling CEF collection
Want to scale CEF or Syslog collection? Use a VM scale set as described here
Querying the Data
Once you satisfy the configuration steps in the Common Event Format data connector, you can then query the data. CEF logs will reside within the CommonSecurityLog table, hence by querying for CommonSecurityLog you will have visibility to your CEF data.
Now that we can see the data in Azure Sentinel, we now can build workbooks, analytic rules, hunting queries, or associate it with other data for correlation.
Conclusion
In this blog, you learned how Common Event Format collection works and the best practices to consider when configuring CEF collection in Azure Sentinel. To learn more about Azure Sentinel, see the following articles: