Enrich your advanced hunting experience using network layer signals from Zeek
Published Apr 17 2023 10:00 AM 43.9K Views

UPDATE : July 9, 2023

The article has been updated to include new signatures added for SSL, DNS and NTLM protocols.

 

UPDATE : May 22, 2023

On July 18, 2023, Microsoft will be deprecating a subset of signatures found in the "NetworkSignaturesInspected" action type of Advanced Hunting. With the recent integration of Zeek providing advanced protocol parsing capabilities, which result in better visibility into full network sessions compared to the raw packet bytes found in the "NetworkSignaturesInspected" action type of Advanced Hunting today, the effort to consolidate will provide a better overall experience for our customers by reducing the signatures that serve similar functions without the added benefits provided by the new Zeek alternative. For customers currently using the "NetworkSignaturesInspected" action type, here is a list of signatures that will be deprecated, referenced alongside their alternatives available in Advanced Hunting: 

 

Protocol / Signature Name 

Old Action Type 

 New Action Type 

SSH 

NetworkSignatureInspected 

SshConnectionInspected 

FTP_Upload 

NetworkSignatureInspected 

FtpConnectionInspected 

FTP_Client 

NetworkSignatureInspected 

FtpConnectionInspected 

HTTP_Client 

NetworkSignatureInspected 

HttpConnectionInspected 

HTTP_Server 

NetworkSignatureInspected 

HttpConnectionInspected 

HTTP_RequestBodyParameters 

NetworkSignatureInspected 

HttpConnectionInspected 

HTTPS_Client 

NetworkSignatureInspected 

SslConnectionInspected 

DNS_Request 

NetworkSignatureInspected 

DnsConnectionInspected 

 

Steps you can take now:

Your organization might be using a "NetworkSignatureInspected" action type in your Advanced Hunting queries and custom detections. Particularly, you might be using a Signature Name that is going to be deprecated soon. Please update your queries with the new action types so that you can leverage this valuable data and avoid breaking your current custom detections.

 

An example of your old query:

 

DeviceNetworkEvents  
| where ActionType == "NetworkSignatureInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| where AdditionalFields.SignatureName == "SSH"

 

 

Your new query:

 

DeviceNetworkEvents  
| where ActionType == "SshConnectionInspected"

 

 

 

-----------

In our previous blog about hunting for network signatures in Microsoft 365 Defender, we described how we used device discovery capabilities to capture some network event information in deeper detail and expose them in advanced hunting with the NetworkSignatureInspected action type. Since then we have made several developments, the most significant being the integration with Zeek. This release has expanded what is possible for generating network detections across Microsoft Defender for Endpoint. That announcement, shared examples of detections created for PrintNightmare and NTLM password spraying attempts.

 

Today, we would like to share a variety of Zeek-based events in advanced hunting that will help you expand your investigation, hunting, and detection capabilities for identifying and addressing network-layer anomalies across HTTP, SSH and ICMP protocols. Using the new Zeek events, we will demonstrate how to perform network threat hunting while also covering some of the MITRE ATT&CK Matrix.

 

Note: As the integration with Zeek continues to mature, more action types will gradually be released over time. With the Zeek integration only supported on Windows devices, these action types will surface for connections to and from Windows device.

 

To identify these action types in your tenant, look for the value ConnectionInspected in the ActionType field of the DeviceNetworkEvents table of advanced hunting. The extra information is stored in the AdditionalFields column as a JSON data structure and has the commonly known Zeek fields per event, which can be parsed. These field names are identical to those that Zeek uses, which are documented on Zeek’s site. You can also check the Schema Reference flyout page on the advanced hunting pages to check for any new action types that were recently released.

 

Link to query

 

DeviceNetworkEvents
| where ActionType contains 'ConnectionInspected'
| distinct ActionType

 

 

The result of this query looks something like this:

 

cventour_0-1681377541830.png

Figure 1 – Sample result upon checking for ConnectionInspected in the ActionType table

 

The format of the action type will follow the [Protocol_Name]ConnectionInspected standard.

 

Inspecting HTTP connections

 

The HttpConnectionInspected action type contains extra information about HTTP connections, inbound or outbound. In cases where you click on an event of the HttpConnectionInspected action type, the page flyout will parse the additional fields and present them in a  format like the example below:

 

cventour_1-1681378349897.png

 

Figure 2 – Sample result of an HttpConnectionInspected action type

 

Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

 

Field Name

Description

direction

The direction of the conversation relevant to the Microsoft Defender for Endpoint-onboarded device, where the values are either ‘In’ or ‘Out’

host

The host header content

method

The HTTP method requested

request_body_len

Length of the HTTP message body in bytes

response_body_len

Length of the HTTP response body in bytes

status_code

The HTTP response code

status_msg

The full text message of the response

tags

A set of indicators of various attributes discovered and related to a particular request/response pair.

trans_depth

Represents the pipelined depth into the connection of the request/response transaction

uri

The complete URI that was requested

user_agent

The user_agent header of the request

version

The HTTP version used

 

Let’s look at a few examples of using the HttpConnectionInspected action type. In the first example, you want to look for rare user agents in the environment to identify potentially suspicious outbound web requests and cover the "T1071.001: (Application Layer Protocol) Web Protocols" technique.

 

Link to query

 

// Identify rare User Agent strings used in http conversations
DeviceNetworkEvents
| where ActionType == 'HttpConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction = tostring(json.direction), user_agent = tostring(json.user_agent)
| where direction == 'Out'
| summarize Devices = dcount(DeviceId) by user_agent
| sort by Devices asc

 

 

 

Suppose you have identified a suspicious-looking user-agent named “TrickXYZ 1.0” and need to determine which user/process/commandline combination had initiated that connection.  Currently, the HttpConnectionInspected events, as with all Zeek-related action types, do not contain that information, so you must execute a follow-up query by joining with events from  ConnectionEstablished action type. Here’s an example of a follow-up query:

 

Link to query

 

// Identify usage of a suspicious user agent
DeviceNetworkEvents
| where Timestamp > ago(1h) and ActionType == "HttpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend user_agent = tostring(json.user_agent)
| where user_agent == "TrickXYZ"
| project ActionType,AdditionalFields, LocalIP,LocalPort,RemoteIP,RemotePort, TimeKey = bin(Timestamp, 5m)
| join kind = inner (
DeviceNetworkEvents
| where Timestamp > ago(1h) and ActionType == "ConnectionSuccess"
| extend TimeKey = bin(Timestamp, 5m)) on LocalIP,RemoteIP,LocalPort,TimeKey
| project DeviceId, ActionType, AdditionalFields, LocalIP,LocalPort,RemoteIP,RemotePort , InitiatingProcessId,InitiatingProcessFileName,TimeKey

 

 

In another example, let’s look for file downloads from HTTP, particularly files of executable and compressed file extensions to cover the "T1105: Ingress tool transfer" technique:

 

Link to query

 

// Detect file downloads
DeviceNetworkEvents
| where ActionType == 'HttpConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction= tostring(json.direction), user_agent=tostring(json.user_agent), uri=tostring(json.uri)
| where uri matches regex @"\.(?:dll|exe|zip|7z|ps1|ps|bat|sh)$"

 

 

The new HTTP action type will unlock a variety of possibilities for detection on this protocol. We  look forward to seeing the queries you come up with by sharing your contributions with the community.

 

Looking at SSH connections

 

The SshConnectionInspected action type will display information on SSH connections. While decrypting the entire SSH traffic is not possible, the cleartext part of the SSH session initiation can provide valuable insights. Let’s look at the data found in the AdditionalFields section.

 

cventour_0-1681379880041.png

Figure 3 - Screenshot of additional fields that SshConnectionInspected generates.

 

The fields depend on the activity that was observed. Some of these fields might not appear depending on the connection. For example, if the client disconnected before completing the authentication, you will not have an auth_success field populated for that event..

 

Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

 

Field Name

Description

direction

The direction of the conversation relevant to the Defender for Endpoint-onboarded device, where the values are either ‘In’ or ‘Out’

auth_attempts

The number of authentication attempts until the success or failure of the attempted session.

auth_success

The success or failure in authentication, where ‘true’ means successful user authentication and ‘false’ means the user-provided credentials are incorrect.

client

The version and type of client used to authenticate to the SSH session.

host_key

Host public key value

server

SSH server information

version

SSH protocol major version used

uid

The unique ID of the SSH session attempt

 

Let’s look at a few advanced hunting examples using this action type. In the first example, you want to look for potentially infected devices trying to perform "T1110: Brute-Force" against remote servers using SSH as an initial step to “T1021.004: Lateral Movement - Remote Services: SSH”.

 

The query below will give you a list of Local/Remote IP combinations with at least 12 failed attempts (three failed authentications on four sessions) of SSH connections in the last hour. Feel free to use this example and adapt it to your needs.

 

Link to query

 

// Detect potential bruteforce/dictionary attacks against SSH
DeviceNetworkEvents
| where ActionType == 'SshConnectionInspected'
| extend json = todynamic(AdditionalFields)
| extend direction=tostring(json.direction), auth_attempts = toint(json.auth_attempts), auth_success=tostring(json.auth_success)
| where auth_success=='false'
| where auth_attempts > 3
| summarize count() by LocalIP, RemoteIP
| where count_ > 4
| sort by count_ desc

 

 

 

In the next example, let’s suppose you are looking to identify potentially vulnerable SSH versions and detect potentially unauthorized client software being used to initiate SSH connections and operating systems that are hosting SSH server services in your environment:

 

Link to query

 

// Identify Server/Client pairs being used for SSH connections
DeviceNetworkEvents
| where  ActionType == "SshConnectionInspected"
| extend json = todynamic(AdditionalFields)
| project Server = tostring(json.server),Client = tostring(json.client)
| distinct Server ,Client

 

 

cventour_1-1681380056116.png

Figure 4 - An example result with a short description of the different components

 

The results above describe breaking down the SSH banners to identify the different components. A short analysis of the banners shows that the server is Ubuntu 22.04, running OpenSSH version 8.9, and the client software is WinSCP version 5.21.3. Now, you can search these versions online to verify if they are vulnerable.

 

Note: The query above can be used to surface potential "T1046: Network Service Discovery" attempts, as attackers may try to search for unpatched or vulnerable SSH services to compromise.

 

Reviewing ICMP connections

 

The IcmpConnectionInspected action type will provide details about ICMP-related activity. The breadth of fields generated creates opportunities for some interesting detections. Here’s an example of the human-readable view of the event as shown on the event flyout page

 

cventour_2-1681380100285.png

Figure 5 – Sample result of an IcmpConnectionInspected action type

 

 Below, you will find a complete list of fields that this action type can expose and the respective descriptions:

 

Field Name

Description

direction

The direction of the conversation relevant to the Defender for Endpoint-onboarded device, where the values are either ‘In’ or ‘Out’

conn_state

The state of the connection. In the screenshot example OTH means that no SYN packet was seen. Read the Zeek documentation for more information on conn_state.

duration

The length of the connection, measured in seconds

missed_bytes

Indicates the number of bytes missed in content gaps, representing packet loss. 

orig_bytes

The number of payload bytes the originator sent. For example, in ICMP this designates the payload size of the ICMP packet.

orig_ip_bytes

The number of IP level bytes that the originator sent as seen on the wire and taken from the IP total_length header field.

orig_pkts

The number of packets that the originator sent.

resp_bytes

The number of payload bytes the responder sent.

resp_ip_bytes

The number of IP level bytes that the responder sent as seen on the wire.

resp_pkts

The number of packets that the responder sent. 

Uid

Unique Zeek ID of the transaction.

 

Let’s explore a few examples of hunting queries that you can use to leverage the ICMP connection information collected by Defender for Endpoint.

 

In the first example, you wish to look for potential data leakage via ICMP to cover the "T1048: Exfiltration Over Alternative Protocol" or "T1041: Exfiltration Over C2 Channel" techniques. The idea is to look for outbound connections and check the payload bytes a device sends in a given timeframe. We will parse the direction, orig_bytes, and duration fields and look for conversations over 100 seconds where more than 500,000 were sent. The numbers are used as an example and do not necessarily indicate malicious activity. Usually, you will see the download and upload are almost equal for ICMP traffic because most devices generate “ICMP reply” with the same payload that was observed on the “ICMP echo” request.

 

Link to query

 

// search for high upload over ICMP
DeviceNetworkEvents
| where ActionType == "IcmpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend Upload = tolong(json['orig_bytes']), Download = tolong(json['resp_bytes']), Direction = tostring(json.direction), Duration = tolong(json.duration)
| where Direction == "Out" and Duration > 100 and Upload > 500000
| top 10 by Upload
| project RemoteIP, LocalIP, Upload = format_bytes(Upload, 2, "MB"), Download = format_bytes(Download, 2, "MB"),Direction,Duration,Timestamp,DeviceId,DeviceName

 

 

 

Below is an example result after exfiltrating a large file over ICMP to another device on the network:

 

cventour_3-1681380100287.png

 

In the last example, you wish to create another hunting query that helps you detect potential Ping sweep activities in your environment to cover the "T1018: Remote System Discovery" and "T1595: Active Scanning" techniques. The query will look for outbound ICMP traffic to internal IP addresses, create an array of the targeted IPs reached from the same source IP, and display them if the same source IP has pinged more than 5 IP Addresses within a 10-minute time window.

 

Link to query

 

// Search for ping scans
DeviceNetworkEvents
| where ActionType == "IcmpConnectionInspected"
| extend json = todynamic(AdditionalFields)
| extend Direction = json.direction
| where Direction == "Out" and ipv4_is_private(RemoteIP)
| summarize IpsList = make_set(RemoteIP) by DeviceId, bin(Timestamp, 10m)
| where array_length(IpsList) > 5

 

 

Identifying the origin process of ICMP traffic can be challenging as ICMP is an IP-Layer protocol. Still, we can use some OS-level indications to narrow down our search. We can use the following query to identify which process-loaded network, or even ICMP-specific, binaries:

 

Link to query

 

DeviceImageLoadEvents
| where FileName =~ "icmp.dll" or FileName =~ "Iphlpapi.dll"

 

 

 

Inspecting SSL connections

 

The SslConnectionInspected action type contains extra information about SSL connections, inbound or outbound. In cases where you click on an event of the SslConnectionInspected action type, the page flyout will parse the additional fields and present them in a  format like the example below:

cventour_0-1688891531222.png

Figure 6 – Sample result of an SslConnectionInspected action type

 

Below, you will find a complete list of fields that this action type can expose and the respective descriptions ( sourced from Zeek SSL log documentation )

 

Field Name

Description

direction

The direction of the conversation relevant to the Microsoft Defender for Endpoint-onboarded device, where the values are either ‘In’ or ‘Out’

version

The SSL/TLS version that the server chose.

cipher

The SSL/TLS cipher suite that the server chose.

curve

The elliptic curve the server chose when using ECDH/ECDHE.

server_name

The value of the Server Name Indicator SSL/TLS extension. It indicates the server name that the client was requesting.

established

Flag which indicate if this ssl session has been established successfully, or if it was aborted during the handshake.

subject

Subject of the X.509 certificate offered by the server.

issuer

Issuer of the signer of the X.509 certificate offered by the server.

resumed

Flag to indicate if the session was resumed reusing the key material exchanged in an earlier connection.

client_issuer

Subject of the signer of the X.509 certificate offered by the client.

client_subject

Subject of the X.509 certificate offered by the client.

 

In the following example query, you wish to search https connections to external IP that use self-signed certificates :

 

Link to query

 

 

DeviceNetworkEvents 
| where ActionType == "SslConnectionInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| extend issuer = tostring(AdditionalFields.issuer), direction = tostring(AdditionalFields.direction)
| where direction == "Out" and not(ipv4_is_private(RemoteIP))
| where AdditionalFields.issuer matches regex @"CN=\S+$"

 

 

 

 

Inspecting DNS

 

The DnsConnectionInspected action type contains extra information about DNS connections, inbound or outbound. In cases where you click on an event of the DnsConnectionInspected action type, the page flyout will parse the additional fields and present them in a format like the example below:

 

cventour_1-1688891802540.png

Figure 7 – Sample result of an DnsConnectionInspected action type

 

Below, you will find a complete list of fields that this action type can expose and the respective descriptions (sourced from the Zeek DNS log documentation)

 

Using this action type we can explore and investigate the DNS traffic of each managed endpoint. In the following example we will try to identify potentially suspicious DNS traffic directed towards a DNS server that is not listed in the DNS server configuration of any managed endpoint:

 

Query Link

 

let knownDnsServers =
 DeviceNetworkInfo
 | where isnotempty( DnsAddresses)
 | mv-expand todynamic(DnsAddresses)
 | where DnsAddresses !in("::", "127.0.0.1","0.0.0.0")
 | distinct tostring(DnsAddresses);
DeviceNetworkEvents
| where ActionType == "DnsConnectionInspected"
| extend AdditionalFields = todynamic(AdditionalFields)
| extend Direction = tostring(AdditionalFields.direction), QueryType = tostring(AdditionalFields.qtype_name), Query = tostring(AdditionalFields.query), Answers = todynamic(AdditionalFields.answers)
| where Direction == "Out" and QueryType == "TXT" and RemoteIP !in(knownDnsServers) and isnotempty(Answers)

 

 

 

Inspecting NTLM traffic

 

For network connection with NTLM authentication we’ve create “NtlmAuthenticationInspected” action type. By clicking on the AdditionalFields the page flyout will parse the additional fields and present them in a format like the example below:

 

cventour_2-1688891934945.png

Figure 2 – Sample result of an NtlmConnectionInspected action type

 

Below, you will find a complete list of fields that this action type can expose and the respective descriptions (sourced from the Zeek NTLM log documentation)

 

 

Field Name

Description

direction

The direction of the conversation relevant to the Microsoft Defender for Endpoint-onboarded device, where the values are either ‘In’ or ‘Out’

username

A 16-bit identifier assigned by the program that generated the DNS query. Also used in responses to match up replies to outstanding queries.

hostname

Hostname given by the client

domainname

A descriptive name for the class of the query.

server_nb_computer_name

NetBIOS name given by the server in a CHALLENGE.

server_nb_domain_name

 

NetBIOS domain name given by the server in a CHALLENGE.

 

server_dns_computer_name

DNS name given by the server in a CHALLENGE.

server_tree_name

Tree name given by the server in a CHALLENGE.

success

Indicate whether or not the authentication was successful.

server_version

The server NTLM version.

version

The client NTLM version.

Using using this action type we can track NTLM authentication that were observed on managed endpoints. In the following example we will try to identify multiple failed logons in short interval against internet-facing :

 

Query Link

 

let devices = DeviceInfo
|summarize arg_max(Timestamp, *) by DeviceId
| where IsInternetFacing
|distinct DeviceId;
DeviceNetworkEvents
| where ActionType == "NtlmAuthenticationInspected" and DeviceId in(devices)
| extend AdditionalFields = todynamic(AdditionalFields)
| extend Direction = tostring(AdditionalFields.direction),username = tostring(AdditionalFields.username), success = iff(isempty(AdditionalFields.success),  false, tobool(AdditionalFields.success))
| where isnotempty( username) and not(success) and Direction == "In"
| summarize Attempts = make_set(username) by SourceIP = LocalIP, bin(Timestamp, 10m),DeviceId
| where array_length(Attempts) > 3

 

 

More information

 

Understand which versions of the Microsoft Defender for Endpoint agent support the new integration here:

Find out more details about the integration in our ZeekWeek 2022 presentations:

View the open-source contribution in Zeek’s GitHub repository:

Previous announcements:

11 Comments
Version history
Last update:
‎Jul 17 2023 04:14 AM
Updated by: