Blog Post

Microsoft Defender XDR Blog
8 MIN READ

Microsoft 365 Defender Streaming API: Identity and CloudApp Events in General Availability

Michael Shalev's avatar
Michael Shalev
Icon for Microsoft rankMicrosoft
May 18, 2022

We're happy to share that Microsoft 365 Defender Streaming API support for the following event types (tables) is General Availability:

  • Identity events
    • IdentityLogonEvents
    • IdentityQueryEvents
    • IdentityDirectoryEvents
  • CloudAppEvents

These new event types enable automating queries and enrichment about user accounts, on-premises and online services authentication activities, and queries about Active Directory objects and cloud app and identity-related events, significantly extending custom enrichment and analytics possibilities from endpoint and email to identity and cloud apps.

 

Identity use cases and queries that are now possible include account brute force attempts, credential access/dump attempts, reconnaissance and discovery activities, sensitive Active Directory group modification, sensitive LDAP queries, and much more.

 

Cloud app use cases and queries include cloud service-based exfiltration attempts such as adding mail permissions to applications, changing ADFS trust settings and other techniques - many of which have been used by actors.

 

Here are some links to related blogs about these tables being added to Microsoft 365 Defender Advanced Hunting that go deeper into queries and use cases these event types enable:

 

These event types are also now available in Microsoft Sentinel - for more information see:

 

Here are the Event Types/Tables' schemas for easy access (event type titles link directly to the respective online documentation):

 

IdentityLogonEvents

The IdentityLogonEvents table in the advanced hunting schema contains information about authentication activities made through your on-premises Active Directory captured by Microsoft Defender for Identity and authentication activities related to Microsoft online services captured by Microsoft Defender for Cloud Apps.

 

Column name

Data type

Description

Timestamp

datetime

Date and time when the event was recorded

ActionType

string

Type of activity that triggered the event. See the in-portal schema reference for details

Application

string

Application that performed the recorded action

LogonType

string

Type of logon session, specifically:

  • Interactive: User physically interacts with the machine using the local keyboard and screen
  • Remote interactive (RDP): User interacts with the machine remotely using Remote Desktop, Terminal Services, Remote Assistance, or other RDP clients
  • Network: Session initiated when the machine is accessed using PsExec or when shared resources on the machine, such as printers and shared folders, are accessed
  • Batch: Session initiated by scheduled tasks
  • Service: Session initiated by services as they start

Protocol

string

Network protocol used

FailureReason

string

Information explaining why the recorded action failed

AccountName

string

User name of the account

AccountDomain

string

Domain of the account

AccountUpn

string

User principal name (UPN) of the account

AccountSid

string

Security Identifier (SID) of the account

AccountObjectId

string

Unique identifier for the account in Azure AD

AccountDisplayName

string

Name of the account user displayed in the address book. Typically a combination of a given or first name, a middle initiation, and a last name or surname.

DeviceName

string

Fully qualified domain name (FQDN) of the device

DeviceType

string

Type of device

OSPlatform

string

Platform of the operating system running on the machine. This indicates specific operating systems, including variations within the same family, such as Windows 11, Windows 10, etc.

IPAddress

string

IP address assigned to the endpoint and used during related network communications

Port

string

TCP port used during communication

DestinationDeviceName

string

Name of the device running the server application that processed the recorded action

DestinationIPAddress

string

IP address of the device running the server application that processed the recorded action

DestinationPort

string

Destination port of related network communications

TargetDeviceName

string

Fully qualified domain name (FQDN) of the device that the recorded action was applied to

TargetAccountDisplayName

string

Display name of the account that the recorded action was applied to

Location

string

City, country, or other geographic location associated with the event

Isp

string

Internet service provider (ISP) associated with the endpoint IP address

ReportId

long

Unique identifier for the event

AdditionalFields

string

Additional information about the entity or event


IdentityQueryEvents

The IdentityQueryEvents table in the advanced hunting schema contains information about queries performed against Active Directory objects, such as users, groups, devices, and domains.

 

Column name

Data type

Description

Timestamp

datetime

Date and time when the event was recorded

ActionType

string

Type of activity that triggered the event. See the in-portal schema reference for details

Application

string

Application that performed the recorded action

QueryType

string

Type of query, such as QueryGroup, QueryUser, or EnumerateUsers

QueryTarget

string

Name of user, group, device, domain, or any other entity type being queried

Query

string

String used to run the query

Protocol

string

Protocol used during the communication

AccountName

string

User name of the account

AccountDomain

string

Domain of the account

AccountUpn

string

User principal name (UPN) of the account

AccountSid

string

Security Identifier (SID) of the account

AccountObjectId

string

Unique identifier for the account in Azure AD

AccountDisplayName

string

Name of the account user displayed in the address book. Typically a combination of a given or first name, a middle initiation, and a last name or surname.

DeviceName

string

Fully qualified domain name (FQDN) of the endpoint

IPAddress

string

IP address assigned to the endpoint and used during related network communications

Port

string

TCP port used during communication

DestinationDeviceName

string

Name of the device running the server application that processed the recorded action

DestinationIPAddress

string

IP address of the device running the server application that processed the recorded action

DestinationPort

string

Destination port of related network communications

TargetDeviceName

string

Fully qualified domain name (FQDN) of the device that the recorded action was applied to

TargetAccountUpn

string

User principal name (UPN) of the account that the recorded action was applied to

TargetAccountDisplayName

string

Display name of the account that the recorded action was applied to

Location

string

City, country, or other geographic location associated with the event

ReportId

long

Unique identifier for the event

AdditionalFields

string

Additional information about the entity or event

 

IdentityDirectoryEvents

The IdentityDirectoryEvents table in the advanced hunting schema contains events involving an on-premises domain controller running Active Directory (AD). This table captures various identity-related events, like password changes, password expiration, and user principal name (UPN) changes. It also captures system events on the domain controller, like scheduling of tasks and PowerShell activity.

 

Column name

Data type

Description

Timestamp

datetime

Date and time when the event was recorded

ActionType

string

Type of activity that triggered the event. See the in-portal schema reference for details

Application

string

Application that performed the recorded action

TargetAccountUpn

string

User principal name (UPN) of the account that the recorded action was applied to

TargetAccountDisplayName

string

Display name of the account that the recorded action was applied to

TargetDeviceName

string

Fully qualified domain name (FQDN) of the device that the recorded action was applied to

DestinationDeviceName

string

Name of the device running the server application that processed the recorded action

DestinationIPAddress

string

IP address of the device running the server application that processed the recorded action

DestinationPort

string

Destination port of the activity

Protocol

string

Protocol used during the communication

AccountName

string

User name of the account

AccountDomain

string

Domain of the account

AccountUpn

string

User principal name (UPN) of the account

AccountSid

string

Security Identifier (SID) of the account

AccountObjectId

string

Unique identifier for the account in Azure Active Directory

AccountDisplayName

string

Name of the account user displayed in the address book. Typically a combination of a given or first name, a middle initiation, and a last name or surname.

DeviceName

string

Fully qualified domain name (FQDN) of the device

IPAddress

string

IP address assigned to the device during communication

Port

string

TCP port used during communication

Location

string

City, country, or other geographic location associated with the event

ISP

string

Internet service provider associated with the IP address

ReportId

long

Unique identifier for the event

AdditionalFields

string

Additional information about the entity or event

 

CloudAppEvents

The CloudAppEvents table in the advanced hunting schema contains information about activities in various cloud apps and services covered by Microsoft Defender for Cloud Apps.

 

Column name

Data type

Description

Timestamp

datetime

Date and time when the event was recorded

ActionType

string

Type of activity that triggered the event

Application

string

Application that performed the recorded action

ApplicationId

string

Unique identifier for the application

AccountObjectId

string

Unique identifier for the account in Azure Active Directory

AccountId

string

An identifier for the account as found by Microsoft Defender for Cloud Apps. Can be Azure Active Directory ID, user principal name, or other identifiers.

AccountDisplayName

string

Name of the account user displayed in the address book. Typically a combination of a given or first name, a middle initiation, and a last name or surname.

IsAdminOperation

string

Indicates whether the activity was performed by an administrator

DeviceType

string

Type of device based on purpose and functionality, such as "Network device", "Workstation", "Server", "Mobile", "Gaming console", or "Printer"

OSPlatform

string

Platform of the operating system running on the device. This column indicates specific operating systems, including variations within the same family, such as Windows 11, Windows 10, etc.

IPAddress

string

IP address assigned to the endpoint and used during related network communications

IsAnonymousProxy

string

Indicates whether the IP address belongs to a known anonymous proxy

CountryCode

string

Two-letter code indicating the country where the client IP address is geolocated

City

string

City where the client IP address is geolocated

Isp

string

Internet service provider (ISP) associated with the IP address

UserAgent

string

User agent information from the web browser or other client application

ActivityType

string

Type of activity that triggered the event

ActivityObjects

dynamic

List of objects, such as files or folders, that were involved in the recorded activity

ObjectName

string

Name of the object that the recorded action was applied to

ObjectType

string

Type of object, such as a file or a folder, that the recorded action was applied to

ObjectId

string

Unique identifier of the object that the recorded action was applied to

ReportId

string

Unique identifier for the event

RawEventData

string

Raw event information from the source application or service in JSON format

AdditionalFields

dynamic

Additional information about the entity or event

AccountType

string

Type of user account, indicating its general role and access levels, such as Regular, System, Admin, DcAdmin, System, Application

IsExternalUser

boolean

Indicates whether a user inside the network doesn't belong to the organization's domain

IsImpersonated

boolean

Indicates whether the activity was performed by one user for another (impersonated) user

IPTags

dynamic

Customer-defined information applied to specific IP addresses and IP address ranges

IPCategory

string

Additional information about the IP address

UserAgentTags

dynamic

More information provided by Microsoft Defender for Cloud Apps in a tag in the user agent field. Possible values: Native client, Outdated browser, Outdated operating system, Robot

 

We would love to hear your feedback about these additions to Microsoft 365 Defender Streaming API: how you're making use of them, what scenarios they enable, and anything else you would like to share by emailing m365dapis@microsoft.com.

 

Microsoft 365 Defender API team

Updated Jul 06, 2022
Version 4.0
automation
microsoft defender for cloud apps
microsoft defender for identity
  • quinzy's avatar
    quinzy
    Copper Contributor
    May 30, 2022

    Hi, I am using powerbi:

    like described 

    https://docs.microsoft.com/en-us/windows/security/threat-protection/microsoft-defender-atp/api-power-bi

    but for the new tables. It gives me an error

    DataSource.Error. Web.Contents failed to get contents from 'https://api.securitycenter.microsoft.com/api/advancedqueries?key=ldentityLogonEvents%20%7C%20where%20ActionType%20%21contains%20%27LogonSuccess%27' (400): Bad Request Details DataSourceKind = Web DataSourcePath = https://api.securitycenter.microsoftcom/api/advancedqueries Url = https://api.securitycenter.microsoft.com/api/advancedqueries?key=ldentityLogonEvents%20%7C%20where%20ActionType%20%21contains%20%27LogonSuccess%27

    this is my query in powerbi

     

    let
      AdvancedHuntingQuery = "IdentityLogonEvents | where ActionType !contains 'LogonSuccess'",
      Response = Json.Document(Web.Contents(HuntingUrl, [Query=[key=AdvancedHuntingQuery]])),
      TypeMap = #table(
                { "Type", "PowerBiType" },
                {
                    { "Double",   Double.Type },
                    { "Int64",    Int64.Type },
                    { "Int32",    Int32.Type },
                    { "Int16",    Int16.Type },
                    { "UInt64",   Number.Type },
                    { "UInt32",   Number.Type },
                    { "UInt16",   Number.Type },
                    { "Byte",     Byte.Type },
                    { "Single",   Single.Type },
                    { "Decimal",  Decimal.Type },
                    { "TimeSpan", Duration.Type },
                    { "DateTime", DateTimeZone.Type },
                    { "String",   Text.Type },
                    { "Boolean",  Logical.Type },
                    { "SByte",    Logical.Type },
                    { "Guid",     Text.Type }
                }),
      Schema = Table.FromRecords(Response[Schema]),
      TypedSchema = Table.Join(Table.SelectColumns(Schema, {"Name", "Type"}), {"Type"}, TypeMap, {"Type"}),
      Results = Response[Results],
      Rows = Table.FromRecords(Results, Schema[Name]),
      Table = Table.TransformColumnTypes(Rows, Table.ToList(TypedSchema, (c) => {c{0}, c{2}})),
      #"Sorted rows" = Table.Sort(Table, {{"count_DeviceId", Order.Descending}}),
      #"Filtered rows" = Table.SelectRows(#"Sorted rows", each [VulnerabilitySeverityLevel] = "Critical")
    in
      #"Filtered rows"
     
    so likely it is not ok 😞
    or is there a new hunting url for it , if so the page need to be refresh. Can you help me with it
    kind regards
    Quinzy
  • JeremyTBradshaw's avatar
    JeremyTBradshaw
    Steel Contributor
    Aug 28, 2024

    Wondering if anyone here can help me to understand, in the Defender Portal (security.microsoft.com), when I'm using Advanced Hunting there, when am I and when am I not querying the streaming API vs the live table (for lack of knowing what else to call the "live table")?  Specifically, I'm looking for EmailEvents table entries, filtering by LatestDeliveryLocation, but what I find here - EmailEvents table in the advanced hunting schema - Microsoft Defender XDR | Microsoft Learn - is this:

     Note

    * The LatestDeliveryLocation and LatestDeliveryAction columns are not available in the Streaming API.

     

    Does it work like this: when I select the time range from the UI menu I'm searching the live table, and when I set timerange in my query, that is searching the streaming API?

     

    Thanks in advance.