Advanced hunting data schema changes
Published Dec 03 2019 04:31 AM 37.4K Views
Microsoft

Update: We've pushed out the date for this change to from Dec 15, 2019 to Dec 29, 2019. Note that saved queries will be automatically updated. Also, existing names will continue to work for at least 1 month after the transition.

Hello there, Hunters!

 

As announced in previous blog postwe will be making changes to how Advanced hunting will expose information through its schema.

 

With the broad initiative to unify Microsoft security capabilities under Microsoft Threat ProtectionAdvanced hunting will eventually support new types of data sets from various productsemail events from Office 365 ATP, app activity from Microsoft Cloud App Security, and richer identity information from Azure ATP. To prepare for these changes and keep the schema intuitive, we want to ensure that data providers are easily identified by customers as they transition to an expanded schema. 

 

On December 22, we will start supporting this initiative by adding “Device” as a prefix to tables populated with device-related information. Moving forward, as the schema expands, corresponding prefixes will be used for tables populated by data from other providers as shown in the table below. 

 

Data provider 

Prefix 

Table name examples 

ETA 

Microsoft Defender ATP 

Device 

DeviceProccessCreationEvents 

DeviceFileEvents 

Dec 29, 2019 

Office 365 ATP 

Email 

EmailEvents 

EmailAttachmentInfo 

TBD 

Identity Threat Protection (Microsoft Cloud App Security + Azure ATP) 

App 

IdentityQueryEvents 

AppFileEvents 

TBD 

 

Here are the actual changes to existing table names that we will apply on December 22. 

 

Old table name 

New table name 

AlertEvents 

DeviceAlertEvents

MachineInfo 

DeviceInfo 

MachineNetworkInfo 

DeviceNetworkInfo 

ProcessCreationEvents 

DeviceProcessEvents 

NetworkCommunicationEvents 

DeviceNetworkEvents 

FileCreationEvents 

DeviceFileEvents 

RegistryEvents 

DeviceRegistryEvents 

LogonEvents 

DeviceLogonEvents 

ImageLoadEvents 

DeviceImageLoadEvents 

MiscEvents 

DeviceEvents 

 

We are also renaming the following columns to ensure that their names remain meaningful when they are used across more tables. 

 

Old column name 

New column name 

EventTime 

Timestamp 

MachineId 

DeviceId 

ComputerName 

DeviceName 

RegistryMachineTag 

RegistryDeviceTag 

RemoteComputerName 

RemoteDeviceName 

 

Automatic updates to saved queries and custom detections 

On December 29, we will automatically update all your saved queries and custom detections with the new table and column names, so nothing is required from your end. Keep in mind, however, that the query you have in the Advanced hunting query editor will not be updated automatically. 

 

Changes to the schema displayed in the portal and the auto-complete functionality will also take full effect on December 22. From that point on, only the new names will be visible in the UI.

 

Deprecation of old names 

To give you more time to transition, old names will continue to work as aliases for a short period. We do recommend that you stop using the old names and manually modify queries you've saved outside the portal.

 

We will deprecate the old names after at least a month, so they will eventually stop working. 

 

Questions? Add a comment below so we can discuss! 

4 Comments
Version history
Last update:
‎Jan 22 2020 05:08 PM
Updated by: