Get full data into Playbook

%3CLINGO-SUB%20id%3D%22lingo-sub-1193346%22%20slang%3D%22en-US%22%3EGet%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1193346%22%20slang%3D%22en-US%22%3E%3CP%3EHi%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWe%20are%20currently%20trying%20to%20automate%20some%20alerts%20through%20Playbooks.%3C%2FP%3E%3CP%3EWe%20created%20a%20custom%20alert%20that%20checks%20for%20Impossible%20Travel%20Alerts%20from%20MCAS.%3C%2FP%3E%3CP%3EThis%20works%20well.%20But%20the%20issue%20is%20that%20some%20data%20that%20is%20in%20the%20SecurityAlerts%20table%20(like%20ExtendedProperties%20and%20Entities)%20isn't%20forwarded%20to%20the%20Logic%20App%20when%20it's%20triggered%20by%20Sentinel.%3C%2FP%3E%3CP%3EI%20attached%20a%20screenshot%20of%20what%20data%20is%20forwarded%20through%20a%20logic%20app%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBecause%20Sentinel%20Entities%20do%20not%20support%20an%20array%2C%20but%20only%20one%20value.%20It's%20not%20possible%20to%20use%20those%2C%20because%20an%20impossible%20travel%20alert%20has%20multiple%20IP's.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20tried%20to%20use%20the%20Log%20Analytics%20connector%20for%20Logic%20Apps%20and%20run%20the%20query%20manually.%20But%20the%20issue%20is%20that%20the%20query%20sometimes%20returns%20an%20array%2Fsometimes%20an%20object.%20And%20the%20parse%20JSON%20fails%20because%20of%20this.%20So%20this%20workaround%20isn't%20valid%20either%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESo%20we%20are%20looking%20for%20another%20way%20to%20get%20all%20the%20data%20of%20a%20Sentinel%20alert%20into%20a%20playbook.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDoes%20anybody%20have%20something%20for%20this%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1194150%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1194150%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F186539%22%20target%3D%22_blank%22%3E%40Thijs%20Lecomte%3C%2FA%3E%26nbsp%3B%20Not%20sure%20what%20you%20mean%20when%20you%20say%20that%20Entities%20do%20not%20support%20arrays.%26nbsp%3B%20If%20the%20alert%20that%20creates%20the%20Incident%20finds%20multiple%20events%20and%20each%20of%20those%20events%20has%20matching%20entities%2C%20then%20the%20incident%20will%20have%26nbsp%3B%20multiple%20entities.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20currently%20have%20one%20incident%20that%20is%20made%20up%20of%2013%20events%20and%20has%205%20IP%20and%206%20Account%20Entities%20in%20it.%26nbsp%3B%20Using%20a%20Playbook%20to%20write%20the%20Entities%20to%20a%20Teams%20message%20I%20see%20that%20it%20writes%20out%20the%20Entities%20in%20a%20JSON%20array.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ELooking%20at%20your%20image%20it%20shows%20the%20same%20thing%20just%20that%2C%20in%20your%20case%2C%20you%20only%20have%201%20Entity%20listed%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1194177%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1194177%22%20slang%3D%22en-US%22%3EThanks%20for%20the%20response.%3CBR%20%2F%3EThe%20alert%20is%20have%20only%20has%20one%20event.%3CBR%20%2F%3E%3CBR%20%2F%3EThe%20event%20that%20comes%20from%20MCAS.%3CBR%20%2F%3EI%20can%20understand%20that%20multiple%20events%20in%20one%20alert%20can%20lead%20to%20multiple%20entities.%3CBR%20%2F%3E%3CBR%20%2F%3EBut%20can%20one%20event%20in%20an%20alert%20lead%20into%20multiple%20entities%3F%3CBR%20%2F%3E%3CBR%20%2F%3EOr%20is%20there%20another%20way%20to%20get%20the%20full%20event%20details%20into%20Playbooks%3F%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1194182%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1194182%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F186539%22%20target%3D%22_blank%22%3E%40Thijs%20Lecomte%3C%2FA%3E%26nbsp%3BOne%20event%20will%20only%20have%20up%20to%20a%20single%20value%20for%20each%20of%20the%20entities.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1194186%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1194186%22%20slang%3D%22en-US%22%3EAny%20chance%20that%20will%20be%20changed%20in%20the%20future%3F%3CBR%20%2F%3E%3CBR%20%2F%3EOr%20any%20way%20to%20get%20the%20full%20details%20through%20the%20playbook%3F%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1194471%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1194471%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F186539%22%20target%3D%22_blank%22%3E%40Thijs%20Lecomte%3C%2FA%3E%26nbsp%3BI%20would%20not%20expect%20this%20to%20change.%26nbsp%3B%20%26nbsp%3B%20What%20do%20you%20mean%20when%20you%20say%20full%20details%3F%26nbsp%3B%20What%20is%20missing%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1196258%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1196258%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F186539%22%20target%3D%22_blank%22%3E%40Thijs%20Lecomte%3C%2FA%3E%26nbsp%3BThis%20is%20something%20I%20have%20done%20and%20probably%20you%20could%20do.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EEntities%20and%20Extended%20properties%20are%20JSON%20parameters.%20Using%20the%20data%20operations%20connector%20parse%20these%20parameters.%20You%20could%20get%20the%20JSON%20schema%20from%20the%20entities%2Fextended%20properties%20logs.%26nbsp%3B%3C%2FP%3E%3CP%3EAlmost%20all%20of%20the%20times%20the%20scheduled%20query%20is%20available%20in%20the%20Extended%20properties.%20By%20parsing%20you%20will%20convert%20JSON%20into%20String%20and%20take%20the%20Query%20parameter%20out.%20Initialize%20the%20variable%20naming%20it%20as%20XYZ%20and%20use%20Query%20with%20a%20time%20period(this%20is%20must).%26nbsp%3B%3C%2FP%3E%3CP%3EUsing%20Azure%20Log%20Analytics%20run%20this%20initialized%20variable%20and%20you%20should%20be%20good%20to%20go..%20Seems%20like%20a%20lot%20of%20pain%20but%20works%20just%20fine.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1196830%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1196830%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F428046%22%20target%3D%22_blank%22%3E%40Pranesh1060%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThat's%20how%20I%20do%20it%20too%20ATM.%3C%2FP%3E%3CP%3EBut%20this%20means%20there%20is%20an%20extra%20query%20to%20log%20analytics%20which%20seems%20unnecessary%20to%20me...%20Shouldn't%20this%20be%20available%20by%20default%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F46875%22%20target%3D%22_blank%22%3E%40Gary%20Bushey%3C%2FA%3E%26nbsp%3B%20Some%20data%20that%20I%20see%20in%20the%20SecurityAlerts%20table%20isn't%20able%20in%20the%20Playbook.%20These%20are%20columns%20like%20'Entities'%20and%20'ExtendedProperties'%20(these%20are%20columns%20that%20contain%20extra%20data%20from%20MCAS).%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1198295%22%20slang%3D%22en-US%22%3ERe%3A%20Get%20full%20data%20into%20Playbook%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1198295%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F186539%22%20target%3D%22_blank%22%3E%40Thijs%20Lecomte%3C%2FA%3E%26nbsp%3BI%20believe%20what%26nbsp%3B%40%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F428046%22%20target%3D%22_blank%22%3EPranesh1060%3C%2FA%3E%26nbsp%3Bmeant%20is%20that%20you%20can%20take%20the%20%3CSTRONG%3EExtended%20properties%3C%2FSTRONG%3E%20and%20the%20%3CSTRONG%3EEntities%3C%2FSTRONG%3E%20fields%20and%20use%20the%20Logic%20Apps%20%3CSTRONG%3EParse%20JSON%3C%2FSTRONG%3E%20action%20to%20extract%20the%20information.%26nbsp%3B%20%26nbsp%3BThere%20is%20no%20reason%20to%20perform%20another%20query%20against%20Log%20Analytics.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Valued Contributor

Hi

 

We are currently trying to automate some alerts through Playbooks.

We created a custom alert that checks for Impossible Travel Alerts from MCAS.

This works well. But the issue is that some data that is in the SecurityAlerts table (like ExtendedProperties and Entities) isn't forwarded to the Logic App when it's triggered by Sentinel.

I attached a screenshot of what data is forwarded through a logic app

 

Because Sentinel Entities do not support an array, but only one value. It's not possible to use those, because an impossible travel alert has multiple IP's.

 

So we are looking for another way to get all the data of a Sentinel alert into a playbook.

 

Should we query log analytics for this? I don't really like doing this, because this seems like an unnecessary step. 

 

Does anybody have something for this?

8 Replies

@Thijs Lecomte  Not sure what you mean when you say that Entities do not support arrays.  If the alert that creates the Incident finds multiple events and each of those events has matching entities, then the incident will have  multiple entities.

 

I currently have one incident that is made up of 13 events and has 5 IP and 6 Account Entities in it.  Using a Playbook to write the Entities to a Teams message I see that it writes out the Entities in a JSON array.

 

Looking at your image it shows the same thing just that, in your case, you only have 1 Entity listed

Thanks for the response.
The alert is have only has one event.

The event that comes from MCAS.
I can understand that multiple events in one alert can lead to multiple entities.

But can one event in an alert lead into multiple entities?

Or is there another way to get the full event details into Playbooks?

@Thijs Lecomte One event will only have up to a single value for each of the entities. 

 

Any chance that will be changed in the future?

Or any way to get the full details through the playbook?

@Thijs Lecomte I would not expect this to change.    What do you mean when you say full details?  What is missing?

@Thijs Lecomte This is something I have done and probably you could do.

 

Entities and Extended properties are JSON parameters. Using the data operations connector parse these parameters. You could get the JSON schema from the entities/extended properties logs. 

Almost all of the times the scheduled query is available in the Extended properties. By parsing you will convert JSON into String and take the Query parameter out. Initialize the variable naming it as XYZ and use Query with a time period(this is must). 

Using Azure Log Analytics run this initialized variable and you should be good to go.. Seems like a lot of pain but works just fine.

@Pranesh1060 

That's how I do it too ATM.

But this means there is an extra query to log analytics which seems unnecessary to me... Shouldn't this be available by default?

 

@Gary Bushey  Some data that I see in the SecurityAlerts table isn't able in the Playbook. These are columns like 'Entities' and 'ExtendedProperties' (these are columns that contain extra data from MCAS).

@Thijs Lecomte I believe what @Pranesh1060 meant is that you can take the Extended properties and the Entities fields and use the Logic Apps Parse JSON action to extract the information.   There is no reason to perform another query against Log Analytics.