Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community

Microsoft Defender ATP and Malware Information Sharing Platform integration

Microsoft

Pull file hashes (SHA1) from Malware Information Sharing Platform (MISP) and push them to Microsoft Defender ATP

image.png5 Minutes 

image.pngLow complexity 

 

Enterprises use threat intelligence to enrich their cyber security telemetry as well as to detect and block attacks. Microsoft Defender ATP supports blocking capabilities through the portal using the indicators page and the indicators API. In a previous blog, we explained how to generally use the indicators API.

 

In our commitment to listen to customer feedback, we’re going to demonstrate how to leverage the power of Malware Information Sharing Platform (MISP) to further augment the protection offering of Microsoft Defender ATP.

 

MISP is a free and open source threat sharing platform. It is designed to help share threat intelligence information such as cyber security indicators, vulnerability information, and others.

 

In this blog, we will demonstrate an easy way to automatically pull the indicators from the MISP platform and push them into the custom IoC list of Microsoft Defender ATP.

 

We’ll guide you on how to pull SHA1 hashes from the past four days from MISP, and push them as indicators for Microsoft Defender ATP to detect. We’ll push the indicators with an expiration date set to seven days from the time the script is run to ensure the freshness of indicators.  

 

Let’s start

  1. Add the required permission to write indicators to Microsoft Defender ATP
  2. Get your MISP URL and Authorization key
  3. Download and use the script to use MISP API to pull SHA1* hashes from your MISP platform and push them into Microsoft Defender ATP

Step 1: Add permission to write indicators to Microsoft Defender ATP

NOTE: This step is identical to the process detailed in our previous blog. If you’ve already done it, skip this step.

If you haven’t created an app: 

  • Create an app using the instructions described in the Hello world
  • Then follow the instructions on how to Add Isolation Permission as described below. 

If you’ve already created an app that you’re going to reuse for this demonstration: 

  • Add the “TI.ReadWrite.All” permission as described below. 
  • We recommend that you follow the detailed steps as described in the “Step 1 - Add the required permission to the application” in the Alert Update API blog.

 

Add TI write permissions

  • Open Azure portal. 
  • Navigate to Azure Active Directory > App registrations
  • Under All Applications, find and select the application, for example, ContosoSIEMConnector
  • Click  API permissions > Add a permission

permission.png

  • Click on “APIs my organization uses” and type WindowDefenderATP in the search box. Then chose the “WindowsDefenderATP” API from the list.

permission2.png

  • Click on “Application permission” button and check the “TI.ReadWrite.All” checkbox.

permission3.png

  • Click Add Permission.
  • On the “API Permission” screen, click on “Grant admin consent for…” button.

permission4.png

Done! You have successfully added the required permissions to the application.

 

Step 2: Get your MISP URL and Authorization key

To use MISP API, you will your MISP URL and authorization key. You can get the URL from MISP web interface and the API key from the Automation section of the MISP web interface. See below:

Misp-Key1.png

Step 3: Run the script

Download the zip file at the bottom of this blog and extract the “Get-MISP-Hash.ps1” script.

Make sure the script in the same directory (folder) as your “Get-Token.ps1” script.

Now lets run the script!

Let’s say your MISP URL is 10.0.0.1 and the key is Yn81mP1ck13Ric4.

Open PowerShell and run the following command:

.\Get-Misp-Hash.ps1 -title “Sha1 from MISP” -mispUrl “10.0.0.1” -authKey “Yn81mP1ck13Ric4”

And that’s it, you are done! You can see the new hashes on your Microsoft Defender Security Portal by clicking on Settings, then look for “Indicators” on the menu on the left of the screen.

The script also offers additional (optional) fields you may want to use to control the indicators and how the MDATP handles them:

Parameter

Description

Default value

-title

The alert’s title

-

-mispUrl

Your MISP URL from the MISP platform

-

-authKey

Your MISP authorization key

-

-action (optional)

Choose one: Alert/AlertAndBlock/Allowed

Alert

-severity (optional)

Choose one: Informational/Low/Medium/High

Informational

-description (optional)

the alert’s description

-

-recommendedAction

(optional)

the recommended action to remediate the alert

-

-expiration (optional)

number of days the hashes will expire and deleted from the MDATP

7

 

Another example that utilizes all of the options is shown below all in the same line:

.\Get-MISP-Hash.ps1 -title "Sha1 from MISP" -mispUrl “10.0.0.1  ” -authKey “Yn81mP1ck13Ric4” -action AlertAndBlock -severity High -description "Sha1 from the MISP platform loaded by automated script" -recommendedActions "Add your own recommended actions" -expiration 14

Conclusion

In this blog, we demonstrated how you can  pull SHA1 hashes from MISP to augment the Microsoft Defender ATP protection capabilities

 

FAQs:

Which file hashes are supported?

This integration supports SHA1. MISP’s API only supports SHA1 and MD5 (which is relatively weak), while Microsoft Defender ATP supports SHA1 and SHA256.

Also note that “Only published events and attributes marked as IDS Signature are exported” (MISP API documentation).

 

How do I modify the MISP filter?

The MISP API we used to get the hashes has some filtering options.

We set the script to get hashes from the past 4 days:

https://<your_MISP_URL>/events/hids/sha1/download/false/false/false/4d

 

You can change it to any other number of days or hours, also you can filter by start and end date or by MISP tags you’ve set up.

 

For the full details, see MISP’s API DOC here: https://www.circl.lu/doc/misp/automation/ and look for the “GET /events/hids Hash” section.

 

Important note:

If the hash you are submitting already exists in Microsoft Defender ATP with a different expiration date or severity etc., it’s OK – it will be updated with the new value. However, if it already exists with a different “action” field value (Alert, AlertAndBlock or Allow), it will not be updated (or submitted), and the script will print the hash with the corresponding message – this is by design.

 

Please share your thoughts and feedback here.

Thanks,

@Itai Zur, program manager, Windows Defender ATP

@Thorsten Henking, senior program manager

@Dan Michelson, program manager, Windows Defender ATP

@Haim Goldshtein, security software engineer, Windows Defender ATP

 

 

 

 

 

 

4 Replies

@Haim Goldshtein - Thank you for the guide, just some clarifications from MISP's side: MISP supports a long list of hashing algorithms, I think you may have ended up on an old API documentation, the /hids endpoints are deprecated. Instead we have transitioned to the restSearch APIs (/events/restSearch and /attributes/restSearch) that allow you to search and export any of the supported types (which for hashes includes sha256, various fuzzy hashing algorithms like ssdeep, import hashes, etc). 

 

An example of using the modern APIs:

 

https://<your_MISP_URL>/events/hids/sha1/download/false/false/false/4d

 

would be

POST to https://<your_MISP_URL>/attributes/restSearch

payload:
{

    "returnFormat": "text",

    "publish_timestamp": "4d",

    "type": "sha1"

}

 

Type of course could be sha256 or ssdeep for example. 

 

If you prefer URL parameters and GET requests it would look like this:

 

GET to https://<your_MISP_URL>/attributes/restSearch/returnFormat:text/type:sha1/publish_timestamp:4d

 

The best resource for the API is the built in documentation under event actions -> automation or alternatively the REST client with the query builder (Event actions -> REST client).

 

Hope this helps and thanks for the integration and associated guide!

@Haim Goldshtein 
Thanks for the example code.

 

We tweaked it a little, as we found the indicator upload API is rate-limited, and with more than about 150 indicators returned from MISP the script would run into this and start to receive 429 error responses. In that case, ours will wait 15s and retry the indicator a few times before giving up.

 

There's also a new batch upload mechanism and we added a parameter (and code) to use that instead - with "-batchUpload 1". This can feed 500 at a time and is much faster.

 

We also added the "Get-Token.ps1" code to the main script to remove the dependency (oh, and changed to the MISP REST GET format URL per the previous Reply.)

 

I've attached the result with any company specifics stripped out in case it's useful. Note that some parameters are renamed: atpAction, atpTitle, atpSeverity, atpDescription, and atpRecommendedActions .

@Haim Goldshtein 

 

I keep getting an error on this script;
"Invoke-WebRequest : A parameter cannot be found that matches parameter name 'k'."

On the "$response = curl -k --header" line. I've tried playing around with the curl/Invoke-WebRequest syntax a bit but no dice.

 

Edit: Using Powershell 7 resolved this.

Is there a way to ignore the security certificate on the misp server when executing this script while in the testing phase?