Minemeld Threat Intel Integration to Sentinel

Occasional Contributor

Hello guys,

 

I have deployed a Minemeld server in Azure, I'm pulling free threat intel in there. Processing it, then using the Microsoft Security Graph extension to forward it to Microsoft. Turned the Threat Intel Connector on and now I have the Threat Intel in the LogAnalytics space.

 

There are two issues I have, in order:

 

1. Currently, with threat intel of type IP, I get the IP in a field called ExternalIndicatorID. A sample value for this is: IPv4:36.119.0.0-36.119.255.255 . As you can see, we have IPv4: then a range of IPs follows. The problem is this is something that's very impractical to use from an analytics point of view. I have to write the query in such a way to ignore the "IPv4:" and then also be able to interpret range. This is impractical and the preview Threat Intel rules offered by Microsoft do not use that field. They instead use NetworkIP, NetworkDestinationIP, NetworkSourceIP ....whichever of the three they find with a value. For me however, those values are empty.

 

Apparently this is something that must be changed with the Minemeld processor so that it does not merge IPs and generate ranges. I have not found a way to do that.

 

Has anyone managed to do that or otherwise any other workarounds to be able to consume Minemeld IP Threat Intel in Sentinel?

 

2. The second thing and I'm not completely sure here as nr 1 was a much bigger priority, is the Microsoft Security Graph extension for Minemeld only able to consume URLs, Domains and IPs? No emails, hashes, etc?

 

I have also asked on Palo Alto's board, however I'm really curious and could use a hand from someone who managed to already do this.

 

Thank you!

30 Replies

Hi @GabrielNecula 

 

Is it possible to remove the "IPv4" bit when you ingest the data through the Graph API? I assume you are using some kind of scripts? I think it will be the easiest to remove it that way.

 

Security Graph supports the following TI's:

- Email

- file

- Network (IP address, CIDR block, URL)

More information can be found here.

@Thijs Lecomte 

 

The data is getting to the Graph via an Mimemeld extension provided by them here https://github.com/PaloAltoNetworks/minemeld-msgraph-secapi.git

 

The how to can be found here https://live.paloaltonetworks.com/t5/MineMeld-Articles/Send-IOCs-to-Microsoft-Graph-API-With-MineMel...

 

You are saying to remove the IPv4 bit after ingestion by the Graph?

Also that would only be part of the problem. There is still the IP range that is problematic to interpret in KQL.

I was thinking about changing the sync script (used by the Mimemeld extension, this is an MS Graph script) so that the IPv4 bit can be removed.

You have two options:
- Create a custom ingestion script which removes the IPv4 bit and calculates the ranges
- Keep adapting your query

@Thijs Lecomte 

 

By any chance is there any solution for this. I just integrated Minemeld with Azure Sentinel and see the similar issue of getting range of IP address which will not help us to identify from which single IP the actual threat is

There haven't been any developments for this. Have you check the mimemeld side of things?

Yup. No luck. Could not find anything related to the IP range

@GabrielNecula 

 

Were you able to find out any workaround or a solution to this ?

@pavankemi nope, will likely be done at query time in Sentinel. Please let me know if you find any other workarounds.
I've checked the python code and it seems like it SHOULD provide single ips, not ranges. No idea how to solve this.

@GabrielNecula I have a column called NetworkCidrBlock that shows me the same information in CIDR notation.  I am using the Mindmeld free stream.

@Gary Bushey 

I know but you'd still have to parse the "/32" at query time. If it's anything other than /32, you will have to interpret that range somehow which is still hard.

@GabrielNecula Take a look at the KQL command ipv4_is_match().  It can match using CIDR notations.

@Gary Bushey 

 

Hi Gary, the TI queries join the SigninLogs or AzureActivity tables with TI using IP address so I don't understand how to use ipv4_is_match() in this scenario.

 

ThreatIntelligenceIndicator
| extend TI_ipEntity = iff(isnotempty(NetworkIP), NetworkIP, NetworkDestinationIP)
| join (
SigninLogs
)
on $left.TI_ipEntity == $right.IPAddress
 
If Minmeld doesnt have NetworkIP and instead have a CIDR value, how would match the values before joining?
 
Regards, 
Joachim

@JoachimLassus Take a look at parse_ipv4_mask().  You would need to use it to convert you IP addresses into the long number and then do the comparison in your query.  It will also take some string manipulation to get the format that you need (you would think the command would be written to do all of that for you already).

@Gary Bushey  Thanks for your reply. I'm still struggling with this, parse_ipv4_mask doesn't seem to be available within Azure Sentinel or am I missing something?

 

@CliveWatson  Is this something you have looked at before? 

 

@JoachimLassus 

 

Sorry which bit isn't working?  https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/parse-ipv4-maskfunction  this will work from any Workspace that Azure Sentinel is associated with.  However you will need a record in the Workspace that has a IP and prefix mask to supply as a parameter:

 

print parse_ipv4_mask('192.1.168.2', 31) == parse_ipv4_mask('192.1.168.3', 31)  

or (you'd pass in the mask as a parameter as well), I just hard-coded "31" as an example

SigninLogs
| summarize by IPAddress, parse_ipv4_mask(IPAddress,31)
 
 

ThreatIntelligenceIndicator
| extend TI_ipEntity = iff(isnotempty(NetworkIP), NetworkIP, NetworkDestinationIP)
| where isnotempty( TI_ipEntity)
//| project NetworkIP = "2.2.2.2/31"
| extend TI_ipEntity = iif(NetworkIP has "/", tostring(split(NetworkIP,"/").[0]),NetworkIP)
| project TI_ipEntity, NetworkIP
| join (
SigninLogs
//| project IPAddress = "2.2.2.2"
)
on $left.TI_ipEntity == $right.IPAddress
| project IPAddress, NetworkIP
| extend NetworkIP = iif(NetworkIP has "/", tostring(split(NetworkIP,"/").[1]),NetworkIP)
| project parse_ipv4_mask(IPAddress, toint(NetworkIP))

You can probably check for a IP with a prefix mask "/" and filter those, something like the above?

@CliveWatson 

 

The TI data from Minemeld have these columns

JoachimLassus_0-1601911337783.png

 

So we get range of bad IP's, wouldn't you have to cycle through each IP to see if it exists in / to join the Signinlogs table?

@JoachimLassus 

 

Ah ok, my entries in that column are empty, so it could work like this?  Checking each TI entry against the Signinlogs  

 

ThreatIntelligenceIndicator
//| where isnotempty(NetworkCidrBlock)
| project NetworkCidrBlock = "193.228.91.0/26"
| project justIP = tostring(split(NetworkCidrBlock,"/").[0]) , prefixIP = toint(split(NetworkCidrBlock,"/").[1])
| join (
SigninLogs
| project IPAddress = "193.228.91.0"
)
on $left.justIP == $right.IPAddress
| project parse_ipv4_mask(justIP, prefixIP)

 You'll need to uncomment line 2 and remove lines 3 & 7 

ThreatIntelligenceIndicator
| where isnotempty(NetworkCidrBlock)
//| project NetworkCidrBlock = "193.228.91.0/26"
| project justIP = tostring(split(NetworkCidrBlock,"/").[0]) , prefixIP = toint(split(NetworkCidrBlock,"/").[1])
| join (
SigninLogs
//| project IPAddress = "193.228.91.0"
)
on $left.justIP == $right.IPAddress
| project parse_ipv4_mask(justIP, prefixIP)

 

@CliveWatson 

 

Thanks for you response, correct me if I'm wrong but I believe that query only considers if 193.228.91.0 exists in the Signinlogs.

 

193.228.91.0/24 would give you a range of 193.228.91.0 - 193.228.91.63 so if someone with IP 193.228.91.5 signs in it wouldn't trigger an alert.