Kusto regex for extracting IP adresses

Copper Contributor

In my AzureDiagnostics for my ResourceType "AzureFirewalls", there's a column named "msg_s". 

It contains information about IP-adresses trying to request access to another adress.

 

Examples include:

 
HTTPS request from 10.192.168.10:10100 to some-text.blob.core.windows.net:443. Action: Allow. Azure internal traffic.
 

HTTPS request from 198.192.100.10:10500. Action: Deny. Reason: SNI TLS extension was missing

 

UDP request from 10.192.100.1:10500 to 10.168.10.20. Action: Allow

 

I'd like to use RegEx to extract the first IP into one column, then extract the second IP if there is one (second example did not have a destination IP), and extract "Allow" or "Deny" into a third column. 

 

Can someone help me solve this?

 

I've already tried using Parse instead of RegEx but I believe RegEx is better because of the optional destination adress in the second example, and optional :port in the third example. 

3 Replies

@Preben902 

 

I can get the first one, but will have to have a think about the other cases 

AzureDiagnostics
| where ResourceType == "AZUREFIREWALLS" 
| where msg_s has "request from"
| extend IPaddr = extract("(([0-9]{1,3})\\.([0-9]{1,3})\\.([0-9]{1,3})\\.(([0-9]{1,3})))",1,msg_s) 
| extend action = iif(msg_s has ": Deny", "Deny", "Allow") 
| project IPaddr , msg_s , action 

In the meantime the above may help 

@Preben902 

 

I also forgot there are some examples on https://docs.microsoft.com/en-us/azure/firewall/log-analytics-samples

AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " to " TargetIP ":" TargetPortInt:int *
| parse msg_s with * ". Action: " Action1a
| parse msg_s with * " was " Action1b " to " NatDestination
| parse msg_s with Protocol2 " request from " SourceIP2 " to " TargetIP2 ". Action: " Action2
| extend SourcePort = tostring(SourcePortInt),TargetPort = tostring(TargetPortInt)
| extend Action = case(Action1a == "", case(Action1b == "",Action2,Action1b), Action1a),Protocol = case(Protocol == "", Protocol2, Protocol),SourceIP = case(SourceIP == "", SourceIP2, SourceIP),TargetIP = case(TargetIP == "", TargetIP2, TargetIP),SourcePort = case(SourcePort == "", "N/A", SourcePort),TargetPort = case(TargetPort == "", "N/A", TargetPort),NatDestination = case(NatDestination == "", "N/A", NatDestination)
| project TimeGenerated, msg_s, Protocol, SourceIP,SourcePort,TargetIP,TargetPort,Action, NatDestination

 

Results

TimeGenerated msg_s Protocol SourceIP SourcePort TargetIP TargetPort Action NatDestination
2019-09-12T15:20:27.718Z TCP request from 10.249.96.136:49925 to 13.94.141.226:12000. Action: Deny TCP 10.249.96.136 49925 13.94.141.226 12000 Deny N/A
2019-09-12T15:20:27.765Z TCP request from 10.249.96.136:49925 to 13.94.141.226:12000. Action: Deny TCP 10.249.96.136 49925 13.94.141.226 12000 Deny N/A
2019-09-12T15:20:27.843Z TCP request from 10.249.96.136:49925 to 13.94.141.226:12000. Action: Deny TCP 10.249.96.136 49925 13.94.141.226 12000 Deny N/A

@Preben902 Here's a basic pattern. It doesn't check for valid IP addresses, for this use case that shouldn't matter.

 

print extract_all("request from (?P<from>.+?)(?: to (?P<to>.+))?\\. Action: (?P<action>[^.]+)","HTTPS request from 198.192.100.10:10500. Action: Deny. Reason: SNI TLS extension was missing")