Log analytics customisation

%3CLINGO-SUB%20id%3D%22lingo-sub-1175533%22%20slang%3D%22en-US%22%3ELog%20analytics%20customisation%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1175533%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20All%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20am%20pretty%20new%20to%20azure%20log%20analytics%20and%20would%20appreciate%20if%20anyone%20would%20help%20me%20with%20below%20questions.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThe%20setting%20on%20my%20subscription%20is%20that%20all%20the%20Azure%20firewall%20logs%20are%20sent%20to%20log%20analytics.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EQ1%20-%20I%20am%20trying%20to%20find%20traffic%20log%20from%20a%20specific%20source%20IP%2C%20but%20I%20am%20not%20sure%20how%20to%20edit%20the%20sample%20provided%20by%20Azure.%26nbsp%3B%20%26nbsp%3B%20%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EAzureDiagnostics%3CBR%20%2F%3E%7C%26nbsp%3Bwhere%26nbsp%3BCategory%20%3D%3D%26nbsp%3B%22AzureFirewallNetworkRule%22%3CBR%20%2F%3E%7C%20parse%20msg_s%20with%20Protocol%20%22%20request%20from%20%22%20SourceIP%26nbsp%3B%22%3A%22%20SourcePortInt%3Aint%26nbsp%3B%22%20to%20%22%26nbsp%3BTargetIP%26nbsp%3B%22%3A%22%26nbsp%3BTargetPortInt%3Aint%20*%3CBR%20%2F%3E%7C%20parse%20msg_s%20with%20*%20%22.%20Action%3A%20%22%20Action1a%3CBR%20%2F%3E%7C%20parse%20msg_s%20with%20*%20%22%20was%20%22%20Action1b%20%22%20to%20%22%20NatDestination%3CBR%20%2F%3E%7C%20parse%20msg_s%20with%20Protocol2%20%22%20request%20from%20%22%20SourceIP2%26nbsp%3B%22%20to%20%22%26nbsp%3BTargetIP2%26nbsp%3B%22.%20Action%3A%20%22%20Action2%3CBR%20%2F%3E%7C%20extend%20SourcePort%20%3D%20tostring(SourcePortInt)%2CTargetPort%20%3D%20tostring(TargetPortInt)%3CBR%20%2F%3E%7C%20extend%20Action%20%3D%20case(Action1a%20%3D%3D%20%22%22%2C%20case(Action1b%20%3D%3D%20%22%22%2CAction2%2CAction1b)%2C%20Action1a)%2CProtocol%20%3D%20case(Protocol%20%3D%3D%20%22%22%2C%20Protocol2%2C%20Protocol)%2CSourceIP%20%3D%20case(SourceIP%20%3D%3D%20%22%22%2C%20SourceIP2%2C%20SourceIP)%2CTargetIP%20%3D%20case(TargetIP%20%3D%3D%20%22%22%2C%20TargetIP2%2C%20TargetIP)%2CSourcePort%20%3D%20case(SourcePort%20%3D%3D%20%22%22%2C%20%22N%2FA%22%2C%20SourcePort)%2CTargetPort%20%3D%20case(TargetPort%20%3D%3D%20%22%22%2C%20%22N%2FA%22%2C%20TargetPort)%2CNatDestination%20%3D%20case(NatDestination%20%3D%3D%20%22%22%2C%20%22N%2FA%22%2C%20NatDestination)%3CBR%20%2F%3E%7C%20project%20TimeGenerated%2C%20msg_s%2C%20Protocol%2C%20SourceIP%2CSourcePort%2CTargetIP%2CTargetPort%2CAction%2C%20NatDestination%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EQ2-%20Is%20there%20any%20way%20i%20can%20change%20the%20log%20search%20so%20that%20it%20shows%20more%20than%2010%2C000%20records%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EQ3-%20How%20can%20I%20export%20everything%20in%20my%20log%20analytics%20workspace%20to%20a%20csv%20or%20json%20file%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20in%20advance%20for%20all%20your%20guidance!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1175533%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3Eazure%20firewall%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1182011%22%20slang%3D%22en-US%22%3ERe%3A%20Log%20analytics%20customisation%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1182011%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F558528%22%20target%3D%22_blank%22%3E%40Simon_y_lee%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EA1%20-%20The%20example%20provided%20below%20is%20great%2C%20because%20if%20parses%20the%20relevant%20fields%20and%20you'll%20only%20need%20to%20add%20a%20%22where%22%20clause%20to%20the%20query%3A%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3E...%0A%7C%20where%20SourceIP%20%3D%3D%20%221.2.3.4%22%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3BA2%20-%20the%2010K%20limit%20is%20a%20UI%20limit%2C%20which%20you%20currently%20can't%20bypass.%3C%2FP%3E%0A%3CP%3EWhat%20you%20can%20do%20is%3A%3C%2FP%3E%0A%3CP%3E-%20Run%20your%20queries%20through%20the%20LA%20API%20instead%2C%20which%20will%20return%20the%20full%20resultset%3C%2FP%3E%0A%3CP%3E-%20Run%20your%20queries%20through%20PowerBI%3C%2FP%3E%0A%3CP%3E-%20Scope%20your%20query%20to%20a%20specific%20time%20frame%20(last%20hour%2C%20last%206%20hours%20etc.)%20which%20may%20produce%20smaller%20resultsets%20and%20not%20reach%20the%2010K%20limit.%3C%2FP%3E%0A%3CP%3EA3%20-%20After%20running%20the%20query%2C%20select%20Export%20from%20the%20action%20bar%20(top%20area)%20and%20export%20to%20CSV.%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Regular Visitor

Hi All,

 

I am pretty new to azure log analytics and would appreciate if anyone would help me with below questions. 

 

The setting on my subscription is that all the Azure firewall logs are sent to log analytics. 

 

Q1 - I am trying to find traffic log from a specific source IP, but I am not sure how to edit the sample provided by Azure.     

 

AzureDiagnostics
| where Category == "AzureFirewallNetworkRule"
| parse msg_s with Protocol " request from " SourceIP ":" SourcePortInt:int " to " TargetIP ":" TargetPortInt:int *
| parse msg_s with * ". Action: " Action1a
| parse msg_s with * " was " Action1b " to " NatDestination
| parse msg_s with Protocol2 " request from " SourceIP2 " to " TargetIP2 ". Action: " Action2
| extend SourcePort = tostring(SourcePortInt),TargetPort = tostring(TargetPortInt)
| extend Action = case(Action1a == "", case(Action1b == "",Action2,Action1b), Action1a),Protocol = case(Protocol == "", Protocol2, Protocol),SourceIP = case(SourceIP == "", SourceIP2, SourceIP),TargetIP = case(TargetIP == "", TargetIP2, TargetIP),SourcePort = case(SourcePort == "", "N/A", SourcePort),TargetPort = case(TargetPort == "", "N/A", TargetPort),NatDestination = case(NatDestination == "", "N/A", NatDestination)
| project TimeGenerated, msg_s, Protocol, SourceIP,SourcePort,TargetIP,TargetPort,Action, NatDestination 

 

Q2- Is there any way i can change the log search so that it shows more than 10,000 records?

 

Q3- How can I export everything in my log analytics workspace to a csv or json file?

 

Thanks in advance for all your guidance!

 

1 Reply
Highlighted

@Simon_y_lee 

A1 - The example provided below is great, because if parses the relevant fields and you'll only need to add a "where" clause to the query:

...
| where SourceIP == "1.2.3.4"

 A2 - the 10K limit is a UI limit, which you currently can't bypass.

What you can do is:

- Run your queries through the LA API instead, which will return the full resultset

- Run your queries through PowerBI

- Scope your query to a specific time frame (last hour, last 6 hours etc.) which may produce smaller resultsets and not reach the 10K limit.

A3 - After running the query, select Export from the action bar (top area) and export to CSV.