SOLVED
Home

Help me to connect from power bi web to azure log analytics

%3CLINGO-SUB%20id%3D%22lingo-sub-825580%22%20slang%3D%22en-US%22%3EHelp%20me%20to%20connect%20from%20power%20bi%20web%20to%20azure%20log%20analytics%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-825580%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI'm%20trying%20to%20connect%20power%20BI%20web%20portal%20from%20log%20analytics.%20I%20read%20a%20lot%20of%20articles%20but%20I%20did%20not%20find%20a%20solid%20solution%20for%20this.%20So%20I%20came%20up%20with%20workaround%20send%20logs%20to%20blob%20storage%20and%20from%20there%20using%20stream%20analytics%20job%20to%20send%20power%20bi%20web%20portal%20datasets.%20Now%20I%20was%20stuck%20in%20sending%20log%20analytics%20query%20results%20to%20blob%20storage.%26nbsp%3B%3C%2FP%3E%3CP%3E1)%20Can%20someone%20help%20me%20with%20this%3F%20Is%20there%20any%20direct%20way%20to%20send%20logs%20from%20log%20analytics%20to%20power%20bi%20web%3F.%20Looking%20for%20live%20streaming%20data.%26nbsp%3B%3C%2FP%3E%3CP%3E2)%20How%20can%20I%20send%20log%20analytics%20query%20results%20to%20blob%20storage%3F.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%20for%20your%20help.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-825580%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EApplication%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EData%20Retention%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMachine%20Learning%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EQuery%20Language%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESolutions%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-825987%22%20slang%3D%22en-US%22%3ERe%3A%20Help%20me%20to%20connect%20from%20power%20bi%20web%20to%20azure%20log%20analytics%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-825987%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F399256%22%20target%3D%22_blank%22%3E%40phanikumar236%3C%2FA%3Efrom%20your%20post%20it%20is%20not%20clear%20if%20you%20have%20read%20about%20this%20approach%3A%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fpowerbi%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fpowerbi%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIt%20is%20way%20straight%20forward.%20It%20is%20not%20live%20stream%20as%20relies%20on%20the%20setting%20refresh%20time.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThere%20is%20no%20out%20of%20the%20box%20method%20to%20live%20stream%20data%20from%20Log%20Analytics%20to%20PowerBI.%20Also%20there%20is%20no%20out%20of%20the%20box%20method%20to%20send%20data%20from%20Log%20Analytics%20to%20Azure%20storage.%20If%20you%20are%20going%20with%20the%20approach%20of%20sending%20data%20from%20Log%20Analytics%20to%20Azure%20storage%20to%20Stream%20Analytics%20and%20than%20to%20PowerBI%20you%20are%20introducing%20a%20lot%20of%20point%20of%20failures%20which%20I%20am%20sure%20you%20are%20aware.%20The%20other%20problem%20is%20that%20to%20move%20data%20between%20so%20many%20points%20will%20have%20additional%20cost%20and%20for%20sure%20will%20not%20be%20live%20stream.%20Moving%20data%20out%20of%20Log%20Analytics%20to%20another%20storage%20like%20Azure%20blob%20has%20the%20following%20challenges%20you%20should%20consider%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EHow%20do%20you%20know%20which%20data%20was%20moved%20and%20which%20not%3F%3C%2FLI%3E%0A%3CLI%3EWhat%20amount%20of%20data%20will%20be%20processed%3F%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3ETaking%20that%20into%20account%20you%20could%20use%20some%20of%20the%20automation%20technologies%20like%20Logic%20App%2C%20Functions%20or%20Azure%20Automation%20to%20automate%20the%20processing%20of%20data%20from%20Log%20Analytics%20to%20Azure%20Storage.%20Taking%20into%20account%20that%20you%20want%20to%20be%20live%20stream%20I%20would%20choose%20Azure%20Functions%20but%20still%20I%20think%20you%20will%20not%20be%20able%20to%20achieve%20truly%20live%20stream.%20Even%20if%20you%20set%20some%20low%20time%20trigger%20on%20the%20Azure%20function%20you%20have%20to%20figure%20out%20which%20data%20you%20will%20query%3F%20If%20you%20choose%20a%20big%20amount%20of%20data%20to%20query%20and%20move%20to%20Azure%20Storage%20you%20will%20hit%20some%20API%20limits%20which%20you%20will%20need%20to%20overcome.%20Also%20you%20have%20to%20think%20at%20what%20interval%20the%20data%20you%20query%20appears%20in%20Log%20Analytics.%20Different%20data%20appears%20in%20different%20intervals.%20Additionally%20in%20some%20rare%20cases%20there%20could%20be%20slight%20delay%20in%20that%20interval.%20My%20point%20is%20that%20you%20have%20to%20build%20some%20complex%20sync%20mechanism%20to%20do%20all%20this%20work%20so%20you%20do%20not%20end%20up%20with%20doable%20data%20or%20missing%20data.%20It%20is%20certainly%20possible%20but%20definitely%20not%20easy%20to%20make.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThis%20is%20probably%20not%20the%20answer%20you%20were%20expecting%20but%20I%20hope%20it%20is%20somehow%20useful%20to%20you.%20I%20would%20definitely%20recommend%20going%20with%20the%20default%20PowerBI%20integration%20if%20you%20can%20overcome%20that%20the%20data%20is%20not%20live%20streamed.%20As%20Log%20Analytics%20is%20the%20monitoring%20tool%20it%20is%20usually%20the%20end%20point%20for%20such%20data%20and%20PowerBI%20in%20such%20cases%20is%20used%20more%20of%20summarizing%20certain%20data%20there%20for%20reporting%20rather%20for%20live%20dashboarding.%20Of%20course%20your%20case%20seems%20to%20be%20different.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-832105%22%20slang%3D%22en-US%22%3ERe%3A%20Help%20me%20to%20connect%20from%20power%20bi%20web%20to%20azure%20log%20analytics%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-832105%22%20slang%3D%22en-US%22%3EI%20have%20a%20log%20analytics%20job%20(for%20my%20api%20management)%20logging%20to%20a%20blob.%20Then%20I%20hook%20up%20with%20the%20powerbi%20azure%20blob%20connector.%20Everything%20is%20coming%20over%20in%20Json.%20Problem%20is%20that%20there%20is%20a%20bug%20where%20the%20Json%20in%20the%20data%20file%20(from%20the%20blob)%20is%20incorrectly%20formatted.%20It%E2%80%99s%20a%20well%20known%20bug.%20Waiting%20for%20a%20fix.%3C%2FLINGO-BODY%3E
phanikumar236
Occasional Visitor

Hi, 

 

I'm trying to connect power BI web portal from log analytics. I read a lot of articles but I did not find a solid solution for this. So I came up with workaround send logs to blob storage and from there using stream analytics job to send power bi web portal datasets. Now I was stuck in sending log analytics query results to blob storage. 

1) Can someone help me with this? Is there any direct way to send logs from log analytics to power bi web?. Looking for live streaming data. 

2) How can I send log analytics query results to blob storage?.

 

Thanks for your help.

 

2 Replies
Solution

Hi @phanikumar236 from your post it is not clear if you have read about this approach:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/powerbi

 

It is way straight forward. It is not live stream as relies on the setting refresh time.

 

There is no out of the box method to live stream data from Log Analytics to PowerBI. Also there is no out of the box method to send data from Log Analytics to Azure storage. If you are going with the approach of sending data from Log Analytics to Azure storage to Stream Analytics and than to PowerBI you are introducing a lot of point of failures which I am sure you are aware. The other problem is that to move data between so many points will have additional cost and for sure will not be live stream. Moving data out of Log Analytics to another storage like Azure blob has the following challenges you should consider:

  • How do you know which data was moved and which not?
  • What amount of data will be processed?

Taking that into account you could use some of the automation technologies like Logic App, Functions or Azure Automation to automate the processing of data from Log Analytics to Azure Storage. Taking into account that you want to be live stream I would choose Azure Functions but still I think you will not be able to achieve truly live stream. Even if you set some low time trigger on the Azure function you have to figure out which data you will query? If you choose a big amount of data to query and move to Azure Storage you will hit some API limits which you will need to overcome. Also you have to think at what interval the data you query appears in Log Analytics. Different data appears in different intervals. Additionally in some rare cases there could be slight delay in that interval. My point is that you have to build some complex sync mechanism to do all this work so you do not end up with doable data or missing data. It is certainly possible but definitely not easy to make.

 

This is probably not the answer you were expecting but I hope it is somehow useful to you. I would definitely recommend going with the default PowerBI integration if you can overcome that the data is not live streamed. As Log Analytics is the monitoring tool it is usually the end point for such data and PowerBI in such cases is used more of summarizing certain data there for reporting rather for live dashboarding. Of course your case seems to be different.

 

I have a log analytics job (for my api management) logging to a blob. Then I hook up with the powerbi azure blob connector. Everything is coming over in Json. Problem is that there is a bug where the Json in the data file (from the blob) is incorrectly formatted. It’s a well known bug. Waiting for a fix.
Related Conversations
Tabs and Dark Mode
cjc2112 in Discussions on
46 Replies
Extentions Synchronization
Deleted in Discussions on
3 Replies
Stable version of Edge insider browser
HotCakeX in Discussions on
35 Replies
How to Prevent Teams from Auto-Launch
chenrylee in Microsoft Teams on
30 Replies
flashing a white screen while open new tab
Deleted in Discussions on
14 Replies
Security Community Webinars
Valon_Kolica in Security, Privacy & Compliance on
13 Replies