SOLVED
Home

Missing custom logs in Log Analytics schema pane

%3CLINGO-SUB%20id%3D%22lingo-sub-307178%22%20slang%3D%22en-US%22%3EMissing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307178%22%20slang%3D%22en-US%22%3E%3CP%3EHi%2C%3C%2FP%3E%3CP%3EI%20submitted%20custom%20logs%20via%20the%20AA%20API%20but%20I'm%20not%20seeing%20my%20custom%20log%20tables%20in%20the%20logs%20app.%3C%2FP%3E%3CP%3EHere's%20my%20code%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fgist.github.com%2FBand-Aid%2Fe74a9e30fcada3e40afb922ac50d19ec%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fgist.github.com%2FBand-Aid%2Fe74a9e30fcada3e40afb922ac50d19ec%3C%2FA%3E%3C%2FP%3E%3CP%3EThe%20returned%20status%20code%20was%20200%20and%20all%20seems%20to%20be%20fine%20so%20I%20was%20expecting%20the%20logs%20to%20show%20up.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBut%26nbsp%3Bwhen%20I%20try%20to%20filter%20for%20columns%20in%20the%20Classic%20Log%20view%2C%20I%26nbsp%3Bdo%20see%20the%20column%20entires%20were%20created.%3C%2FP%3E%3CP%3EAny%20ideas%20why%20my%20logs%20aren't%20showing%20up%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20363px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F64299i82DE7EDE92FEE341%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20alt%3D%22aa.png%22%20title%3D%22aa.png%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EClassic%20logs%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20116px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Fgxcuf89792.i.lithium.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F64300i7C705BA50E1A4C66%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20alt%3D%22aa2.png%22%20title%3D%22aa2.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-307178%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307511%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307511%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20Daichi%2C%3C%2FP%3E%0A%3CP%3E8%20hours%20for%20the%20initial%20ingestion%20and%201%20hour%20for%20appended%20data%20are%20indeed%20much%20higher%20than%20expected.%3C%2FP%3E%0A%3CP%3EAre%20you%20still%20experiencing%20that%20delay%3F%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307465%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307465%22%20slang%3D%22en-US%22%3E%3CP%3EThank%20you%20very%20much%20for%20the%20follow%20up.%20These%20links%20really%20help!%3C%2FP%3E%3CP%3EI'll%20give%20PS%20a%20try%20and%20see%20if%20there's%20any%20difference.%20Thanks%20again!%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307258%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307258%22%20slang%3D%22en-US%22%3E%3CP%3EMy%20experience%20with%20PowerShell%20is%20it%20only%20takes%20a%20few%20seconds%20for%20data%20to%20show%20with%20the%20data%20collector%20API%2C%20after%20the%20initial%20setup%20delay.%26nbsp%3B%20The%20code%20I%20used%20can%20be%20found%20at%26nbsp%3Bthe%20bottom%20of%20this%20link.%3C%2FP%3E%3CP%3E%3CA%20href%3D%22http%3A%2F%2Fwww.ciraltos.com%2Fazure-oms-log-analytics-step-by-step-data-collector-api%2F%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttp%3A%2F%2Fwww.ciraltos.com%2Fazure-oms-log-analytics-step-by-step-data-collector-api%2F%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThis%20article%20goes%20deeper%20into%20ingestion%20time%20that%20may%20help%20with%20troubleshooting.%3C%2FP%3E%3CP%3E%3CA%20href%3D%22http%3A%2F%2Fcontoso.se%2Fblog%2F%3Fp%3D4493%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttp%3A%2F%2Fcontoso.se%2Fblog%2F%3Fp%3D4493%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307234%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307234%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20for%20the%20link.%20I%20checked%20back%20every%2030%20mins%20or%20so%26nbsp%3Bbut%26nbsp%3Bit%20literally%20did%20take%20that%20long.%20(and%20i%20did%20refresh%20my%20browser%20from%20time%20to%20time)%3C%2FP%3E%3CP%3EIt's%20improved%20since%20but%20still%20takes%20around%201%20-%202%20hours%20for%20new%20tables%20to%20show%20up...%26nbsp%3B%3C%2FP%3E%3CP%3EAdding%20data%20to%20existing%20tables%20takes%20around%2030%20mins%20to%20process.%3C%2FP%3E%3CP%3EThis%20is%20very%20disappointing%20given%20how%20robust%20the%20features%20are%20compared%20to%20other%20competitor%20products.%26nbsp%3B%3C%2FP%3E%3CP%3EI%20guess%20I'll%20just%20have%20to%20live%20with%20the%20limitations%20and%20come%20up%20with%20a%20cleaver%20solution%20%3Bp%3C%2Fimg%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307232%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307232%22%20slang%3D%22en-US%22%3E%3CP%3EI've%20see%20it%20take%20over%20an%20hour%2C%20but%208%20seems%20longer%20than%20normal.%26nbsp%3B%20Below%20is%20a%20link%20that%20goes%20over%20ingestion%20time.%26nbsp%3B%20In%20my%20testing%2C%20I%20also%20noticed%20that%20refreshing%20the%20browser%20will%20help%20if%20data%20is%20not%20showing%20up%20after%20about%20an%20hour.%20%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%3C%2FA%3E%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-307202%22%20slang%3D%22en-US%22%3ERe%3A%20Missing%20custom%20logs%20in%20Log%20Analytics%20schema%20pane%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-307202%22%20slang%3D%22en-US%22%3E%3CP%3E8%20hours%20later...%20Now%20it's%20showing%20up%20in%20the%20console.%3C%2FP%3E%3CP%3EIt%20is%20normal%20for%20custom%20logs%20to%20take%20this%20long%20to%20process%3F%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
daichi
New Contributor

Hi,

I submitted custom logs via the AA API but I'm not seeing my custom log tables in the logs app.

Here's my code: https://gist.github.com/Band-Aid/e74a9e30fcada3e40afb922ac50d19ec

The returned status code was 200 and all seems to be fine so I was expecting the logs to show up.

 

But when I try to filter for columns in the Classic Log view, I do see the column entires were created.

Any ideas why my logs aren't showing up?

 

aa.pngClassic logs

aa2.png

6 Replies

8 hours later... Now it's showing up in the console.

It is normal for custom logs to take this long to process? 

I've see it take over an hour, but 8 seems longer than normal.  Below is a link that goes over ingestion time.  In my testing, I also noticed that refreshing the browser will help if data is not showing up after about an hour.  

 

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-ingestion-time

 

Thanks for the link. I checked back every 30 mins or so but it literally did take that long. (and i did refresh my browser from time to time)

It's improved since but still takes around 1 - 2 hours for new tables to show up... 

Adding data to existing tables takes around 30 mins to process.

This is very disappointing given how robust the features are compared to other competitor products. 

I guess I'll just have to live with the limitations and come up with a cleaver solution ;p

Solution

My experience with PowerShell is it only takes a few seconds for data to show with the data collector API, after the initial setup delay.  The code I used can be found at the bottom of this link.

http://www.ciraltos.com/azure-oms-log-analytics-step-by-step-data-collector-api/

 

This article goes deeper into ingestion time that may help with troubleshooting.

http://contoso.se/blog/?p=4493

 

 

Thank you very much for the follow up. These links really help!

I'll give PS a try and see if there's any difference. Thanks again! 

Hi Daichi,

8 hours for the initial ingestion and 1 hour for appended data are indeed much higher than expected.

Are you still experiencing that delay?

 

Related Conversations
Tabs and Dark Mode
cjc2112 in Discussions on
46 Replies
Extentions Synchronization
Deleted in Discussions on
3 Replies
Stable version of Edge insider browser
HotCakeX in Discussions on
35 Replies
flashing a white screen while open new tab
Deleted in Discussions on
14 Replies
How to Prevent Teams from Auto-Launch
chenrylee in Microsoft Teams on
29 Replies
Security Community Webinars
Valon_Kolica in Security, Privacy & Compliance on
13 Replies