Logging and Log Analytics, Pretty confusing.

%3CLINGO-SUB%20id%3D%22lingo-sub-1236168%22%20slang%3D%22en-US%22%3ELogging%20and%20Log%20Analytics%2C%20Pretty%20confusing.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1236168%22%20slang%3D%22en-US%22%3E%3CP%3EI%20code%20out%20a%20lot%20of%20Infrastructure%20as%20Code%20in%20Terraform%20for%20our%20customers%20so%20they%20can%20have%20well%20documented%2C%20reliable%20ways%20to%20stand%20up%20or%20recreate%20any%20portion%20or%20all%20of%20there%20Azure%20environment.%26nbsp%3B%20I%20want%20to%20use%20Log%20Analytics%20Workspaces%20to%20trigger%20alerts%20for%20my%20Azure%20Automation%20Accounts%20and%20to%20monitor%20my%20Windows%20servers.%26nbsp%3B%20It%20seems%20like%20there%20are%20100%20different%20things%20in%20Azure%20related%20to%20logging%20and%20diagnostics.%26nbsp%3B%20And%20I%20am%20not%20sure%20if%20they%20are%20related%2C%20or%20separate.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EWhen%20I%20create%20a%20server%20I%20see%20Insights%2C%20Alerts%2C%20Diagnostic%20Settings%20and%20Logs%20all%20under%20monitoring.%26nbsp%3B%20These%20seem%20to%20be%20related%20in%20some%20way%20to%20the%20extensions.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20see%20things%20like%3A%3C%2FP%3E%3CP%3EMicrosoft.Azure.Diagnostics.IaaSDiagnostics%3CBR%20%2F%3EMicrosoft.EnterpriseCloud.Monitoring.MicrosoftMonitoringAgent%3CBR%20%2F%3EMicrosoft.VisualStudio.Services.TeamServicesAgent%3CBR%20%2F%3EMicrosoft.Compute.VMAccessAgent%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20think%20I%20want%20to%20focus%20on%26nbsp%3BMicrosoft.Azure.Diagnostics.IaaSDiagnostics.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EDo%20I%20have%20to%20make%20a%20storage%20account%20for%20my%20logs%20to%20go%20to%3F%26nbsp%3B%20How%20do%20I%20get%20those%20to%20work%20with%20Log%20Analytics%20Workspace%3F%26nbsp%3B%20I%20have%20been%20able%20to%20get%20the%20server%20to%20show%20up%20under%20virtual%20machines%2C%20but%20even%20when%20I%20do%20Perf%20query%2C%20I%20don't%20get%20anything.%20Can%20someone%20help%20me%20single%20out%20the%20pieces%20I%20need%20to%20get%20the%20logging%20from%20my%20servers%20and%20Automation%20Account%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1236168%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAgents%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ECustom%20Logs%20and%20Custom%20Fields%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EQuery%20Language%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-1236262%22%20slang%3D%22en-US%22%3ERe%3A%20Logging%20and%20Log%20Analytics%2C%20Pretty%20confusing.%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1236262%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F586032%22%20target%3D%22_blank%22%3E%40integraDan%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ELets%20break%20this%20down%20a%20little.%26nbsp%3B%20A%20workspace%20is%20created%20in%20a%20Resource%20Group%26nbsp%3B%20%2F%20Azure%20Region%20(ideally%20the%20same%20region%20as%20your%20Windows%20Servers).%26nbsp%3B%20The%20workspace%20is%20the%20storage%20(essentially%20it's%20a%20cloud%20database)%20for%20logs%20and%20other%20data%20types.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EYou%20then%20install%20the%26nbsp%3B%3CSPAN%3EMicrosoft%20Monitoring%20Agent%20(MMA)%20in%20your%20Server%20(Azure%20market%20place%20images%20have%20the%20extension%26nbsp%3Bbuilt-in).%26nbsp%3B%20For%20non%20IaaS%20resources%20you%20typically%26nbsp%3Buse%20the%20Diagnostic%20blade%20to%20send%20logs%20to%20Storage%2C%20EventHubs%20or%20Log%20Analytics%20(so%20if%20you%20select%20Storage%20you%20will%20need%20a%20storage%20account)%2C%20if%20you%20just%20select%20Log%20Analytics%20you%20can%20use%20your%20workspace.%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20agent%20(MMA)%20when%20installed%20or%20activated%20will%20send%20%3CSTRONG%3EHeartbeat%3C%2FSTRONG%3E%20data%20to%20Log%20Analytics.%26nbsp%3B%20For%20any%20other%20data%20like%20%3CSTRONG%3EPerf%3C%2FSTRONG%3E%20you%20have%20to%20go%20to%20the%20%5BAdvanced%20Settings%5D%20in%20Log%20Analytics%20and%20select%20the%20Perf%20counters%20(or%20Event%20logs%2C%20Syslogs%20etc...)%20you%20wish%20to%20collect%20from%20any%20server%20with%20the%20MMA%20on%20it%20(its%20managed%20centrally).%3CBR%20%2F%3E%3CBR%20%2F%3EPlease%20read%3A%26nbsp%3B%20%26nbsp%3B%3C%2FP%3E%0A%3CP%3EOfficial%20docs%3A%20%3CU%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdesign-logs-deployment%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdesign-logs-deployment%3C%2FA%3E%3C%2FU%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CU%3EHow%20to%20run%20a%20POC%20and%20design%3A%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Sentinel%2FBest-practices-for-designing-an-Azure-Sentinel-or-Azure-Security%2Fba-p%2F832574%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzure-Sentinel%2FBest-practices-for-designing-an-Azure-Sentinel-or-Azure-Security%2Fba-p%2F832574%3C%2FA%3E%3C%2FU%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E----------------------------%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Foverview%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Monitor%3C%2FA%3E%20(the%20product%20group%20in%20Microsoft%20that%20runs%20Logging%2C%20Metrics%2C%20Alerts...)%20also%20has%20a%20Metrics%20technology%20alongside%20the%20Logs%20(it%20collects%20a%20subset%20of%20perf%20counters)%20that%20are%20stored%20on%20your%20behalf.%26nbsp%3B%20See%20Azure%20Monitor%20Metrics%20(in%20the%20diagram)%20-%20these%20don't%20require%20an%20agent%20if%20your%20servers%20are%20in%20Azure.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Occasional Visitor

I code out a lot of Infrastructure as Code in Terraform for our customers so they can have well documented, reliable ways to stand up or recreate any portion or all of there Azure environment.  I want to use Log Analytics Workspaces to trigger alerts for my Azure Automation Accounts and to monitor my Windows servers.  It seems like there are 100 different things in Azure related to logging and diagnostics.  And I am not sure if they are related, or separate.  

 

When I create a server I see Insights, Alerts, Diagnostic Settings and Logs all under monitoring.  These seem to be related in some way to the extensions.  

 

I see things like:

Microsoft.Azure.Diagnostics.IaaSDiagnostics
Microsoft.EnterpriseCloud.Monitoring.MicrosoftMonitoringAgent
Microsoft.VisualStudio.Services.TeamServicesAgent
Microsoft.Compute.VMAccessAgent

 

I think I want to focus on Microsoft.Azure.Diagnostics.IaaSDiagnostics.  

 

Do I have to make a storage account for my logs to go to?  How do I get those to work with Log Analytics Workspace?  I have been able to get the server to show up under virtual machines, but even when I do Perf query, I don't get anything. Can someone help me single out the pieces I need to get the logging from my servers and Automation Account 

1 Reply
Highlighted

@integraDan

 

Lets break this down a little.  A workspace is created in a Resource Group  / Azure Region (ideally the same region as your Windows Servers).  The workspace is the storage (essentially it's a cloud database) for logs and other data types.

 

You then install the Microsoft Monitoring Agent (MMA) in your Server (Azure market place images have the extension built-in).  For non IaaS resources you typically use the Diagnostic blade to send logs to Storage, EventHubs or Log Analytics (so if you select Storage you will need a storage account), if you just select Log Analytics you can use your workspace. 

 

The agent (MMA) when installed or activated will send Heartbeat data to Log Analytics.  For any other data like Perf you have to go to the [Advanced Settings] in Log Analytics and select the Perf counters (or Event logs, Syslogs etc...) you wish to collect from any server with the MMA on it (its managed centrally).

Please read:   

Official docs: https://docs.microsoft.com/en-us/azure/azure-monitor/platform/design-logs-deployment

 

How to run a POC and design: https://techcommunity.microsoft.com/t5/Azure-Sentinel/Best-practices-for-designing-an-Azure-Sentinel...

 

----------------------------

 

Azure Monitor (the product group in Microsoft that runs Logging, Metrics, Alerts...) also has a Metrics technology alongside the Logs (it collects a subset of perf counters) that are stored on your behalf.  See Azure Monitor Metrics (in the diagram) - these don't require an agent if your servers are in Azure.