Announcing Log Monitor 2.0 Release Candidate
Published Mar 08 2023 04:38 PM 6,463 Views
Microsoft

Log Monitor is an open-source tool that helps customers expose their Windows logs to the STDOUT pipeline. After our previous release of Log Monitor, we’ve heard feedback from our customers around the lack of structured formatting for logs that affect querying and diagnosability. We’ve enabled JSON log format to improve Log Monitor’s interoperability and experience along with starting to improve extensibility to log analysis tools including Azure Monitor, ELK, etc.

 

We’re excited to announce the latest release candidate of Log Monitor that comes with the following improvements:

  • All monitors, ETW, Process, Events and File now output JSON logs as specified in this schema discussion
  • ETW provider name now included in ETW Log Output
  • Stability and Quality Improvements:
    • Docs Fix: specify default level as Error for Event Log
    • Fixes a bug where file monitor source tagging skips some log lines

 

We also encourage you to provide feedback on the following issues we’re currently working on:

  • Process Monitoring component does not support multi-byte characters: Issue: #121
  • Log output format is not configurable, discussions on additional log formats: Discussion #123

 

Log Monitor 2.0.png

 

How to Use Log Monitor with Azure Monitor Container Insights

This example will show the changes to the end-to-end experience using Log Monitor with Azure Monitor Container Insights. For a more detailed step by step please see this blog post.

 

Config File

In order for Log Monitor to pass logs from ETW, Event Logs, and Custom Log Files, the Log Monitor tool needs to be configured through a LogMonitorConfig.json file. Additional documentation can be found here.

 

Build a container image with Log Monitor

Log Monitor can be used in a SHELL or ENTRYPOINT usage pattern. It can also be used nested with Log Monitor as shown below.

 

FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019
WORKDIR /LogMonitor
COPY LogMonitorConfig.json .
RUN powershell.exe -command wget \
  -uri https://github.com/microsoft/windows-container-tools/releases/download/v2.0-rc0/LogMonitor.exe \
  -outfile LogMonitor.exe
# Change the startup type of the IIS service from Automatic to Manual
RUN sc config w3svc start=demand
# Enable ETW logging for Default Web Site on IIS
RUN c:\windows\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/sites /"[name='Default Web Site'].logFile.logTargetW3C:"File,ETW"" /commit:apphost
EXPOSE 80
# Start "C:\LogMonitor\LogMonitor.exe C:\ServiceMonitor.exe w3svc"
ENTRYPOINT ["C:\\LogMonitor\\LogMonitor.exe", "C:\\ServiceMonitor.exe", "w3svc"]

 

Enable Azure Monitor Container Insights

Azure Monitor Container Insights can be used to monitor logs when deploying to AKS. Additional documentation can be found here.

 

lucillexiong_3-1678314188480.png

 

Sample Query for ETW

With JSON output format, customers can extract specific data points from logs to analyze on. In Azure Monitor, the LogEntry field can be queried as shown below with the sample Kusto query which crunches the data from IIS ETW provider.

 

ContainerLog
| where LogEntry has "Microsoft-Windows-IIS-Logging" // for optimization of parse_json
| extend d = parse_json(LogEntry)
| extend Source = d.Source
| extend Status = d.LogEntry.EventData["sc-status"]
| extend Method = d.LogEntry.EventData["cs-method"]
| extend Path = d.LogEntry.EventData["cs-uri-stem"]
| extend UserAgent = d.LogEntry.EventData["csUser-Agent"]
| extend cIP = d.LogEntry.EventData["c-ip"]
| extend sIP = d.LogEntry.EventData["s-ip"]
| extend Port = d.LogEntry.EventData["s-port"]
| extend PodName = d.LogEntry.EventData["s-computername"]
| extend TimeTaken = d.LogEntry.EventData["time-taken"]
| where Status > 400
| project TimeGenerated, Source, PodName, Method, Status, UserAgent, cIP, sIP, Port, Path, LogEntry
| take 30

 

 

lucillexiong_5-1678314188488.png

Closing

We encourage you to use, provide feedback, and contribute to Window Container Tools at microsoft/windows-container-tools: Collection of tools to improve the Windows Containers experience ... Please look out for upcoming features and tools that enhance the Window Containers experience.

 

 

Co-Authors
Version history
Last update:
‎Mar 08 2023 03:06 PM
Updated by: