Log Monitor is an open-source tool that helps customers expose their Windows logs to the STDOUT pipeline. After our previous release of Log Monitor, we’ve heard feedback from our customers around the lack of structured formatting for logs that affect querying and diagnosability. We’ve enabled JSON log format to improve Log Monitor’s interoperability and experience along with starting to improve extensibility to log analysis tools including Azure Monitor, ELK, etc.
We’re excited to announce the latest release candidate of Log Monitor that comes with the following improvements:
- All monitors, ETW, Process, Events and File now output JSON logs as specified in this schema discussion
- ETW provider name now included in ETW Log Output
- Stability and Quality Improvements:
- Docs Fix: specify default level as Error for Event Log
- Fixes a bug where file monitor source tagging skips some log lines
We also encourage you to provide feedback on the following issues we’re currently working on:
- Process Monitoring component does not support multi-byte characters: Issue: #121
- Log output format is not configurable, discussions on additional log formats: Discussion #123
How to Use Log Monitor with Azure Monitor Container Insights
This example will show the changes to the end-to-end experience using Log Monitor with Azure Monitor Container Insights. For a more detailed step by step please see this blog post.
Config File
In order for Log Monitor to pass logs from ETW, Event Logs, and Custom Log Files, the Log Monitor tool needs to be configured through a LogMonitorConfig.json file. Additional documentation can be found here.
Build a container image with Log Monitor
Log Monitor can be used in a SHELL or ENTRYPOINT usage pattern. It can also be used nested with Log Monitor as shown below.
FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019
WORKDIR /LogMonitor
COPY LogMonitorConfig.json .
RUN powershell.exe -command wget \
-uri https://github.com/microsoft/windows-container-tools/releases/download/v2.0-rc0/LogMonitor.exe \
-outfile LogMonitor.exe
# Change the startup type of the IIS service from Automatic to Manual
RUN sc config w3svc start=demand
# Enable ETW logging for Default Web Site on IIS
RUN c:\windows\system32\inetsrv\appcmd.exe set config -section:system.applicationHost/sites /"[name='Default Web Site'].logFile.logTargetW3C:"File,ETW"" /commit:apphost
EXPOSE 80
# Start "C:\LogMonitor\LogMonitor.exe C:\ServiceMonitor.exe w3svc"
ENTRYPOINT ["C:\\LogMonitor\\LogMonitor.exe", "C:\\ServiceMonitor.exe", "w3svc"]
Enable Azure Monitor Container Insights
Azure Monitor Container Insights can be used to monitor logs when deploying to AKS. Additional documentation can be found here.
Sample Query for ETW
With JSON output format, customers can extract specific data points from logs to analyze on. In Azure Monitor, the LogEntry field can be queried as shown below with the sample Kusto query which crunches the data from IIS ETW provider.
ContainerLog
| where LogEntry has "Microsoft-Windows-IIS-Logging" // for optimization of parse_json
| extend d = parse_json(LogEntry)
| extend Source = d.Source
| extend Status = d.LogEntry.EventData["sc-status"]
| extend Method = d.LogEntry.EventData["cs-method"]
| extend Path = d.LogEntry.EventData["cs-uri-stem"]
| extend UserAgent = d.LogEntry.EventData["csUser-Agent"]
| extend cIP = d.LogEntry.EventData["c-ip"]
| extend sIP = d.LogEntry.EventData["s-ip"]
| extend Port = d.LogEntry.EventData["s-port"]
| extend PodName = d.LogEntry.EventData["s-computername"]
| extend TimeTaken = d.LogEntry.EventData["time-taken"]
| where Status > 400
| project TimeGenerated, Source, PodName, Method, Status, UserAgent, cIP, sIP, Port, Path, LogEntry
| take 30
Closing
We encourage you to use, provide feedback, and contribute to Window Container Tools at microsoft/windows-container-tools: Collection of tools to improve the Windows Containers experience ... Please look out for upcoming features and tools that enhance the Window Containers experience.