TL;DR: Windows Containers Log Monitor is opensource released on Github.
In this blog post we announced our efforts to improve the tooling experience for containers. The first step in this journey was with a log tool concept we demoed during our container session at //build. This demo was met with clear excitement from attendees and interest from the community. After gathering customer feedback, we set out to bring this tool to production quality and make it open source available for our Windows Containers users.
Today, we are happy to announce the opensource release of the Windows Container Log Monitor, now available on Github! This blog offers a deep dive into the architecture and usage of the tool.
To recap, unlike Linux applications that log to STDOUT, Windows applications log to Windows log locations such as ETW, Event Log, and custom log files. Since many container ecosystem logging solutions are built to pull from the STDOUT pipeline as standard with Linux, Windows containers app logs historically have not been accessible via these solutions. The Log Monitor bridges this gap between Windows log locations and STDOUT, as depicted in the diagram below. The scope of the Log Monitor tool is to bridge Windows application logs to the STDOUT pipeline.
The Log Monitor tool consists of two primary elements: LogMonitor.exe and LogMonitorConfig.json. Like the //build demo POC, the Log Monitor is authored with an observer, publisher-subscriber design pattern; however, instead of subscribing to hardcoded log source providers as in the original concept, the Log Monitor is configured to subscribe to a set of sources defined by the user in the Log Monitor Config file.
The log sources supported by the Log Monitor tool include:
Note that these sources differ from the original concept, which collected Perf Counters and did not collect Event Logs. From feedback we concluded that Perf Counters, while useful, fall closer to system metrics than application logs, and were thus out of scope for this release. Additionally, the Log Monitor tool does not monitor the lifecycle of services within the container as in the original concept, but instead provides a nesting pattern so users can define their own service monitor or desired application.
The log tool is supported for Windows, Server Core, and Nano images.
Usage
A key point of feedback we received at //build and in the post-build survey was that the entry point-only solution restricts users from using their own app as the entry point process. We mitigate this with several options on usage:
An example dockerfile for the SHELL usage pattern (with nesting):
FROM mcr.mt.com/windows/servercore:1903
#LogMonitor directory contains LogMonitor.exe and LogMonitorConfig.json file
COPY LogMonitor/ C:/LogMonitor
WORKDIR /LogMonitor
SHELL ["C:\\LogMonitor\\LogMonitor.exe", "cmd", "/S", "/C"]
CMD c:\windows\system32\ping.exe -n 20 localhost
An example dockerfile for the ENTRYPOINT usage pattern (with nesting):
FROM mcr.microsoft.com/windows/servercore:1903
#LogMonitor directory contains LogMonitor.exe and LogMonitorConfig.json file
COPY LogMonitor/ C:/LogMonitor
WORKDIR /LogMonitor
ENTRYPOINT C:\LogMonitor\LogMonitor.exe c:\windows\system32\ping.exe -n 20 localhost
Note that in the SHELL usage pattern the CMD/ENTRYPOINT instruction should be specified in the SHELL form and not exec form. When exec form of the CMD/ENTRYPOINT instruction is used, SHELL is not launched, and the Log Monitor tool will not be launched inside the container.
Additionally, if you are building from an image that pre-defines an entrypoint (i.e. an IIS image that pre-defines IIS.ServiceMonitor as the entry point), you must redefine the entrypoint in your dockerfile, either as the Log Monitor if you are using the ENTRYPOINT pattern, or as the pre-defined entrypoint. If you do not redefine the ENTRYPOINT of your dockerfile and are using the SHELL usage pattern, the Log Monitor tool will not work. This is because the SHELL is tied to the entrypoint definition.
Both example usages wrap the ping.exe application. Other applications such as IIS.ServiceMonitor can be nested with Log Monitor in a similar fashion:
COPY LogMonitor.exe LogMonitorConfig.json C:\LogMonitor\
WORKDIR /LogMonitor
SHELL ["C:\\LogMonitor\\LogMonitor.exe", "powershell.exe"]
# Start IIS Remote Management and monitor IIS
ENTRYPOINT Start-Service WMSVC; `
C:\ServiceMonitor.exe w3svc;
Configuration
A sample Log Monitor Config file would be structured as follows:
{
"LogConfig": {
"sources": [
{
"type": "EventLog",
"startAtOldestRecord": true,
"eventFormatMultiLine": false,
"channels": [
{
"name": "system",
"level": "Error"
}
]
},
{
"type": "File",
"directory": "c:\\inetpub\\logs",
"filter": "*.log",
"includeSubdirectories": true
},
{
"type": "ETW",
"providers": [
{
"providerName": "IIS: WWW Server",
"ProviderGuid": "3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83",
"level": "Information"
},
{
"providerName": "Microsoft-Windows-IIS-Logging",
"ProviderGuid ": "7E8AD27F-B271-4EA2-A783-A47BDE29143B",
"level": "Information" ,
"keywords": "0xFF"
}
]
}
]
}
}
In the config file there are the three log types you can configure the Log Monitor tool to pull from:
Note that which events, files, and ETW providers an application logs to is defined by the application. There are some standard patterns for some common applications such as IIS. Some sample config files for these popular scenarios are included in the Github repo. If information about which events, files, and ETW providers a particular application logs to is not readily available, wevutil or Event Viewer can help.
What is next?
While we are excited to bring this functionality to our users, this is just the beginning of our journey. By open sourcing this tool, we hope to continuously gather feedback and improve the tool to meet customer needs. Some areas we have already identified and are investigating for future releases:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.