Windows Server Summit 2024
Mar 26 2024 08:00 AM - Mar 28 2024 04:30 PM (PDT)
Microsoft Tech Community
LIVE
Containers for ITPros - Resource limits on containers
Published Oct 22 2019 11:39 AM 3,451 Views

One of the first things that struck admins when making the transition from managing Virtual Machines (VMs) to containers is resource constraint. Whenever I demonstrate containers to an audience of ITPros, and I use the docker run command to spin up a container, the question that comes back is "How much memory is this container using?" or something like "How do I set how many processors this container can use?". This is a natural concern for admins since ensuring their applications and hosts have the proper amount of resources and don't interfere with one another is one of the primary tasks of a System Admin.

The main thing that is different from VMs and containers in terms of resource utilization is that for VMs you set a limit of how much CPU and memory that VM can get/see. Over the years we evolved that to over provision, share, quota limit, etc. This is primarily because VMs emulate the hardware that we presented to them to the OS running on that given VM. With containers however, this is not true. Instead, since containers are a virtualization of the OS, not the hardware, the container can use as much resources the container host can provide. Of course, you don't want to get to a case where one container uses all the resources from a host - especially if you are running multiple containers on a on a single node. In today's blog, we'll look at simple ways to limit how much CPU and memory a container can use.

 

Setting CPU limits

The simplest way to limit the CPU of a container is by specifying the --cpus parameter on docker run:

 

docker run -d --cpus=1 --name testcontainer mcr.microsoft.com/windows/servercore:ltsc2019

 

In this example, we are limiting the container to use only 1 CPU from the container host. You can set a limit of up to the number of CPUs the host has. If you try to configure more CPUs than the host, you'll get an error.

Another option is to set half, or another percentage of a CPU:

 

docker run -d --cpus=1.5 --name testcontainer mcr.microsoft.com/windows/servercore:ltsc2019

 

In this case the container can use 1 CPU and 50% of another CPU. The lowest amount you can specify for --cpus is .1 which gives the container 10% of one CPU.

 

Setting memory limits

To set memory limits, the simplest way is to specify the -m or --memory parameter on docker run:

 

docker run -d -m 2048 --name testcontainer mcr.microsoft.com/windows/servercore:ltsc2019

 

This command will enforce a limit of 2Gb of memory for that container. Just like CPUs you can't over provision memory for containers. In fact, docker will let you create the container but you won't be able to start it.

 

Checking your configuration on running containers

Once you configure the limits and your containers are running, you can check if the limits are being enforced. The way to do that on docker is by using the command docker inspect:

 

docker inspect testcontainer

 

The command above will return a very unfriendly JSON log:

 

[
    {
        "Id": "109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe",
        "Created": "2019-10-22T17:08:59.6228671Z",
        "Path": "c:\\windows\\system32\\cmd.exe",
        "Args": [],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2019-10-22T17:16:15.5314257Z",
            "FinishedAt": "2019-10-22T10:16:15.56805-07:00"
        },
        "Image": "sha256:739b21bd02e70d010b6b75d41aad12b941ab49aeda66d7930dd7b3745511ca9c",
        "ResolvConfPath": "",
        "HostnamePath": "",
        "HostsPath": "",
        "LogPath": "C:\\ProgramData\\docker\\containers\\109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe\\109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe-json.log",
        "Name": "/testcontainer",
        "RestartCount": 0,
        "Driver": "windowsfilter",
        "Platform": "windows",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "no",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 0,
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "process",
            "CpuShares": 0,
            "Memory": 2048,
            "NanoCpus": 2000000000,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": [],
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": null,
            "ReadonlyPaths": null
        },
        "GraphDriver": {
            "Data": {
                "dir": "C:\\ProgramData\\docker\\windowsfilter\\109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe"
            },
            "Name": "windowsfilter"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "109d65ac9ab7",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": [
                "c:\\windows\\system32\\cmd.exe"
            ],
            "Image": "mcr.microsoft.com/windows/servercore:ltsc2019",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "109d65ac9ab7cdc9de5996e2e0853b67ea0d461d723a7c18920f9c176a45ffbe",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "nat": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "6994e855c400ea285d24e30bb9f2c6d5d749014ab5a8771e011c506b24edff0b",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]

 

Luckily, there's an option to extract only the information you want:

 

docker inspect --format='{{.HostConfig.Memory}}' testcontainer

 

The command above will return the total memory allocated for this container. If you want to check the CPU limit, you can use:

 

docker inspect --format='{{.HostConfig.NanoCpus}}' testcontainer

 

See that this will return the calculation of CPU period * CPU quota. Although you can't configure these on a Windows container host, these are still present. In our case, the result for the configuration of this container (2 CPUs) is "2000000000"

 

Windows limitations on resource constraints

When checking the Docker documentation, you'll notice that there are a few other options for limiting CPU and memory on your containers. Today, these are available for Linux only.  If you have a strong case for using one of these options that is not available on Windows, please let us know.

 

Hopefully this is useful. Let us know in the comments!

Version history
Last update:
‎Oct 22 2019 11:43 AM
Updated by: