Memory Management - Demystifying /3GB

CraigMarcho · ‎Mar 15 2019

First published on TECHNET on Mar 23, 2007

As promised - here's the long awaited post on the infamous /3GB switch. At least once a week we have this discussion with a Systems Administrator somewhere who has this set in the boot.ini file on all of the servers but doesn't know why. Maybe someone added it to the server build process at some point or perhaps someone read about the /3GB switch somewhere and thought that it would improve performance or enable them to see the full 4GB or 8GB of physical memory installed on the system. So let's start by dispelling a few /3GB myths:

/3GB won't enable you to see the additional 4GB or 8GB of RAM you added to your new server
/3GB doesn't necessarily make your application 50% more efficient
/3GB should not be a standard for your environment (there are exceptions, and we'll get to those)

OK - so what does the /3GB switch really do? If you recall, from the Memory Management 101 post, Windows 32-bit Operating Systems implement a virtual memory system based on a flat 32-bit address space. This address space translates into 4GB of virtual memory - no more, and no less. By default, the address space is divided into two equal chunks. 2GB for the Kernel and 2GB for User-mode processes. The Kernel space is common for all applications and the User-mode processes each get their own 2GB address space to work with.

So where does the /3GB switch come in? Windows 2000 Advanced Server, Windows 2000 Datacenter Server, Windows XP SP2 and later and all versions of Windows Server 2003 support the /3GB boot-time option to allow the user mode address space to grow to 3GB. The /3GB option was intended as a short term solution to allow applications such as database servers to maintain more data in memory than a 2GB address space allowed. However, using the /3GB method to increase the user-mode memory space comes at a cost.

Remember that we only have a 4GB total address space to work with. If we have to allocate an additional 1GB of this address space to the user-mode space, then the System space is cut in half. Drivers, Heap, Paged & NonPaged Memory all have only half the resources to work with now. However, because of the way memory mapping works, cutting the kernel space in half does a lot more than just reducing the address space. Many of the structures within the kernel virtual memory space are cut back by far more than 50%. For example, we took a Windows Server 2003 Enterprise R2 machine with 1GB of RAM installed and compared some values with and without the /3GB switch enabled.

Default OS Build:
Free System PTE's	251,980 (1,007,920 kb)
NonPaged Pool Max	206,848 kb

With /3GB enabled:
Free System PTE's	34,884 (139,536 kb)
NonPaged Pool Max	129,312 kb

As you can see, the Free System PTE's drops by over 200,000. Keep in mind that this is only a test server that isn't under any sort of load. A machine under medium to heavy load could quite easily run out of free PTE's - meaning that the system can no longer map system pages such as I/O space, kernel stacks and memory descriptor lists. In addition, look at NonPaged Pool after the /3GB parameter is enabled. The NonPaged Pool maximum is only 130MB. Drivers use the NonPaged Pool for many of their requirements because they can be accessed at any IRQL. Once we run into NonPaged pool depletion, we're looking at our old friend, the Event ID 2019.

OK - so let's quickly recap what we've discussed so far. The /3GB switch is not related to the amount of physical memory installed in a system. It is useful if you have an application that can take advantage of a larger address space. For a process to access the full 3GB address space, the image file must have the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in the image header.

If the flag is not set in the image header, then the OS reserves the third gigabyte so that the application won't see virtual addresses greater than 0x7FFFFFFF. You set this flag by specifying the linker flag /LARGEADDRESSAWARE when building the executable. This flag has no effect when running the application on a system with a 2-GB user address space. Therefore if you enable the /3GB switch, then applications that do not have this flag set can only use the standard 2GB of User mode memory, and the Kernel is still limited to the 1GB space - which means that 1GB of virtual memory is basically wasted!

Earlier, we mentioned that there were some applications that benefit from the use of the /3GB switch. The predominant scenario where the /3GB switch is not only recommended, but actually required is with Microsoft Exchange servers that house public folders and / or mailboxes. Due to the way that Exchange handles memory management, the additional 1GB of user mode memory is required to ensure that the Store.exe process does not run out of virtual address space. However, in order to guard against System PTE depletion, the system can be tuned using the /USERVA switch in conjunction with the /3GB switch. This tunes the actual amount of memory for the Address space. For example, setting USERVA=3030 means that the process space is actually only 3,030MB and not 3,072MB (which would be the process space with only the /3GB switch present). The additional 42MB is used for System PTE usage. The USERVA value can safely be tweaked as low as 2800 - however, if it is necessary to set USERVA this low, then you probably want to start thinking about scaling your Exchange environment to spread the load!

Ideally, there should always be at least 24,000 Free System PTE's at boot time. Depending on server workload there may be wide variances in the amount of Free System PTE's during the course of a normal duty cycle, so it may be necessary to implement some long-term monitoring to ensure that the server does not fall below 10,000 Free PTE's.

So there you have it - the /3GB switch demystified. Hopefully this post, as well as the others in our Memory Management series will help you understand a bit more about how and why the Operating System behaves the way it does. Remember that the /3GB switch is intended to be used in very specific instances - and now you know why!

Additional Resources:

- Tim Newton

Products (50)

Special Topics (27)

Video Hub (462)

Most Active Hubs

Most Active Hubs

Video Hub

Memory Management - Demystifying /3GB