Hi all,
Windows Server 2003 x86 systems running Exchange require the use of the
/3GB
switch in the boot.ini to ensure that there is plenty of address space available for the
Store.exe
process. In brief, the
/3GB
switch is used to provide more virtual memory to user mode processing. It signals the OS to use 3 GB of the 4 GB virtual address space for each user mode process, as opposed to the normal 2 GB.. This leaves only 1 GB of the virtual address space for kernel mode processing, therefore the amount of Paged Pool Memory, Non-Paged Pool Memory and Available System PTEs on the server are reduced. As a result, there is often fine tuning required for problem free operations.
The theoretical maximums for Paged and Non-Paged Pool on Server 2003 32bit with the /3GB switch are:
Paged Pool = 256MB
Non-Paged Pool = 128MB
For a full explanation of how the
/3GB
switch works, please refer to the following AskPerf Blog article:
Memory Management - Demystifying /3GB
Issues caused by the
/3GB
switch will typically show up as some type of system resource depletion. A vast majority of the cases we see in this area are due to either Paged or Non-Paged Pool depletion. If this is occurring, there will be Event 2020 (Paged Pool depletion) or Event 2019 (Non-Paged Pool depletion) events in the System Event Log. We can also run low on Available System PTEs (Page Table Entries). Customers are often first made aware of problems in the Exchange environment when they see errors that HTTP.sys is refusing connections.
Making the following changes will almost always get the server back to a serviceable state. This is
NOT
to be considered a resolution, but is intended to get the server back online and serving mail quickly – additional data collection and analysis (perfmon, poolmon and possibly a complete memory dump) will be required to properly diagnose the underlying issue:
1. Make sure there is no more than 4GB of physical RAM in the system (i.e. do not have more RAM and use /MAXMEM or /BURNMEMORY as memory structures are still created at boot time in order to hide/remove memory when using these parameters).
2. Change the system to use
/USERVA=2970
setting in the Boot.ini.
3. Set the system to use
/BASEVIDEO
in the Boot.ini and the display driver to Standard VGA.
4. To allow the Operating System to gracefully manage and allocate memory, set Paged Pool and System Pages to 0 for “system managed” by setting the following registry keys:
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetcontrolSession ManagerMemory Management
SystemPages=0
PagedPoolSize=0
PoolUsageMaximum=60 (decimal)
5. Ensure you have the most current hotfix updates for
Store.exe
and
Exifs.sys.
6.
HTTP.sys
is designed to stop servicing requests if the system comes within 20 MB of the non-paged pool limitation. As per
KB 934878
, you can set the
EnableAggressiveMemoryUsage
registry key to allow the system to get closer to the Non-Paged Pool limit before
HTTP.sys
stops servicing requests:
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesHTTPParameters
EnableAggressiveMemoryUsage=1 (DWORD)
7. Ensure settings are in place to capture a complete memory dump if necessary as per the following KB article:
972110 How to generate a kernel dump file or a complete memory dump file in Windows Server 200...
8. Reboot the system
When Microsoft is engaged for troubleshooting these issues, we often find that the highest consumers of Non-Paged Pool are Exchange / EXIFS related pool tags (AuxL or Flst). This is typical of a heavily loaded Exchange server. When you see this it is a good indicator that there are more users than the server can handle. You should consider the following:
- Bring another Exchange server online and redistribute the user load
- Move to a Server 2008 64bit solution for your Exchange server
- If there is plenty of paged pool available, you can move AuxL and Flst to paged pool as was the default behavior in SP1.
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesEXIFSParameters
AuxFreeListInPagedPool=1 (DWORD)
FreeListInPagedPool=1 (DWORD)
Helpful Articles / References
934878 Users receive a "The page cannot be displayed" error message, and "Connections_refused" entries are logged in the Httperr.log file on a server that is running Windows Server 2003, Exchange 2003, and IIS 6.0
http://support.microsoft.com/default.aspx?scid=kb;EN-US;934878
912376 How to monitor and troubleshoot the use of paged pool memory in Exchange Server 2003 or in Exchange 2000 Server
http://support.microsoft.com/default.aspx?scid=kb;EN-US;912376
.
823440 Use of the /3GB switch in Exchange Server 2003 on a Windows Server 2003-based system
http://support.microsoft.com/default.aspx?scid=kb;EN-US;823440
810371 Using the /Userva switch on Windows Server 2003-based computers that are running Exchange Server
http://support.microsoft.com/default.aspx?scid=kb;EN-US;810371
AskPerf Blog - Memory Management - Demystifying /3GB
http://blogs.technet.com/b/askperf/archive/2007/03/23/memory-management-demystifying-3gb.aspx
...
Robin Aylesworth
