cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
PRF: Memory Management (Event ID 333)
By
CraigMarcho
Published Mar 15 2019 08:27 PM 2,042 Views
CraigMarcho
Microsoft
‎Mar 15 2019 08:27 PM

PRF: Memory Management (Event ID 333)

‎Mar 15 2019 08:27 PM
First published on TECHNET on Apr 10, 2009


MEMORY MANAGEMENT – EVENT ID 333




Description: The first thing to understand is what exactly an Event ID 333 is.  The event ID 333 is a System event error log that occurs when the registry is unable to complete a flush operation to the disk.  There are several reasons that this can fail and we'll discuss them below.  So, what does the Event ID 333 look like?  In the system event log, you see at least one, and more likely multiple instances of the following:



Event ID 333’s are new to Windows Server 2003 Service Pack 1 and are written to the system event log if the Operating System is not able to flush out or write to the registry hive.  The symptoms that accompany an Event ID 333 can vary between server hangs, “Insufficient resources exist to complete the requested service” errors, SQL Databases stopping and starting, database queries are slow and sluggish operating system performance.  The apparent slow server performance is the reason why the Performance team is engaged on the majority of the issues.  As mentioned previously, the Event ID 333 can occur on both x86 and x64 systems.


Now that we have a little background on what the Event ID 333 is and why it exists, we’re going to take a look at the causes.



Scoping the Issue:


There are a number of different reasons for why a server is logging the Event ID 333's - the majority of the issues are caused by one of the following:



  1. The disk hosting the system partition is not keeping up with the load (high disk queue lengths).  This can be determined using a Performance Monitor capture.  This can occur on both x64 and x86 systems.

  2. There is memory pressure.  In many cases there will also be issues with low Paged (Event 2020) or Non-Paged (Event 2019) pool memory or low System PTE's.  Using Performance Monitor can help to determine if you are low on System PTE's.  This is more likely to occur on x86 systems dues to the memory limitations of the 32-bit address space.

  3. A filter driver is keeping the registry from being flushed.  This can occur on both x64 and x86 systems.

  4. A User account granted the “Lock system pages in memory” user right.  This can occur on both x64 and x86 systems.


Data Gathering: In all instances, collecting either MPS Reports with the General, Internet and Networking, Business Networks and Server Components diagnostics, or a Performance-oriented MSDT manifest must be done.  Additional data required may include the following:




Troubleshooting / Resolution: After you have gathered this data, review the following:


Disk issues:



  1. Enable Enhanced Performance Options for the physical disks that host the Operating System itself

  2. Update your disk controller firmware and drivers.  Engage your hardware vendor to ensure that you are running the latest supported version and also to verify if there are any known issues.

  3. Determine what process is taxing the disk by using Performance Monitor logging.  Examine the Process Object and review each processes I/O Data bytes/sec.  If a process exhibits an unusual amount of I/O compared to your server's baseline performance, then investigate that process.  For non-Microsoft processes, contact the software vendor.

  4. As a workaround, add the RegistryLazyFlushInterval value and set it to 60 using Article 324446.


Maximize Kernel Memory and System PTE's:



  1. If you are using the /3GB switch on a 32-bit system, do you really need it?  As we discussed in our post on /3GB this switch may not be necessary.

    • Some software vendors recommend implementing the switch for their applications - verify with them that they have the IMAGE_FILE_LARGE_ADDRESS_AWARE flag set in their image header.  If they do not, then the application is only able to use the standard 2GB of user-mode memory - but the kernel is now limited to 1GB for no reason.

    • You can also check the virtual memory of the executable that is believed to take advantage of /3GB.  If the process is not exceeding 1.8GB of Virtual Memory then it is unlikely to be able to take advantage of the extra virtual memory.

    • If the /3GB switch is required (for example Exchange Servers that are configured as Mailbox Servers ) then add the /USERVA=2970 switch to give a portion of the User-mode memory to the Kernel to help guard against PTE Depletion. Microsoft KB Article 316739 discusses the use of the /USERVA switch.

  2. Adjust the allocation and use of PagedPool memory by making some registry changes that are outlined in Microsoft KB Article 312362 .

  3. Disable the Hot Add Memory feature to re-allocate reserved Paged Pool memory back to the Operating System.  When the Hot Add Memory feature is enabled, the operating system pre-allocates kernel resources to handle any future memory that may be added to the computer.  Kernel resources are allocated based on the capabilities of the computer instead of on the RAM that is actually installed.  The kernel may allocate significant resources to RAM that may never be installed.  Therefore, the Hot Add Memory feature may cause the maximum size of the paged pool to be much smaller than expected.


A filter driver is preventing the registry from being flushed:


A quick note on filter drivers - removing or disabling filter drivers without understanding the impact to the system can result in unexpected system behaviors including system hangs or bugchecks.  Examples of common programs that use filter and kernel drivers include Anti-virus software, Backup Software (including Microsoft Volume Shadow Copy) and Version-2 printer drivers .  For information on how to temporarily disable filter drivers, refer to Microsoft KB Article 816071 .  Before disabling the filter drivers however, check to see if some of the 3rd party drivers are simply outdated.  The easiest way to check on these 3rd party filter drivers is to use our MPS Reporting utility to capture this information.



"Lock Pages in Memory" user right:


On x64 systems, the likelihood of a server running out of kernel memory or System PTE's is far less than on an x86 system.  However, we have seen the Event 333 error occur on x64 systems as well.  Using the "!vm" debugger command when reviewing a memory dump of the server may indicate that the server is low on physical memory, even though performance monitor data indicates that there is plenty of available memory.  The sample output below illustrates this:


15: kd> !vm
*** Virtual Memory Usage ***
Physical Memory: 2095394 ( 8381576 Kb)
Page File: \??\C:\pagefile.sys
Current: 4249600 Kb Free Space: 4154172 Kb
Minimum: 4249600 Kb Maximum: 4249600 Kb
Available Pages: 868200 ( 3472800 Kb)
ResAvail Pages: 250 ( 1000 Kb)
********** Running out of physical memory **********

To work around this behavior on x64 platforms or on servers with 4GB or less of physical RAM use the following steps:



  1. Click on Start --> Run

  2. Type Secpol.msc and click OK

  3. Expand "Local Policies" then click on "User Rights Assignment"

  4. Double click on "Lock pages in memory"

  5. Highlight any user accounts listed and click on "Remove"

  6. Click OK after all accounts have been removed

  7. Reboot the system

NOTE: By default, the Windows Operating system does not grant this user right to any accounts.  The “Lock pages in memory” right is granted to the account used for SQL Services by the SQL 2005 RTM/SP1 Enterprise Edition install on 32bit systems.  If you are using SQL Enterprise on 32-bit servers with more than 4GB of RAM, the Lock Pages in Memory right is needed.  To help reduce the occurrence of the Event ID 333's on these systems, ensure the user account you are using for the SQL services is only used for SQL. Check the access right for “Lock pages in memory” and only list the account used for SQL.  If System, or any other accounts are listed, remove them.  For x64 systems, remove all accounts listed.




Additional Resources:


0 Likes
Like
Version history
Last update:
‎Mar 15 2019 08:27 PM
Updated by:
Labels