How to use Pageheap.exe in Windows XP, Windows 2000, and Windows Server 2003

This article describes how to use the Page Heap tool (Pageheap.exe) in Microsoft Windows XP, Microsoft Windows 2000, and Microsoft Windows Server 2003.

Pageheap.exe sets page heap flags that help to find heap-related corruption. It can also help detect leaks in programs that are running on Windows 2000 Professional Service Pack 2 (SP2) and Windows XP Professional systems.

Pageheap.exe introduces a software validation layer (Page Heap manager) between the application and the system that verifies all dynamic memory operations (allocations, frees, and other heap operations). When Page Heap manager is enabled, the application that is being tested is then started under a debugger. If a problem is encountered, it will cause a debugger break.

Important Pageheap.exe does not specify what the bug is, but it will crash the system when a problem is encountered. It enables a verification layer that already exists in the Ntdll.dll system libraries in Windows 2000 Professional SP2 and Windows XP Professional. Pageheap.exe will not work on previous versions of Microsoft Windows.

If the application being tested is not started under a debugger and a bug is encountered, it will just crash without any feedback.

Concepts

A common problem in application development is heap corruption. This typically occurs when an application allocates a block of heap memory of a given size and then writes to memory addresses beyond the requested size of the heap block. Heap corruption can also occur when an application writes to block of memory that has already been freed.

Two concepts are central to understanding the commands related to Pageheap.exe and the way to use it:

• Corruptions in heap blocks are discovered by either placing a non-accessible page at the end of the allocation, or by checking fill patterns when the block is freed.
• There are two heaps (full-page heap and normal page heap) for each heap created within a process that has page heap enabled.
  • Full-page heap reveals corruptions in heap blocks by placing a non-accessible page at the end of the allocation. The advantage of this approach is that you achieve "sudden death," meaning that the process will access violate (AV) exactly at the point of failure. This behavior makes failures easy to debug. The disadvantage is that every allocation uses at least one page of committed memory. For a memory-intensive process, system resources can be quickly exhausted.
  • Normal page heap can be used in situations where memory limitations render full-page heap unusable. It checks fill patterns when a heap block is freed. The advantage of this method is that it drastically reduces memory consumption. The disadvantage is that corruptions will only be detected when the block is freed. This makes failures harder to debug.

Download Location for the Pageheap Tool

To download the latest debug tools package, click the following link:

Select the latest release of the debug tools. When you install the tools, select the Custom installation, and then install to a directory with an appropriate name. For example, install to tools to C:\Debug or C:\Debugtools.

Choosing a Method to Investigate Heap Block Corruptions

Most of the corruptions in heap blocks can be discovered in one of two ways:

• Full-page heap: Place a non-accessible page at the end of the allocation.
• Normal page heap: Check fill patterns when the block gets freed.

Full-Page Heap

Full-page heap should be enabled for individual processes, or under limited parameters for large processes, because of its high memory requirements. It cannot be enabled system-wide, because it is difficult to evaluate the required page file size. Using a page file that is too small with system-wide full-page heap renders the system unbootable.

The advantage of full-page heap is that it causes a process to access violate (AV) exactly at the point of failure. This makes the failure easy to debug. In order to be able to pinpoint failures, first use normal page heap to determine the range where a process is failing, and then use full-page heap on individual large-scale processes for that restricted class of allocations (that is, a specific size range or a specific library).

Normal Page Heap

Normal page heap can be used for the testing of large-scale processes without the high memory consumption that full-page heap requires. However, normal page heap delays detection until blocks are freed, thus making failures more difficult to debug.

In general, use normal page heap for initial large-scale processes testing. Then, if problems are detected, enable full-page heap for a restricted class of allocations in those processes.

Normal page heap can be safely enabled system-wide for all processes. This is very useful on test benches that perform general system validation, rather than component focused testing. Normal page heap can also be enabled for a single process.

Using GFlags with System-Wide Page Heap

The GFlags tool is used to enable system-wide page heap. In order for a GFlags command to take effect, you must restart your computer after you issue the command.

To enable system-wide normal page heap:

1. Type the following at the command line:gflags -r +hpa
   
   
2. Restart your computer.

To disable system-wide normal page heap:

1. Type the following at the command line:gflags -r -hpa
   
   
2. Restart your computer.

Note No other GFlags settings are useful when you enable page heap. If other settings that appear to be related to the heap are enabled, then page heap bugs can be introduced due to conflicts between page heap manager and these "harmless" heap flags.

Using GFlags With a Single Process Page Heap

You can enable the page heap to monitor one specific process. To do this, follow these steps:

1. At a command prompt, change the directory where you installed the debug tools.
2. At the command prompt, type the following, and then press ENTER:
   Gflags.exe /p /enable lsass.exe
   Note lsass.exe stands for the name of the process that you want to monitor with the Pageheap tool.
3. When you no longer need page heap monitoring, disable the monitoring. To do this, type the following at a command prompt, and then press ENTER:
   Gflags.exe /p /disable lsass.exe
   Note lsass.exe stands for the name of the process that you want to monitor with the Pageheap tool.
4. To list all programs that currently have Pageheap verification enabled, type the following at a command prompt, and then press ENTER:
   Gflags.exe /p

Unaligned Allocations

The Windows heap managers (all versions) have always guaranteed that the heap allocations have a start address that is 8-byte aligned (on 64-bit platforms the alignment is 16-bytes). The page heap manager makes the same guarantee. This is impossible, however, if you want to have the end-of-the-allocation exactly at the end of a page. The exact end-of-page allocation is needed so that an off-by-one-byte error will trigger a read or write into the non-accessible page and cause an immediate fault.

If the user-requested size for the block is not 8-byte aligned, then page heap cannot meet both the "start address 8-byte aligned" and the "end address page aligned" constraints. The solution is to meet the first constraint and make the start of the block 8-byte aligned. Then use a small fill pattern between the end of the block and the start of the non-accessible page. This fill pattern can be from 0 bytes through 7 bytes in length on 32-bit architectures. The fill pattern is checked upon free.

If immediate fault detection is needed for these allocations that otherwise will have a fill pattern at the end, make the page heap manager ignore the 8-byte alignment rule, and always align the end of the allocation at a page boundary by using the /unaligned and /full parameters. For more information, see the /unaligned parameter.

NOTE: Some programs make assumptions about 8-byte alignment and they stop working correctly with the /unaligned parameter. Microsoft Internet Explorer is one such program.

Uncommitted Pages for Full-Page Heap Allocations

The core full-page heap implementation commits two pages for any allocation smaller than one page. One page is used for the user allocation, and the other is made non-accessible at the end of the buffer.

End-of-buffer overruns can be detected by using a zone of reserved virtual space, instead of a non-accessible committed page. An access violation exception occurs when the process touches that reserved virtual space. This approach can reduce memory consumption by up to 50 percent. For more information, see the /decommit parameter.

Fault Injection

You can control page heap manager so that some allocations are deliberately failed. This is useful in simulating low memory conditions without actually using all system resources.

Specify a number from 1 through 10,000 to represent the probability that an allocation will fail. Using a probability of 10,000 ensures that 100 percent of allocations will fail. A probability of 2,000 specifies that approximately 20 percent of allocations will fail.

The page heap manager takes special care to avoid fault injection in both the first 5 seconds of the process' life, and the Windows NT loader code paths (for exampole, LoadLibrary, FreeLibrary). If 5 seconds is not sufficient to allow your process to complete startup, then you can specify a longer timeout at the beginning of the process. For more information, see the /fault parameter.

When you use the /fault parameter and the process being tested has a bug, an exception will be raised. Generally, the reason for this is that the allocate operation returned a NULL, and the application later tries to access the allocated memory. Because the allocate failed, however, the memory cannot be accessed, and therefore an access violation occurs.

The other reason that an exception is raised is that the application tries to deal with the allocation failure, but does not release some resources. This manifests as a memory leak and is more difficult to debug.

In order to help diagnose these failures, the page heap manager keeps a history of stack traces from the moment of fault injection. These traces can be displayed with the following debugger command:

!heap -p -f [NUMBER-OF-TRACES]

By default the extension will display only the last four traces.

Automatically Attaching a Debugger When the Application Starts

Some applications are difficult to start from a command prompt, or they are spawned from other processes. For these applications, specify that, whenever they are started, a debugger will automatically be attached to them. This is useful if page heap is enabled for that process and heap failures occur. For more information, see the /debug parameter.

Pageheap.exe is effective when used to verify any memory allocation process, including C++ style allocations new and delete, as long as the custom allocation/free functions eventually call into NT heap management interfaces (that is, RtlAllocateHeap, RtlFreeHeap). The following functions are guaranteed to do that:

• Functions like HeapAlloc, HeapFree, HeapReAlloc: These functions are exported by kernel32.dll and call directly into the NT heap interfaces. Functions like GlobalAlloc, GlobalFree, GlobalReAlloc: These functions are exported by kernel32.dll and call either directly or indirectly into the NT heap interfaces.
• Functions like LocalAlloc, LocalFree, LocalReAlloc: These functions are exported by kernel32.dll and call either directly or indirectly into the NT heap interfaces.
• Functions malloc, free, realloc, msize, expand: These functions are exported by msvcrt.dll and call either directly or indirectly into NT heap. This has not always been the case. The C run-time used to have a different heap implementation, but the current C run-time calls directly into NT heap.
• Operators new, delete, new[ ] , delete[ ] : These functions are exported by msvcrt.dll and call either directly or indirectly into NT heap.

Any other allocation/free set of functions probably is a custom scheme and is not guaranteed to call either directly or indirectly into NT heap. Only source code inspection or running under debugger can reveal the actual implementation.

Avoid using static linking. Some applications have been statically linked to old versions of the C runtime. These old versions do not call Windows NT heap APIs, and Pageheap.exe cannot be used to verify these allocations. Dynamic linking ensures that you get the latest C runtime library (msvcrt.dll).

Classes of Bugs Found by Pageheap.exe

Pageheap.exe detects most heap-related bugs; however, it is focused on heap corruption problems and not focused on leaks. Pageheap.exe has limited success with finding heap leaks, although it has functionality to target this.

One of the advantages of Pageheap.exe is that many errors are detected when they happen. For example, an off-by-one-byte error at the end of a dynamically allocated buffer might cause an instant access violation. There are few types of errors that cannot be detected when they occur. In those cases, the error report is delayed until the block is freed.

• Invalid heap pointer: All Win32 and Windows NT level heap interfaces take as a first parameter a pointer to the heap where the operation should happen. The page heap manager detects an invalid heap pointer at the moment the call is made.
• Invalid heap block pointer: After a block is allocated, it can be used as a parameter for several heap interfaces, notably the free() class of interfaces. The page heap manager immediately detects an invalid heap block pointer. See Debugging Page Heap Failures for a way to determine if the invalid address is a few bytes off, or is completely incorrect.
• Multithreaded unsynchronized access to the heap: Some applications call into a heap from multiple threads. This type of scenario requires setting a flag (by the user) which will trigger acquiring a heap lock. The page heap manager will detect this type of violation when two threads attempt to call simultaneously into the heap.
• Assumptions about reallocation of a block at the same address: A reallocation operation is not guaranteed to return the same address. This is especially true when the reallocation reduces the size of the block. Some applications assume that reallocation will return the same address. The page heap manager always allocates a new block during a reallocation and frees the old block. The free block is protected for read/write access, and therefore any access to it will raise an access violation.
• Double free: This bug, where the same heap blocks are freed several times, is common in some applications. This is detected immediately by the page heap manager because, on the second free, the block will not have the proper prefix header and cannot be found among the allocated blocks. See Debugging Page Heap Failures for ways to analyze the stack trace of the first free operation. This error can be a variant of the reallocation problem because, when the application frees what it thinks it is the address of the block, that block was already freed as part of the reallocation.
• Access of block after free: Freed memory blocks are kept for a short time by the page heap manager in a pool of protected memory. Any access to those blocks will raise an access violation. Based on the "locality" principle, most of the problems should be caught if the free protected pool is large enough. If the freed block is still in the protected pool, the bug is caught instantly. However, if the memory was reused, then there is less chance of finding the bug or identifying the code that caused it.
• Access after the end of allocated block: The page heap manager places an inaccessible page immediately following the allocated block. Any access past the end of the block will raise an access violation. Some applications expect allocations to be 8-byte aligned. This feature has been supported since Windows NT 3.5 heap managers. A request size that is not 8-byte aligned will still get an 8-byte aligned address, but this leaves a few bytes after the end of the block that are still accessible. If the application corrupts only those few bytes, then the error will be caught only by checking the block suffix pattern when the block is freed.
• Access before the start of allocated block: The page heap manager can be instructed through a settable flag to place the inaccessible page at the beginning of the block, rather than at the end. Any access before the start of the block will raise an access violation.

Debugging Page Heap Failures

For more information on Debugging Page Heap Failures, please look at Application Compatibility Tookit Reference available inside of the Application Compatibility Toolkit.

For the Syntax of Pageheap.exe and examples of using Pageheap.exe, please look at Application Compatibility Tookit Reference available inside of the Application Compatibility Toolkit.

For additional information please see the following Microsoft Knowledge Base article: