PRB: MSXML Performance Bottleneck in ShareMutex Under Stress

When you use the MSXML parser in a high-stress environment such as Microsoft Internet Information Services (IIS), a bottleneck may occur while the MSXML parser performs garbage collection, such as around the msxml3!ShareMutex__SleepWaitingForLock internal MSXML function. This may cause unusually long run times on XML requests, and the application or service can appear to stop responding (hang). However, if the stress is removed from the application or service, the requests are eventually run and the process stabilizes.
For additional information about the garbage collection mechanism with the MSXML parser, click the article number below to view the article in the Microsoft Knowledge Base:

This behavior is a result of contention on locks in the DOMDocument object class factory. The locks must be acquired whenever a DOMDocument object is created or destroyed. Each time the internal reference count of active objects for MSXML drops to zero and a DOMDocument object is unloaded, extensive locking and processor activity occurs. Because Active Server Pages (ASP) pages run in single threaded apartments (STAs), each Web page that uses a local DOMDocument object instantiates its own copy, meaning that every page that is using the local DOMDocument object incurs the overhead of object creation and deletion.

When numerous threads simultaneously create and destroy Document Object Model (DOM) objects, the locks in the class factory may experience significant contention. The threads enter a wait state, with each thread waiting for its turn at the locks. Although this is not a true deadlock, a thread that executes a client request can take an unexpectedly long time to complete.

WARNING: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system. Microsoft cannot guarantee that you can solve problems that result from using Registry Editor incorrectly. Use Registry Editor at your own risk.

To resolve this problem, upgrade to the latest version of the MSXML parser.

With MSXML 3.0 Service Pack 2 (SP2) and later, changes to the implementation of DOMDocument are included with other performance improvements. This upgrade may reduce the bottleneck that is caused by this issue.

NOTE: MSXML 3.0 SP2 only installs in Replace mode. For additional information, click the article number below to view the article in the Microsoft Knowledge Base: With MSXML 4.0 Service Pack 1 (SP1) and later, there is a new feature is implemented in the parser to turn off FullGC automatically if a certain number of threads are waiting for garbage collection.

NOTE: MSXML 4.0 only installs and runs in side-by-side mode. MSXML 4.0 is more compliant with World Wide Web Consortium (W3C) XML specifications. An application that relies on behavior that does not comply with the W3C specifications may need additional compatibility tests.

In addition to upgrading to the latest version of MSXML parser, there are two recommended workarounds for this problem, and each workaround is appropriate depending upon the situation in which it is used. These workarounds are generalized, and may not work perfectly in every situation. Unforeseen results may occur, and you must consider each situation carefully before you implement either of these workarounds.

For each workaround, you must use the FreeThreadedDOMDocument object instead of the DOMDocument object, because the DOM must be loaded into a multi-threaded apartment (MTA) to keep it from unloading after each ASP page is complete. The DOMDocument is not supported in an MTA, but the FreeThreadedDOMDocument is.

Using Component Services or MTS to Load the DOM Out of Process

This technique loads the XML DOM object in its own out-of-process server package and places it in an MTA for the hosting process. This approach affects the creation of all DOM objects system-wide.

Consider this technique in cases in which the following are true:

• The application has already been written, so that changes to avoid the object creation require a complete rewrite.
• The Web server sustains a moderate to high number (15 or more) of ASP pages that each use XML simultaneously.
• XML is integrated into an existing Web application that precludes the use of a global XML object.

Do not use this workaround if less than 15 threads access the DOM at the same time. The overhead of marshalling calls to the out-of-process DOM actually results in performance degradation with less than 15 threads.

To use Component Services or MTS to load the DOM out of process, follow these steps:

1. Create a Component Services or MTS package and import the MSXML FreeThreadedDOMDocument into the new package.
2. Change the threading model for the FreeThreadedDOMDocument in the registry to Free. This registry key is found in the following location:
   HKEY_CLASSES_ROOT\CLSID\<Class ID of the version of the FreeThreadedDOMDocument being used>\InProcServer32\ThreadingModel
   The default value for this key is Both. This is required for the object to run in the Component Services or MTS package.
3. Make sure that the package is marked for server activation (out-of-process) instead of library activation (in-process). This is the default setting on Microsoft Windows 2000, but must be manually configured on Microsoft Windows NT 4.0.

Using an Application Scope DOM to Prevent Unnecessary Garbage Collection

Consider this technique in cases in which forcing all XML DOM objects to load in the MTA of a separate process is not a valid option. This may apply in situations in which you have Web servers that have numerous Web sites and you want to control the lifetime of the XML DOM on a case-by-case basis, or in which only a few long-lived XML documents are created.

This workaround is not system-wide and must be applied independently to each Web site that experiences the problem.

Although this workaround does not incur the marshalling overhead of the first workaround, it has the disadvantage that if the DOM crashes for any reason, the Web site stops responding (hangs). If the DOM crashes in the first workaround, its host process dies and Component Services or MTS instantiates a new host process when the next request comes through. Also, using a global DOM causes all calls to be serialized, which may affect performance.

To use an application-scope DOM to prevent unnecessary garbage collection, follow these steps:

1. Create a global instance of the FreeThreadedDOMDocument in Application_OnStart for the Web site that has the problem. This global "dummy" object prevents the reference count on the DOM from reaching zero, so that the DOM does not unload until the application shuts down.
2. Change the threading model for the FreeThreadedDOMDocument in the registry to Free. This registry key is found in the following location:
   HKEY_CLASSES_ROOT\CLSID\Class ID of the version of the FreeThreadedDOMDocument being used\InProcServer32\ThreadingModel
   The default value for this key is Both. This is required for the object to run in application scope.

NOTE: The Class IDs to be substituted in these registry keys are as follows, depending on the version of MSXML that is being used:

• 3.0 - {F6D90F12-9C73-11D3-B32E-00C04F990BB4}
• 2.6 - {F5078F1C-C551-11D3-89B9-0000F81FE221}

NOTE: For this workaround, the main purpose of creating a global FreeThreadedDOMDocument at application scope is to prevent the global count of objects tracked by MSXML parser from ever reaching zero (that is, to prevent the expensive full garbage collection from occurring). ASP pages can still use the regular DOMDocument object without modification.

If a debugger is attached to the process and the threads are examined, a number of threads have call stacks that look like one or both of the following sample call stacks:

To download MSXML 4.0 SP2, visit the following Microsoft Web site: For additional information, click the article numbers below to view the articles in the Microsoft Knowledge Base: