VS403406 error when you have many team projects using a single Hosted XML process

When you create a project in Microsoft Visual Studio Team Services (VSTS), you may receive an error message that resembles the following:

Some customers created dozens of team projects that were based on a single Hosted XML process. When they modify this Hosted XML process, all process metadata has to be copied to each of these team projects, which can take several hours. During this process, other users in the account experience decreased performance.

To limit this problem, Microsoft added a limit to VSTS of the number of team projects that a single Hosted XML process can have. You receive the VS403406 error message when your Hosted XML process reaches that limit.

To minimize the performance effect, we strongly recommend that you:

• Minimize the number of times that you upload process changes by batching them.
• Upload the process after business hours to reduce the number of affected users.

If you experience the VS403406 error, try the following methods to work around this issue:

• The recommended approach is to start using the Inheritance process model for new projects. You can create an Inherited process from Agile, Scrum, or CMMI to recreate your process manually. It's a one-time effort to create this process. The benefit is that you then can create as many team projects as you want without any implications, and you get a much better user experience when changing your process. In the future, Microsoft will let you migrate your existing projects to the Inheritance process model.
• Create a copy of the Hosted XML process. Then, you can create another set of team projects until you hit the limit. It will help the system spread the load, and so any process change will have less impact on other users in the account. 
• The least favorable solution is to request an increase of the limit. To submit your request, contact vstsprcssprt@microsoft.com. Only requests that include valid business justification will be reviewed. If the request is approved, the same recommendations as in the previous option apply: limit the number of times that you upload the process, and upload the process after business hours.

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

When you create an account on Visual Studio Team Services, you have accessed to the new Inheritance process model. This process model has a nice "what you see is what you get" (WYSIWYG) editing experience and includes the concept of "sharing." With sharing, when you modify the process, all the team projects that are based on the process automatically get these changes.

Microsoft Team Foundation Server (TFS, on-premise version) uses the "on-prem XML" process model. As the name suggests, this process model is based on a set of XML files. However, this process model doesn't have the concept of sharing. When the user creates a team project, the process metadata is copied into the team project. To change the process, the user must change the metadata in the team project. Modifications to the process template itself are applied to new projects only. That's also why it’s "Process Templates" in TFS and "Processes" in VSTS.  

The third process model is available only for those who have used the TFS Database Import Service. This process model is called the Hosted XML process model. This process model uses XML files to define the process, such as the "on-prem XML" process model. As opposed to the TFS version, Hosted XML supports the concept of "sharing," such as the Inheritance process model. The difference between Inheritance and Hosted XML is where the process metadata is stored. In the Inheritance process model, team projects read their process metadata directly from the process. However, team projects still store the process metadata locally in Hosted XML. To support the concept of sharing in Hosted XML, the process metadata is copied to each team project when an Hosted XML process is changed.