SMTP traffic between an Edge Transport server and an internal Hub Transport server is blocked if the Hub server is published by using ISA Server 2006 and if SMTP filtering is enabled

Consider the following scenario:

• You use Microsoft Internet Security and Acceleration (ISA) Server 2006 to publish an internal Hub Transport server to a Microsoft Exchange 2007 Edge Transport server.
• In ISA Server 2006, you enable Simple Mail Transfer Protocol (SMTP) filtering.
• When the Edge Transport server tries to send e-mail messages through ISA Server 2006 to the internal Hub Transport server, the SMTP traffic may be blocked.
• You configure ISA Server 2006 by using the method that is described on the following Microsoft TechNet Web site:
  How to Add SMTP Verb Commands to ISA Server 2006
  http://technet.microsoft.com/en-us/library/bb851508.aspx

In this scenario, the SMTP traffic between the Edge Transport server and the internal Hub Transport server may still be blocked even though you have added the X-AnonymousTLS verb and the X-EXPs verb to the SMTP filter settings in ISA Server.

This problem occurs when Exchange 2007 uses the proprietary verb X-AnonymousTLS to switch to Transport Layer Security (TLS) encryption. The SMTP filter in ISA Server 2006 is not aware of the usage of this verb. Therefore, the SMTP filter inspects the traffic, even though it is encrypted. Intermittantly, the SMTP filter detects malformed traffic and ends the session.

To resolve this problem, apply the hotfix that is mentioned in the following Microsoft Knowledge Base article:

Note You must still manually add the X-AnonymousTLS verb and the X-EXPs verb to the SMTP filter settings in ISA Server. For more information, visit the following Microsoft TechNet Web site:

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

After you apply this hotfix, the SMTP filter in ISA Server 2006 uses passthrough mode for the X-AnonymousTLS verb, and the filter does not inspect traffic. This is identical to how the TLS verb and the STARTTLS verb are treated.

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base: