Notice: This website is an unofficial Microsoft Knowledge Base (hereinafter KB) archive and is intended to provide a reliable access to deleted content from Microsoft KB. All KB articles are owned by Microsoft Corporation. Read full disclaimer for more details.

"ASN1 out of memory" error when the CryptMsgUpdate function decodes a message that is larger than 5 MB in Windows 7 or in Windows Server 2008 R2


View products that this article applies to.

Symptoms

Consider the following scenario:
  • You have a computer that is running Windows 7 or Windows Server 2008 R2.
  • You use the EnvelopedCms.Decode method in a Microsoft .NET Framework application to decode a message that is larger than 5 megabytes (MB).
In this scenario, the application cannot decode the message. Additionally, you receive an error message that resembles the following:
Unhandled Exception: System.Security.Cryptography.CryptographicException: ASN1 out of memory.
at System.Security.Cryptography.Pkcs.EnvelopedCms.OpenToDecode(Byte[] encodedMessage)
at System.Security.Cryptography.Pkcs.EnvelopedCms.Decode(Byte[] encodedMessage)

↑ Back to the top


Cause

This issue occurs because a buffer size is incremented incorrectly in the CryptMsgUpdate function when the EnvelopedCms.Decode method adds decoded content to a buffer during the decoding process. Therefore, integer overflow occurs for a large file.

↑ Back to the top


Resolution

Hotfix information

A supported hotfix is available from Microsoft. However, this hotfix is intended to correct only the problem that is described in this article. Apply this hotfix only to systems that are experiencing the problem described in this article. This hotfix might receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next software update that contains this hotfix.

If the hotfix is available for download, there is a "Hotfix download available" section at the top of this Knowledge Base article. If this section does not appear, contact Microsoft Customer Service and Support to obtain the hotfix.

Note If additional issues occur or if any troubleshooting is required, you might have to create a separate service request. The usual support costs will apply to additional support questions and issues that do not qualify for this specific hotfix. For a complete list of Microsoft Customer Service and Support telephone numbers or to create a separate service request, visit the following Microsoft Web site: Note The "Hotfix download available" form displays the languages for which the hotfix is available. If you do not see your language, it is because a hotfix is not available for that language.

Prerequisites

To apply this hotfix, you must be running one of the following operating systems:
  • Windows 7
  • Windows 7 Service Pack 1 (SP1)
  • Windows Server 2008 R2
  • Windows Server 2008 R2 Service Pack 1 (SP1)

Registry information

To use the hotfix in this package, you do not have to make any changes to the registry.

Restart requirement

You must restart the computer after you apply this hotfix.

Hotfix replacement information

This hotfix does not replace a previously released hotfix.

File information

The global version of this hotfix installs files that have the attributes that are listed in the following tables. The dates and the times for these files are listed in Coordinated Universal Time (UTC). The dates and the times for these files on your local computer are displayed in your local time together with your current daylight saving time (DST) bias. Additionally, the dates and the times may change when you perform certain operations on the files.
Windows 7 and Windows Server 2008 R2 file information notes
Important Windows 7 hotfixes and Windows Server 2008 R2 hotfixes are included in the same packages. However, hotfixes on the Hotfix Request page are listed under both operating systems. To request the hotfix package that applies to one or both operating systems, select the hotfix that is listed under "Windows 7/Windows Server 2008 R2" on the page. Always refer to the "Applies To" section in articles to determine the actual operating system that each hotfix applies to.
  • The files that apply to a specific product, milestone (RTM, SPn), and service branch (LDR, GDR) can be identified by examining the file version numbers as shown in the following table:
    VersionProductMilestoneService branch
    6.1.760
    0.20xxx
    Windows 7 and Windows Server 2008 R2RTMLDR
    6.1.760
    1.21xxx
    Windows 7 and Windows Server 2008 R2SP1LDR
  • The MANIFEST files (.manifest) and the MUM files (.mum) that are installed for each environment are listed separately in the "Additional file information for Windows Server 2008 R2 and for Windows 7" section. MUM and MANIFEST files, and the associated security catalog (.cat) files, are extremely important to maintain the state of the updated components. The security catalog files, for which the attributes are not listed, are signed with a Microsoft digital signature.
For all supported x86-based versions of Windows 7
File nameFile versionFile sizeDateTimePlatform
Msasn1.dll6.1.7600.2086134,81618-Dec-201005:28x86
Msasn1.dll6.1.7601.2162434,30418-Dec-201007:57x86
For all supported x64-based versions of Windows 7 and of Windows Server 2008 R2
File nameFile versionFile sizeDateTimePlatform
Msasn1.dll6.1.7600.2086146,59218-Dec-201006:08x64
Msasn1.dll6.1.7601.2162446,59218-Dec-201008:51x64
Msasn1.dll6.1.7600.2086134,81618-Dec-201005:28x86
Msasn1.dll6.1.7601.2162434,30418-Dec-201007:57x86
For all supported IA-64-based versions of Windows Server 2008 R2
File nameFile versionFile sizeDateTimePlatform
Msasn1.dll6.1.7600.20861106,49618-Dec-201005:17IA-64
Msasn1.dll6.1.7601.21624106,49618-Dec-201009:35IA-64
Msasn1.dll6.1.7600.2086134,81618-Dec-201005:28x86
Msasn1.dll6.1.7601.2162434,30418-Dec-201007:57x86

↑ Back to the top


Status

Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.

↑ Back to the top


More Information

For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:

824684 Description of the standard terminology that is used to describe Microsoft software updates

For more information about the CryptMsgUpdate function, visit the following Microsoft Developer Network (MSDN) website:
For more information about the EnvelopedCms.Decode method, visit the following MSDN website:

Additional file information

Additional file information for Windows 7 and for Windows Server 2008 R2

Additional files for all supported x86-based versions of Windows 7
File nameUpdate.mum
File versionNot applicable
File size2,773
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameX86_2c068eab9a6df4c6041136270272d599_31bf3856ad364e35_6.1.7600.20861_none_cc9efa6e4750ded6.manifest
File versionNot applicable
File size694
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameX86_c104271f6ef210218498515d829eaa9a_31bf3856ad364e35_6.1.7601.21624_none_38cc5a63c230cc86.manifest
File versionNot applicable
File size694
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7600.20861_none_c7cbaf011fc5efc4.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)06:02
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7601.21624_none_c9e04c951cc937f1.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)08:36
PlatformNot applicable
Additional files for all supported x64-based versions of Windows 7 and of Windows Server 2008 R2
File nameAmd64_2c068eab9a6df4c6041136270272d599_31bf3856ad364e35_6.1.7600.20861_none_28bd95f1ffae500c.manifest
File versionNot applicable
File size696
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_4d2b7eebd9b882574669f4089bdf1f32_31bf3856ad364e35_6.1.7600.20861_none_227d42950d3bf1f0.manifest
File versionNot applicable
File size1,036
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_56d44b0d6c65f56a1048af741db8484f_31bf3856ad364e35_6.1.7600.20861_none_82219e64c125677d.manifest
File versionNot applicable
File size698
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_afeadb967959b177102d85264cf80950_31bf3856ad364e35_6.1.7601.21624_none_833ce1b22dea6dfd.manifest
File versionNot applicable
File size698
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_c104271f6ef210218498515d829eaa9a_31bf3856ad364e35_6.1.7601.21624_none_94eaf5e77a8e3dbc.manifest
File versionNot applicable
File size696
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_c4c7dab0a2493fb080bae437447408a1_31bf3856ad364e35_6.1.7601.21624_none_5b635005064fc7ef.manifest
File versionNot applicable
File size1,036
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameAmd64_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7600.20861_none_23ea4a84d82360fa.manifest
File versionNot applicable
File size2,411
Date (UTC)18-Dec-2010
Time (UTC)07:00
PlatformNot applicable
File nameAmd64_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7601.21624_none_25fee818d526a927.manifest
File versionNot applicable
File size2,411
Date (UTC)18-Dec-2010
Time (UTC)09:30
PlatformNot applicable
File nameUpdate.mum
File versionNot applicable
File size4,059
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7600.20861_none_c7cbaf011fc5efc4.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)06:02
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7601.21624_none_c9e04c951cc937f1.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)08:36
PlatformNot applicable
Additional files for all supported IA-64-based versions of Windows Server 2008 R2
File nameIa64_2c068eab9a6df4c6041136270272d599_31bf3856ad364e35_6.1.7600.20861_none_cca09e64474ee7d2.manifest
File versionNot applicable
File size695
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameIa64_312f83e28a17dca0435713b9f34bb380_31bf3856ad364e35_6.1.7600.20861_none_578a87e4e5e232fb.manifest
File versionNot applicable
File size696
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameIa64_c104271f6ef210218498515d829eaa9a_31bf3856ad364e35_6.1.7601.21624_none_38cdfe59c22ed582.manifest
File versionNot applicable
File size695
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameIa64_e4b9f84b918b15db4246152f640aca26_31bf3856ad364e35_6.1.7601.21624_none_97b9f3ccdadc562a.manifest
File versionNot applicable
File size696
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameIa64_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7600.20861_none_c7cd52f71fc3f8c0.manifest
File versionNot applicable
File size2,410
Date (UTC)18-Dec-2010
Time (UTC)06:55
PlatformNot applicable
File nameIa64_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7601.21624_none_c9e1f08b1cc740ed.manifest
File versionNot applicable
File size2,410
Date (UTC)18-Dec-2010
Time (UTC)10:01
PlatformNot applicable
File nameUpdate.mum
File versionNot applicable
File size2,169
Date (UTC)21-Dec-2010
Time (UTC)02:39
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7600.20861_none_c7cbaf011fc5efc4.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)06:02
PlatformNot applicable
File nameX86_microsoft-windows-msasn1_31bf3856ad364e35_6.1.7601.21624_none_c9e04c951cc937f1.manifest
File versionNot applicable
File size2,409
Date (UTC)18-Dec-2010
Time (UTC)08:36
PlatformNot applicable

↑ Back to the top


Keywords: kb, kbautohotfix, kbqfe, kbhotfixserver, kbfix, kbsurveynew, kbexpertiseadvanced

↑ Back to the top

Article Info
Article ID : 2480994
Revision : 1
Created on : 1/7/2017
Published on : 2/10/2011
Exists online : False
Views : 424