AssemblyBinder role instance is throwing System.IO.IOException exception

This article provides information about troubleshoot issues in which AssemblyBinder role instance of Compressor application is stuck between Busy and Restarting state and throwing System.IO.IOException exception stating: There is not enough space on the disk.

Original product version:   API Management Service
Original KB number:   4464907

Symptoms

AssemblyBinder role instance of Compressor application is throwing the below unhandled exception in the Azure portal blade and stuck between Busy and Restarting state.

Unhandled Exception: There is not enough space on the disk. at System.IO.__Error.WinIOError  (Int32 errorCode, String maybeFullPath) at System.IO.FileStream.WriteCore(Byte[] buffer, Int32 offset, Int32 count) at Ionic.Zip.ZipEntry.ExtractAndCrc(Stream archiveStream, Stream targetOutput, Int16 compressionMethod, Int64 compressedFileDataSize, Int64 uncompressedSize) at Ionic.Zip.ZipEntry.ExtractToStream(Stream archiveStream, Stream output, EncryptionAlgorithm encryptionAlgorithm, Int32 expectedCrc32) at Ionic.Zip.ZipEntry.InternalExtractToBaseDir(String baseDir, String password, ZipContainer zipContainer, ZipEntrySource zipEntrySource, String fileName) at Ionic.Zip.ZipFile._InternalExtractAll(String path, Boolean overrideExtractExistingProperty) at AssemblyBinder.WorkerRole.OnStart() in D:\compressor\AssemblyBinder\WorkerRole.cs:line 56 at Microsoft.WindowsAzure.ServiceRuntime.RoleEnvironment.InitializeRoleInternal(RoleType roleTypeEnum) at Microsoft.WindowsAzure.ServiceRuntime.Implementation.Loader.RoleRuntimeBridge. <InitializeRole> b__0() at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart()'[2018-08-12T14:47:25Z] Last exit time: [2018/08/12, 14:47:25.965].

Troubleshoot steps

From the error call stack, it looks like this WorkerRole is performing some unzip or extract operation at OnStart() method but it is failing due to insufficient storage. The next obvious question that will come to your mind is where exactly WaWorkerHost.exe process is extracting the file.

In order to find out the answer, use Process Monitor tool to take a ProcMon trace and see what you can find.

WaWorkerHost.exe process is writing some file in its default temporary directory, which has a maximum size of 100 MB, which may become full at some point. Upon navigating to the RoleTemp directory, you might find that disk space quota is indeed getting exhausted for that directory.

You might have figured out the cause of the problem, but what should you do when running out of space in RoleTemp location? Here is the answer.

Therefore, you should configure a local storage resource, and point the TEMP and TMP directories to the path of the local storage resource like below:

The local resource declaration must have been added to the service definition file for AssemblyBinder role.

<?xml version="1.0" encoding="UTF-8"?>
<LocalResources>
   <LocalStorage name="FileStorage" cleanOnRoleRecycle="true" sizeInMB="200" />
</LocalResources>

This modification should be performed within the RoleEntryPoint.OnStart method.

localResource = RoleEnvironment.GetLocalResource("FileStorage");
Environment.SetEnvironmentVariable("TMP", localResource.RootPath);
Environment.SetEnvironmentVariable("TEMP", localResource.RootPath);

Contact us for help

If you have questions or need help, create a support request, or ask Azure community support. You can also submit product feedback to Azure feedback community.