ASP.NET 4.8 WebAPI - streaming large file upload fails with HTTP 2

Question

Hello,

I have problems with streaming large file upload in ASP.NET 4.8 Web API project.

I'm using custom WebHostBufferPolicySelector to disable buffering of input stream (as described here: https://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/) and I'm sending file in HTTP POST request with multipart/form-data. I read data in controller using HttpContext.Current.Request.Files["key"].InputStream. I'm using it with FileStream to write uploaded data to file on server. Basically I use InputStream.CopyToAsync method with 80kB buffer size (this seems to be the default value) to copy data.

This seems to work for HTTP 1.1 but as soon I switch to HTTP 2, I start getting following exceptions:

System.Web.HttpException (0x80004005): An error occurred while communicating with the remote host. The error code is 0x800703E3. --->     System.Runtime.InteropServices.COMException (0x800703E3): The I/O operation has been aborted because of either a thread exit or an application request. (Exception from HRESULT: 0x800703E3)
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityBody(Byte[] buffer, Int32 size)
at System.Web.HttpRequest.GetEntireRawContent()
at System.Web.HttpRequest.GetMultipartContent()
at System.Web.HttpRequest.FillInFilesCollection()
at System.Web.HttpRequest.EnsureFiles()
at System.Web.HttpRequest.get_Files()
at Lic.Web.Controllers.FileUploadController.<UploadFileAsync>d__6.MoveNext() in C:\projects\emclient\license-manager\Lic.Web\Controllers\FileUploadController.cs:line 93

sometimes I get this exception after just few first megabytes of files are transfered.

Is this the right approach for large file upload? Basically I want to avoid loading whole file into memory, I'd like to use streams to process it with small buffer to avoid memory congestion problems.

Here is sample code:

    public class FileUploadChunkResBindingModel
    { 
        public string Status { get; set; }
        public string FileName { get; set; }
        public string Guid { get; set; }
    }

    [RoutePrefix("api/FileUpload")]
    public class FileUploadController : ApiController
    {
        private ILogger<FileUploadController > _logger;

        [HttpPost]
        [Route("UploadFile")]
        [Authorize(Roles = "WriteAdmin")]
        public async Task<FileUploadChunkResBindingModel> UploadFileAsync(CancellationToken ct)
        {
            var req = HttpContext.Current.Request;

            if (req.Files.Count != 1)
            {
                throw new HttpResponseException(new HttpResponseMessage
                {
                    StatusCode = HttpStatusCode.BadRequest,
                    Content = new StringContent(string.Format("Request must contain 1 file, but contains {0}", req.Files.Count)),
                });
            }

            var guid = req.Form["guid"];
            var file = req.Files[req.Files.Keys.Get(0)];
            var ret = await UploadFileAsync(guid, file, ct);

            return ret;
        }

        public async Task<FileUploadChunkResBindingModel> UploadFileAsync(string guid, HttpPostedFile file, CancellationToken ct = default)
        {
            var path = Path.Combine("~", "FileUpload", guid);
            var uploadDir = HostingEnvironment.MapPath(path);

            Directory.CreateDirectory(uploadDir);

            try
            {
                var filePath = Path.Combine(uploadDir, file.FileName);

                using (var fileReadStream = file.InputStream)
                using (var fileWriteStream = File.OpenWrite(filePath))
                {
                    var bufferSize = 81920; // 80kB should be default size based on documentation
                    await fileReadStream.CopyToAsync(fileWriteStream, bufferSize, ct);
                }

                return new FileUploadChunkResBindingModel
                {
                    Status = "Done",
                    Guid = guid,
                    FileName = file.FileName
                };
            }
            catch (Exception exc)
            {
                _logger.LogError(exc, "File upload failed with exception.");

                Directory.Delete(uploadDir, true);

                return new FileUploadChunkResBindingModel
                {
                    Status = "Cancelled"
                };
            }
        }
    }

This is code for WebHostBufferPolicySelector:

   public class NoBufferPolicySelector : WebHostBufferPolicySelector
    {
        public override bool UseBufferedInputStream(object hostContext)
        {
            if (hostContext is HttpContextBase context)
            {
                if (context.Request.CurrentExecutionFilePath.EndsWith("UploadFile"))
                {
                    // Do not use buffered input stream for file uploads - this reduces memory footprint
                    return false;
                }
            }

            return true;
        }
    }

And it is registered in Global.asax.cs

public class Global : HttpApplication
{
        void Application_Start(object sender, EventArgs e)
        {
            GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicySelector());
        }
}

(it's little bit simplified code example, I hope I didn't remove any important part for this issue).

Answer 1

Hi @David Klingenberg ,

1. > sometimes I get this exception

This usually happens when the website takes a while to respond or if the client has closed/disconnected from your site before finishing processing the request. 2.

> Is this the right approach for large file upload? You can upload large files using MultipartFormDataStreamProvider.You can click this link to view the code.

---

If the answer is helpful, please click "Accept Answer" and upvote it. Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Best Regards, YihuiSun