Share via

EasyAuth middleware on App Service: System.IO.IOException - How to disable it?

CAMPOLMI-EXT Oscar 0 Reputation points
2024-02-22T11:03:05.54+00:00

Question

We have a Flask application deployed on Linux Azure App Service, Plan P1v2. The authorization (EasyAuth) is not enabled.

We do POST requests with a payload => ~8kb and the reposes are a 500 instead an expected 401 or 400. Inspecting the log of our application it returns the proper code. Inspecting other logs we notice the EasyAuth logs, they report the following error:enter image description here So even if EasyAuth is disabled the EasyAuth middleware is forwarding our requests and for some reasons it is crashing returning 500.

How we could disable definitely EasyAuth middleware?

Details

Inspecting with Diagnose and solve problems tool the EasyAuth section report the following stack trace.

The problem seems that the Flask application answer too fast before the payload is completely streamed from the EasyAut middleware.

	
|  | HTTP proxy request encountered exception.
Type: System.Net.Http.HttpRequestException
Message: Error while copying content to a stream.
StackTrace: at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 117
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 92

InnerException
Type: System.IO.IOException
Message: Unable to write data to the transport connection: Connection reset by peer.
StackTrace: at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)<br> at System.IO.Pipelines.PipeReader.<CopyToAsync>g__Awaited|15_1(ValueTask writeTask)<br> at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)
at System.Net.Http.StreamToStreamCopy.g__DisposeSourceAsync|1_0(Task copyTask, Stream source)
at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask) | 11 | |  | HTTP proxy request encountered exception.
Type: System.Net.Http.HttpRequestException
Message: Error while copying content to a stream.
StackTrace: at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 117
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 92

InnerException
Type: System.IO.IOException
Message: Unable to write data to the transport connection: Broken pipe.
StackTrace: at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)<br> at System.IO.Pipelines.PipeReader.<CopyToAsync>g__Awaited|15_1(ValueTask writeTask)<br> at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)
at System.Net.Http.StreamToStreamCopy.g__DisposeSourceAsync|1_0(Task copyTask, Stream source)
at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask) | 2 | |  | HTTP proxy request encountered exception.
Type: System.Net.Http.HttpRequestException
Message: Error while copying content to a stream.
StackTrace: at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 117
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 92

InnerException
Type: System.IO.IOException
Message: Unable to write data to the transport connection: Connection reset by peer.
StackTrace: at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)<br> at System.IO.Pipelines.PipeReader.<CopyToAsync>g__Awaited|15_1(ValueTask writeTask)<br> at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)
at System.Net.Http.StreamToStreamCopy.g__DisposeSourceAsync|1_0(Task copyTask, Stream source)
at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask) | 1 | |  | HTTP proxy request encountered exception.
Type: System.Net.Http.HttpRequestException
Message: Error while copying content to a stream.
StackTrace: at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 110
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 74

InnerException
Type: System.IO.IOException
Message: Unable to write data to the transport connection: Connection reset by peer.
StackTrace: at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)<br> at System.IO.Pipelines.PipeReader.<CopyToAsync>g__Awaited|15_1(ValueTask writeTask)<br> at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)
at System.Net.Http.StreamToStreamCopy.g__DisposeSourceAsync|1_0(Task copyTask, Stream source)
at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask) | 1 | |  | HTTP proxy request encountered exception.
Type: System.Net.Http.HttpRequestException
Message: Error while copying content to a stream.
StackTrace: at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 117
at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 92

InnerException
Type: Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException
Message: Unexpected end of request content.
StackTrace: at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1ContentLengthMessageBody.ReadAsyncInternal(CancellationToken cancellationToken)
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder1.StateMachineBox1.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)<br> at System.Net.Http.StreamToStreamCopy.<CopyAsync>g__DisposeSourceAsync|1_0(Task copyTask, Stream source)<br> at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask) | 1 | | <i class='fa fa-exclamation-circle critical-color' aria-hidden='true'></i> | HTTP proxy request encountered exception.<br>Type: System.Net.Http.HttpRequestException<br>Message: Error while copying content to a stream.<br> StackTrace: at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)<br> at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)<br> at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)<br> at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 110<br> at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 74<br><br>InnerException<br>Type: System.IO.IOException<br>Message: Unable to write data to the transport connection: Connection reset by peer.<br> StackTrace: at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)
at System.IO.Pipelines.PipeReader.g__Awaited|15_1(ValueTask writeTask)
at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func4 writeAsync, CancellationToken cancellationToken)<br> at System.Net.Http.StreamToStreamCopy.<CopyAsync>g__DisposeSourceAsync|1_0(Task copyTask, Stream source)<br> at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask) | 1 | | <i class='fa fa-exclamation-circle critical-color' aria-hidden='true'></i> | HTTP proxy request encountered exception.<br>Type: System.Net.Http.HttpRequestException<br>Message: Error while copying content to a stream.<br> StackTrace: at System.Net.Http.HttpContent.<CopyToAsync>g__WaitAsync|56_0(ValueTask copyTask)<br> at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)<br> at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)<br> at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)<br> at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.ForwardRequestAsync(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 117<br> at Microsoft.Azure.AppService.Middleware.Forwarding.RequestForwarder.OnRequest(HttpContext context) in /__w/1/s/src/EasyAuth/Middleware.Forwarding/RequestForwarder.cs:line 92<br><br>InnerException<br>Type: System.IO.IOException<br>Message: Unable to write data to the transport connection: Connection reset by peer.<br> StackTrace: at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source, Boolean async)
at System.IO.Pipelines.PipeReader.g__Awaited|15_1(ValueTask writeTask)
at System.IO.Pipelines.PipeReader.CopyToAsyncCore[TStream](TStream destination, Func`4 writeAsync, CancellationToken cancellationToken)
at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask) | 1 |
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
8,939 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. brtrach-MSFT 17,731 Reputation points Microsoft Employee Moderator
    2024-02-22T21:39:47.1+00:00

    @CAMPOLMI-EXT Oscar I understand you are receiving 500 errors and are concerned about resolving them.

    The last time I saw this error was in December and the customer was indeed using EasyAuth. The solution was to turn off EasyAuth. The difference though is that the customer was having issues with files larger than 1GB, which is different than your experience so this might not be the solution.

    The other time I saw this error the customer needed to expand their timeout to be beyond 30 seconds in their gunicorn.conf.py file. I wonder if this might be part of your issue since you said the response is coming in too quickly.

    Can you please check any timeout values for your application that you might think need adjusting? Let us know the outcome and if you need further assistance, we can help you.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.