Android HttpClient get SocketException (Connection reset by peer) after 2-3 minutes when sending larger files

Question

Android HttpClient get SocketException (Connection reset by peer) after 2-3 minutes when sending larger files

Andreas Brosten 116

I'm uploading files to azure blob storage with a normal HttpClient from my Xamarin forms project.
I set a timeout value of 10 minutes for the HttpClient.
Sending files (up to around 100MB) works perfect on iOS and UWP. On Android however, smaller files works great, but when trying to send 50-100MB large files, I get SocketException at the end of each send session. The exception is reproducable all the time, but depending on filesize and bandwidth, it comes after different amount of time. Usually 2-4 minutes.
So, the failing request is a call to httpClient.SendAsync(myRequest)
It's tested on three different Android devices and all behaves the same (different brands).
HttpClient implementation = Android
SSL/TLS implementation = Native TLS 1.2+

I guess something gets disposed too early, but seems out of my control. I am not doing any other http calls in the meantime.
Any idéas?

Stack track:
at System.Net.Http.HttpContent.CopyToAsyncCore (System.Threading.Tasks.ValueTask copyTask) [0x00089] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/HttpContent.cs:375
at System.Net.Http.HttpConnection.SendRequestContentAsync (System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpConnection+HttpContentWriteStream stream, System.Threading.CancellationToken cancellationToken) [0x00058] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:796
at System.Net.Http.HttpConnection.SendAsyncCore (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x012e6] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs:748
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync (System.Net.Http.HttpConnection connection, System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) [0x000e6] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:330
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync (System.Net.Http.HttpRequestMessage request, System.Boolean doRequestAuth, System.Threading.CancellationToken cancellationToken) [0x00101] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:296
at System.Net.Http.RedirectHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x00070] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:32
at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x000d7] in <6735d06e4e7e46e0b8693d25d7699ab4>:0
at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancellationToken) [0x000ef] in <6735d06e4e7e46e0b8693d25d7699ab4>:0
at System.Net.Http.HttpClient.FinishSendAsyncBuffered (System.Threading.Tasks.Task1[TResult] sendTask, System.Net.Http.HttpRequestMessage request, System.Threading.CancellationTokenSource cts, System.Boolean disposeCts) [0x0017e] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Net.Http/src/System/Net/Http/HttpClient.cs:506 at MyApp.Client.ApiClients.ApiClientBase.SendAsync (System.Net.Http.HttpRequestMessage request, System.Threading.CancellationToken cancelToken) [0x00037] in C:\.....\MyApp.Client\ApiClients\ApiClientBase.cs:25 at MyApp.Client.Operations.FileOperation.SaveCompleteFile (MyApp.Client.InternalModel.Inspection localInspection, MyApp.Client.InternalModel.Inspection originalInspection, MyApp.Client.DbModels.FileDataEntity file, MyApp.Client.Common.FileWorkerCommand command) [0x00111] in C:\...\Source\MyApp.Client\Operations\FileOperation.cs:165 at MyApp.Client.Operations.FileOperation.UploadFilesToServer (MyApp.Client.InternalModel.Inspection localInspection, MyApp.Client.InternalModel.Inspection originalInspection, System.Collections.Generic.List1[T] commands) [0x00130] in C:....\Source\MyApp.Client\Operations\FileOperation.cs:92

Anonymous

2021-11-10T06:30:13.733+00:00

Do you used background service or thread to send larger files? Based on your exception, I find these information from this thread: The connection to the machine (remote host/server/PC that the service runs at) was made but since the service was not available on that machine, the machine didn't know what to do with the request. Please check your background services or thread was killed by Android OS. You can try to upload files in foreground service
Andreas Brosten 116 Reputation points

2021-11-10T07:37:25.407+00:00

It's done in the main application (not a service), and while the app is visually running on screen. However, it's done in a separate thread (not the MainThread).
But all files taking less then two minutes to upload works fine. Also small files taking long time on week connections works well. But large files ( above 50 MB) taking over ~2 minutes fails. Seems like some "end processing" of the send call fails, due to some data is eaten by the GC or so...
Anonymous

2021-11-11T09:58:54.367+00:00

Please make a test in the main thread when you upload the file, if it worked. To exclude the situation where the separate thread is killed
Andreas Brosten 116 Reputation points

2021-11-12T10:09:11.98+00:00

I just tried with the main thread, but with the same result (Connection reset). :(
Anonymous

2021-11-15T08:01:17.553+00:00

Could you share a mini demo to here？

Andreas Brosten 116

@Anonymous sorry about the late answer, took some time to get things in order.

When preparing the mini demo, we found something very interesting!
The problem only occurs when the http-client is fetched from the DI container.
The DI pattern is taken from here
https://montemagno.com/add-asp-net-cores-dependency-injection-into-xamarin-apps-with-hostbuilder/

When doing var client = new HttpClient() ..... the upload takes significatly shorter time and without any issues.
This forum doesn't seem to accept zip uploads, but I could paste some code snippets of interest...

// Startup.cs  
  
      public static void ConfigureServices(HostBuilderContext ctx, IServiceCollection services)  
      {  
         services.AddHttpClient();  
      }  
  
// Using the httpclient  
            if (useDI)  
            {  
               using (var httpClient = Startup.ServiceProvider.GetService<HttpClient>())  
               {  
                  httpClient.Timeout = TimeSpan.FromMinutes(10);  
                  response = await httpClient.SendAsync(req);  
               }  
            }  
            else  
            {  
               using (var httpClient = new HttpClient())  
               {  
                  httpClient.Timeout = TimeSpan.FromMinutes(10);  
                  response = await httpClient.SendAsync(req);  
               }  
            }

Andreas Brosten 116 Reputation points

2021-11-22T14:21:02.08+00:00

So,
For Android
D1 fails (connection reset) and takes significatly longer time
D2 runs like a charm

For UWP
Both D1 and D2 takes equaly long time and works without any conneciton reset

Accepted answer

0 additional answers

Your answer

Anonymous

2021-11-10T06:30:13.733+00:00

Do you used background service or thread to send larger files? Based on your exception, I find these information from this thread: The connection to the machine (remote host/server/PC that the service runs at) was made but since the service was not available on that machine, the machine didn't know what to do with the request. Please check your background services or thread was killed by Android OS. You can try to upload files in foreground service
Andreas Brosten 116 Reputation points

2021-11-10T07:37:25.407+00:00

It's done in the main application (not a service), and while the app is visually running on screen. However, it's done in a separate thread (not the MainThread).
But all files taking less then two minutes to upload works fine. Also small files taking long time on week connections works well. But large files ( above 50 MB) taking over ~2 minutes fails. Seems like some "end processing" of the send call fails, due to some data is eaten by the GC or so...
Anonymous

2021-11-11T09:58:54.367+00:00

Please make a test in the main thread when you upload the file, if it worked. To exclude the situation where the separate thread is killed
Andreas Brosten 116 Reputation points

2021-11-12T10:09:11.98+00:00

I just tried with the main thread, but with the same result (Connection reset). :(
Anonymous

2021-11-15T08:01:17.553+00:00

Could you share a mini demo to here？
Andreas Brosten 116 Reputation points

2021-11-22T14:20:49.233+00:00

@Anonymous sorry about the late answer, took some time to get things in order.

When preparing the mini demo, we found something very interesting!
The problem only occurs when the http-client is fetched from the DI container.
The DI pattern is taken from here
https://montemagno.com/add-asp-net-cores-dependency-injection-into-xamarin-apps-with-hostbuilder/

When doing var client = new HttpClient() ..... the upload takes significatly shorter time and without any issues.
This forum doesn't seem to accept zip uploads, but I could paste some code snippets of interest...

// Startup.cs public static void ConfigureServices(HostBuilderContext ctx, IServiceCollection services) { services.AddHttpClient(); } // Using the httpclient if (useDI) { using (var httpClient = Startup.ServiceProvider.GetService<HttpClient>()) { httpClient.Timeout = TimeSpan.FromMinutes(10); response = await httpClient.SendAsync(req); } } else { using (var httpClient = new HttpClient()) { httpClient.Timeout = TimeSpan.FromMinutes(10); response = await httpClient.SendAsync(req); } }
Andreas Brosten 116 Reputation points

2021-11-22T14:21:02.08+00:00

So,
For Android
D1 fails (connection reset) and takes significatly longer time
D2 runs like a charm

For UWP
Both D1 and D2 takes equaly long time and works without any conneciton reset

Answer 1

I think I found and solved the problem!
I use the ServiceHost to handle my HttpClient cration, like this

  services.AddHttpClient<IMyApiClient, MyApiClient>()
  .ConfigureHttpClient(client =>
            {
               client.Timeout = TimeSpan.FromMinutes(10);
            })
  // This line is what fixed the issue!
  .ConfigurePrimaryHttpMessageHandler(() => Startup.ServiceProvider.GetService<IMyPlatformSpecificService>().GetMessageHandler());

And the implementation on android is just:

  public HttpMessageHandler GetMessageHandler()
  {
     return new Xamarin.Android.Net.AndroidClientHandler();
  }

So, I guess, without this line, the message handler on android will be incorrectly set! (I guess it will use the System.Net.Http version).

Share via

Android HttpClient get SocketException (Connection reset by peer) after 2-3 minutes when sending larger files

0 additional answers

Your answer