How to improve (very poor) file read performance with C# Core on an Azure Kubernetes PVC

Chris Ahern 1 Reputation point
2021-10-29T00:22:20.463+00:00

After having done a number of tests, I've narrowed the performance problem down to a combination of C# and PVC.

Problem: Read a large(ish) file (140MB) using C# Core from a PVC. Here's the (simple) test program:

using System;
using System.IO;

namespace cmaApp
{
    class Program
    {
            private static void readStream(String filePath) {
                Console.WriteLine($"Reading {filePath}");
                using (var sr = new StreamReader(new BufferedStream(new FileStream(filePath, FileMode.Open))))
                {
                        int len = sr.ReadToEnd().Length;
                        Console.WriteLine($"dat len={len}");
                }
            }

            private static void copyFile(String filePath) {
                    File.Copy(filePath, "./t2.dat");
            }

        static void Main(string[] args)
        {
            Console.WriteLine("Start!");
            String filePath = "/uploads/3864.dat";    // PVC file
            //String filePath = "./3864.dat";         // Local file

            readStream(filePath);
            //copyFile(filePath);

            Console.WriteLine("Done!");
        }
    }
}

Here's what I did, on the command line in the pod:

Test 1. Read the file with C# directly from the PVC (takes 2 mins!!):

/cmaApp# time dotnet run
Start!
Reading /uploads/3864.dat
dat len=153508096
Done!

real    2m2.172s
user    0m3.437s
sys     0m3.357s

Test 2. Copy the file to a local folder in the pod and read with C# (take 3 seconds):

cmaApp# time dotnet run
Start!
Reading ./3864.dat
dat len=153508096
Done!

real    0m2.765s
user    0m2.910s
sys     0m0.762s

Test 3. Read the file with cat directly from the PVC (takes 4 secs):

cmaApp# time cat /uploads/3864.dat |wc
 144187 5543913 153508096

real    0m4.615s
user    0m0.755s
sys     0m0.293s

Test 4. Copy the file to a local folder using cp (takes 4 secs):

cmaApp# time cp /uploads/3864.dat .

real    0m4.358s
user    0m0.000s
sys     0m0.213s

Test 5. Copy the file to a local folder using C# File.Copy (takes 13 secs):

cmaApp# time dotnet run
Start!
Done!

real    0m13.042s
user    0m2.687s
sys     0m0.913s

Seeing that the linux level cp copies the file fast, it's not a throttling / network issue.

Why is C# Core so slow? What can be done to improve it?

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,156 questions
Azure Kubernetes Service (AKS)
Azure Kubernetes Service (AKS)
An Azure service that provides serverless Kubernetes, an integrated continuous integration and continuous delivery experience, and enterprise-grade security and governance.
1,848 questions
.NET CLI
.NET CLI
A cross-platform toolchain for developing, building, running, and publishing .NET applications.
322 questions
C#
C#
An object-oriented and type-safe programming language that has its roots in the C family of languages and includes support for component-oriented programming.
10,199 questions
{count} votes