System.Text.Json.JsonSerializer.Deserialize slowness in Azure Kubernetes Service

Question

In a .NET 6.0 Web API running in Azure Kubernetes Service (using a Docker Image in Azure Container Registry), System.Text.Json.JsonSerializer.Deserialize<T>(jsonToDeserialize, options) is taking 2-3 seconds. Locally on a Windows 10 computer, it takes about .40 seconds. Does anyone know how to solve the slowness with JsonSerializer.Deserialize in Azure Kubernetes Service?

Answer 1

Hi @shiva patpi ,

Thank you for your detailed response.

The problem was solved in AKS by re-using the same JsonSerializerOptions instance for each JsonSerializer.Serialize and JsonSerializer.Deserialize. I'm not sure why it behaves differently locally running the code in VS 2022 on Windows 10.

The code makes a call to an API and performs JsonSerializer.Deserialize on the data that is returned. The data that is returned is very large.

It is a Linux container.

Seth

Answer 2

Hello @sdobbi ,
We would like to know more details about your problem statement but based upon the available information

From AKS point of I was thinking below suggestions:

1) Validate the worker Nodes VM size which is being used , by default AKS gets deployed with Standard_DS2_v2 - try to have a new nodepool with higher configuration (Like more CPU/Memory)
2) When the Deserializer statement is being executed , in other window check the utilization of the node ( kubectl top nodes (or) kubectl top pods)
3) Consider using Ephemeral OS disks for AKS node pools https://learn.microsoft.com/en-us/samples/azure-samples/aks-ephemeral-os-disk/aks-ephemeral-os-disk/
4) Is this running on Linux/Windows container? If it is a windows container - I would suggest higher VM size for the worker nodes (Just play around with available options while creating the node pool)

Coming to Code point of view:

1) The data which you are trying to serialize where exactly it is getting downloaded? (When deployed in AKS and when running locally)
2) Can you provide a sample piece of code which we can test it from our end ?
3) How the docker file is being generated , is that from VS Code ?

Hope that helps !

Regards,
Shiva

Answer 3

Hi @shiva patpi ,

After I wrote that the problem was solved, the problem came back. Therefore, I restarted the Pod in AKS and ran tests where 106 JsonSerializer.Deserialize operations were performed and all of them were less than 1 second which is good. Any idea why this would be solved by restarting the Pod in AKS? My concern is that this will keep happening and the Pod will need to be restarted.

Thanks,
Seth