WFC service cpu suddently drops

asked 2021-11-19T19:03:31.22+00:00
Jakob Aagesen 136 Reputation points

hi
I have a netTcpBinding service that is called from another service and it works more or less. I can stream audio from one service to the other but at some point the CPU drops from about 26% to 0% and the service does not receive any more audio.
The way it is setup, is that the first service is getting audio streamed via UDP from a client. The service, upon receiving the audio, converts it to another format and send it to the other service using netTcpBinding.
So what I see it that, when increase the audio flow, to the first service, it goes as expected op till about 15 Mbps. the CPU follows and goes op steadily. At some point, a unknown limit is hit and the service stops sending anymore audio to the other service.
These are my observations:

  • From performance counters, I see that the second that the CPU drops, the counter called "Pool Nonpaged Bytes" sky rockets.
  • If I drop the audio instead of converting and sending it to the second service, there is no drop in CPU and no problems, other than the system is broken :D.
  • CPU and Memory is not stressed at all.

I am not sure if this is caused by the fact that audio is propagated from the receiving thread all the way to the receiving thread on the second service in one go. What I mean is, I don't have a queue where audio is put upon receiving it and then another thread fetching and sending it to the other service. The service that receives the converted audio has this behavior

[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]

i have the following config setting:
<serviceThrottling maxConcurrentSessions="4000" maxConcurrentCalls="4000" />

The number of connections between the two services is 1 so i'm not hitting any maximum TCP port limit.

this is the configured binding that I have:

<binding name="RSRStreamingServiceBinding" maxReceivedMessageSize="2147483647" maxBufferSize="2147483647" maxBufferPoolSize="2147483647" receiveTimeout="00:10:00">
<reliableSession inactivityTimeout="00:10:00" />
<readerQuotas maxArrayLength="2147483647" maxStringContentLength="6291456" />
</binding>

I don't understand how the CPU usage of the service can drop to 0 in a split second. I think that the reason that the "Pool Nonpaged Bytes" goes op is because the first service is trying to send audio but can't for some reason. I guess that it is because the second service is refusing to receive anymore, but i don't understand why.
I have looked at a lot of performance counter and could not find any that shed some light on the matter.

Can anybody help me understand this problem a little bit better and maybe point me to the next point of investigation.

Windows Communication Foundation
Windows Communication Foundation
A Microsoft runtime and set of application programming interfaces in the .NET Framework for building service-oriented applications.
258 questions
No comments
{count} votes

2 answers

Sort by: Most helpful
  1. answered 2021-11-22T08:55:45.2+00:00
    Lan Huang-MSFT 8,881 Reputation points Microsoft Employee

    Hi @Jakob Aagesen ,
    You dont need maxBufferSize or maxBufferPoolSize set so high these are possibly causing your out of memory exception. The defaults should be fine.
    Check out the following article,specifically the section Special Security Considerations for Large Data this piece of the text is important.
    https://learn.microsoft.com/en-us/dotnet/framework/wcf/feature-details/large-data-and-streaming?redirectedfrom=MSDN,
    Best regards,
    Lan Huang


    If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    No comments

  2. answered 2021-11-24T14:32:15.227+00:00
    Jakob Aagesen 136 Reputation points

    Thank you for the suggestion. I changed so that when audio is send from one service to the other, it spins up a thread to do the sending. This resulted in a faster through put and the CPU does not drop, rather it follows the load. I still don't understand why the CPU dropped to 0, but it doesn't now :-)

    No comments