Hello Aaron Ching,Thanks for you patience on this.
I have reached out to Azure Functions engineering team on this issue and sharing the below.
Upon investigating further, team has confirmed that this is a know issue and a fix related to this issue has been published in preview version 4.*. NuGet Gallery | Microsoft.Azure.Functions.Worker.Extensions.Mcp 1.0.0-preview.4
Also engineer team has suggested to the latest version preview5 which helps in avoiding hanging and opening a bunch of connections. NuGet Gallery | Microsoft.Azure.Functions.Worker.Extensions.Mcp 1.0.0-preview.5