Long running 101 responses when querying Cosmos using Gremlin

Question

Trying to diagnose some odd behaviour from CosmosDB when querying using Gremlin:

I've got a Blazor site that is querying Cosmos. It's regularly timing out, or taking a long time to pull back results from Cosmos. I've reduced the code down to:

var gremlinServer = new GremlinServer(
    Hostname, 
    Port, 
    true, 
    $"/dbs/{Database}/colls/{Collection}", 
    AuthKey
);

using var gremlinClient = new GremlinClient(
    gremlinServer, 
    new GraphSON2Reader(), 
    new GraphSON2Writer(), 
    GremlinClient.GraphSON2MimeType
);

var query = $"g.V().has('itemid', '{PartitionKey}')";

StringBuilder sb = new StringBuilder();

try
{
    var results = await gremlinClient.SubmitAsync<dynamic>(query);
    foreach (var result in results)
    {
        sb.AppendLine(result);
    }

    Logger.LogInformation(sb.ToString());
}
catch (Exception ex)
{
    Logger.LogError($"Error executing Gremlin query: {ex.Message}");
}

When checking App Insights to see what is going on behind the scenes, I noticed a number of long running requests/redirects which seem to be causing the issues:

When it's working correctly those 101 responses return in < 100ms and the query executes correctly. However, fairly regularly they can take > 30 secs to run and the query times out.

In terms of the database, there is < 1000 records in it. Running the query directly on the DB returns in < 1 sec. I've disabled any other processes that interact with the DB, so it doesn't seem to be a rate-limit issue.

Any ideas?

Answer 1

Managed to resolve this - somewhere else in the project the WebSockets.KeepAliveInterval and ConnectionPoolSettings had been set to a non-default value which was causing this behaviour. Removing that stopped the long running 101 requests.

Thanks

Chris

Answer 2

Hi Chris Bampton •,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.

Solution Adopted:

Somewhere in the project the WebSockets.KeepAliveInterval and ConnectionPoolSettings had been set to a non-default value which was causing this behaviour.

Removing that stopped the long running 101 requests.

Thanks