Parallel.ForEachAsync Problem

Question

Hi There, I am calling a 3rd party web service in a worker background service. Here is the web service calling method. MaxDegreeOfParallelism for Parallel.ForEachAsync is 2. This makes 2 calls for the 3rd party web service.

private async Task MakeRequestsToRemoteService(string productCode, long id, int amount, int bulkId)
{
            if (id <= 0) throw new ArgumentOutOfRangeException(nameof(id));
            try
            {
                var httpClient = _httpClientFactory.CreateClient("RazerClient");

                //Token
                httpClient.DefaultRequestHeaders.Accept.Add(
                    new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));

                var tokenContent = new FormUrlEncodedContent(new[]
                {
                    new KeyValuePair<string, string>("username", _configuration["Razer:User"]),
                    new KeyValuePair<string, string>("password", _configuration["Razer:Password"])
                });

                using var tokenResponse = await httpClient.PostAsync(_configuration["Token:Production"], tokenContent);


                if ((int)tokenResponse.StatusCode == 200)
                {
                    var tokenResult = await tokenResponse.Content.ReadAsStringAsync();
                    var dynamicObject = JsonConvert.DeserializeObject<TokenReturnDto>(tokenResult)!;
                    var token = dynamicObject.Token;

                    //Call Razer Multi Requests

                    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer",
                        token);
                    httpClient.DefaultRequestHeaders.Accept.Add(
                        new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));

                    #region Calculate quantity

                    var num = amount;
                    var firstNum = 10;
                    var secondNum = 10;

                    if (num < 20)
                    {
                        firstNum = (num + 1) / 2;
                        secondNum = num - firstNum;
                    }

                    #endregion

                    var quantities = new List<int> { firstNum, secondNum };
                    var cts = new CancellationTokenSource();
                    ParallelOptions parallelOptions = new()
                    {
                        MaxDegreeOfParallelism = Convert.ToInt32(_configuration["MaxDegreeOfParallelism:Max"]),
                        CancellationToken = cts.Token
                    };
                    try
                    {
                        await Parallel.ForEachAsync(quantities, parallelOptions, async (quantity, ct) =>
                        {
                            var content = new FormUrlEncodedContent(new[]
                            {
                                new KeyValuePair<string, string>("productCode", productCode),
                                new KeyValuePair<string, string>("quantity", quantity.ToString()),
                                new KeyValuePair<string, string>("clientTrxRef", bulkId.ToString())
                            });

                            try
                            {
                                using var response =
                                    await httpClient.PostAsync(_configuration["Razer:Production"], content, ct);

                                if ((int)response.StatusCode == 200)
                                {
                                    var coupon = await response.Content.ReadFromJsonAsync<Root>(cancellationToken: ct);

                                    _logger.LogInformation("REFERENCE ID: {referenceId}", coupon.ReferenceId);

                                    await UpdateData(id);
                                }
                                else
                                {
                                    _logger.LogError("Purchase ServiceError: {statusCode}",
                                        (int)response.StatusCode);
                                }
                            }
                            catch (HttpRequestException ex)
                            {
                                _logger.LogError("HTTP client exception: {Message}", ex.Message);
                            }
                            catch (JsonException ex)
                            {
                                _logger.LogError("JSON serialization exception: {Message}", ex.Message);
                            }
                            catch (Exception ex)
                            {
                                _logger.LogError("Unexpected exception: {Message}", ex.Message);
                            }
                        });
                    }
                    catch (OperationCanceledException ex)
                    {
                        _logger.LogError("Operation canceled: {Message}", ex.Message);
                    }
                }
                else
                {
                    _logger.LogError("Token ServiceError: {statusCode}",
                        (int)tokenResponse.StatusCode);
                }
            }
            catch (Exception e)
            {
                _logger.LogError("Error: {Error} ", e.Message);
            }
}

I am getting this error once in a while;

CallRazer failed with error A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

My question is, even if I get an error from the 3rd party web service is it possible await UpdateData(id) to be called? Shouldn't both requests be successful in order to execute await UpdateData(id)?

How can I know if one of the parallel threads is NOT successful? So I can implement business logic.

Answer 1

you are calling UpdateData(id) in the for loop, so it called on every successful httpclient call. it is successful if the post did not throw an error and response status is OK if you want to call UpdateData(id) in failed loops, place in finally so it runs whether the httpclient call works or not. if you only wanted to call once after all httpclient calls complete, then place after the foreach.

Answer 2

Hi @Cenk

I mean even though I got an error from 3rd party web service, how come update data was called?

As Bruce said, from your code, we can see the UpdataData(id)was in the if((int)response.StatusCode == 200) block, so this method will be called once the http request is success.

If you want to call the UpdateData(id) method no matter the request is success or not, you can put this command at the end of try block, or put it in the finally block.

How can I know if one of the parallel threads is NOT successful? So I can implement business logic.

Before calling the Parallel.ForEachAsync method, you can define a counter or list (to store the success request quantity), then in the Parallel.ForEachAsync method, if the request succeeds, the count can be incremented. After that, based on the count result to implement business. Code like this:

            var quantities = new List<int> { firstNum, secondNum };
            var cts = new CancellationTokenSource();
            ParallelOptions parallelOptions = new()
            {
                MaxDegreeOfParallelism = Convert.ToInt32(_configuration["MaxDegreeOfParallelism:Max"]),
                CancellationToken = cts.Token
            };
            try
            {
                var successcount = 0; //define a counter
                var successquantity = new List<int>();
                await Parallel.ForEachAsync(quantities, parallelOptions, async (quantity, ct) =>
                {
                    var content = new FormUrlEncodedContent(new[]
                    {
                                new KeyValuePair<string, string>("productCode", productCode),
                                new KeyValuePair<string, string>("quantity", quantity.ToString()),
                                new KeyValuePair<string, string>("clientTrxRef", bulkId.ToString())
                            });

                    try
                    {
                        using var response =
                            await httpClient.PostAsync(_configuration["Razer:Production"], content, ct);

                        if ((int)response.StatusCode == 200)
                        {
                            var coupon = await response.Content.ReadFromJsonAsync<Root>(cancellationToken: ct);

                            _logger.LogInformation("REFERENCE ID: {referenceId}", coupon.ReferenceId);

                            await UpdateData(id);

                            successcount++; //success,
                            successquantity.Add(quantity); // add the success quantity in the list.
                        }
                        ...
                    }
                    ...
                 }

                //add business logic based on the result.
                if (successcount == quantities.Count)
                {
                    // all request is success.
                }
                //or using the following code. by using this method, you can know which request is send failed.
                if(successquantity.Count == quantities.Count)
                {

                }

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.
Best regards,
Dillion