IHostedService - How to implement Parallel.ForEachAsync to call web API?

Question

Hello,

In my current solution, a user triggers my web API from a web application first, then my web API calls the 3rd party web service, and the results/codes are displayed on the web application. This current design (web application <-> web API <-> 3rd party web service) forces the user to stay in front of the screen and keep constantly sending requests to the API until successfully fulfilling 20K codes. I want to call a web API in the background with IHostedService. I am not familiar with using Parallel.ForEachAsync. How can I implement it into my IHostedService while calling this web API?

Here is the sample IHostedService:

 public class BusinessService : IHostedService  
        {  
            private readonly IHttpClientFactory _httpClientFactory;  
            private readonly IHostApplicationLifetime _applicationLifetime;  
            private readonly IConfiguration _configuration;  
            private readonly ILogger _logger;  
      
            public BusinessService(IHttpClientFactory httpClientFactory, IHostApplicationLifetime applicationLifetime,  
                IConfiguration configuration, ILogger<BusinessService> logger)  
            {  
                _httpClientFactory = httpClientFactory;  
                _applicationLifetime = applicationLifetime;  
                _configuration = configuration;  
                _logger = logger;  
            }  
      
            public async Task StartAsync(CancellationToken cancellationToken)  
            {  
                _logger.LogInformation("Application starts: ");  
                //await MakeRequestsToRemoteService();  
                await GetData();  
                //Stop Application  
                _applicationLifetime.StopApplication();  
                _logger.LogInformation("Application stops: ");  
            }  
      
            private async Task MakeRequestsToRemoteService(string productCode, long id)  
            {  
                try  
                {  
                    var httpClient = _httpClientFactory.CreateClient("GameClient");  
      
                    var authenticationBytes = Encoding.ASCII.GetBytes("Test:R08GO1234^");  
      
                    httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic",  
                        Convert.ToBase64String(authenticationBytes));  
                    httpClient.DefaultRequestHeaders.Accept.Add(  
                        new MediaTypeWithQualityHeaderValue("application/x-www-form-urlencoded"));  
      
                    var content = new FormUrlEncodedContent(new[]  
                    {  
                        new KeyValuePair<string, string>("productCode", productCode),  
                        new KeyValuePair<string, string>("quantity", "1"),  
                          
                    });  
      
      
                    using var response = await httpClient.PostAsync(_configuration["Service:Production"], content);  
                    if ((int) response.StatusCode == 200)  
                    {  
                        var coupons = await response.Content.ReadAsAsync<Root>();  
      
      
                        _logger.LogInformation("Order {referenceId} {pin}, {Serial} ", coupons.ReferenceId,  
                            coupons.Coupons[0].Pin, coupons.Coupons[0].Serial);  
      
                        await UpdateData(coupons, id);  
                    }  
                    else  
                    {  
                        _logger.LogError("Game Code Service Application Error {statusCode} for {productCode}",  
                            (int) response.StatusCode, productCode);  
                    }  
                }  
                catch (Exception e)  
                {  
                    _logger.LogError("Error: {Error} ", e.Message);  
                    throw;  
                }  
            }  
      
            private async Task GetData()  
            {  
                var connString = _configuration["ConnectionStrings:Production"];  
                await using var sqlConnection = new SqlConnection(connString);  
                sqlConnection.Open();  
      
                await using var command = new SqlCommand {Connection = sqlConnection};  
                const string sql = @"Select top 100 id, customerId, firstName, LastName, phone, productCode, productName from TestOrders where status = 0 ORDER BY NEWID()";  
                command.CommandText = sql;  
      
      
                try  
                {  
                    await using var reader = await command.ExecuteReaderAsync();  
                    while (reader.Read())  
                    {  
                        _logger.LogInformation(  
                            "Order {ID} {CustomerId}, {Name} {LastName}, {Phone}, {ProductCode}, {ProductName} ",  
                            reader.GetInt64(0), reader.GetInt32(1), reader.GetString(2), reader.GetString(3),  
                            reader.GetString(4), reader.GetString(5), reader.GetString(6));  
      
                        await MakeRequestsToRemoteService(reader.GetString(5).Trim(), reader.GetInt64(0));  
                    }  
                }  
                catch (SqlException exception)  
                {  
                    _logger.LogError("Error: {Error} ", exception.Message);  
                    throw; // Throw exception if this exception is unexpected  
                }  
            }  
      
            private async Task UpdateData(Root root, long id)  
            {  
                var connString = _configuration["ConnectionStrings:Production"];  
                await using var sqlConnection = new SqlConnection(connString);  
                sqlConnection.Open();  
      
                await using var command = new SqlCommand {Connection = sqlConnection};  
                const string sql = @"Update TestOrders set referenceId = @referenceId, Serial = @serial, Pin = @pin, status = 1 where id = @id";  
                command.CommandText = sql;  
      
                command.Parameters.Add(new SqlParameter("referenceId", root.ReferenceId));  
                command.Parameters.Add(new SqlParameter("serial", root.Coupons[0].Serial));  
                command.Parameters.Add(new SqlParameter("pin", root.Coupons[0].Pin));  
                command.Parameters.Add(new SqlParameter("id", id));  
      
                try  
                {  
                    await command.ExecuteNonQueryAsync();  
                }  
                catch (SqlException exception)  
                {  
                    _logger.LogError("Error: {Error} ", exception.Message);  
                    throw; // Throw exception if this exception is unexpected  
                }  
            }  
      
            public Task StopAsync(CancellationToken cancellationToken)  
            {  
                return Task.CompletedTask;  
            }  
        }  

Answer 1

Hi,
Because there are sql queries, it is necessary to be careful that the transaction passes or does not pass, so you need to use TransactionScope.

Parallel.ForEach is a method in the System.Threading.Tasks namespace that allows you to perform a specific action on each item in a collection in parallel.

Here is an example of how you can use Parallel.ForEach to update a list of users in a database using Entity Framework Core:

using (var context = new MyDbContext())  
{  
    var users = context.Users.ToList();  
  
    Parallel.ForEach(users, user =>  
    {  
        user.Name = "Updated Name";  
        context.SaveChanges();  
    });  
}  

TransactionScope is a class in the System.Transactions namespace that allows you to group a set of operations into a single transaction, so that either all of the operations are completed or none of them are. This can be useful when you want to make sure that a series of database updates are either all completed or all rolled back if an error occurs.

Here is an example of how you can use TransactionScope with Parallel.ForEach to update a list of users in a database using Entity Framework Core:

 using (var context = new MyDbContext())  
{  
    var users = context.Users.ToList();  
  
    using (var transaction = new TransactionScope())  
    {  
        Parallel.ForEach(users, user =>  
        {  
            user.Name = "Updated Name";  
            context.SaveChanges();  
        });  
  
        transaction.Complete();  
    }  
}  

Note that you will need to include the following using statement at the top of your file to use TransactionScope:

using System.Transactions;  

Answer 2

By the way, my intention is to use IHostedService to schedule this process in the background. My goal is to call the 3rd party web service 100 records/codes per 1 minutes interval. Estimated time 3.5 hours for 20000 codes.