Problem using Parallel.ForEachAsync

Marc Hillman 21 Reputation points
2022-10-10T02:53:31.75+00:00

I have a problem using Parallel.ForEachAsync.

I have a list of 3500 url's, and associated data. They are stored in "links" which is a List(Of link).I wish to simply validate each url to confirm they exist. I would like to exploit parallelism and test them (say) 10 at a time. I have developed the following code, but I cannot get it to work.

At the statement "Async Sub(t)" I get a BC36670 error, and nothing I can do seems to clear it. I'm new to parallelism, and I can't understand what the error message is trying to tell me. Any insights?

            httpclient.DefaultRequestHeaders.Clear()  
            Try  
                count = 0  
                Dim options As New ParallelOptions()  
                options.MaxDegreeOfParallelism = 10  
                Await Parallel.ForEachAsync(links, options,  
                       Async Sub(t)  
                           count += 1  
                           'AppendText(TextBox2, $"Processing index {t.index}{vbCrLf}")  
                           If t.url = "" Then  
                               t.status = "<font color='orange'>Missing</font>"  
                               Missing += 1  
                           Else  
                               'Dim r = Await TestURL(t.url)  
                               Dim httpResult = Await httpclient.GetAsync(t.url)  
                               Dim r = httpResult.IsSuccessStatusCode  
                               t.OK = r  
                               If r Then  
                                   t.status = "<font color='green'>OK</font>"  
                                   OK += 1  
                               Else  
                                   t.status = "<font color='red'>Not found</font>"  
                                   NotFound += 1  
                               End If  
                               SetText(TextBox1, $"Processed {count}/{links.Count}")  
                           End If  
  
                       End Sub)  
            Catch ex As Exception  
                MsgBox(ex.Message & vbCrLf & ex.StackTrace, vbCritical + vbOK, "Parallel web access failed")  
            End Try  
  
            ' Produce results  
            logWriter.WriteLine("<table border=1>")  
            logWriter.WriteLine("<tr><th>WWFFID</th><th>Name</th><th>State</th><th>HTTPLink</th><th>Status</th></tr>")  
            For Each lnk In links  
                logWriter.WriteLine($"<tr><td>{lnk.WWFFID}</td><td>{lnk.LongName}</td><td>{lnk.State}</td><td><a href='{lnk.url}'>{lnk.url}</a></td><td>{lnk.status}</td></tr>")  
            Next  
            logWriter.WriteLine("</table>")  
            logWriter.WriteLine($"<br>{links.Count} total links validated. {OK} OK, {Missing} Missing, {NotFound} Not Found")  
            logWriter.Close()  
            SetText(TextBox1, "Done")  
        End Using  
        Return True  
    End Function  
    Public Class link  
        Property index As Integer  
        Property url As String  
        Property WWFFID As String  
        Property LongName As String  
        Property State As String  
        Property status As String  
        Property OK As Boolean  
        Sub New(index As String, url As String, WWFFID As String, LongName As String, State As String)  
            _index = index  
            _url = url  
            _WWFFID = WWFFID  
            _LongName = LongName  
            _State = State  
            _status = status  
            _OK = False  
        End Sub  
    End Class  
.NET Runtime
.NET Runtime
.NET: Microsoft Technologies based on the .NET software framework.Runtime: An environment required to run apps that aren't compiled to machine language.
1,130 questions
0 comments No comments
{count} votes

6 answers

Sort by: Most helpful
  1. Jack J Jun 24,296 Reputation points Microsoft Vendor
    2022-10-11T03:22:42.967+00:00

    @Marc Hillman , Welcome to Microsoft Q&A, based on my test, I reproduced your problem.

    According to my search, I recommend that you could refer to the answer from the stackoverflow.

    It seems that we need to remove async from the Lambda because vb.net doesn't support an awaitable ValueTask.

    Here is a code example you could refer to.

      Public Async Function Test() As Task  
      
      
            Dim options As New ParallelOptions()  
            options.MaxDegreeOfParallelism = 10  
            Dim links As New List(Of String)  
      
      
            Await Parallel.ForEachAsync(Of String)(links, options, New Func(Of String, CancellationToken, ValueTask) _  
                (Function(uri, token)  
      
                 End Function))  
        End Function  
    

    Best Regards,
    Jack


    If the answer is the right solution, please click "Accept Answer" and upvote it.If you have extra questions about this answer, please click "Comment".
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.
    0 comments No comments

  2. Bruce (SqlWork.com) 57,641 Reputation points
    2022-10-10T15:30:19.273+00:00

    You have a couple issues

    The variable count is not thread safe, you need a lock to update

    You can not call the UI from a separate thread. Also the call is not thread safe.

    0 comments No comments

  3. Marc Hillman 21 Reputation points
    2022-10-10T22:56:39.367+00:00

    Thanks for the advice. I have put a SyncLock around the count variable (and a few others). I believe my UI calls are thread safe, see code for SetText below.

    This still doesn't solve the fundamental problem that it won't compile :(

        Private Sub SetText(tb As System.Windows.Forms.TextBox, ByVal text As String)  
      
            ' InvokeRequired required compares the thread ID of the  
            ' calling thread to the thread ID of the creating thread.  
            ' If these threads are different, it returns true.  
            If tb.InvokeRequired Then  
                tb.Invoke(New SetTextCallback(AddressOf SetText), New Object() {tb, text})  
            Else  
                tb.Text = text  
            End If  
            System.Windows.Forms.Application.DoEvents()  
        End Sub  
    
    0 comments No comments

  4. Bruce (SqlWork.com) 57,641 Reputation points
    2022-10-11T18:25:16.66+00:00

    the issue is with VB syntax defining the difference between methods and functions even with lambda.

    awaitable methods return a Task, so they are always functions. as the parallel foreach does not use the result, your VB lambda function can return any type. in C# we would use Task<void>, which would be inferred in a lambda.

    note: unless SetText is thread safe, you would need a lock.

    0 comments No comments

  5. Marc Hillman 21 Reputation points
    2022-10-11T04:51:41.323+00:00

    I think I might be screwed. Inside Function(uri, token) I need to do an Async operation, so the lambda function needs to be Async, but you say "It seems that we need to remove async from the Lambda because vb.net doesn't support an awaitable ValueTask. What I have is below, but I get BC37059 on the Await httpclient.GetAsync(lnk.Url). I seem to have a Catch-22 - the lambda function must be Async because it contains Await, but VB doesn't support Awaitable lambda.

    Using Parallel.ForEachAsync was a nice idea, but I can't see it's possible :(

                Try  
    Dim Links as List(Of Link), OK as Integer, Missing as Integer, NotFound as Integer  
                    Count = 0  
                    Dim Options As New ParallelOptions With {  
                        .MaxDegreeOfParallelism = 10  
                    }  
                    Await Parallel.ForEachAsync(Of Link)(Links, Options,  
                              New Func(Of Link, CancellationToken, ValueTask) _  
                              (Function(lnk, token)  
                                   Dim httpResult As Net.Http.HttpResponseMessage  
                                   SyncLock Links  
                                       Count += 1  
                                   End SyncLock  
                                   If lnk.Url = "" Then  
                                       lnk.Status = "<font color='orange'>Missing</font>"  
                                       Missing += 1  
                                   Else  
                                       httpResult = Await httpclient.GetAsync(lnk.Url)  
                                       Dim r = httpResult.IsSuccessStatusCode  
                                       lnk.OK = r  
                                       If r Then  
                                           lnk.Status = "<font color='green'>OK</font>"  
                                           SyncLock Links  
                                               OK += 1  
                                           End SyncLock  
                                       Else  
                                           lnk.Status = "<font color='red'>Not found</font>"  
                                           SyncLock Links  
                                               NotFound += 1  
                                           End SyncLock  
                                       End If  
                                       SetText(TextBox1, $"Processed {Count}/{Links.Count}")  
                                   End If  
                                   Return ???  
                               End Function))  
      
                Catch ex As Exception  
                    MsgBox(ex.Message & vbCrLf & ex.StackTrace, vbCritical + vbOK, "Parallel web access failed")  
                End Try  
            Return True  
        End Function  
        Public Class Link  
            Property Index As Integer  
            Property Url As String  
            Property WWFFID As String  
            Property LongName As String  
            Property State As String  
            Property Status As String  
            Property OK As Boolean  
            Sub New(Index As String, Url As String, WWFFID As String, LongName As String, State As String)  
                _Index = Index  
                _Url = Url  
                _WWFFID = WWFFID  
                _LongName = LongName  
                _State = State  
                _Status = Status  
                _OK = False  
            End Sub  
        End Class  
    
    0 comments No comments