Problem using Parallel.ForEachAsync

Question

I have a problem using Parallel.ForEachAsync.

I have a list of 3500 url's, and associated data. They are stored in "links" which is a List(Of link).I wish to simply validate each url to confirm they exist. I would like to exploit parallelism and test them (say) 10 at a time. I have developed the following code, but I cannot get it to work.

At the statement "Async Sub(t)" I get a BC36670 error, and nothing I can do seems to clear it. I'm new to parallelism, and I can't understand what the error message is trying to tell me. Any insights?

            httpclient.DefaultRequestHeaders.Clear()  
            Try  
                count = 0  
                Dim options As New ParallelOptions()  
                options.MaxDegreeOfParallelism = 10  
                Await Parallel.ForEachAsync(links, options,  
                       Async Sub(t)  
                           count += 1  
                           'AppendText(TextBox2, $"Processing index {t.index}{vbCrLf}")  
                           If t.url = "" Then  
                               t.status = "<font color='orange'>Missing</font>"  
                               Missing += 1  
                           Else  
                               'Dim r = Await TestURL(t.url)  
                               Dim httpResult = Await httpclient.GetAsync(t.url)  
                               Dim r = httpResult.IsSuccessStatusCode  
                               t.OK = r  
                               If r Then  
                                   t.status = "<font color='green'>OK</font>"  
                                   OK += 1  
                               Else  
                                   t.status = "<font color='red'>Not found</font>"  
                                   NotFound += 1  
                               End If  
                               SetText(TextBox1, $"Processed {count}/{links.Count}")  
                           End If  
  
                       End Sub)  
            Catch ex As Exception  
                MsgBox(ex.Message & vbCrLf & ex.StackTrace, vbCritical + vbOK, "Parallel web access failed")  
            End Try  
  
            ' Produce results  
            logWriter.WriteLine("<table border=1>")  
            logWriter.WriteLine("<tr><th>WWFFID</th><th>Name</th><th>State</th><th>HTTPLink</th><th>Status</th></tr>")  
            For Each lnk In links  
                logWriter.WriteLine($"<tr><td>{lnk.WWFFID}</td><td>{lnk.LongName}</td><td>{lnk.State}</td><td><a href='{lnk.url}'>{lnk.url}</a></td><td>{lnk.status}</td></tr>")  
            Next  
            logWriter.WriteLine("</table>")  
            logWriter.WriteLine($"<br>{links.Count} total links validated. {OK} OK, {Missing} Missing, {NotFound} Not Found")  
            logWriter.Close()  
            SetText(TextBox1, "Done")  
        End Using  
        Return True  
    End Function  
    Public Class link  
        Property index As Integer  
        Property url As String  
        Property WWFFID As String  
        Property LongName As String  
        Property State As String  
        Property status As String  
        Property OK As Boolean  
        Sub New(index As String, url As String, WWFFID As String, LongName As String, State As String)  
            _index = index  
            _url = url  
            _WWFFID = WWFFID  
            _LongName = LongName  
            _State = State  
            _status = status  
            _OK = False  
        End Sub  
    End Class  

Answer 1

@Marc Hillman , Welcome to Microsoft Q&A, based on my test, I reproduced your problem.

According to my search, I recommend that you could refer to the answer from the stackoverflow.

It seems that we need to remove async from the Lambda because vb.net doesn't support an awaitable ValueTask.

Here is a code example you could refer to.

  Public Async Function Test() As Task  
  
  
        Dim options As New ParallelOptions()  
        options.MaxDegreeOfParallelism = 10  
        Dim links As New List(Of String)  
  
  
        Await Parallel.ForEachAsync(Of String)(links, options, New Func(Of String, CancellationToken, ValueTask) _  
            (Function(uri, token)  
  
             End Function))  
    End Function  

Best Regards,
Jack

---

If the answer is the right solution, please click "Accept Answer" and upvote it.If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

You have a couple issues

The variable count is not thread safe, you need a lock to update

You can not call the UI from a separate thread. Also the call is not thread safe.

Answer 3

Thanks for the advice. I have put a SyncLock around the count variable (and a few others). I believe my UI calls are thread safe, see code for SetText below.

This still doesn't solve the fundamental problem that it won't compile :(

    Private Sub SetText(tb As System.Windows.Forms.TextBox, ByVal text As String)  
  
        ' InvokeRequired required compares the thread ID of the  
        ' calling thread to the thread ID of the creating thread.  
        ' If these threads are different, it returns true.  
        If tb.InvokeRequired Then  
            tb.Invoke(New SetTextCallback(AddressOf SetText), New Object() {tb, text})  
        Else  
            tb.Text = text  
        End If  
        System.Windows.Forms.Application.DoEvents()  
    End Sub  

Answer 4

the issue is with VB syntax defining the difference between methods and functions even with lambda.

awaitable methods return a Task, so they are always functions. as the parallel foreach does not use the result, your VB lambda function can return any type. in C# we would use Task<void>, which would be inferred in a lambda.

note: unless SetText is thread safe, you would need a lock.

Answer 5

I think I might be screwed. Inside Function(uri, token) I need to do an Async operation, so the lambda function needs to be Async, but you say "It seems that we need to remove async from the Lambda because vb.net doesn't support an awaitable ValueTask. What I have is below, but I get BC37059 on the Await httpclient.GetAsync(lnk.Url). I seem to have a Catch-22 - the lambda function must be Async because it contains Await, but VB doesn't support Awaitable lambda.

Using Parallel.ForEachAsync was a nice idea, but I can't see it's possible :(

            Try  
Dim Links as List(Of Link), OK as Integer, Missing as Integer, NotFound as Integer  
                Count = 0  
                Dim Options As New ParallelOptions With {  
                    .MaxDegreeOfParallelism = 10  
                }  
                Await Parallel.ForEachAsync(Of Link)(Links, Options,  
                          New Func(Of Link, CancellationToken, ValueTask) _  
                          (Function(lnk, token)  
                               Dim httpResult As Net.Http.HttpResponseMessage  
                               SyncLock Links  
                                   Count += 1  
                               End SyncLock  
                               If lnk.Url = "" Then  
                                   lnk.Status = "<font color='orange'>Missing</font>"  
                                   Missing += 1  
                               Else  
                                   httpResult = Await httpclient.GetAsync(lnk.Url)  
                                   Dim r = httpResult.IsSuccessStatusCode  
                                   lnk.OK = r  
                                   If r Then  
                                       lnk.Status = "<font color='green'>OK</font>"  
                                       SyncLock Links  
                                           OK += 1  
                                       End SyncLock  
                                   Else  
                                       lnk.Status = "<font color='red'>Not found</font>"  
                                       SyncLock Links  
                                           NotFound += 1  
                                       End SyncLock  
                                   End If  
                                   SetText(TextBox1, $"Processed {Count}/{Links.Count}")  
                               End If  
                               Return ???  
                           End Function))  
  
            Catch ex As Exception  
                MsgBox(ex.Message & vbCrLf & ex.StackTrace, vbCritical + vbOK, "Parallel web access failed")  
            End Try  
        Return True  
    End Function  
    Public Class Link  
        Property Index As Integer  
        Property Url As String  
        Property WWFFID As String  
        Property LongName As String  
        Property State As String  
        Property Status As String  
        Property OK As Boolean  
        Sub New(Index As String, Url As String, WWFFID As String, LongName As String, State As String)  
            _Index = Index  
            _Url = Url  
            _WWFFID = WWFFID  
            _LongName = LongName  
            _State = State  
            _Status = Status  
            _OK = False  
        End Sub  
    End Class