JavaScript to select data from a specific cell in HTML table loaded in WebView2

Question

I am trying to select data from a webpage that is loaded into WebView2. This was very easy with WebBrowser, but WebView2 forces me to execute JavaScripts in WebView2.ExecuteScriptAsync() which is a pain since I don't know JavaScripts.

Here is the URL for example page: https://qpublic.schneidercorp.com/Application.aspx?AppID=694&LayerID=11394&PageTypeID=4&PageID=4834&Q=872542282&KeyValue=0650010028

When WebView2 loads this URL there is a table with ID "ctlBodyPane_ctl11_ctl01_gvwSales". I want to select the data from each cell in the first row. I use Chrome DevTool > Copy > JS path to get the selector script: document.querySelector("#ctlBodyPane_ctl12_ctl01_gvwSales > tbody > tr:nth-child(1) > th"). The code below always returns "null", why?

        Dim result As String = Await webView.ExecuteScriptAsync("document.querySelector(""#ctlBodyPane_ctl12_ctl01_gvwSales > tbody > tr:nth-child(1) > th"").innerText;")

Answer 1

Hi @Ryan Hennings ,

According to your description and part of the code provided, I tried to reproduce your problem but failed. The code can work normally (successfully get the date data in the first row) in the WebView2NavigationComplete event.

This is a screenshot of the test result:

So I'm not sure how you wrote the code that caused this problem. If possible, could you provide a complete sample code that reproduces the problem? Thank you for understanding.

Best regards,
Xudong Peng

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 2

I'm facing another issue.

Here's an updated snippet of my project. Place a Button and WebView2 control on a Form and test code below.

I need to load multiple URL's and "scrape" data from each URL one at a time. Unfortunately, WebView2 requires the URL to be loaded from the UI thread, so I can't use threading. This is problematic because the UI won't load the URL until the Button1_Click event is done executing. As a workaround, I tried Application.DoEvents, but my code crashes when processing the 3rd URL, why?

I'm confused and need help. I want to force WebView2 to fully load a URL, run a series of functions to get the data from the page, save to database, and continue to process the next URL. How do I do that?

Imports System.Threading.Tasks
Imports Microsoft.Web.WebView2.Core

Public Class Form1

    Private _isNavigating As Boolean

    Public Sub New()
        InitializeComponent()
        Me.InitializeAsync()
    End Sub

    Private Async Sub InitializeAsync()
        Await Me.WebView21.EnsureCoreWebView2Async
    End Sub

    Private Async Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click

        Dim uris As New List(Of Uri)
        uris.Add(New Uri("https://qpublic.schneidercorp.com/Application.aspx?AppID=794&PageTypeID=4&KeyValue=C035000070032000"))
        uris.Add(New Uri("https://qpublic.schneidercorp.com/Application.aspx?AppID=794&PageTypeID=4&KeyValue=0049B00000026000"))
        uris.Add(New Uri("https://qpublic.schneidercorp.com/Application.aspx?AppID=794&PageTypeID=4&KeyValue=X006000000030000"))
        uris.Add(New Uri("https://qpublic.schneidercorp.com/Application.aspx?AppID=794&PageTypeID=4&KeyValue=8000000371000"))

        For Each uri As Uri In uris
            Me.WebView21.CoreWebView2.Navigate(uri.ToString)

            ' force WebView2 to load webpage before continuing
            Me._isNavigating = True
            Do
                Application.DoEvents()
            Loop While Me._isNavigating
        Next
    End Sub

    Private Async Function GetAPN(webView As WebView2) As Task(Of String)
        Dim apn As String = Await webView.ExecuteScriptAsync("document.querySelector('#ctlBodyPane_ctl00_ctl01_lblParcelID').innerText;")
        If apn = "null" Then
            Throw New Exception("APN not found in HTMLDocument")
        Else
            apn = Trim(apn.Replace("""", ""))
            Return apn
        End If
    End Function

    Private Async Function GetAddress(webView As WebView2) As Task(Of String)
        Dim address As String = Await webView.ExecuteScriptAsync("document.querySelector('#ctlBodyPane_ctl00_ctl01_lblLocationAddress').innerText;")
        If address = "null" Then
            Throw New Exception("Address not found in HTMLDocument")
        Else
            address = Trim(address.Replace("""", ""))
            Return address
        End If
    End Function

    Private Sub WebView21_NavigationStarting(sender As Object, e As CoreWebView2NavigationStartingEventArgs) Handles WebView21.NavigationStarting
        Debug.Print("NavigationStarting")
    End Sub

    Private Async Sub WebView21_NavigationCompleted(sender As Object, e As CoreWebView2NavigationCompletedEventArgs) Handles WebView21.NavigationCompleted
        Debug.Print("NavigationCompleted")
        Me._isNavigating = False

        ' get data from multiple elements on loaded webpage
        Debug.Print("APN:  " & Await Me.GetAPN(Me.WebView21))
        Debug.Print("Address:  " & Await Me.GetAddress(Me.WebView21))
    End Sub

End Class