Copy Data from investing.com return Error

Malik Asad Mahmood 126 Reputation points
2021-03-20T16:25:27.147+00:00

Hi,
I want to copy data from investing.com upon executing following code in vb.net its return following error "The underlying connection was closed: An unexpected error occurred on a send." please help me

Dim webClient As WebClient = New WebClient()
Dim page As String = webClient.DownloadString("https://www.investing.com/equities/wrldcal-teleco-earnings")
Dim doc As HtmlAgilityPack.HtmlDocument = New HtmlAgilityPack.HtmlDocument()
doc.LoadHtml(page)

    Dim lstNode As List(Of HtmlNode) = New List(Of HtmlNode)
    Dim lstName As List(Of String) = New List(Of String)
    Dim lstTable As List(Of DataTable) = New List(Of DataTable)

    For Each thed As HtmlNode In doc.DocumentNode.SelectNodes("//thead")
        lstNode.Add(thed.ParentNode)
        Dim text = thed.SelectSingleNode("tr").SelectSingleNode("th").SelectSingleNode("h4").InnerText
        Console.WriteLine(text)
        lstName.Add(text)
    Next
    For i As Integer = 0 To lstNode.Count - 1
        Dim dt As DataTable = New DataTable
        dt.TableName = lstName(i)
        For Each trNode As HtmlNode In lstNode(i).SelectNodes("tr")
            If trNode.Attributes("class") Is Nothing Then
                For Each colNode As HtmlNode In trNode.SelectNodes("td")
                    dt.Columns.Add(colNode.InnerText)
                Next
            Else
                Dim j As Integer = 0
                Dim row As DataRow = dt.NewRow()
                For Each rowNode As HtmlNode In trNode.SelectNodes("td")
                    row(j) = rowNode.InnerText
                    j += 1
                Next
                dt.Rows.Add(row)
                '  MsgBox("test")
            End If
        Next
        lstTable.Add(dt)
        DataGridView1.DataSource = dt
        save_trx(dt)
    Next

    MsgBox("final finished...")
VB
VB
An object-oriented programming language developed by Microsoft that is implemented on the .NET Framework. Previously known as Visual Basic .NET.
2,659 questions
0 comments No comments
{count} votes

Accepted answer
  1. Karen Payne MVP 35,386 Reputation points
    2021-03-22T11:28:51.417+00:00

    These work, try the second one

    Imports System.Net
    
    Public Class Form1
        Private siteAddress As String =
                    "https://www.investing.com/equities/wrldcal-teleco-earnings"
    
        Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            Using client As New WebClient()
                client.Headers.Add("user-agent", "karen payne")
                Dim htmlCode As String = client.DownloadString(siteAddress)
            End Using
        End Sub
    
        Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
            Using client As New WebClient()
                client.Headers.Add("user-agent", "karen payne")
                ServicePointManager.SecurityProtocol = CType(3072, SecurityProtocolType)
                ServicePointManager.DefaultConnectionLimit = 9999
                Dim htmlCode As String = client.DownloadString(siteAddress)
            End Using
        End Sub
    End Class
    
    1 person found this answer helpful.
    0 comments No comments

5 additional answers

Sort by: Most helpful
  1. Viorel 114.5K Reputation points
    2021-03-20T17:58:54.377+00:00

    Try adding a header:

    . . .
    webClient.Headers.Add("user-agent", "My WebClient")
    Dim page As String = webClient.DownloadString( 
    . . .
    

    But sometimes the pages are built by browsers, not by WebClient, therefore the results could be incomplete.

    1 person found this answer helpful.

  2. Malik Asad Mahmood 126 Reputation points
    2021-03-20T18:31:49.467+00:00

    same error further referred to attached snapshot79816-errorinv.jpg

    1 person found this answer helpful.
    0 comments No comments

  3. Xingyu Zhao-MSFT 5,361 Reputation points
    2021-03-22T07:43:21.537+00:00

    Hi @Malik Asad Mahmood ,
    What's the version of your .NET Framework?
    In my test with .NET Framework 4.8, the following code worked for me.

        Using client As WebClient = New WebClient()  
                client.Headers.Add("user-agent", "test")  
                Dim htmlCode As String = client.DownloadString("https://www.investing.com/equities/wrldcal-teleco-earnings")  
      
                ...      
        End Using  
    

    Also check : https://stackoverflow.com/a/57571745/12666543
    Hope it could be helpful.

    Best Regards,
    Xingyu Zhao
    *
    If the answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    1 person found this answer helpful.
    0 comments No comments

  4. Malik Asad Mahmood 126 Reputation points
    2021-03-22T14:17:49.62+00:00

    Thank you , Button2 code working fine and connected.

    1 person found this answer helpful.