Copy Data from investing.com return Error

Question

Hi,
I want to copy data from investing.com upon executing following code in vb.net its return following error "The underlying connection was closed: An unexpected error occurred on a send." please help me

Dim webClient As WebClient = New WebClient()
Dim page As String = webClient.DownloadString("https://www.investing.com/equities/wrldcal-teleco-earnings")
Dim doc As HtmlAgilityPack.HtmlDocument = New HtmlAgilityPack.HtmlDocument()
doc.LoadHtml(page)

    Dim lstNode As List(Of HtmlNode) = New List(Of HtmlNode)
    Dim lstName As List(Of String) = New List(Of String)
    Dim lstTable As List(Of DataTable) = New List(Of DataTable)

    For Each thed As HtmlNode In doc.DocumentNode.SelectNodes("//thead")
        lstNode.Add(thed.ParentNode)
        Dim text = thed.SelectSingleNode("tr").SelectSingleNode("th").SelectSingleNode("h4").InnerText
        Console.WriteLine(text)
        lstName.Add(text)
    Next
    For i As Integer = 0 To lstNode.Count - 1
        Dim dt As DataTable = New DataTable
        dt.TableName = lstName(i)
        For Each trNode As HtmlNode In lstNode(i).SelectNodes("tr")
            If trNode.Attributes("class") Is Nothing Then
                For Each colNode As HtmlNode In trNode.SelectNodes("td")
                    dt.Columns.Add(colNode.InnerText)
                Next
            Else
                Dim j As Integer = 0
                Dim row As DataRow = dt.NewRow()
                For Each rowNode As HtmlNode In trNode.SelectNodes("td")
                    row(j) = rowNode.InnerText
                    j += 1
                Next
                dt.Rows.Add(row)
                '  MsgBox("test")
            End If
        Next
        lstTable.Add(dt)
        DataGridView1.DataSource = dt
        save_trx(dt)
    Next

    MsgBox("final finished...")

Answer 1

These work, try the second one

Imports System.Net

Public Class Form1
    Private siteAddress As String =
                "https://www.investing.com/equities/wrldcal-teleco-earnings"

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Using client As New WebClient()
            client.Headers.Add("user-agent", "karen payne")
            Dim htmlCode As String = client.DownloadString(siteAddress)
        End Using
    End Sub

    Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
        Using client As New WebClient()
            client.Headers.Add("user-agent", "karen payne")
            ServicePointManager.SecurityProtocol = CType(3072, SecurityProtocolType)
            ServicePointManager.DefaultConnectionLimit = 9999
            Dim htmlCode As String = client.DownloadString(siteAddress)
        End Using
    End Sub
End Class

Answer 2

Try adding a header:

. . .
webClient.Headers.Add("user-agent", "My WebClient")
Dim page As String = webClient.DownloadString( 
. . .

But sometimes the pages are built by browsers, not by WebClient, therefore the results could be incomplete.

Answer 3

same error further referred to attached snapshot

Answer 4

Hi @Malik Asad Mahmood ,
What's the version of your .NET Framework?
In my test with .NET Framework 4.8, the following code worked for me.

    Using client As WebClient = New WebClient()  
            client.Headers.Add("user-agent", "test")  
            Dim htmlCode As String = client.DownloadString("https://www.investing.com/equities/wrldcal-teleco-earnings")  
  
            ...      
    End Using  

Also check : https://stackoverflow.com/a/57571745/12666543
Hope it could be helpful.

Best Regards,
Xingyu Zhao
*
If the answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Answer 5

Thank you , Button2 code working fine and connected.