Get data from web table

Question

Ho to get web table value from:https://www.xamig.com/superenalotto/sistema-consigliato-prossima-estrazione.phpthe table Is to the bottom of Page. Tks

Answer 1

What you're basically asking to do is scrape a website. For that you're going to need get the HTML. Then you'll need to parse the HTML looking for the element you care about. How easy or hard it is completely depends on the page in question.

To get the raw HTML then use HttpClient as provided in the example code given. This is a standard GET request to the URL in question. Once you get the response back, assuming it is successful then you can read the response as a string, since that is what HTML is.

' Note: Not tested
Dim client as HttpClient = New HttpClient
client.BaseAddress = New Uri("https://www.xamig.com")

' Assuming this is an Async function
Dim html As String = Await client.GetStringAsync("superenalotto/sistema-consigliato-prossima-estrazione.php")

' Parse the HTML

Inspecting the HTML in the browser will help narrow down exactly what you're looking for. Just a cursory glance shows that maybe you can search for tables with id tabcol. Once you have that element then you can get to the table contents. HTML parsing can be notoriously tricky so I strongly recommend you use a parsing library. I've used HtmlAgilityPack (HAP) over the years and it works well. Furthermore it already has helper methods to get the HTML from the web so you don't even need to call HttpClient directly.

Dim web = New HtmlWeb
Dim html = web.Load("url")

Dim table = html.DocumentNode.SelectSingleNode("//table[@id='tabcol'[")

Of course you'll want to do validation such as you actually get HTML back and the table is found.

Now the bad news. Most every site these days loads data after the page loads. You can generally look at the HTML and figure this out pretty quick. That means you cannot just get the HTML and then parse it. The HTML you get is pre-data. The only way to get the final HTML is to actually let the scripts run and that requires a browser. Libraries like HAP don't run scripts as scripts require a browser. If that is the case for the site you're trying to load then you have little choice but to load the site into a browser and then get the final HTML. If this is a one-off thing then you could use WebView2 or similar browser control to load the site into the (hidden) browser window. Then after it is fully loaded you can grab the HTML out and then use HAP to parse the final HTML. This is going to be slow but it is the best you can do in most cases.