Copy Web Data into datbase

Question

Copy Web Data into datbase

Malik Asad Mahmood 126

Hi Friends,
I am trying to copy get webpage values into my databases table using vb.net 2017, I am using following code but its not working

Dim Web As New HtmlAgilityPack.HtmlWeb
Dim Doc As New HtmlAgilityPack.HtmlDocument
Doc = Web.Load("https://scstrade.com/stockscreening/SS_TechSnapShot.aspx?symbol=NCPL")
For Each table As HtmlAgilityPack.HtmlNode In Doc.DocumentNode.SelectNodes("//*[@id='pricing']/tr/td")
MsgBox(table.InnerText)
Next

the website content I want to copy the URL as followed "https://scstrade.com/stockscreening/SS_TechSnapShot.aspx?symbol=NCPL" further refer to snapshot of required table content

thank you in advance

regards,

asad webdata

Mason Wheare 70 Reputation points

2023-09-05T06:03:11.9733333+00:00
It seems like you're trying to scrape data from a webpage using VB.NET and the HtmlAgilityPack library. However, it's important to note that web scraping can be subject to the terms of service of the website you're scraping from, and some websites may employ measures to prevent scraping. Always ensure you have permission to scrape a website's content, and respect their terms of service.

Here's how you can modify your code to scrape the table data from the given URL using HtmlAgilityPack:

Imports HtmlAgilityPack Sub Main() Dim Web As New HtmlWeb() Dim Doc As New HtmlDocument() ' Load the webpage content Doc = Web.Load("https://scstrade.com/stockscreening/SS_TechSnapShot.aspx?symbol=NCPL") ' Locate the table element using its HTML structure or attributes Dim table As HtmlNode = Doc.DocumentNode.SelectSingleNode("//table[@id='pricing']") If table IsNot Nothing Then ' Iterate through rows of the table For Each row As HtmlNode In table.SelectNodes(".//tr") ' Iterate through cells (td) in each row For Each cell As HtmlNode In row.SelectNodes(".//td") ' Output the cell content to the console (you can save it to your database here) Console.WriteLine(cell.InnerText.Trim()) Next Next Else Console.WriteLine("Table not found on the page.") End If End Sub

This code should locate the table with the id "pricing" on the webpage and extract its data. Make sure you have the HtmlAgilityPack library properly installed in your VB.NET project.

Remember that websites may change their structure over time, so if the webpage structure changes, you may need to update your code accordingly. Additionally, always respect the website's terms of use and scraping policies.

Your answer

Mason Wheare 70 Reputation points

2023-09-05T06:03:11.9733333+00:00

It seems like you're trying to scrape data from a webpage using VB.NET and the HtmlAgilityPack library. However, it's important to note that web scraping can be subject to the terms of service of the website you're scraping from, and some websites may employ measures to prevent scraping. Always ensure you have permission to scrape a website's content, and respect their terms of service.

Here's how you can modify your code to scrape the table data from the given URL using HtmlAgilityPack:

Imports HtmlAgilityPack Sub Main() Dim Web As New HtmlWeb() Dim Doc As New HtmlDocument() ' Load the webpage content Doc = Web.Load("https://scstrade.com/stockscreening/SS_TechSnapShot.aspx?symbol=NCPL") ' Locate the table element using its HTML structure or attributes Dim table As HtmlNode = Doc.DocumentNode.SelectSingleNode("//table[@id='pricing']") If table IsNot Nothing Then ' Iterate through rows of the table For Each row As HtmlNode In table.SelectNodes(".//tr") ' Iterate through cells (td) in each row For Each cell As HtmlNode In row.SelectNodes(".//td") ' Output the cell content to the console (you can save it to your database here) Console.WriteLine(cell.InnerText.Trim()) Next Next Else Console.WriteLine("Table not found on the page.") End If End Sub

This code should locate the table with the id "pricing" on the webpage and extract its data. Make sure you have the HtmlAgilityPack library properly installed in your VB.NET project.

Remember that websites may change their structure over time, so if the webpage structure changes, you may need to update your code accordingly. Additionally, always respect the website's terms of use and scraping policies.

Share via

Copy Web Data into datbase

Your answer