How to do WebScraping with ASP.NET Core ?

Valenciano8 116 Reputation points
2021-03-17T10:57:25.02+00:00

Hi everyone, I'd like to know if there is a built-in way to do webscraping (like fetching basic data from a website and use that data on my ASP.NET Core website) with ASP.NET Core ? Because from what I've understood webscraping basically works like this :

  1. Fetch the HTML page
  2. Analyse the HTML
  3. Get the data that matches classnames, divs or whatever you've specified

Best regards

ASP.NET Core
ASP.NET Core
A set of technologies in the .NET Framework for building web applications and XML web services.
4,360 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Sam of Simple Samples 5,526 Reputation points
    2021-03-17T21:46:18.38+00:00

    I wrote the article Introduction to Web Site Scraping but most of that is not useful here.

    See Html Agility pack. The responses to asp.net - Find div at run time at server side in c# - Stack Overflow indicate it can be used in C# in ASP.Net. You can fiddle with it in a console program in [Fork] Web Scraping using HtmlAgilityPack | C# Online Compiler | .NET Fiddle]4.

    0 comments No comments

  2. sharpegypt 1 Reputation point
    2021-04-17T03:49:57.203+00:00

    python is the best for scrap data

    0 comments No comments