httpwebrequest: WebException return "the remote server returned an error 403 forbidden".

Question

Am I trying to create a URI validation tool in C#.net console application.

When I place the uri link in browser, it's working but through program, error exception occuring.

Just I want to confirm that the uri address is valid and exists.

Placing my code below and the required details. Please help on this to resolve. Thanks.

uri path: https://www.parliament.uk/globalassets/documents/fair-society-healthy-lives-full-report.pdf

C#.net code:

 public static bool IsValidURL(string url)
        {
            try
            {
                ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
                HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
                req.AllowAutoRedirect = false;
                req.AuthenticationLevel = System.Net.Security.AuthenticationLevel.None;
                req.Timeout = 10000;
                req.UseDefaultCredentials = true;
                req.MaximumAutomaticRedirections = 5;
                req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8";
                req.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36";
                req.CookieContainer = new CookieContainer();
                HttpWebResponse res = (HttpWebResponse)req.GetResponse();
                if (res.StatusCode == HttpStatusCode.OK)
                {
                    res.Close();
                    return true;
                }
                else
                {
                    res.Close();
                    return false;
                }
            }
            catch (WebException ex)
            {
                return false;
            }
        }

Answer 1

It appears that the site is using cookies/javascript to combat bot's.

I tried the site with Powershell's Invoke-Webrequest and also got the 403. Then I tried curl.exe noticed these comments in the returned data.

In Chrome I cleared my browsing data and got this.

It appears to be "working as designed".