VB.NET Winform HttpWebRequest / XmlDocument Error: The remote server returned an error: (403) Forbidden.

jim brown 271 Reputation points
2023-01-25T16:42:03.8733333+00:00

So the below code works on my home network but at work my .NET application gets a 403 Forbidden. I believe ZScaler firewall is blocking traffic, but I can view XML via browser without issue just not in a .NET application. I have read that it's trying to block BOT traffic so adding a .UserAgent should fix the issue, It did not.

I am just trying to read an XLM file from the internet and can view from a webrowser.

What alternatives are there for this issue?


Try
            Dim url As String = "https://www.espn.com/espn/rss/news"
            Dim rssFeed As HttpWebRequest = DirectCast(WebRequest.Create(url), HttpWebRequest)

            'rssFeed.Method = "POST"
            'rssFeed.Accept = "application/xml"
            'rssFeed.UserAgent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"
            'rssFeed.Credentials = CredentialCache.DefaultNetworkCredentials
            'rssFeed.Proxy.Credentials = CredentialCache.DefaultCredentials

            Dim response = rssFeed.GetResponse()
            Dim rssStream = response.GetResponseStream()

            Dim rssDoc As New XmlDocument()
            rssDoc.Load(rssStream)
            Dim rssItems As XmlNodeList = rssDoc.SelectNodes("rss/channel/item")
            Dim i As Integer = 0
            Dim dt As DataTable = New DataTable("table")
            dt.Columns.Add("Title", Type.GetType("System.String"))
            dt.Columns.Add("Description", Type.GetType("System.String"))
            dt.Columns.Add("Link", Type.GetType("System.String"))

            While i < rssItems.Count
                Dim node As XmlNode = rssItems.Item(i).SelectSingleNode("title")
                Dim title As String
                Dim description As String
                Dim link As String
                If node IsNot Nothing Then
                    title = node.InnerText
                Else
                    title = ""
                End If

                node = rssItems.Item(i).SelectSingleNode("description")
                If node IsNot Nothing Then
                    description = node.InnerText
                Else
                    description = ""
                End If

                node = rssItems.Item(i).SelectSingleNode("link")
                If node IsNot Nothing Then
                    link = node.InnerText
                Else
                    link = ""
                End If
                Dim dr As DataRow = dt.NewRow()
                dr("Title") = title
                dr("Description") = description
                dr("Link") = link

                dt.Rows.Add(dr)

                i += 1
            End While

            DataGridView1.DataSource = dt
            WebBrowser1.ScriptErrorsSuppressed() = True
            WebBrowser1.Navigate("https://www.espn.com/espn/rss/news")

        Catch ex As Exception
            MsgBox(ex.Message)
        End Try

screenshot

Windows 10
Windows 10
A Microsoft operating system that runs on personal computers and tablets.
10,201 questions
.NET
.NET
Microsoft Technologies based on the .NET software framework.
3,179 questions
Visual Studio
Visual Studio
A family of Microsoft suites of integrated development tools for building applications for Windows, the web and mobile devices.
4,428 questions
VB
VB
An object-oriented programming language developed by Microsoft that is implemented on the .NET Framework. Previously known as Visual Basic .NET.
2,501 questions
Windows 11
Windows 11
A Microsoft operating system designed for productivity, creativity, and ease of use.
7,622 questions
0 comments No comments
{count} votes