C# does not encode Uri properly/Web client does not load page when uri has a unicode character

Question

Hi,

I were trying to get data from the url https://hanziyuan.net/#字 . This in percent encoding is

https://hanziyuan.net/#%E5%AD%97
.

No matter what I do, the data that loads is from the default page
https://hanziyuan.net/#%E8%BD%A6
https://hanziyuan.net/#车

The code I used is given below. It seems the encoded part is not getting passed on to the server
by the C# client.

// Online C# Editor for free
// Write, Edit and Run your C# code using C# Online Compiler

using System;

public class HelloWorld
{
public static void Main(string[] args)
{

        System.Net.WebClient wc = new System.Net.WebClient();  

          

        
        byte[] raw = wc.DownloadData(new System.Uri("https://hanziyuan.net/#%E5%AD%97"));  


          
        string webData = System.Text.Encoding.UTF8.GetString(raw);     
          

    Console.WriteLine (webData);  
}  

}

   The data that loads is from the default page: https://hanziyuan.net/#%E8%BD%A6  
   https://hanziyuan.net/#车  
     
   While **the expected data on that code **is from:   
   https://hanziyuan.net/#字  
     
   https://hanziyuan.net/#%E5%AD%97  
     
   I have tried with the string "https://hanziyuan.net/#字" as well. Nothing seems to work!  
   "

Answer 1

Both https://hanziyuan.net/#字 and https://hanziyuan.net/ refer to the same page, except the former has a document fragment (note the # before 字).

This means that the HTML you're downloading with your DownloadData call is identical for both, and the page routing is done on the client side (using hash routing, see link: https://blog.bitsrc.io/using-hashed-vs-nonhashed-url-paths-in-single-page-apps-a66234cefc96)

Answer 2

the # in a url is a bookmark. it is not supposed to be sent to the server, after the response, its used to position the page. in the case of javascript routing, its used to navigate to an internal page. in the sample url, javascript uses the bookmark to makes an ajax call to fetch custom display content.

if you read the javascript you can probably reverse engineer how the page works, and do it in c#.