Use UTF8Encoding.GetBytes get some error

Question

Use UTF8Encoding.GetBytes get some error

济田胡 1

My netcore version is 2.0.0
I run it as a server,and I get a error when I write my log: Unable to translate Unicode character \uD83C at index 267 to specified code page.
however it happen just only once,I find the uD83C mean the signal �
first,I check my code,it not change.I also do same action on my test server,it's ok too.
then i search this question online; I find many similar question: https://github.com/dotnet/runtime/issues/594;https://github.com/dotnet/runtime/issues/12183

I want to know how to deal this question,should I use unicode encode?or I should up my version of netcore;
when this error happen,my program will stop,it surely confuse me ,please help me !!!!!

0 comments

2 answers

Your answer

Answer 1

Daniel Zhang-MSFT 9,661

Hi 21422491,
You can try to replace characters before you return them or save them using Regex.Replace method.
More discussion in this link you can refer to.
Best Regards,
Daniel Zhang

If the response is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

0 comments

Answer 2

Viorel 127K

I think that D83C is the first part of a “surrogate pair”. Did you determine which is the code after this character in the string?

Consider using Unicode or UTF-8 encoding and show some details about the program that has this problem.

济田胡 1 Reputation point

2020-12-30T07:26:20.57+00:00

Ok，I think it's possible, I will write a log when it happen again,but I create a utf-8 flush,May the user can input UTF-16 something?
the dll can't translate utf-16 to utf-8? I have a idea that may be input a emoji? it can influences the result?
Viorel 127K Reputation points

2020-12-30T13:06:23.303+00:00
For example, in case of the next valid surrogate pairs, the function works well returning the expected bytes:

string example = "\uD83C\uDF85"; byte[] utf8 = Encoding.UTF8.GetBytes( example );

But if the sequence is invalid or incomplete, GetBytes seems to accept it too. However, the exception is thrown in case of customised encoding:

string example = "\uD83C"; var encoding = new UTF8Encoding( encoderShouldEmitUTF8Identifier: true, throwOnInvalidBytes: true ); byte[] utf8 = encoding.GetBytes( example );

Therefore, to accept even invalid characters, specify false for throwOnInvalidBytes or use the predefined encoding object.

Share via

Use UTF8Encoding.GetBytes get some error

2 answers

Your answer