Decoder.Convert Method

Definition

Namespace:

System.Text

Assembly:

System.Runtime.dll

Converts an encoded byte sequence to a string or array of characters.

Overloads

Convert(ReadOnlySpan<Byte>, Span<Char>, Boolean, Int32, Int32, Boolean)	Converts a span of encoded bytes to UTF-16 encoded characters and stores the result in another span buffer.
Convert(Byte*, Int32, Char*, Int32, Boolean, Int32, Int32, Boolean)	Converts a buffer of encoded bytes to UTF-16 encoded characters and stores the result in another buffer.
Convert(Byte[], Int32, Int32, Char[], Int32, Int32, Boolean, Int32, Int32, Boolean)	Converts an array of encoded bytes to UTF-16 encoded characters and stores the result in a character array.

Remarks

Remember that the Decoder object saves state between calls to Convert. When the application is done with a stream of data, it should set the flush parameter to true to make sure that the state information is flushed. With this setting, the decoder ignores invalid bytes at the end of the data block and clears the internal buffer. Any remaining processed data that is part of a logical unit, such as the high surrogate of a surrogate pair, is converted according to the current fallback settings.

The Convert method is designed to be used in a loop to decode an arbitrary amount of input, such as data read from a file or stream. It stores the output of the decoding operation in a fixed-size buffer. GetChars will throw an exception if the output buffer isn't large enough, but Convert will fill as much space as possible and return the bytes read and chars written, provided the output array allows for at least two characters. Also see Encoding.GetChars for more comments.

Convert(ReadOnlySpan<Byte>, Span<Char>, Boolean, Int32, Int32, Boolean)

Source:

Decoder.cs

Converts a span of encoded bytes to UTF-16 encoded characters and stores the result in another span buffer.

public:
 virtual void Convert(ReadOnlySpan<System::Byte> bytes, Span<char> chars, bool flush, [Runtime::InteropServices::Out] int % bytesUsed, [Runtime::InteropServices::Out] int % charsUsed, [Runtime::InteropServices::Out] bool % completed);

public virtual void Convert (ReadOnlySpan<byte> bytes, Span<char> chars, bool flush, out int bytesUsed, out int charsUsed, out bool completed);

abstract member Convert : ReadOnlySpan<byte> * Span<char> * bool * int * int * bool -> unit
override this.Convert : ReadOnlySpan<byte> * Span<char> * bool * int * int * bool -> unit

Public Overridable Sub Convert (bytes As ReadOnlySpan(Of Byte), chars As Span(Of Char), flush As Boolean, ByRef bytesUsed As Integer, ByRef charsUsed As Integer, ByRef completed As Boolean)

Parameters

bytes

ReadOnlySpan<Byte>

A read-only bytes span containing the sequence to convert.

chars

Span<Char>

The span to store the converted characters.

flush

Boolean

true to indicate no further data is to be converted; otherwise, false.

bytesUsed

Int32

When this method returns, contains the number of bytes that were produced by the conversion. This parameter is passed uninitialized.

charsUsed

Int32

When this method returns, contains the number of characters from chars that were used in the conversion. This parameter is passed uninitialized.

completed

Boolean

When this method returns, contains true if all the specified characters were converted; otherwise, false. This parameter is passed uninitialized.

Remarks

The completed output parameter indicates whether all the data in the input bytes span was converted and stored in the chars span. This parameter is set to false if the number of bytes contained in the input bytes span cannot be converted without exceeding the number of characters in the chars span. In that situation, the application should use the contents of the output buffer or provide a new output buffer, increment the bytes parameter by the number of bytes specified by the bytesUsed parameter, then call the Convert method again to process the remaining input.

The completed parameter can also be set to false, even though the bytesUsed parameter and bytes span length are equal. This situation occurs if there is still data in the Decoder object that has not been stored in the bytes span.

Convert(Byte*, Int32, Char*, Int32, Boolean, Int32, Int32, Boolean)

Source:

Decoder.cs

Important

This API is not CLS-compliant.

CLS-compliant alternative

System.Text.Decoder.Convert(Byte[], Int32, Int32, Char[], Int32, Int32, Boolean, Int32, Int32, Boolean)

Converts a buffer of encoded bytes to UTF-16 encoded characters and stores the result in another buffer.

public:
 virtual void Convert(System::Byte* bytes, int byteCount, char* chars, int charCount, bool flush, [Runtime::InteropServices::Out] int % bytesUsed, [Runtime::InteropServices::Out] int % charsUsed, [Runtime::InteropServices::Out] bool % completed);

[System.CLSCompliant(false)]
public virtual void Convert (byte* bytes, int byteCount, char* chars, int charCount, bool flush, out int bytesUsed, out int charsUsed, out bool completed);

[<System.CLSCompliant(false)>]
abstract member Convert : nativeptr<byte> * int * nativeptr<char> * int * bool * int * int * bool -> unit
override this.Convert : nativeptr<byte> * int * nativeptr<char> * int * bool * int * int * bool -> unit

Parameters

bytes

Byte*

The address of a buffer that contains the byte sequences to convert.

byteCount

Int32

The number of bytes in bytes to convert.

chars

Char*

The address of a buffer to store the converted characters.

charCount

Int32

The maximum number of characters in chars to use in the conversion.

flush

Boolean

true to indicate no further data is to be converted; otherwise, false.

bytesUsed

Int32

When this method returns, contains the number of bytes that were produced by the conversion. This parameter is passed uninitialized.

charsUsed

Int32

When this method returns, contains the number of characters from chars that were used in the conversion. This parameter is passed uninitialized.

completed

Boolean

When this method returns, contains true if all the characters specified by byteCount were converted; otherwise, false. This parameter is passed uninitialized.

Attributes

CLSCompliantAttribute

Exceptions

ArgumentNullException

chars or bytes is null (Nothing).

ArgumentOutOfRangeException

charCount or byteCount is less than zero.

ArgumentException

The output buffer is too small to contain any of the converted input. The output buffer should be at least 2 chars in size to accomodate at least one surrogate character pair.

DecoderFallbackException

A fallback occurred (for more information, see Character Encoding in .NET)

-and-

Fallback is set to DecoderExceptionFallback.

Remarks

The completed output parameter indicates whether all the data in the input buffer was converted and stored in the output buffer. This parameter is set to false if the number of bytes specified by the byteCount parameter cannot be converted without exceeding the number of characters specified by the charCount parameter. In that situation, the application should use the contents of the output buffer or provide a new output buffer, increment the bytes parameter by the number of bytes specified by the bytesUsed parameter, then call the Convert method again to process the remaining input.

The completed parameter can also be set to false, even though the bytesUsed and byteCount parameters are equal. This situation occurs if there is still data in the Decoder object that has not been stored in the bytes buffer.

Convert(Byte[], Int32, Int32, Char[], Int32, Int32, Boolean, Int32, Int32, Boolean)

Source:

Decoder.cs

Converts an array of encoded bytes to UTF-16 encoded characters and stores the result in a character array.

public:
 virtual void Convert(cli::array <System::Byte> ^ bytes, int byteIndex, int byteCount, cli::array <char> ^ chars, int charIndex, int charCount, bool flush, [Runtime::InteropServices::Out] int % bytesUsed, [Runtime::InteropServices::Out] int % charsUsed, [Runtime::InteropServices::Out] bool % completed);

public virtual void Convert (byte[] bytes, int byteIndex, int byteCount, char[] chars, int charIndex, int charCount, bool flush, out int bytesUsed, out int charsUsed, out bool completed);

abstract member Convert : byte[] * int * int * char[] * int * int * bool * int * int * bool -> unit
override this.Convert : byte[] * int * int * char[] * int * int * bool * int * int * bool -> unit

Public Overridable Sub Convert (bytes As Byte(), byteIndex As Integer, byteCount As Integer, chars As Char(), charIndex As Integer, charCount As Integer, flush As Boolean, ByRef bytesUsed As Integer, ByRef charsUsed As Integer, ByRef completed As Boolean)

Parameters

bytes

Byte[]

A byte array to convert.

byteIndex

Int32

The first element of bytes to convert.

byteCount

Int32

The number of elements of bytes to convert.

chars

Char[]

An array to store the converted characters.

charIndex

Int32

The first element of chars in which data is stored.

charCount

Int32

The maximum number of elements of chars to use in the conversion.

flush

Boolean

true to indicate that no further data is to be converted; otherwise, false.

bytesUsed

Int32

When this method returns, contains the number of bytes that were used in the conversion. This parameter is passed uninitialized.

charsUsed

Int32

When this method returns, contains the number of characters from chars that were produced by the conversion. This parameter is passed uninitialized.

completed

Boolean

When this method returns, contains true if all the characters specified by byteCount were converted; otherwise, false. This parameter is passed uninitialized.

Exceptions

ArgumentNullException

chars or bytes is null (Nothing).

ArgumentOutOfRangeException

charIndex, charCount, byteIndex, or byteCount is less than zero.

-or-

The length of chars - charIndex is less than charCount.

-or-

The length of bytes - byteIndex is less than byteCount.

ArgumentException

The output buffer is too small to contain any of the converted input. The output buffer should be at least 2 chars in size to accomodate at least one surrogate character pair.

DecoderFallbackException

A fallback occurred (for more information, see Character Encoding in .NET)

-and-

Fallback is set to DecoderExceptionFallback.

Examples

The following example uses the Convert method to convert a file of UTF-16 characters to UTF-8. It then uses the Convert method to convert the UTF-8 characters back to UTF-16 characters.

// This code example demonstrates the Encoder.Convert() and Decoder.Convert methods.
// This example uses files for input and output, but any source that can be expressed
// as a stream can be used instead.

    using System;
    using System.Text;
    using System.IO;

    public class Sample
    {
    static void Main(string[] args)
        {
// Create a large file of UTF-16 encoded Unicode characters. The file is named Example.txt,
// and is used as input to the Encoder.Convert() method.

            CreateTestFile("Example.txt");

// Using an input file of UTF-16 encoded characters named Example.txt, create an output file
// of UTF-8 encoded bytes named UTF8.txt.

            EncoderConvert("Example.txt", "UTF8.txt", Encoding.UTF8);

// Using an input file of UTF-8 encoded bytes named UTF8.txt, create an output file
// of UTF-16 encoded characters named UTF16.txt.

            DecoderConvert("UTF8.txt", "UTF16.txt", Encoding.UTF8);
        }

// --------------------------------------------------------------------------------------------
// Use the Encoder.Convert() method to convert a file of characters to a file of encoded bytes.
// --------------------------------------------------------------------------------------------
        static void EncoderConvert(String inputFileName, String outputFileName, Encoding enc)
        {
// Convert an input file of characters to an output file of encoded bytes.
// StreamWriter could convert the input file for us, but we'll perform the conversion
// ourselves.

            FileStream fs = new FileStream(outputFileName, FileMode.Create);
            BinaryWriter outputFile = new BinaryWriter(fs);

// StreamReader will detect Unicode encoding from the Byte Order Mark that heads the input file.
            StreamReader inputFile = new StreamReader(inputFileName);

// Get an Encoder.
            Encoder encoder = enc.GetEncoder();

// Guarantee the output buffer large enough to convert a few characters.
            int UseBufferSize = 64;
            if (UseBufferSize < enc.GetMaxByteCount(10))
                    UseBufferSize = enc.GetMaxByteCount(10);
            byte[] bytes = new byte[UseBufferSize];

// Intentionally make the input character buffer larger than the output byte buffer so the
// conversion loop executes more than one cycle.

            char[] chars = new char[UseBufferSize * 4];
            int charsRead;
            do
            {
// Read at most the number of characters that will fit in the input buffer. The return
// value is the actual number of characters read, or zero if no characters remain.
                charsRead = inputFile.Read(chars, 0, UseBufferSize * 4);

                bool completed = false;
                int charIndex = 0;
                int charsUsed;
                int bytesUsed;

                while (!completed)
                {
// If this is the last input data, flush the encoder's internal buffer and state.

                    bool flush = (charsRead == 0);
                    encoder.Convert(chars, charIndex, charsRead - charIndex,
                                    bytes, 0, UseBufferSize, flush,
                                    out charsUsed, out bytesUsed, out completed);

// The conversion produced the number of bytes indicated by bytesUsed. Write that number
// of bytes to the output file.
                    outputFile.Write(bytes, 0, bytesUsed);

// Increment charIndex to the next block of characters in the input buffer, if any, to convert.
                    charIndex += charsUsed;
                }
            }
            while(charsRead != 0);

            outputFile.Close();
            fs.Close();
            inputFile.Close();
        }

// --------------------------------------------------------------------------------------------
// Use the Decoder.Convert() method to convert a file of encoded bytes to a file of characters.
// --------------------------------------------------------------------------------------------
        static void DecoderConvert(String inputFileName, String outputFileName, Encoding enc)
        {
// Convert an input file of of encoded bytes to an output file characters.
// StreamWriter could convert the input file for us, but we'll perform the conversion
// ourselves.

            StreamWriter outputFile = new StreamWriter(outputFileName, false, Encoding.Unicode);

// Read the input as a binary file so we can detect the Byte Order Mark.
            FileStream fs = new FileStream(inputFileName, FileMode.Open);
            BinaryReader inputFile = new BinaryReader(fs);

// Get a Decoder.
            Decoder decoder = enc.GetDecoder();

// Guarantee the output buffer large enough to convert a few characters.
            int UseBufferSize = 64;
            if (UseBufferSize < enc.GetMaxCharCount(10))
                    UseBufferSize = enc.GetMaxCharCount(10);
            char[] chars = new char[UseBufferSize];

// Intentionally make the input byte buffer larger than the output character buffer so the
// conversion loop executes more than one cycle.

            byte[] bytes = new byte[UseBufferSize * 4];
            int bytesRead;
            do
            {
// Read at most the number of bytes that will fit in the input buffer. The
// return value is the actual number of bytes read, or zero if no bytes remain.

                bytesRead = inputFile.Read(bytes, 0, UseBufferSize * 4);

                bool completed = false;
                int byteIndex = 0;
                int bytesUsed;
                int charsUsed;

                while (!completed)
                {
// If this is the last input data, flush the decoder's internal buffer and state.

                    bool flush = (bytesRead == 0);
                    decoder.Convert(bytes, byteIndex, bytesRead - byteIndex,
                                    chars, 0, UseBufferSize, flush,
                                    out bytesUsed, out charsUsed, out completed);

// The conversion produced the number of characters indicated by charsUsed. Write that number
// of characters to the output file.

                    outputFile.Write(chars, 0, charsUsed);

// Increment byteIndex to the next block of bytes in the input buffer, if any, to convert.
                    byteIndex += bytesUsed;
                }
            }
            while(bytesRead != 0);

            outputFile.Close();
            fs.Close();
            inputFile.Close();
        }

// --------------------------------------------------------------------------------------------
// Create a large file of UTF-16 encoded Unicode characters.
// --------------------------------------------------------------------------------------------
        static void CreateTestFile(String FileName)
        {
// StreamWriter defaults to UTF-8 encoding so explicitly specify Unicode, that is,
// UTF-16, encoding.
            StreamWriter file = new StreamWriter(FileName, false, Encoding.Unicode);

// Write a line of text 100 times.
            for (int i = 0; i < 100; i++)
            {
                file.WriteLine("This is an example input file used by the convert example.");
            }

// Write Unicode characters from U+0000 to, but not including, the surrogate character range.
            for (char c = (char)0; c < (char)0xD800; c++)
            {
                file.Write(c);
            }
            file.Close();
        }
    }

/*
This code example produces the following results:

(Execute the -dir- console window command and examine the files created.)

Example.txt, which contains 122,594 bytes (61,297 UTF-16 encoded characters).
UTF8.txt, which contains 169,712 UTF-8 encoded bytes.
UTF16.txt, which contains 122,594 bytes (61,297 UTF-16 encoded characters).

(Execute the -comp- console window command and compare the two Unicode files.)

>comp example.txt utf16.txt /L
Comparing example.txt and utf16.txt...
Files compare OK

(The two files are equal.)

*/

' This code example demonstrates the Encoder.Convert() and Decoder.Convert methods. 
' This example uses files for input and output, but any source that can be expressed
' as a stream can be used instead.

Imports System.Text
Imports System.IO

Public Class Sample
    Shared Sub Main(ByVal args() As String) 
        ' Create a large file of UTF-16 encoded Unicode characters. The file is named Example.txt, 
        ' and is used as input to the Encoder.Convert() method. 
        CreateTestFile("Example.txt")
        
        
        ' Using an input file of UTF-16 encoded characters named Example.txt, create an output file 
        ' of UTF-8 encoded bytes named UTF8.txt.
        EncoderConvert("Example.txt", "UTF8.txt", Encoding.UTF8)
        
        ' Using an input file of UTF-8 encoded bytes named UTF8.txt, create an output file 
        ' of UTF-16 encoded characters named UTF16.txt.
        DecoderConvert("UTF8.txt", "UTF16.txt", Encoding.UTF8)
    
    End Sub
    
    ' --------------------------------------------------------------------------------------------
    ' Use the Encoder.Convert() method to convert a file of characters to a file of encoded bytes.
    ' --------------------------------------------------------------------------------------------
    Shared Sub EncoderConvert(ByVal inputFileName As String, ByVal outputFileName As String, ByVal enc As Encoding) 
        ' Convert an input file of characters to an output file of encoded bytes.
        ' StreamWriter could convert the input file for us, but we'll perform the conversion 
        ' ourselves.
        Dim fs As New FileStream(outputFileName, FileMode.Create)
        Dim outputFile As New BinaryWriter(fs)
        
        ' StreamReader will detect Unicode encoding from the Byte Order Mark that heads the input file.
        Dim inputFile As New StreamReader(inputFileName)
        
        ' Get an Encoder.
        Dim encoder As Encoder = enc.GetEncoder()
        
        ' Guarantee the output buffer large enough to convert a few characters.
        Dim UseBufferSize As Integer = 64
        If UseBufferSize < enc.GetMaxByteCount(10) Then
            UseBufferSize = enc.GetMaxByteCount(10)
        End If
        Dim bytes(UseBufferSize) As Byte
        
        ' Intentionally make the input character buffer larger than the output byte buffer so the 
        ' conversion loop executes more than one cycle. 
        Dim chars(UseBufferSize * 4) As Char
        Dim charsRead As Integer
        Do
            ' Read at most the number of characters that will fit in the input buffer. The return 
            ' value is the actual number of characters read, or zero if no characters remain. 
            charsRead = inputFile.Read(chars, 0, UseBufferSize * 4)
            
            Dim completed As Boolean = False
            Dim charIndex As Integer = 0
            Dim charsUsed As Integer
            Dim bytesUsed As Integer
            
            While Not completed
                ' If this is the last input data, flush the encoder's internal buffer and state.
                Dim flush As Boolean = charsRead = 0
                encoder.Convert(chars, charIndex, charsRead - charIndex, bytes, 0, UseBufferSize, flush, charsUsed, bytesUsed, completed)
                
                ' The conversion produced the number of bytes indicated by bytesUsed. Write that number
                ' of bytes to the output file.
                outputFile.Write(bytes, 0, bytesUsed)
                
                ' Increment charIndex to the next block of characters in the input buffer, if any, to convert.
                charIndex += charsUsed
            End While
        Loop While charsRead <> 0
        
        outputFile.Close()
        fs.Close()
        inputFile.Close()
    
    End Sub
    
    ' --------------------------------------------------------------------------------------------
    ' Use the Decoder.Convert() method to convert a file of encoded bytes to a file of characters.
    ' --------------------------------------------------------------------------------------------
    Shared Sub DecoderConvert(ByVal inputFileName As String, ByVal outputFileName As String, ByVal enc As Encoding) 
        ' Convert an input file of of encoded bytes to an output file characters.
        ' StreamWriter could convert the input file for us, but we'll perform the conversion 
        ' ourselves.
        Dim outputFile As New StreamWriter(outputFileName, False, Encoding.Unicode)
        
        ' Read the input as a binary file so we can detect the Byte Order Mark.
        Dim fs As New FileStream(inputFileName, FileMode.Open)
        Dim inputFile As New BinaryReader(fs)
        
        ' Get a Decoder.
        Dim decoder As Decoder = enc.GetDecoder()
        
        ' Guarantee the output buffer large enough to convert a few characters.
        Dim UseBufferSize As Integer = 64
        If UseBufferSize < enc.GetMaxCharCount(10) Then
            UseBufferSize = enc.GetMaxCharCount(10)
        End If
        Dim chars(UseBufferSize) As Char
        
        ' Intentionally make the input byte buffer larger than the output character buffer so the 
        ' conversion loop executes more than one cycle. 
        Dim bytes(UseBufferSize * 4) As Byte
        Dim bytesRead As Integer
        Do
            ' Read at most the number of bytes that will fit in the input buffer. The 
            ' return value is the actual number of bytes read, or zero if no bytes remain. 
            bytesRead = inputFile.Read(bytes, 0, UseBufferSize * 4)
            
            Dim completed As Boolean = False
            Dim byteIndex As Integer = 0
            Dim bytesUsed As Integer
            Dim charsUsed As Integer
            
            While Not completed
                ' If this is the last input data, flush the decoder's internal buffer and state.
                Dim flush As Boolean = bytesRead = 0
                decoder.Convert(bytes, byteIndex, bytesRead - byteIndex, chars, 0, UseBufferSize, flush, bytesUsed, charsUsed, completed)
                
                ' The conversion produced the number of characters indicated by charsUsed. Write that number
                ' of characters to the output file.
                outputFile.Write(chars, 0, charsUsed)
                
                ' Increment byteIndex to the next block of bytes in the input buffer, if any, to convert.
                byteIndex += bytesUsed
            End While
        Loop While bytesRead <> 0
        
        outputFile.Close()
        fs.Close()
        inputFile.Close()
    
    End Sub
    
    ' --------------------------------------------------------------------------------------------
    ' Create a large file of UTF-16 encoded Unicode characters. 
    ' --------------------------------------------------------------------------------------------
    Shared Sub CreateTestFile(ByVal FileName As String) 
        ' StreamWriter defaults to UTF-8 encoding so explicitly specify Unicode, that is, 
        ' UTF-16, encoding.
        Dim file As New StreamWriter(FileName, False, Encoding.Unicode)
        
        ' Write a line of text 100 times.
        Dim i As Integer
        For i = 0 To 99
            file.WriteLine("This is an example input file used by the convert example.")
        Next i
        
        ' Write Unicode characters from U+0000 to, but not including, the surrogate character range.
        Dim c As Integer
        For c = 0 To &HD800
            file.Write(ChrW(c))
        Next c
        file.Close()
    
    End Sub
End Class

'
'This code example produces the following results:
'
'(Execute the -dir- console window command and examine the files created.)
'
'Example.txt, which contains 122,594 bytes (61,297 UTF-16 encoded characters).
'UTF8.txt, which contains 169,712 UTF-8 encoded bytes.
'UTF16.txt, which contains 122,594 bytes (61,297 UTF-16 encoded characters).
'
'(Execute the -comp- console window command and compare the two Unicode files.)
'
'>comp example.txt utf16.txt /L
'Comparing example.txt and utf16.txt...
'Files compare OK
'
'(The two files are equal.)
'

Remarks

The completed output parameter indicates whether all the data in the input buffer was converted and stored in the output buffer. This parameter is set to false if the number of bytes specified by the byteCount parameter cannot be converted without exceeding the number of characters specified by the charCount parameter. In that situation, the application should use the contents of the output buffer or provide a new output buffer, increment the bytes parameter by the number of bytes specified by the bytesUsed parameter, then call the Convert method again to process the remaining input.

The completed parameter can also be set to false, even though the bytesUsed and byteCount parameters are equal. This situation occurs if there is still data in the Decoder object that has not been stored in the bytes buffer.