Base64.DecodeFromUtf8 methods ignore whitespace

Article
06/13/2023

The Convert.FromBase64String(String), Convert.FromBase64CharArray(Char[], Int32, Int32), and corresponding Try methods on System.Convert ignore the ASCII whitespace characters ' ', '\t', '\r', and '\n' and allow any amount of such whitespace to be in the input. However, when the Base64.DecodeFromUtf8(ReadOnlySpan<Byte>, Span<Byte>, Int32, Int32, Boolean) and Base64.DecodeFromUtf8InPlace(Span<Byte>, Int32) methods were added, they didn't ignore these whitespace characters and instead failed to decode any input that included whitespace. That made the behavior of the UTF16-based APIs different from that of the UTF8-based APIs. It also meant that:

The Base64.DecodeFromUtf8 and Base64.DecodeFromUtf8InPlace methods couldn't roundtrip the UTF-encoded base-64 encoded data produced by Convert.FromBase64String(String) with the Base64FormattingOptions.InsertLineBreaks option.
The new IsValid(ReadOnlySpan<Char>) and IsValid(ReadOnlySpan<Byte>) methods would either need to have behavior inconsistent with each other or with their corresponding methods for UTF-16 and UTF-8 data on Convert and Base64.

With this change, the DecodeFromUtf8(ReadOnlySpan<Byte>, Span<Byte>, Int32, Int32, Boolean) and DecodeFromUtf8InPlace(Span<Byte>, Int32) methods now ignore whitespace in the input.

Previous behavior

Base64.DecodeFromUtf8(ReadOnlySpan<Byte>, Span<Byte>, Int32, Int32, Boolean) and Base64.DecodeFromUtf8InPlace(Span<Byte>, Int32) failed to process input that contained whitespace and returned OperationStatus.InvalidData if any whitespace was encountered.

New behavior

Base64.DecodeFromUtf8(ReadOnlySpan<Byte>, Span<Byte>, Int32, Int32, Boolean) and Base64.DecodeFromUtf8InPlace(Span<Byte>, Int32) now ignore whitespace (specifically ' ', '\t', '\r', and '\n') in the input, which matches the behavior of Convert.FromBase64String(String).

Version introduced

.NET 8 Preview 5

Type of breaking change

This change is a behavioral change.

Reason for change

The change was made so that:

The Base64 methods can decode a wider range of input data, including:
- Data produced by Convert.ToBase64String with the Base64FormattingOptions.InsertLineBreaks option.
- Common formatting of data in configuration files and other real data sources.
The Base64 methods are consistent with the corresponding decoding APIs on Convert.
The new Base64.IsValid(ReadOnlySpan<Char>) and Base64.IsValid(ReadOnlySpan<Byte>) APIs could be added in a manner where their behavior is consistent with each other and with the existing Convert and Base64 APIs.

Recommended action

If the new behavior is problematic for your code, you can call IndexOfAny(" \t\r\n"u8) to search the input for the whitespace that previously would have triggered an InvalidData result.