Table of Contents

EncodingExtensions Class

Definition

Namespace
Bodu.Text
Assembly
Bodu.Core.dll
Package
Bodu.Core 1.0.1
Source
EncodingExtensions.Chunked.cs

Provides span-friendly, allocation-aware, and pool-aware extension methods on Encoding, ReadOnlySpan<T> of char and byte, and on Encoder / Decoder.

public static class EncodingExtensions
Inheritance
EncodingExtensions
Inherited Members

Examples

// Encode a chunk of UTF-16 text to UTF-8 bytes via the span receiver.
ReadOnlySpan<char> chars = "héllo".AsSpan();
byte[] utf8 = chars.ToUtf8Bytes();

// Round-trip via the encoding receiver, using the pooled builder for zero unnecessary allocations.
using PooledBufferBuilder<byte> pooled = System.Text.Encoding.UTF8.GetBytesPooled(chars);
ReadOnlySpan<byte> encoded = pooled.WrittenSpan;

// Detect a UTF-8 byte-order-mark and decode the remainder.
if (System.Text.Encoding.UTF8.StartsWithPreamble(encoded))
    encoded = System.Text.Encoding.UTF8.StripPreamble(encoded);

string decoded = encoded.DecodeToString(System.Text.Encoding.UTF8);

Remarks

The extensions are split into focused partial files by concern (encode, decode, UTF-8 fast paths, transcoding, fallback handling, preamble / BOM, pooled allocation, IBufferWriter<T> writes, classification, and chunked encoder/decoder operations). Two receiver shapes coexist:

  • ReadOnlySpan<char> / ReadOnlySpan<byte> receivers express the data-first form - discoverable via IntelliSense from a span variable.
  • Encoding receivers express the encoding-first form - useful when an encoding instance is the obvious starting point (for example, encoder-fluent fallback configuration).

Pooled allocation surfaces are aligned with PooledBufferBuilder<T> so that all pool-returning extensions share a single implementation path and the same lifetime semantics.

Methods

DecodeChunk(Decoder, ReadOnlySpan<byte>, Span<char>, bool, out int, out int)

Decodes a chunk of bytes from source into destination via the stateful decoder and returns an OperationStatus indicating whether the destination was sufficient.

public static OperationStatus DecodeChunk(this Decoder decoder, ReadOnlySpan<byte> source, Span<char> destination, bool flush, out int bytesConsumed, out int charsWritten)

Parameters

decoder Decoder

The decoder used to interpret the bytes. Carries cross-chunk state.

source ReadOnlySpan<byte>

The byte span to decode.

destination Span<char>

The destination buffer.

flush bool

true when source is the final chunk and any pending state should be drained; false when more chunks may follow.

bytesConsumed int

The number of bytes from source that were consumed by the decoder.

charsWritten int

The number of characters written into destination.

Returns

OperationStatus

Done when the entire chunk was decoded; DestinationTooSmall when more output remains.

Exceptions

ArgumentNullException

Thrown when decoder is null.

DecoderFallbackException

Thrown when decoder uses DecoderExceptionFallback and the chunk contains a sequence that cannot be decoded.

DecodeExactlyTo(ReadOnlySpan<byte>, Encoding, Span<char>)

Decodes bytes into destination using encoding and asserts that destination is exactly the size required.

public static int DecodeExactlyTo(this ReadOnlySpan<byte> bytes, Encoding encoding, Span<char> destination)

Parameters

bytes ReadOnlySpan<byte>

The byte span to decode.

encoding Encoding

The encoding used to interpret the bytes.

destination Span<char>

The destination buffer. Must be exactly the size required by the encoding.

Returns

int

The number of characters written, which equals destination.Length on success.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is not exactly the character count required by encoding for bytes.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

DecodeTo(ReadOnlySpan<byte>, Encoding, Span<char>)

Decodes bytes into destination using encoding and returns the number of characters written.

public static int DecodeTo(this ReadOnlySpan<byte> bytes, Encoding encoding, Span<char> destination)

Parameters

bytes ReadOnlySpan<byte>

The byte span to decode.

encoding Encoding

The encoding used to interpret the bytes.

destination Span<char>

The destination buffer. Must be large enough to hold the decoded output.

Returns

int

The number of characters written to destination.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is too small to hold the decoded output.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

DecodeToString(ReadOnlySpan<byte>, Encoding)

Decodes bytes into a new string using encoding.

public static string DecodeToString(this ReadOnlySpan<byte> bytes, Encoding encoding)

Parameters

bytes ReadOnlySpan<byte>

The byte span to decode.

encoding Encoding

The encoding used to interpret the bytes.

Returns

string

The decoded string.

Remarks

The method is named DecodeToString rather than ToString to avoid shadowing ToString() when invoked through IntelliSense on a span variable.

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

DecodeUtf8To(ReadOnlySpan<byte>, Span<char>)

Decodes bytes as UTF-8 into destination and returns the number of characters written.

public static int DecodeUtf8To(this ReadOnlySpan<byte> bytes, Span<char> destination)

Parameters

bytes ReadOnlySpan<byte>

The UTF-8 encoded byte span.

destination Span<char>

The destination buffer.

Returns

int

The number of characters written to destination.

Exceptions

ArgumentException

Thrown when destination is too small to hold the decoded output.

EncodeChunk(Encoder, ReadOnlySpan<char>, Span<byte>, bool, out int, out int)

Encodes a chunk of characters from source into destination via the stateful encoder and returns an OperationStatus indicating whether the destination was sufficient.

public static OperationStatus EncodeChunk(this Encoder encoder, ReadOnlySpan<char> source, Span<byte> destination, bool flush, out int charsConsumed, out int bytesWritten)

Parameters

encoder Encoder

The encoder used to produce the bytes. Carries cross-chunk state.

source ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer.

flush bool

true when source is the final chunk and any pending state should be drained; false when more chunks may follow.

charsConsumed int

The number of characters from source that were consumed by the encoder.

bytesWritten int

The number of bytes written into destination.

Returns

OperationStatus

Done when the entire chunk was encoded; DestinationTooSmall when more output remains; never returns NeedMoreData or InvalidData from this extension (invalid data surfaces as EncoderFallbackException when the encoder's fallback is the exception fallback).

Examples

// Encode a long string into a buffer writer one chunk at a time.
ReadOnlySpan<char> source = text.AsSpan();
System.Text.Encoder encoder = System.Text.Encoding.UTF8.GetEncoder();

while (!source.IsEmpty)
{
    Span<byte> chunk = writer.GetSpan(sizeHint: 256);
    OperationStatus status = encoder.EncodeChunk(
        source,
        chunk,
        flush: source.Length <= chunk.Length,
        out int charsConsumed,
        out int bytesWritten);

    writer.Advance(bytesWritten);
    source = source.Slice(charsConsumed);
}

Exceptions

ArgumentNullException

Thrown when encoder is null.

EncoderFallbackException

Thrown when encoder uses EncoderExceptionFallback and the chunk contains a code point that cannot be represented.

EncodeExactlyTo(ReadOnlySpan<char>, Encoding, Span<byte>)

Encodes chars into destination using encoding and asserts that destination is exactly the size required.

public static int EncodeExactlyTo(this ReadOnlySpan<char> chars, Encoding encoding, Span<byte> destination)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

encoding Encoding

The encoding used to produce the bytes.

destination Span<byte>

The destination buffer. Must be exactly the size required by the encoding.

Returns

int

The number of bytes written, which equals destination.Length on success.

Remarks

Use this overload when the caller has pre-sized the destination to match the exact encoded length and wants the encode call itself to enforce that contract.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is not exactly the byte count required by encoding for chars.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

EncodeTo(ReadOnlySpan<char>, Encoding, Span<byte>)

Encodes chars into destination using encoding and returns the number of bytes written.

public static int EncodeTo(this ReadOnlySpan<char> chars, Encoding encoding, Span<byte> destination)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

encoding Encoding

The encoding used to produce the bytes.

destination Span<byte>

The destination buffer. Must be large enough to hold the encoded output.

Returns

int

The number of bytes written to destination.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is too small to hold the encoded output.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

EncodeUtf8To(ReadOnlySpan<char>, Span<byte>)

Encodes chars as UTF-8 into destination and returns the number of bytes written.

public static int EncodeUtf8To(this ReadOnlySpan<char> chars, Span<byte> destination)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer. Must be large enough to hold the UTF-8 encoded output.

Returns

int

The number of bytes written to destination.

Exceptions

ArgumentException

Thrown when destination is too small to hold the UTF-8 encoded output.

FromUtf8(ReadOnlySpan<byte>)

Decodes bytes as UTF-8 into a new string.

public static string FromUtf8(this ReadOnlySpan<byte> bytes)

Parameters

bytes ReadOnlySpan<byte>

The UTF-8 encoded byte span.

Returns

string

The decoded string.

Exceptions

DecoderFallbackException

Thrown when bytes contains an invalid UTF-8 sequence (only when the default UTF8 instance has been configured with exception fallbacks).

GetByteCountWithPreamble(Encoding, ReadOnlySpan<char>)

Returns the exact number of bytes required to encode chars together with encoding's preamble.

public static int GetByteCountWithPreamble(this Encoding encoding, ReadOnlySpan<char> chars)

Parameters

encoding Encoding

The encoding used to compute the byte count.

chars ReadOnlySpan<char>

The character span to measure.

Returns

int

The preamble length plus the encoded byte count for chars.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetBytesExactly(Encoding, ReadOnlySpan<char>, Span<byte>)

Encodes chars into destination using encoding and asserts that destination is exactly the size required.

public static int GetBytesExactly(this Encoding encoding, ReadOnlySpan<char> chars, Span<byte> destination)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer. Must be exactly the size required by the encoding.

Returns

int

The number of bytes written, which equals destination.Length on success.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is not exactly the byte count required by encoding for chars.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetBytesOwner(Encoding, ReadOnlySpan<char>, MemoryPool<byte>?)

Encodes chars into a pool-backed IMemoryOwner<T> using encoding and the supplied memoryPool.

public static IMemoryOwner<byte> GetBytesOwner(this Encoding encoding, ReadOnlySpan<char> chars, MemoryPool<byte>? memoryPool = null)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

memoryPool MemoryPool<byte>

The pool to rent from. Defaults to Shared when null.

Returns

IMemoryOwner<byte>

An IMemoryOwner<T> whose Memory length equals exactly the number of encoded bytes. Dispose to return the rented buffer to the pool.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetBytesPooled(Encoding, ReadOnlySpan<char>)

Encodes chars into a PooledBufferBuilder<T> using encoding.

public static PooledBufferBuilder<byte> GetBytesPooled(this Encoding encoding, ReadOnlySpan<char> chars)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

Returns

PooledBufferBuilder<byte>

A PooledBufferBuilder<T> with WrittenCount equal to the exact number of encoded bytes. The builder is both an IBufferWriter<T> and an IMemoryOwner<T>; dispose to return the rented buffer to the Shared.

Examples

// Encode into a pooled buffer, hand the written span to a downstream consumer, then dispose to
// return the rented array.
using PooledBufferBuilder<byte> pooled = System.Text.Encoding.UTF8.GetBytesPooled("hello");
ReadOnlySpan<byte> bytes = pooled.WrittenSpan;
downstream.Process(bytes);
// Disposed at the end of the using scope - the underlying byte[] returns to ArrayPool<byte>.Shared.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetBytesRented(Encoding, ReadOnlySpan<char>, out int)

Encodes chars into a buffer rented from Shared using encoding and reports the exact byte count via bytesWritten.

public static byte[] GetBytesRented(this Encoding encoding, ReadOnlySpan<char> chars, out int bytesWritten)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

bytesWritten int

The number of bytes written into the returned array.

Returns

byte[]

A rented byte array that may be larger than bytesWritten. The caller is responsible for returning the array to Shared via Return(T[], bool).

Remarks

Prefer GetBytesPooled(Encoding, ReadOnlySpan<char>) or GetBytesOwner(Encoding, ReadOnlySpan<char>, MemoryPool<byte>?) over this method - both wrap the rented buffer in an IDisposable so a leak cannot occur when the caller uses a using block. This method is provided for interop with APIs that require a bare byte array.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetBytesWithPreamble(Encoding, ReadOnlySpan<char>)

Encodes chars into a freshly allocated byte array preceded by encoding's preamble.

public static byte[] GetBytesWithPreamble(this Encoding encoding, ReadOnlySpan<char> chars)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

Returns

byte[]

A new byte array containing the preamble followed by the encoded bytes.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetBytesWithPreamble(Encoding, ReadOnlySpan<char>, Span<byte>)

Encodes chars into destination preceded by encoding 's preamble and returns the total number of bytes written.

public static int GetBytesWithPreamble(this Encoding encoding, ReadOnlySpan<char> chars, Span<byte> destination)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer. Must be large enough to hold preamble plus encoded output.

Returns

int

The preamble length plus the encoded byte count.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is too small to hold the preamble and encoded bytes.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

GetCharsExactly(Encoding, ReadOnlySpan<byte>, Span<char>)

Decodes bytes into destination using encoding and asserts that destination is exactly the size required.

public static int GetCharsExactly(this Encoding encoding, ReadOnlySpan<byte> bytes, Span<char> destination)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

destination Span<char>

The destination buffer. Must be exactly the size required by the encoding.

Returns

int

The number of characters written, which equals destination.Length on success.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination is not exactly the character count required by encoding for bytes.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

GetCharsOwner(Encoding, ReadOnlySpan<byte>, MemoryPool<char>?)

Decodes bytes into a pool-backed IMemoryOwner<T> using encoding and the supplied memoryPool.

public static IMemoryOwner<char> GetCharsOwner(this Encoding encoding, ReadOnlySpan<byte> bytes, MemoryPool<char>? memoryPool = null)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

memoryPool MemoryPool<char>

The pool to rent from. Defaults to Shared when null.

Returns

IMemoryOwner<char>

An IMemoryOwner<T> whose Memory length equals exactly the number of decoded characters. Dispose to return the rented buffer to the pool.

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

GetCharsPooled(Encoding, ReadOnlySpan<byte>)

Decodes bytes into a PooledBufferBuilder<T> using encoding.

public static PooledBufferBuilder<char> GetCharsPooled(this Encoding encoding, ReadOnlySpan<byte> bytes)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

Returns

PooledBufferBuilder<char>

A PooledBufferBuilder<T> with WrittenCount equal to the exact number of decoded characters. The builder is both an IBufferWriter<T> and an IMemoryOwner<T>; dispose to return the rented buffer to the Shared.

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

GetCharsRented(Encoding, ReadOnlySpan<byte>, out int)

Decodes bytes into a buffer rented from Shared using encoding and reports the exact character count via charsWritten.

public static char[] GetCharsRented(this Encoding encoding, ReadOnlySpan<byte> bytes, out int charsWritten)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

charsWritten int

The number of characters written into the returned array.

Returns

char[]

A rented character array that may be larger than charsWritten. The caller is responsible for returning the array to Shared via Return(T[], bool).

Remarks

Prefer GetCharsPooled(Encoding, ReadOnlySpan<byte>) or GetCharsOwner(Encoding, ReadOnlySpan<byte>, MemoryPool<char>?) over this method - both wrap the rented buffer in an IDisposable so a leak cannot occur when the caller uses a using block. This method is provided for interop with APIs that require a bare char array.

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

GetDecodedCharCount(ReadOnlySpan<byte>, Encoding)

Returns the exact number of characters produced by decoding bytes with encoding.

public static int GetDecodedCharCount(this ReadOnlySpan<byte> bytes, Encoding encoding)

Parameters

bytes ReadOnlySpan<byte>

The byte span to measure.

encoding Encoding

The encoding used to compute the character count.

Returns

int

The exact number of characters required to decode bytes.

Remarks

This routes through GetCharCount(ReadOnlySpan<byte>) and walks the input to produce the exact value, unlike GetMaxCharCount(int) which returns a worst-case upper bound.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetDisplayName(Encoding)

Returns a human-readable display name for encoding that distinguishes BOM-emitting UTF variants from their plain counterparts (for example, "UTF-8-BOM" versus "UTF-8").

public static string GetDisplayName(this Encoding encoding)

Parameters

encoding Encoding

The encoding to describe.

Returns

string

A short, stable name. UTF variants return the canonical short form with a -BOM suffix when the encoding emits a preamble; other encodings fall back to WebName.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetEncodedByteCount(ReadOnlySpan<char>, Encoding)

Returns the exact number of bytes produced by encoding chars with encoding.

public static int GetEncodedByteCount(this ReadOnlySpan<char> chars, Encoding encoding)

Parameters

chars ReadOnlySpan<char>

The character span to measure.

encoding Encoding

The encoding used to compute the byte count.

Returns

int

The exact number of bytes required to encode chars.

Remarks

This routes through GetByteCount(ReadOnlySpan<char>) and walks the input to produce the exact value, unlike GetMaxByteCount(int) which returns a worst-case upper bound.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetPreambleLength(Encoding)

Returns the length in bytes of encoding's preamble.

public static int GetPreambleLength(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

int

The preamble length in bytes; zero when the encoding has no preamble.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetStringSkippingPreamble(Encoding, ReadOnlySpan<byte>)

Decodes bytes after skipping encoding's preamble.

public static string GetStringSkippingPreamble(this Encoding encoding, ReadOnlySpan<byte> bytes)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

Returns

string

The decoded string with any matching preamble removed before decoding.

Exceptions

ArgumentNullException

Thrown when encoding is null.

GetTranscodedByteCount(Encoding, ReadOnlySpan<byte>, Encoding)

Returns the exact number of bytes that source would produce when re-encoded from sourceEncoding into destinationEncoding.

public static int GetTranscodedByteCount(this Encoding sourceEncoding, ReadOnlySpan<byte> source, Encoding destinationEncoding)

Parameters

sourceEncoding Encoding

The encoding of source.

source ReadOnlySpan<byte>

The encoded byte span to measure.

destinationEncoding Encoding

The encoding the bytes would be re-encoded into.

Returns

int

The exact transcoded byte count.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

DecoderFallbackException

Thrown when sourceEncoding uses DecoderExceptionFallback and source contains a sequence that cannot be decoded.

GetUtf8ByteCount(ReadOnlySpan<char>)

Returns the exact number of UTF-8 bytes required to encode chars.

public static int GetUtf8ByteCount(this ReadOnlySpan<char> chars)

Parameters

chars ReadOnlySpan<char>

The character span to measure.

Returns

int

The exact UTF-8 byte count.

HasPreamble(Encoding)

Returns a value indicating whether encoding emits a byte-order-mark preamble.

public static bool HasPreamble(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when Preamble is non-empty; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsAnyUtf(Encoding)

Returns a value indicating whether encoding is any UTF encoding (UTF-8, UTF-16 LE, UTF-16 BE, UTF-32 LE, or UTF-32 BE).

public static bool IsAnyUtf(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is one of 65001, 1200, 1201, 12000, or 12001; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsAscii(Encoding)

Returns a value indicating whether encoding is US-ASCII (Windows codepage 20127).

public static bool IsAscii(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 20127; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsUtf16BigEndian(Encoding)

Returns a value indicating whether encoding is UTF-16 big endian (Windows codepage 1201, the value of BigEndianUnicode).

public static bool IsUtf16BigEndian(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 1201; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsUtf16LittleEndian(Encoding)

Returns a value indicating whether encoding is UTF-16 little endian (Windows codepage 1200, the value of Unicode).

public static bool IsUtf16LittleEndian(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 1200; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsUtf32BigEndian(Encoding)

Returns a value indicating whether encoding is UTF-32 big endian (Windows codepage 12001).

public static bool IsUtf32BigEndian(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 12001; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsUtf32LittleEndian(Encoding)

Returns a value indicating whether encoding is UTF-32 little endian (Windows codepage 12000, the value of UTF32).

public static bool IsUtf32LittleEndian(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 12000; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

IsUtf8(Encoding)

Returns a value indicating whether encoding is UTF-8 (Windows codepage 65001).

public static bool IsUtf8(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when the codepage is 65001; otherwise false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

StartsWithPreamble(Encoding, ReadOnlySpan<byte>)

Returns a value indicating whether bytes begins with the preamble of encoding.

public static bool StartsWithPreamble(this Encoding encoding, ReadOnlySpan<byte> bytes)

Parameters

encoding Encoding

The encoding whose preamble is compared.

bytes ReadOnlySpan<byte>

The byte span to inspect.

Returns

bool

true when bytes starts with the preamble; false when the encoding has no preamble or when the bytes do not match.

Exceptions

ArgumentNullException

Thrown when encoding is null.

StripPreamble(Encoding, ReadOnlySpan<byte>)

Returns bytes with encoding's preamble removed if present.

public static ReadOnlySpan<byte> StripPreamble(this Encoding encoding, ReadOnlySpan<byte> bytes)

Parameters

encoding Encoding

The encoding whose preamble is stripped.

bytes ReadOnlySpan<byte>

The byte span to inspect.

Returns

ReadOnlySpan<byte>

A ReadOnlySpan<T> equal to bytes when no preamble is present, or the sub-span starting after the preamble when one is matched at the start.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ThrowIfTooSmallForDecoding(Span<char>, ReadOnlySpan<byte>, Encoding, string?)

Throws an ArgumentException when destination is too small to receive the characters produced by decoding bytes with encoding.

public static void ThrowIfTooSmallForDecoding(this Span<char> destination, ReadOnlySpan<byte> bytes, Encoding encoding, string? paramName = null)

Parameters

destination Span<char>

The destination span to validate.

bytes ReadOnlySpan<byte>

The byte span whose decoded length is computed.

encoding Encoding

The encoding used to compute the required size.

paramName string

The parameter name reported in the exception; inferred from the call site when not specified.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination.Length is less than GetCharCount(ReadOnlySpan<byte>) for bytes.

ThrowIfTooSmallForEncoding(Span<byte>, ReadOnlySpan<char>, Encoding, string?)

Throws an ArgumentException when destination is too small to receive the bytes produced by encoding chars with encoding.

public static void ThrowIfTooSmallForEncoding(this Span<byte> destination, ReadOnlySpan<char> chars, Encoding encoding, string? paramName = null)

Parameters

destination Span<byte>

The destination span to validate.

chars ReadOnlySpan<char>

The character span whose encoded length is computed.

encoding Encoding

The encoding used to compute the required size.

paramName string

The parameter name reported in the exception; inferred from the call site when not specified.

Exceptions

ArgumentNullException

Thrown when encoding is null.

ArgumentException

Thrown when destination.Length is less than GetByteCount(ReadOnlySpan<char>) for chars.

ToBytes(ReadOnlySpan<char>, Encoding)

Encodes chars into a freshly allocated byte array using encoding.

public static byte[] ToBytes(this ReadOnlySpan<char> chars, Encoding encoding)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

encoding Encoding

The encoding used to produce the bytes.

Returns

byte[]

A new byte array whose length is exactly GetEncodedByteCount(ReadOnlySpan<char>, Encoding).

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

ToChars(ReadOnlySpan<byte>, Encoding)

Decodes bytes into a freshly allocated char array using encoding.

public static char[] ToChars(this ReadOnlySpan<byte> bytes, Encoding encoding)

Parameters

bytes ReadOnlySpan<byte>

The byte span to decode.

encoding Encoding

The encoding used to interpret the bytes.

Returns

char[]

A new character array whose length is exactly GetDecodedCharCount(ReadOnlySpan<byte>, Encoding).

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

ToOwnedBytes(ReadOnlySpan<char>, Encoding, MemoryPool<byte>?)

Encodes chars into a pool-backed IMemoryOwner<T> using encoding and the supplied memoryPool.

public static IMemoryOwner<byte> ToOwnedBytes(this ReadOnlySpan<char> chars, Encoding encoding, MemoryPool<byte>? memoryPool = null)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

encoding Encoding

The encoding used to produce the bytes.

memoryPool MemoryPool<byte>

The pool to rent from. Defaults to Shared when null.

Returns

IMemoryOwner<byte>

An IMemoryOwner<T> whose Memory length equals exactly the number of encoded bytes. Dispose to return the rented buffer to the pool.

Remarks

Prefer the Encoding-receiver variant GetBytesPooled when the caller already needs the concrete PooledBufferBuilder<T> capabilities ( WrittenSpan, IBufferWriter<T> integration). This method returns the same buffer behind the IMemoryOwner<T> abstraction.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

ToUtf8Bytes(ReadOnlySpan<char>)

Encodes chars to a freshly allocated UTF-8 byte array.

public static byte[] ToUtf8Bytes(this ReadOnlySpan<char> chars)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

Returns

byte[]

A new byte array containing the UTF-8 representation of chars.

Remarks

Equivalent to chars.ToBytes(Encoding.UTF8); provided as a discoverable shortcut and to avoid the Encoding indirection at the call site for the very common UTF-8 case.

Transcode(ReadOnlySpan<byte>, Encoding, Encoding)

Transcodes source from sourceEncoding to destinationEncoding and returns the result as a freshly allocated byte array.

public static byte[] Transcode(this ReadOnlySpan<byte> source, Encoding sourceEncoding, Encoding destinationEncoding)

Parameters

source ReadOnlySpan<byte>

The encoded byte span to transcode.

sourceEncoding Encoding

The encoding of source.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

Returns

byte[]

A new byte array containing the transcoded representation.

Remarks

The transcoding pipeline routes through an intermediate char buffer rented from Shared so that no char array is allocated when the input is non-trivial.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

DecoderFallbackException

Thrown when sourceEncoding uses DecoderExceptionFallback and source contains a sequence that cannot be decoded.

EncoderFallbackException

Thrown when destinationEncoding uses EncoderExceptionFallback and the intermediate characters contain a code point that cannot be re-encoded.

Transcode(Encoding, ReadOnlySpan<byte>, Encoding)

Transcodes source from sourceEncoding to destinationEncoding and returns the result as a freshly allocated byte array.

public static byte[] Transcode(this Encoding sourceEncoding, ReadOnlySpan<byte> source, Encoding destinationEncoding)

Parameters

sourceEncoding Encoding

The encoding of source.

source ReadOnlySpan<byte>

The encoded byte span to transcode.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

Returns

byte[]

A new byte array containing the transcoded representation.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

DecoderFallbackException

Thrown when sourceEncoding uses DecoderExceptionFallback and source contains a sequence that cannot be decoded.

EncoderFallbackException

Thrown when destinationEncoding uses EncoderExceptionFallback and the intermediate characters contain a code point that cannot be re-encoded.

TranscodeTo(ReadOnlySpan<byte>, Encoding, Encoding, Span<byte>)

Transcodes source from sourceEncoding to destinationEncoding into destination and returns the byte count.

public static int TranscodeTo(this ReadOnlySpan<byte> source, Encoding sourceEncoding, Encoding destinationEncoding, Span<byte> destination)

Parameters

source ReadOnlySpan<byte>

The encoded byte span to transcode.

sourceEncoding Encoding

The encoding of source.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

destination Span<byte>

The destination buffer.

Returns

int

The number of bytes written to destination.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

ArgumentException

Thrown when destination is too small to receive the transcoded bytes.

DecoderFallbackException

Thrown when sourceEncoding uses DecoderExceptionFallback and source contains a sequence that cannot be decoded.

EncoderFallbackException

Thrown when destinationEncoding uses EncoderExceptionFallback and the intermediate characters contain a code point that cannot be re-encoded.

TranscodeTo(Encoding, ReadOnlySpan<byte>, Encoding, Span<byte>)

Transcodes source from sourceEncoding to destinationEncoding into destination and returns the byte count.

public static int TranscodeTo(this Encoding sourceEncoding, ReadOnlySpan<byte> source, Encoding destinationEncoding, Span<byte> destination)

Parameters

sourceEncoding Encoding

The encoding of source.

source ReadOnlySpan<byte>

The encoded byte span to transcode.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

destination Span<byte>

The destination buffer.

Returns

int

The number of bytes written to destination.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

ArgumentException

Thrown when destination is too small to receive the transcoded bytes.

TryDecodeTo(ReadOnlySpan<byte>, Encoding, Span<char>, out int)

Attempts to decode bytes into destination using encoding without throwing when the destination is too small.

public static bool TryDecodeTo(this ReadOnlySpan<byte> bytes, Encoding encoding, Span<char> destination, out int charsWritten)

Parameters

bytes ReadOnlySpan<byte>

The byte span to decode.

encoding Encoding

The encoding used to interpret the bytes.

destination Span<char>

The destination buffer.

charsWritten int

When this method returns true, contains the number of characters written; otherwise zero.

Returns

bool

true if the decoding completed successfully; false when destination is too small.

Exceptions

ArgumentNullException

Thrown when encoding is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

TryDecodeUtf8To(ReadOnlySpan<byte>, Span<char>, out int)

Attempts to decode bytes as UTF-8 into destination without throwing when the destination is too small.

public static bool TryDecodeUtf8To(this ReadOnlySpan<byte> bytes, Span<char> destination, out int charsWritten)

Parameters

bytes ReadOnlySpan<byte>

The UTF-8 encoded byte span.

destination Span<char>

The destination buffer.

charsWritten int

When this method returns true, contains the number of characters written; otherwise zero.

Returns

bool

true if the decoding completed successfully; false when destination is too small.

TryEncodeTo(ReadOnlySpan<char>, Encoding, Span<byte>, out int)

Attempts to encode chars into destination using encoding without throwing when the destination is too small.

public static bool TryEncodeTo(this ReadOnlySpan<char> chars, Encoding encoding, Span<byte> destination, out int bytesWritten)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

encoding Encoding

The encoding used to produce the bytes.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the number of bytes written; otherwise zero.

Returns

bool

true if the encoding completed successfully; false when destination is too small.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

TryEncodeUtf8To(ReadOnlySpan<char>, Span<byte>, out int)

Attempts to encode chars as UTF-8 into destination without throwing when the destination is too small.

public static bool TryEncodeUtf8To(this ReadOnlySpan<char> chars, Span<byte> destination, out int bytesWritten)

Parameters

chars ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the number of bytes written; otherwise zero.

Returns

bool

true if the encoding completed successfully; false when destination is too small.

TryGetBytesWithPreamble(Encoding, ReadOnlySpan<char>, Span<byte>, out int)

Attempts to encode chars into destination preceded by encoding's preamble without throwing when the destination is too small.

public static bool TryGetBytesWithPreamble(this Encoding encoding, ReadOnlySpan<char> chars, Span<byte> destination, out int bytesWritten)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the total number of bytes written (preamble plus encoded output); otherwise zero.

Returns

bool

true when the preamble and encoded bytes fit; false when destination is too small.

Exceptions

ArgumentNullException

Thrown when encoding is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

TryTranscodeTo(ReadOnlySpan<byte>, Encoding, Encoding, Span<byte>, out int)

Attempts to transcode source from sourceEncoding to destinationEncoding into destination without throwing when the destination is too small.

public static bool TryTranscodeTo(this ReadOnlySpan<byte> source, Encoding sourceEncoding, Encoding destinationEncoding, Span<byte> destination, out int bytesWritten)

Parameters

source ReadOnlySpan<byte>

The encoded byte span to transcode.

sourceEncoding Encoding

The encoding of source.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the number of transcoded bytes written; otherwise zero.

Returns

bool

true if the transcode completed successfully; false when destination is too small.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

DecoderFallbackException

Thrown when sourceEncoding uses DecoderExceptionFallback and source contains a sequence that cannot be decoded.

EncoderFallbackException

Thrown when destinationEncoding uses EncoderExceptionFallback and the intermediate characters contain a code point that cannot be re-encoded.

TryTranscodeTo(Encoding, ReadOnlySpan<byte>, Encoding, Span<byte>, out int)

Attempts to transcode source from sourceEncoding to destinationEncoding into destination without throwing when the destination is too small.

public static bool TryTranscodeTo(this Encoding sourceEncoding, ReadOnlySpan<byte> source, Encoding destinationEncoding, Span<byte> destination, out int bytesWritten)

Parameters

sourceEncoding Encoding

The encoding of source.

source ReadOnlySpan<byte>

The encoded byte span to transcode.

destinationEncoding Encoding

The encoding the bytes should be re-encoded into.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the number of transcoded bytes written; otherwise zero.

Returns

bool

true if the transcode completed successfully; false when destination is too small.

Exceptions

ArgumentNullException

Thrown when sourceEncoding or destinationEncoding is null.

TryWritePreamble(Encoding, Span<byte>, out int)

Attempts to write encoding's preamble into destination.

public static bool TryWritePreamble(this Encoding encoding, Span<byte> destination, out int bytesWritten)

Parameters

encoding Encoding

The encoding whose preamble is written.

destination Span<byte>

The destination buffer.

bytesWritten int

When this method returns true, contains the number of preamble bytes written; otherwise zero.

Returns

bool

true if the preamble fit (including the zero-length case when the encoding has no preamble); false when destination is too small.

Exceptions

ArgumentNullException

Thrown when encoding is null.

UsesExceptionFallbacks(Encoding)

Returns a value indicating whether encoding uses EncoderExceptionFallback and DecoderExceptionFallback.

public static bool UsesExceptionFallbacks(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when both the encoder and decoder fallbacks are the BCL exception fallbacks; otherwise false.

Remarks

Only the built-in EncoderExceptionFallback / DecoderExceptionFallback types are recognised. Custom fallback subclasses return false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

UsesReplacementFallbacks(Encoding)

Returns a value indicating whether encoding uses EncoderReplacementFallback and DecoderReplacementFallback.

public static bool UsesReplacementFallbacks(this Encoding encoding)

Parameters

encoding Encoding

The encoding to inspect.

Returns

bool

true when both the encoder and decoder fallbacks are the BCL replacement fallbacks; otherwise false.

Remarks

Only the built-in EncoderReplacementFallback / DecoderReplacementFallback types are recognised. Custom fallback subclasses return false.

Exceptions

ArgumentNullException

Thrown when encoding is null.

WithExceptionFallbacks(Encoding)

Returns a clone of encoding whose encoder and decoder fallbacks both throw EncoderFallbackException / DecoderFallbackException on invalid sequences.

public static Encoding WithExceptionFallbacks(this Encoding encoding)

Parameters

encoding Encoding

The source encoding to clone.

Returns

Encoding

A new Encoding instance configured with EncoderExceptionFallback and DecoderExceptionFallback.

Examples

// Configure a single call-site to fail fast on bad input without mutating the global default.
System.Text.Encoding strictAscii = System.Text.Encoding.ASCII.WithExceptionFallbacks();
try
{
    byte[] bytes = strictAscii.GetBytes(userSuppliedText);
}
catch (System.Text.EncoderFallbackException ex)
{
    // Surface the offending character at ex.Index for diagnostics.
}

Remarks

The returned encoding is a fresh instance - singleton encodings such as UTF8 are not mutated. Use this method to opt a specific call-site into strict validation without altering the global default.

Exceptions

ArgumentNullException

Thrown when encoding is null.

WithReplacementFallbacks(Encoding, string, string)

Returns a clone of encoding whose encoder fallback substitutes encoderReplacement and whose decoder fallback substitutes decoderReplacement for invalid sequences.

public static Encoding WithReplacementFallbacks(this Encoding encoding, string encoderReplacement = "?", string decoderReplacement = "?")

Parameters

encoding Encoding

The source encoding to clone.

encoderReplacement string

The replacement string used when an encoder cannot represent a character. Defaults to "?", which every supported encoding can represent.

decoderReplacement string

The replacement string used when a decoder cannot interpret a sequence. Defaults to "?" rather than the Unicode replacement character (U+FFFD) because non-Unicode encodings - including ASCII - cannot represent U+FFFD.

Returns

Encoding

A new Encoding instance configured with EncoderReplacementFallback and DecoderReplacementFallback.

Exceptions

ArgumentNullException

Thrown when encoding is null, or when either replacement is null.

ArgumentException

Thrown when encoderReplacement cannot be encoded by encoding. The exception's InnerException is the EncoderFallbackException reported by a strict-fallback clone of encoding when asked to encode the replacement.

WriteBytes(Encoding, ReadOnlySpan<char>, IBufferWriter<byte>)

Encodes chars with encoding and writes the bytes into writer.

public static void WriteBytes(this Encoding encoding, ReadOnlySpan<char> chars, IBufferWriter<byte> writer)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

writer IBufferWriter<byte>

The buffer writer to receive the encoded bytes.

Exceptions

ArgumentNullException

Thrown when encoding or writer is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

WriteBytesWithPreamble(Encoding, ReadOnlySpan<char>, IBufferWriter<byte>)

Writes encoding's preamble (BOM) followed by the bytes produced by encoding chars into writer.

public static void WriteBytesWithPreamble(this Encoding encoding, ReadOnlySpan<char> chars, IBufferWriter<byte> writer)

Parameters

encoding Encoding

The encoding used to produce the bytes.

chars ReadOnlySpan<char>

The character span to encode.

writer IBufferWriter<byte>

The buffer writer to receive the preamble and encoded bytes.

Exceptions

ArgumentNullException

Thrown when encoding or writer is null.

EncoderFallbackException

Thrown when encoding uses EncoderExceptionFallback and chars contains a code point that cannot be represented.

WriteChars(Encoding, ReadOnlySpan<byte>, IBufferWriter<char>)

Decodes bytes with encoding and writes the characters into writer.

public static void WriteChars(this Encoding encoding, ReadOnlySpan<byte> bytes, IBufferWriter<char> writer)

Parameters

encoding Encoding

The encoding used to interpret the bytes.

bytes ReadOnlySpan<byte>

The byte span to decode.

writer IBufferWriter<char>

The buffer writer to receive the decoded characters.

Exceptions

ArgumentNullException

Thrown when encoding or writer is null.

DecoderFallbackException

Thrown when encoding uses DecoderExceptionFallback and bytes contains a sequence that cannot be decoded.

WritePreamble(Encoding, IBufferWriter<byte>)

Writes encoding's preamble (BOM) into writer. No bytes are written when the encoding has no preamble.

public static void WritePreamble(this Encoding encoding, IBufferWriter<byte> writer)

Parameters

encoding Encoding

The encoding whose preamble is written.

writer IBufferWriter<byte>

The buffer writer to receive the preamble bytes.

Exceptions

ArgumentNullException

Thrown when encoding or writer is null.

Applies to

ProductVersions
.NET8, 10