BpeDecoder Class
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Allows decoding Original BPE by joining all the tokens and then replacing the suffix used to identify end-of-words by white spaces
public sealed class BpeDecoder : Microsoft.ML.Tokenizers.TokenizerDecodertype BpeDecoder = class
inherit TokenizerDecoderPublic NotInheritable Class BpeDecoder
Inherits TokenizerDecoder- Inheritance
Constructors
| BpeDecoder() |
Construct a new Bpe decoder object. |
| BpeDecoder(String) |
Construct a new Bpe decoder object. |
Methods
| Decode(IEnumerable<String>) |
Decode the original BPE by joining all the tokens and then replacing the suffix used to identify end-of-words by white spaces. |