BpeDecoder.Decode(IEnumerable<String>) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Decode the original BPE by joining all the tokens and then replacing the suffix used to identify end-of-words by white spaces.
public override string Decode (System.Collections.Generic.IEnumerable<string> tokens);override this.Decode : seq<string> -> stringPublic Overrides Function Decode (tokens As IEnumerable(Of String)) As StringParameters
- tokens
- IEnumerable<String>
The list of tokens to merge.
Returns
The string containing all merged tokens.