AzureOpenAITokenizerParameters.AllowedSpecialTokens Property

Definition

Namespace:

Azure.Search.Documents.Indexes.Models

Assembly:

Azure.Search.Documents.dll

Package:

Azure.Search.Documents v11.7.0-beta.1

Source:

AzureOpenAITokenizerParameters.cs

(Optional) Only applies if the unit is set to azureOpenAITokens. This parameter defines a collection of special tokens that are permitted within the tokenization process.

public System.Collections.Generic.IList<string> AllowedSpecialTokens { get; }

member this.AllowedSpecialTokens : System.Collections.Generic.IList<string>

Public ReadOnly Property AllowedSpecialTokens As IList(Of String)

Property Value

IList<String>