LexicalTokenizer type

يحتوي على الحالات المحتملة للرمز المميز.

type LexicalTokenizer =
  | ClassicTokenizer
  | EdgeNGramTokenizer
  | KeywordTokenizer
  | MicrosoftLanguageTokenizer
  | MicrosoftLanguageStemmingTokenizer
  | NGramTokenizer
  | PathHierarchyTokenizer
  | PatternTokenizer
  | LuceneStandardTokenizer
  | UaxUrlEmailTokenizer