LexicalTokenizer type
يحتوي على الحالات المحتملة للرمز المميز.
type LexicalTokenizer =
| ClassicTokenizer
| EdgeNGramTokenizer
| KeywordTokenizer
| MicrosoftLanguageTokenizer
| MicrosoftLanguageStemmingTokenizer
| NGramTokenizer
| PathHierarchyTokenizer
| PatternTokenizer
| LuceneStandardTokenizer
| UaxUrlEmailTokenizer