Token interface
Represents a token extracted from a text string.
Properties
end | The end index of the token in the original text. |
normalized | The normalized (lowercase) version of the token text. |
start | The start index of the token in the original text. |
text | The original text of the token. |
Property Details
end
The end index of the token in the original text.
end: number
Property Value
number
normalized
The normalized (lowercase) version of the token text.
normalized: string
Property Value
string
start
The start index of the token in the original text.
start: number
Property Value
number
text
The original text of the token.
text: string
Property Value
string