WordsSegmenter.GetTokens(String) Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Determines and returns all of the words or word stems in the provided text.
public:
virtual IVectorView<WordSegment ^> ^ GetTokens(Platform::String ^ text) = GetTokens;IVectorView<WordSegment> GetTokens(winrt::hstring const& text);public IReadOnlyList<WordSegment> GetTokens(string text);function getTokens(text)Public Function GetTokens (text As String) As IReadOnlyList(Of WordSegment)Parameters
- text
-
String
Platform::String
winrt::hstring
Provided text containing words or word stems to be returned.
Returns
A collection of WordSegment objects that represent the words or word stems.
Remarks
Note that some languages do not use spaces (such as Japanese or Chinese) and some languages may return multiple word stems for compound words (such as German).