Share via


WordsSegmenter.GetTokens(String) Method

Definition

Determines and returns all of the words or word stems in the provided text.

public:
 virtual IVectorView<WordSegment ^> ^ GetTokens(Platform::String ^ text) = GetTokens;
IVectorView<WordSegment> GetTokens(winrt::hstring const& text);
public IReadOnlyList<WordSegment> GetTokens(string text);
function getTokens(text)
Public Function GetTokens (text As String) As IReadOnlyList(Of WordSegment)

Parameters

text
String

Platform::String

winrt::hstring

Provided text containing words or word stems to be returned.

Returns

A collection of WordSegment objects that represent the words or word stems.

Remarks

Note that some languages do not use spaces (such as Japanese or Chinese) and some languages may return multiple word stems for compound words (such as German).

Applies to