TextChunker.SplitPlainTextParagraphs Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Split plain text into paragraphs.
public static System.Collections.Generic.List<string> SplitPlainTextParagraphs (System.Collections.Generic.IEnumerable<string> lines, int maxTokensPerParagraph, int overlapTokens = 0, string? chunkHeader = default, Microsoft.SemanticKernel.Text.TextChunker.TokenCounter? tokenCounter = default);static member SplitPlainTextParagraphs : seq<string> * int * int * string * Microsoft.SemanticKernel.Text.TextChunker.TokenCounter -> System.Collections.Generic.List<string>Public Shared Function SplitPlainTextParagraphs (lines As IEnumerable(Of String), maxTokensPerParagraph As Integer, Optional overlapTokens As Integer = 0, Optional chunkHeader As String = Nothing, Optional tokenCounter As TextChunker.TokenCounter = Nothing) As List(Of String)Parameters
- lines
- IEnumerable<String>
Lines of text.
- maxTokensPerParagraph
- Int32
Maximum number of tokens per paragraph.
- overlapTokens
- Int32
Number of tokens to overlap between paragraphs.
- chunkHeader
- String
Text to be prepended to each individual chunk.
- tokenCounter
- TextChunker.TokenCounter
Function to count tokens in a string. If not supplied, the default counter will be used.
Returns
List of paragraphs.