TextCatalog.NormalizeText Methode

Definition

Namespace:

Microsoft.ML

Assembly:

Microsoft.ML.Transforms.dll

Paket:

Microsoft.ML v5.0.0-preview.1.25125.4

Quelle:

TextCatalog.cs

Erstellt einen TextNormalizingEstimator, der eingehenden Text normalisiert, indem optional die Groß-/Kleinschreibung inputColumnName geändert wird, wobei diakritische Markierungen, Satzzeichen, Zahlen und ausgabe neuer Text als outputColumnName.

public static Microsoft.ML.Transforms.Text.TextNormalizingEstimator NormalizeText(this Microsoft.ML.TransformsCatalog.TextTransforms catalog, string outputColumnName, string inputColumnName = default, Microsoft.ML.Transforms.Text.TextNormalizingEstimator.CaseMode caseMode = Microsoft.ML.Transforms.Text.TextNormalizingEstimator+CaseMode.Lower, bool keepDiacritics = false, bool keepPunctuations = true, bool keepNumbers = true);

static member NormalizeText : Microsoft.ML.TransformsCatalog.TextTransforms * string * string * Microsoft.ML.Transforms.Text.TextNormalizingEstimator.CaseMode * bool * bool * bool -> Microsoft.ML.Transforms.Text.TextNormalizingEstimator

<Extension()>
Public Function NormalizeText (catalog As TransformsCatalog.TextTransforms, outputColumnName As String, Optional inputColumnName As String = Nothing, Optional caseMode As TextNormalizingEstimator.CaseMode = Microsoft.ML.Transforms.Text.TextNormalizingEstimator+CaseMode.Lower, Optional keepDiacritics As Boolean = false, Optional keepPunctuations As Boolean = true, Optional keepNumbers As Boolean = true) As TextNormalizingEstimator

Parameter

catalog

TransformsCatalog.TextTransforms

Der Katalog der textbezogenen Transformation.

outputColumnName

String

Name der Spalte, die sich aus der Transformation von inputColumnName. Der Datentyp dieser Spalte ist je nach Datentyp der Eingabespalte ein Skalar oder ein Textvektor.

inputColumnName

String

Name der zu transformierenden Spalte. Bei Festlegung auf null, wird der Wert des Werts outputColumnName als Quelle verwendet. Dieser Schätzer arbeitet auf Text oder Vektor von Textdatentypen.

caseMode

TextNormalizingEstimator.CaseMode

Großschreibung von Text mithilfe der Regeln der invarianten Kultur.

keepDiacritics

Boolean

Gibt an, ob diakritische Markierungen beibehalten oder entfernt werden sollen.

keepPunctuations

Boolean

Gibt an, ob Satzzeichen beibehalten oder entfernt werden sollen.

keepNumbers

Boolean

Gibt an, ob Zahlen beibehalten oder entfernt werden sollen.

Gibt zurück

TextNormalizingEstimator

Beispiele

using System;
using System.Collections.Generic;
using Microsoft.ML;
using Microsoft.ML.Transforms.Text;

namespace Samples.Dynamic
{
    public static class NormalizeText
    {
        public static void Example()
        {
            // Create a new ML context, for ML.NET operations. It can be used for
            // exception tracking and logging, as well as the source of randomness.
            var mlContext = new MLContext();

            // Create an empty list as the dataset. The 'NormalizeText' API does not
            // require training data as the estimator ('TextNormalizingEstimator')
            // created by 'NormalizeText' API is not a trainable estimator. The
            // empty list is only needed to pass input schema to the pipeline.
            var emptySamples = new List<TextData>();

            // Convert sample list to an empty IDataView.
            var emptyDataView = mlContext.Data.LoadFromEnumerable(emptySamples);

            // A pipeline for normalizing text.
            var normTextPipeline = mlContext.Transforms.Text.NormalizeText(
                "NormalizedText", "Text", TextNormalizingEstimator.CaseMode.Lower,
                keepDiacritics: false,
                keepPunctuations: false,
                keepNumbers: false);

            // Fit to data.
            var normTextTransformer = normTextPipeline.Fit(emptyDataView);

            // Create the prediction engine to get the normalized text from the
            // input text/string.
            var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData,
                TransformedTextData>(normTextTransformer);

            // Call the prediction API.
            var data = new TextData()
            {
                Text = "ML.NET's NormalizeText API " +
                "changes the case of the TEXT and removes/keeps diâcrîtîcs, " +
                "punctuations, and/or numbers (123)."
            };

            var prediction = predictionEngine.Predict(data);

            // Print the normalized text.
            Console.WriteLine($"Normalized Text: {prediction.NormalizedText}");

            //  Expected output:
            //   Normalized Text: mlnets normalizetext api changes the case of the text and removeskeeps diacritics punctuations andor numbers
        }

        private class TextData
        {
            public string Text { get; set; }
        }

        private class TransformedTextData : TextData
        {
            public string NormalizedText { get; set; }
        }
    }
}