TextCatalog.FeaturizeText Metoda
Definice
Důležité
Některé informace platí pro předběžně vydaný produkt, který se může zásadně změnit, než ho výrobce nebo autor vydá. Microsoft neposkytuje žádné záruky, výslovné ani předpokládané, týkající se zde uváděných informací.
Přetížení
FeaturizeText(TransformsCatalog+TextTransforms, String, String) |
Vytvořte TextFeaturizingEstimator, která transformuje textový sloupec na featurizovaný vektor Single , který představuje normalizované počty n-gramů a char-gramů. |
FeaturizeText(TransformsCatalog+TextTransforms, String, TextFeaturizingEstimator+Options, String[]) |
Vytvořte TextFeaturizingEstimator, která transformuje textový sloupec na featurizovaný vektor Single , který představuje normalizované počty n-gramů a char-gramů. |
FeaturizeText(TransformsCatalog+TextTransforms, String, String)
Vytvořte TextFeaturizingEstimator, která transformuje textový sloupec na featurizovaný vektor Single , který představuje normalizované počty n-gramů a char-gramů.
public static Microsoft.ML.Transforms.Text.TextFeaturizingEstimator FeaturizeText (this Microsoft.ML.TransformsCatalog.TextTransforms catalog, string outputColumnName, string inputColumnName = default);
static member FeaturizeText : Microsoft.ML.TransformsCatalog.TextTransforms * string * string -> Microsoft.ML.Transforms.Text.TextFeaturizingEstimator
<Extension()>
Public Function FeaturizeText (catalog As TransformsCatalog.TextTransforms, outputColumnName As String, Optional inputColumnName As String = Nothing) As TextFeaturizingEstimator
Parametry
- catalog
- TransformsCatalog.TextTransforms
Katalog transformace související s textem
- outputColumnName
- String
Název sloupce, který je výsledkem transformace inputColumnName
.
Datový typ tohoto sloupce bude vektorem Single.
- inputColumnName
- String
Název sloupce, který se má transformovat. Pokud je nastavená hodnota null
, použije se jako zdroj hodnota outputColumnName
.
Tento estimátor pracuje s textovými daty.
Návraty
Příklady
using System;
using System.Collections.Generic;
using Microsoft.ML;
namespace Samples.Dynamic
{
public static class FeaturizeText
{
public static void Example()
{
// Create a new ML context, for ML.NET operations. It can be used for
// exception tracking and logging, as well as the source of randomness.
var mlContext = new MLContext();
// Create a small dataset as an IEnumerable.
var samples = new List<TextData>()
{
new TextData(){ Text = "ML.NET's FeaturizeText API uses a " +
"composition of several basic transforms to convert text " +
"into numeric features." },
new TextData(){ Text = "This API can be used as a featurizer to " +
"perform text classification." },
new TextData(){ Text = "There are a number of approaches to text " +
"classification." },
new TextData(){ Text = "One of the simplest and most common " +
"approaches is called “Bag of Words”." },
new TextData(){ Text = "Text classification can be used for a " +
"wide variety of tasks" },
new TextData(){ Text = "such as sentiment analysis, topic " +
"detection, intent identification etc." },
};
// Convert training data to IDataView.
var dataview = mlContext.Data.LoadFromEnumerable(samples);
// A pipeline for converting text into numeric features.
// The following call to 'FeaturizeText' instantiates
// 'TextFeaturizingEstimator' with default parameters.
// The default settings for the TextFeaturizingEstimator are
// * StopWordsRemover: None
// * CaseMode: Lowercase
// * OutputTokensColumnName: None
// * KeepDiacritics: false, KeepPunctuations: true, KeepNumbers:
// true
// * WordFeatureExtractor: NgramLength = 1
// * CharFeatureExtractor: NgramLength = 3, UseAllLengths = false
// The length of the output feature vector depends on these settings.
var textPipeline = mlContext.Transforms.Text.FeaturizeText("Features",
"Text");
// Fit to data.
var textTransformer = textPipeline.Fit(dataview);
// Create the prediction engine to get the features extracted from the
// text.
var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData,
TransformedTextData>(textTransformer);
// Convert the text into numeric features.
var prediction = predictionEngine.Predict(samples[0]);
// Print the length of the feature vector.
Console.WriteLine($"Number of Features: {prediction.Features.Length}");
// Print the first 10 feature values.
Console.Write("Features: ");
for (int i = 0; i < 10; i++)
Console.Write($"{prediction.Features[i]:F4} ");
// Expected output:
// Number of Features: 332
// Features: 0.0857 0.0857 0.0857 0.0857 0.0857 0.0857 0.0857 0.0857 0.0857 0.1715 ...
}
private class TextData
{
public string Text { get; set; }
}
private class TransformedTextData : TextData
{
public float[] Features { get; set; }
}
}
}
Platí pro
FeaturizeText(TransformsCatalog+TextTransforms, String, TextFeaturizingEstimator+Options, String[])
Vytvořte TextFeaturizingEstimator, která transformuje textový sloupec na featurizovaný vektor Single , který představuje normalizované počty n-gramů a char-gramů.
public static Microsoft.ML.Transforms.Text.TextFeaturizingEstimator FeaturizeText (this Microsoft.ML.TransformsCatalog.TextTransforms catalog, string outputColumnName, Microsoft.ML.Transforms.Text.TextFeaturizingEstimator.Options options, params string[] inputColumnNames);
static member FeaturizeText : Microsoft.ML.TransformsCatalog.TextTransforms * string * Microsoft.ML.Transforms.Text.TextFeaturizingEstimator.Options * string[] -> Microsoft.ML.Transforms.Text.TextFeaturizingEstimator
<Extension()>
Public Function FeaturizeText (catalog As TransformsCatalog.TextTransforms, outputColumnName As String, options As TextFeaturizingEstimator.Options, ParamArray inputColumnNames As String()) As TextFeaturizingEstimator
Parametry
- catalog
- TransformsCatalog.TextTransforms
Katalog transformace související s textem
- outputColumnName
- String
Název sloupce, který je výsledkem transformace inputColumnNames
.
Datový typ tohoto sloupce bude vektorem Single.
- options
- TextFeaturizingEstimator.Options
Pokročilé možnosti algoritmu.
- inputColumnNames
- String[]
Název sloupců, které se mají transformovat. Pokud je nastavená hodnota null
, použije se jako zdroj hodnota outputColumnName
.
Tento estimátor pracuje s textovými daty a může transformovat několik sloupců najednou, což dává jeden vektor Single jako výsledné funkce pro všechny sloupce.
Návraty
Příklady
using System;
using System.Collections.Generic;
using Microsoft.ML;
using Microsoft.ML.Transforms.Text;
namespace Samples.Dynamic
{
public static class FeaturizeTextWithOptions
{
public static void Example()
{
// Create a new ML context, for ML.NET operations. It can be used for
// exception tracking and logging, as well as the source of randomness.
var mlContext = new MLContext();
// Create a small dataset as an IEnumerable.
var samples = new List<TextData>()
{
new TextData(){ Text = "ML.NET's FeaturizeText API uses a " +
"composition of several basic transforms to convert text into " +
"numeric features." },
new TextData(){ Text = "This API can be used as a featurizer to " +
"perform text classification." },
new TextData(){ Text = "There are a number of approaches to text " +
"classification." },
new TextData(){ Text = "One of the simplest and most common " +
"approaches is called “Bag of Words”." },
new TextData(){ Text = "Text classification can be used for a " +
"wide variety of tasks" },
new TextData(){ Text = "such as sentiment analysis, topic " +
"detection, intent identification etc." },
};
// Convert training data to IDataView.
var dataview = mlContext.Data.LoadFromEnumerable(samples);
// A pipeline for converting text into numeric features.
// The following call to 'FeaturizeText' instantiates
// 'TextFeaturizingEstimator' with given parameters. The length of the
// output feature vector depends on these settings.
var options = new TextFeaturizingEstimator.Options()
{
// Also output tokenized words
OutputTokensColumnName = "OutputTokens",
CaseMode = TextNormalizingEstimator.CaseMode.Lower,
// Use ML.NET's built-in stop word remover
StopWordsRemoverOptions = new StopWordsRemovingEstimator.Options()
{
Language = TextFeaturizingEstimator.Language.English
},
WordFeatureExtractor = new WordBagEstimator.Options()
{
NgramLength
= 2,
UseAllLengths = true
},
CharFeatureExtractor = new WordBagEstimator.Options()
{
NgramLength
= 3,
UseAllLengths = false
},
};
var textPipeline = mlContext.Transforms.Text.FeaturizeText("Features",
options, "Text");
// Fit to data.
var textTransformer = textPipeline.Fit(dataview);
// Create the prediction engine to get the features extracted from the
// text.
var predictionEngine = mlContext.Model.CreatePredictionEngine<TextData,
TransformedTextData>(textTransformer);
// Convert the text into numeric features.
var prediction = predictionEngine.Predict(samples[0]);
// Print the length of the feature vector.
Console.WriteLine($"Number of Features: {prediction.Features.Length}");
// Print feature values and tokens.
Console.Write("Features: ");
for (int i = 0; i < 10; i++)
Console.Write($"{prediction.Features[i]:F4} ");
Console.WriteLine("\nTokens: " + string.Join(",", prediction
.OutputTokens));
// Expected output:
// Number of Features: 282
// Features: 0.0941 0.0941 0.0941 0.0941 0.0941 0.0941 0.0941 0.0941 0.0941 0.1881 ...
// Tokens: ml.net's,featurizetext,api,uses,composition,basic,transforms,convert,text,numeric,features.
}
private class TextData
{
public string Text { get; set; }
}
private class TransformedTextData : TextData
{
public float[] Features { get; set; }
public string[] OutputTokens { get; set; }
}
}
}
Poznámky
Tato transformace může pracovat s několika sloupci.