Usage
microsoftml.categorical(cols: [str, dict, list], output_kind: ['Bag', 'Ind',
'Key', 'Bin'] = 'Ind', max_num_terms: int = 1000000,
terms: int = None, sort: ['Occurrence', 'Value'] = 'Occurrence',
text_key_values: bool = False, **kargs)
Description
可以在训练模型前对数据进行的分类变换。
详细信息
变 categorical 换通过一个数据集，操作文本列，构建类别词典。 对于每一行，输入列中出现的整个文本字符串都定义为一个类别。 范畴变换的输出是一个指示向量。
此向量中的每个槽对应于字典中的一个类别，因此其长度是生成的字典的大小。 范畴变换可以应用于一个或多个列，此时它为每个应用的列构建一个独立的词典。
categorical 目前不支持处理因子数据。
Arguments
cols
一个用于转换的变量字符串或变量列表。 如果 dict，这些键代表要创建的新变量名称。
output_kind
一个字符字符串，指定输出类型的类型。
"Bag"输出多集向量。 如果输入列是范畴向量，输出包含一个向量，每个槽位的值是该范畴在输入向量中的出现次数。 如果输入列包含单一类别，指示向量和袋向量是等价的"Ind"输出指示向量。 输入列是类别向量，输出列中每个槽包含一个指示向量。"Key"输出索引。 输出是该类别的整数 ID（介于 1 和字典中类别数量之间）。"Bin"输出一个向量，该向量是该范畴的二进制表示。
默认值为 "Ind"。
max_num_terms
一个整数，指定词典中包含的最大类别数量。 默认值是1000000。
条款
可选的词语或类别字符向量。
排序
一个字符字符串，用于指定排序条件。
"Occurrence"：按事件分类。 最常见的是第一。"Value"：按数值排序类别。
text_key_values
键值元数据是否应为文本，无论实际输入类型如何。
kargs
发送到计算引擎的其他参数。
Returns
定义转换的对象。
Example
'''
Example on rx_logistic_regression and categorical.
'''
import numpy
import pandas
from microsoftml import rx_logistic_regression, categorical, rx_predict
train_reviews = pandas.DataFrame(data=dict(
review=[
"This is great", "I hate it", "Love it", "Do not like it", "Really like it",
"I hate it", "I like it a lot", "I kind of hate it", "I do like it",
"I really hate it", "It is very good", "I hate it a bunch", "I love it a bunch",
"I hate it", "I like it very much", "I hate it very much.",
"I really do love it", "I really do hate it", "Love it!", "Hate it!",
"I love it", "I hate it", "I love it", "I hate it", "I love it"],
like=[True, False, True, False, True, False, True, False, True, False,
True, False, True, False, True, False, True, False, True, False, True,
False, True, False, True]))
test_reviews = pandas.DataFrame(data=dict(
review=[
"This is great", "I hate it", "Love it", "Really like it", "I hate it",
"I like it a lot", "I love it", "I do like it", "I really hate it", "I love it"]))
# Use a categorical transform: the entire string is treated as a category
out_model = rx_logistic_regression("like ~ reviewCat",
data=train_reviews,
ml_transforms=[categorical(cols=dict(reviewCat="review"))])
# Note that 'I hate it' and 'I love it' (the only strings appearing more than once)
# have non-zero weights.
print(out_model.coef_)
# Use the model to score.
source_out_df = rx_predict(out_model, data=test_reviews, extra_vars_to_write=["review"])
print(source_out_df.head())
输出：
Beginning processing data.
Rows Read: 25, Read Time: 0, Transform Time: 0
Beginning processing data.
Not adding a normalizer.
Beginning processing data.
Rows Read: 25, Read Time: 0, Transform Time: 0
Beginning processing data.
Beginning processing data.
Rows Read: 25, Read Time: 0, Transform Time: 0
Beginning processing data.
LBFGS multi-threading will attempt to load dataset into memory. In case of out-of-memory issues, turn off multi-threading by setting trainThreads to 1.
Warning: Too few instances to use 4 threads, decreasing to 1 thread(s)
Beginning optimization
num vars: 20
improvement criterion: Mean Improvement
L1 regularization selected 3 of 20 weights.
Not training a calibrator because it is not needed.
Elapsed time: 00:00:01.6550695
Elapsed time: 00:00:00.2259981
OrderedDict([('(Bias)', 0.21317288279533386), ('I hate it', -0.7937591671943665), ('I love it', 0.19668534398078918)])
Beginning processing data.
Rows Read: 10, Read Time: 0, Transform Time: 0
Beginning processing data.
Elapsed time: 00:00:00.1385248
Finished writing 10 rows.
Writing completed.
review PredictedLabel Score Probability
0 This is great True 0.213173 0.553092
1 I hate it False -0.580586 0.358798
2 Love it True 0.213173 0.553092
3 Really like it True 0.213173 0.553092
4 I hate it False -0.580586 0.358798
相关内容
- microsoftml.categorical_hash：将文本列进行哈希处理并转换为类别