microsoftml.rx_predict: Scores using a Microsoft machine learning model

Article
03/03/2023

Usage

microsoftml.rx_predict(model,
    data: typing.Union[revoscalepy.datasource.RxDataSource.RxDataSource,
    pandas.core.frame.DataFrame],
    output_data: typing.Union[revoscalepy.datasource.RxDataSource.RxDataSource,
    str] = None, write_model_vars: bool = False,
    extra_vars_to_write: list = None, suffix: str = None,
    overwrite: bool = False, data_threads: int = None,
    blocks_per_read: int = None, report_progress: int = None,
    verbose: int = 1,
    compute_context: revoscalepy.computecontext.RxComputeContext.RxComputeContext = None,
    **kargs)

Description

Reports per-instance scoring results in a data frame or revoscalepy data source using a trained Microsoft ML Machine Learning model with arevoscalepydata source.

Details

The following items are reported in the output by default: scoring on three variables for the binary classifiers: PredictedLabel, Score, and Probability; the Score for oneClassSvm and regression classifiers; PredictedLabel for Multi-class classifiers, plus a variable for each category prepended by the Score.

Arguments

model

A model information object returned from a microsoftml model. For example, an object returned from rx_fast_trees or rx_logistic_regression.

data

A revoscalepy data source object, a data frame, or the path to a .xdf file.

output_data

Output text or xdf file name or an RxDataSource with write capabilities in which to store transformed data. If None, a data frame is returned. The default value is None.

write_model_vars

If True, variables in the model are written to the output data set in addition to the scoring variables. If variables from the input data set are transformed in the model, the transformed variables are also included. The default value is False.

extra_vars_to_write

None or character vector of additional variables names from the input data to include in the output_data. If write_model_vars is True, model variables are included as well. The default value is None.

suffix

A character string specifying suffix to append to the created scoring variable(s) or None in there is no suffix. The default value is None.

overwrite

If True, an existing output_data is overwritten; if False an existing output_data is not overwritten. The default value is False.

data_threads

An integer specifying the desired degree of parallelism in the data pipeline. If None, the number of threads used is determined internally. The default value is None.

blocks_per_read

Specifies the number of blocks to read for each chunk of data read from the data source.

report_progress

An integer value that specifies the level of reporting on the row processing progress:

0: no progress is reported.
1: the number of processed rows is printed and updated.
2: rows processed and timings are reported.
3: rows processed and all timings are reported.

The default value is 1.

verbose

An integer value that specifies the amount of output wanted. If 0, no verbose output is printed during calculations. Integer values from 1 to 4 provide increasing amounts of information. The default value is 1.

compute_context

Sets the context in which computations are executed, specified with a valid revoscalepy.RxComputeContext. Currently local and revoscalepy.RxInSqlServer compute contexts are supported.

kargs

Additional arguments sent to compute engine.

Returns

A data frame or an revoscalepy.RxDataSource object representing the created output data. By default, output from scoring binary classifiers include three variables: PredictedLabel, Score, and Probability; rx_oneclass_svm and regression include one variable: Score; and multi-class classifiers include PredictedLabel plus a variable for each category prepended by Score. If a suffix is provided, it is added to the end of these output variable names.

Binary classification example

'''
Binary Classification.
'''
import numpy
import pandas
from microsoftml import rx_fast_linear, rx_predict
from revoscalepy.etl.RxDataStep import rx_data_step
from microsoftml.datasets.datasets import get_dataset

infert = get_dataset("infert")

import sklearn
if sklearn.__version__ < "0.18":
    from sklearn.cross_validation import train_test_split
else:
    from sklearn.model_selection import train_test_split

infertdf = infert.as_df()
infertdf["isCase"] = infertdf.case == 1
data_train, data_test, y_train, y_test = train_test_split(infertdf, infertdf.isCase)

forest_model = rx_fast_linear(
    formula=" isCase ~ age + parity + education + spontaneous + induced ",
    data=data_train)
    
# RuntimeError: The type (RxTextData) for file is not supported.
score_ds = rx_predict(forest_model, data=data_test,
                     extra_vars_to_write=["isCase", "Score"])
                     
# Print the first five rows
print(rx_data_step(score_ds, number_rows_read=5))

Output:

Automatically adding a MinMax normalization transform, use 'norm=Warn' or 'norm=No' to turn this behavior off.
Beginning processing data.
Rows Read: 186, Read Time: 0, Transform Time: 0
Beginning processing data.
Beginning processing data.
Rows Read: 186, Read Time: 0.001, Transform Time: 0
Beginning processing data.
Beginning processing data.
Rows Read: 186, Read Time: 0.001, Transform Time: 0
Beginning processing data.
Using 2 threads to train.
Automatically choosing a check frequency of 2.
Auto-tuning parameters: maxIterations = 8064.
Auto-tuning parameters: L2 = 2.666837E-05.
Auto-tuning parameters: L1Threshold (L1/L2) = 0.
Using best model from iteration 590.
Not training a calibrator because it is not needed.
Elapsed time: 00:00:00.6058289
Elapsed time: 00:00:00.0084728
Beginning processing data.
Rows Read: 62, Read Time: 0, Transform Time: 0
Beginning processing data.
Elapsed time: 00:00:00.0302359
Finished writing 62 rows.
Writing completed.
Rows Read: 5, Total Rows Processed: 5, Total Chunk Time: 0.001 seconds 
  isCase PredictedLabel     Score  Probability
0  False           True  0.576775     0.640325
1  False          False -2.929549     0.050712
2   True          False -2.370090     0.085482
3  False          False -1.700105     0.154452
4  False          False -0.110981     0.472283

Regression example

'''
Regression.
'''
import numpy
import pandas
from microsoftml import rx_fast_trees, rx_predict
from revoscalepy.etl.RxDataStep import rx_data_step
from microsoftml.datasets.datasets import get_dataset

airquality = get_dataset("airquality")

import sklearn
if sklearn.__version__ < "0.18":
    from sklearn.cross_validation import train_test_split
else:
    from sklearn.model_selection import train_test_split

airquality = airquality.as_df()


######################################################################
# Estimate a regression fast forest
# Use the built-in data set 'airquality' to create test and train data

df = airquality[airquality.Ozone.notnull()]
df["Ozone"] = df.Ozone.astype(float)

data_train, data_test, y_train, y_test = train_test_split(df, df.Ozone)

airFormula = " Ozone ~ Solar_R + Wind + Temp "

# Regression Fast Forest for train data
ff_reg = rx_fast_trees(airFormula, method="regression", data=data_train)

# Put score and model variables in data frame
score_df = rx_predict(ff_reg, data=data_test, write_model_vars=True)
print(score_df.head())

# Plot actual versus predicted values with smoothed line
# Supported in the next version.
# rx_line_plot(" Score ~ Ozone ", type=["p", "smooth"], data=score_df)