Dart Class
Dropouts meet Multiple Additive Regresion Trees.
- Inheritance
-
nimbusml.internal.core.ensemble.booster._dart.DartDart
Constructor
Dart(tree_drop_fraction=0.1, maximum_number_of_dropped_trees_per_round=1, skip_drop_fraction=0.5, xgboost_dart_mode=False, uniform_drop=False, minimum_split_gain=0.0, maximum_tree_depth=0, minimum_child_weight=0.1, subsample_frequency=0, subsample_fraction=1.0, feature_fraction=1.0, l2_regularization=0.01, l1_regularization=0.0, **params)
Parameters
- tree_drop_fraction
The drop ratio for trees. Range:(0,1).
- maximum_number_of_dropped_trees_per_round
Maximum number of dropped trees in a boosting round.
- skip_drop_fraction
Probability for not dropping in a boosting round.
- xgboost_dart_mode
True will enable xgboost dart mode.
- uniform_drop
True will enable uniform drop.
- minimum_split_gain
Minimum loss reduction required to make a further partition on a leaf node of the tree. the larger, the more conservative the algorithm will be.
- maximum_tree_depth
Maximum depth of a tree. 0 means no limit. However, tree still grows by best-first.
- minimum_child_weight
Minimum sum of instance weight(hessian) needed in a child. If the tree partition step results in a leaf node with the sum of instance weight less than min_child_weight, then the building process will give up further partitioning. In linear regression mode, this simply corresponds to minimum number of instances needed to be in each node. The larger, the more conservative the algorithm will be.
- subsample_frequency
Subsample frequency for bagging. 0 means no subsample. Specifies the frequency at which the bagging occurs, where if this is set to N, the subsampling will happen at every N iterations.This must be set with Subsample as this specifies the amount to subsample.
- subsample_fraction
Subsample ratio of the training instance. Setting it to 0.5 means that LightGBM randomly collected half of the data instances to grow trees and this will prevent overfitting. Range: (0,1].
- feature_fraction
Subsample ratio of columns when constructing each tree. Range: (0,1].
- l2_regularization
L2 regularization term on weights, increasing this value will make model more conservative.
- l1_regularization
L1 regularization term on weights, increase this value will make model more conservative.
- params
Additional arguments sent to compute engine.
Examples
###############################################################################
# LightGbmClassifier
from nimbusml import Pipeline, FileDataStream
from nimbusml.datasets import get_dataset
from nimbusml.ensemble import LightGbmClassifier
from nimbusml.ensemble.booster import Dart
from nimbusml.feature_extraction.categorical import OneHotVectorizer
# data input (as a FileDataStream)
path = get_dataset('infert').as_filepath()
data = FileDataStream.read_csv(path)
print(data.head())
# age case education induced parity ... row_num spontaneous ...
# 0 26 1 0-5yrs 1 6 ... 1 2 ...
# 1 42 1 0-5yrs 1 1 ... 2 0 ...
# 2 39 1 0-5yrs 2 6 ... 3 0 ...
# 3 34 1 0-5yrs 2 4 ... 4 0 ...
# 4 35 1 6-11yrs 1 3 ... 5 1 ...
# define the training pipeline
pipeline = Pipeline([
OneHotVectorizer(columns={'edu': 'education'}),
LightGbmClassifier(feature=['parity', 'edu'], label='induced',
booster=Dart(reg_lambda=0.1))
])
# train, predict, and evaluate
metrics, predictions = pipeline.fit(data).test(data, output_scores=True)
# print predictions
print(predictions.head())
# PredictedLabel Score.0 Score.1 Score.2
# 0 2 0.070722 0.145439 0.783839
# 1 0 0.737733 0.260116 0.002150
# 2 2 0.070722 0.145439 0.783839
# 3 0 0.490715 0.091749 0.417537
# 4 0 0.562419 0.197818 0.239763
# print evaluation metrics
print(metrics)
# Accuracy(micro-avg) Accuracy(macro-avg) Log-loss Log-loss reduction ...
# 0 0.641129 0.462618 0.772996 19.151269 ...
Remarks
Multiple Additive Regression Trees (MART) is an ensemble method of boosted regression trees. The Dropouts meet Multiple Additive Regression Trees (DART) employs dropouts in MART and overcomes the issues of over- specialization of MART, achieving better performance in many tasks.
Reference
https://arxiv.org/abs/1505.01866
Methods
get_params |
Get the parameters for this operator. |
get_params
Get the parameters for this operator.
get_params(deep=False)
Parameters
- deep