ModelBatchDeploymentSettings Class

Note

This is an experimental class, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.

Model Batch Deployment Settings entity.

Inheritance

builtins.object

ModelBatchDeploymentSettings

Constructor

ModelBatchDeploymentSettings(*, mini_batch_size: int | None, instance_count: int | None = None, max_concurrency_per_instance: int | None = None, output_action: BatchDeploymentOutputAction | None = None, output_file_name: str | None = None, retry_settings: BatchRetrySettings | None = None, environment_variables: Dict[str, str] | None = None, error_threshold: int | None = None, logging_level: str | None = None, **kwargs: Any)

Parameters

Name	Description
mini_batch_size Required	int Size of the mini-batch passed to each batch invocation, defaults to 10
instance_count Required	int Number of instances the interfering will run on. Equivalent to resources.instance_count.
output_action Required	str or BatchDeploymentOutputAction Indicates how the output will be organized. Possible values include: "summary_only", "append_row". Defaults to "append_row"
output_file_name Required	str Customized output file name for append_row output action, defaults to "predictions.csv"
max_concurrency_per_instance Required	int Indicates maximum number of parallelism per instance, defaults to 1
retry_settings Required	BatchRetrySettings Retry settings for a batch inference operation, defaults to None
environment_variables Required	dict Environment variables that will be set in deployment.
error_threshold Required	int Error threshold, if the error count for the entire input goes above this value, the batch inference will be aborted. Range is [-1, int.MaxValue] -1 value indicates, ignore all failures during batch inference For FileDataset count of file failures For TabularDataset, this is the count of record failures, defaults to -1
logging_level Required	str Logging level for batch inference operation, defaults to "info"

Examples

Creating a Model Batch Deployment Settings object.

   from azure.ai.ml.entities._deployment.model_batch_deployment_settings import ModelBatchDeploymentSettings

   modelBatchDeploymentSetting = ModelBatchDeploymentSettings(
       mini_batch_size=256,
       instance_count=5,
       max_concurrency_per_instance=2,
       output_file_name="output-file-name",
       environment_variables={"env1": "value1", "env2": "value2"},
       error_threshold=2,
       logging_level=1,
   )