Not Monitored
Tag not monitored by Microsoft.
42,838 questions
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
I have an Azure cluster instance with 100 nodes and I want to perform batch inference on that cluster.
I have more than 1 model that will be used to perform inferencing. These models will be accessed in scoring file. I am passing the list of models to Batch Deployment Class using code attached. I get an error as below
Error creating models asset: must be type Optional[Union[str, Asset]]
modelsList = ['model1', 'model2', 'model3']
deployment = BatchDeployment(
name="ml-inference",
description="Batch deployment for scoring",
endpoint_name=endpoint_name,
model=modelsList,
code_path=".",
scoring_script="batch_inference.py",
environment="ml-batch-env:1",
compute=compute_name,
instance_count=2,
max_concurrency_per_instance=2,
mini_batch_size=10,
output_action=BatchDeploymentOutputAction.APPEND_ROW,
output_file_name="predictions.csv",
retry_settings=BatchRetrySettings(max_retries=3, timeout=30),
logging_level="info",
)