Azure Machine Learning
An Azure machine learning service for building and deploying models.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 54%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENK%3C/text%3E%3C/svg%3E)
Any Update on this? I am having the same issue?
Models registered with AML CLI cause ACI deployments to fail
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 26%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEJ%3C/text%3E%3C/svg%3E)
Eu Jing Chua
6
Reputation points Microsoft Employee
We were registering a local file ts_scalar.pickle as a model using AML CLI's create az model create -n test-model -l ./ts_scalar.pickle ...
This registered model test-model works fine when deployed to a LocalWebService, but does not work for a AciWebService.
We either get a timeout during the deployment (does not reach stage where containers start running) or this error:
Traceback (most recent call last):
File "./download.py", line 353, in <module>
init_container_assets(args.config_json, args.conn_string, args.container, args.appinsights_key, args.config_folder)
File "./download.py", line 327, in init_container_assets
downloader.download(config_file_content)
File "./download.py", line 128, in download
self.download_artifact(blob_path, local_path, unpack_type)
File "./download.py", line 92, in download_artifact
file_path=local_path)
File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/baseblobservice.py", line 2014, in get_blob_to_path
cpk=cpk)
File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/baseblobservice.py", line 2193, in get_blob_to_stream
raise ex
File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/baseblobservice.py", line 2160, in get_blob_to_stream
cpk=cpk)
File "/usr/local/lib/python3.6/site-packages/azure/storage/blob/baseblobservice.py", line 1887, in _get_blob
operation_context=_context)
File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 446, in _perform_request
raise ex
File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 374, in _perform_request
raise ex
File "/usr/local/lib/python3.6/site-packages/azure/storage/common/storageclient.py", line 360, in _perform_request
HTTPError(response.status, response.message, response.headers, response.body))
File "/usr/local/lib/python3.6/site-packages/azure/storage/common/_error.py", line 115, in _http_error_handler
raise ex
azure.common.AzureMissingResourceHttpError: The specified blob does not exist. ErrorCode: BlobNotFound
<?xml version="1.0" encoding="utf-8"?><Error><Code>BlobNotFound</Code><Message>The specified blob does not exist.
RequestId:59b52a82-301e-005a-51c8-c52ce4000000
Time:2021-10-20T15:40:04.2703282Z</Message></Error>
However if we register the model through AML's UI model registration, both kinds of deployments are successful.
Is there an issue with using the CLI to register models for ACI deployment?
Azure Machine Learning
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 77%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EY9%3C/text%3E%3C/svg%3E)
YutongTie-9091 54,026 Reputation points Moderator2021-10-21T03:47:17.75+00:00 Hello,
Could you please share how you deployed your ACI model? Any code sample you used?
According to the error message, this is related to wrong blob path or wrong storage setting. How you set it?
This is the guidance works for me, could you please take a look?
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-and-where?tabs=azcli#re-deploy-to-cloudHope this helps.
Regards,
Yutong -
Eu Jing Chua 6 Reputation points Microsoft Employee
2021-10-25T16:30:10.377+00:00 Hi @Anonymous !
We are also deploying the ACI model using the python SDK based on the guide you posted, just that our models were registered with the CLI.
The wrong blob path and storage setting were not set by us, I think it is something being done by AML's internals with downloading the registered models into ACI. Here is a code sample of our ACI deployment, where both modelstest-scalerandtest-rnnwere registered using the CLI:from azureml.core import Workspace, Environment from azureml.core.model import Model, InferenceConfig from azureml.core.webservice import AciWebservice aml_ws = Workspace.from_config() # Inference Environment env = Environment.from_conda_specification( name="rnn-env", file_path="./conda_dependencies.yml") inference_config = InferenceConfig(...) deployment_config = AciWebservice.deploy_configuration(...) sc = Model(aml_ws, "test-scaler") rnn = Model(aml_ws, "test-rnn") service = Model.deploy( workspace=aml_ws, name="rnn-service", models=[sc, rnn], inference_config=inference_config, deployment_config=deployment_config, overwrite=True, ) service.wait_for_deployment(show_output=True) print("Service State:", service.state) print("Scoring URI:", service.scoring_uri) -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 57.99999999999999%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQ0%3C/text%3E%3C/svg%3E)
ql-0154 1 Reputation point2022-06-08T14:19:24.337+00:00 Any update on this? I am having a similar issue.
Sign in to comment