Model.Package fails when attempting to build image for ML model

MathBarbarian 21 Reputation points
2021-07-26T15:35:51.78+00:00

Previously my code was completing while running on a machine in the cloud.

I am using azureml.core.Model to .package models and script which are in a /source/ subdirectory.

After connecting to the Workspace remotely I started to get the following error even when running unchanged code from the machine in the cloud.

Message: Package creation reached non-successful terminal state.

State: Failed
Error:
StatusCode: 400
Message: Failed to fetch details for Environment with Name: package Version: Autosave_2021-07-26T15:25:15Z_5cb0d061 Reason: Workspace connections are not supported for registry: viennaglobal.azurecr.io.

InnerException None
ErrorResponse 
Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
3,332 questions
{count} votes

2 answers

Sort by: Most helpful
  1. MathBarbarian 21 Reputation points
    2021-07-27T17:39:10.783+00:00

    Here is the code which was executing on the cloud compute Jupyter Notebool:

    from azureml.__version__ import __version__ as _azml_version_
    from azureml.core import __version__ as _azmlcore_version_
    from azureml.train.automl import __version__ as _azmltrain_version_
    
    print('azure', _azml_version_)
    print(' core', _azmlcore_version_)
    print('train', _azmltrain_version_)
    
    from azureml.core.experiment import Experiment
    from azureml.core import Workspace, Datastore
    from azureml.core.webservice import AciWebservice
    from azureml.core.model import InferenceConfig
    from azureml.core.environment import Environment
    from azureml.core.conda_dependencies import CondaDependencies
    
    from azureml.core.webservice import LocalWebservice
    from azureml.core.model import Model
    
    ws = Workspace.from_config()
    
    PROJECT = 'qtc'
    data_date = '2021-06-08'
    experiment_date = "2021-07-25"
    dd_name = data_date.replace('-', '')
    ed_name = experiment_date.replace('-', '')
    detail = 'conditional-select'
    
    # Choose a name for the experiment and specify the project folder.
    experiment_name = '-'.join(['qtc', ed_name, detail.replace(' ' , '-')])
    experiment = Experiment(ws, experiment_name)
    aciconfig = AciWebservice.deploy_configuration(cpu_cores=1, 
                                                   memory_gb=2, 
                                                   tags={"data": "qtc segment trimmed",  "method" : "selective model"}, 
                                                   description='Estimate order completion time')
    deployment_config = LocalWebservice.deploy_configuration(port=8890)
    env = Environment.get(ws, "AzureML-AutoML").clone('qtc_env')
    inference_config = InferenceConfig(entry_script="score_20210726.py",
                                       environment=env, source_directory=os.path.join(os.getcwd(), 'source'))
    
    service = Model.deploy(workspace=ws, 
                           name='selective-model', 
                           models=[], 
                           inference_config=inference_config, 
                           deployment_config=aciconfig,
                           #deployment_config=deployment_config,
                           overwrite=True)
    
    service.wait_for_deployment(show_output=True)
    service.delete()
    
    package = Model.package(ws, [], inference_config)
    package.wait_for_creation(show_output=True)
    
    0 comments No comments

  2. MathBarbarian 21 Reputation points
    2021-07-27T17:41:25.297+00:00

    And here is the error that I am now getting in the same environment without changing anything in there.

    azure 1.0.1b6.post1
     core 1.31.0
    train 1.31.0
    Tips: You can try get_logs(): https://aka.ms/debugimage#dockerlog or local deployment: https://aka.ms/debugimage#debug-locally to debug if deployment takes longer than 10 minutes.
    Running
    2021-07-27 16:29:00+00:00 Registering the environment.
    2021-07-27 16:29:01+00:00 Use the existing image.
    2021-07-27 16:29:01+00:00 Generating deployment configuration.
    2021-07-27 16:29:02+00:00 Submitting deployment to compute..
    2021-07-27 16:29:06+00:00 Checking the status of deployment selective-model..
    2021-07-27 16:34:06+00:00 Checking the status of inference endpoint selective-model.
    Succeeded
    ACI service creation operation finished, operation "Succeeded"
    Package creation Failed
    ---------------------------------------------------------------------------
    WebserviceException                       Traceback (most recent call last)
    <ipython-input-1-ac8fa2a0144c> in <module>
         59 
         60 package = Model.package(ws, [], inference_config)
    ---> 61 package.wait_for_creation(show_output=True)
    
    /anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/core/model.py in wait_for_creation(self, show_output)
       2735                                       'State: {}\n'
       2736                                       'Error:\n'
    -> 2737                                       '{}\n'.format(self.state, error_message))
       2738 
       2739     def serialize(self):
    
    WebserviceException: WebserviceException:
        Message: Package creation reached non-successful terminal state.
    State: Failed
    Error:
    StatusCode: 400
    Message: Failed to fetch details for Environment with Name: package Version: Autosave_2021-07-27T16:34:16Z_e22f01a7 Reason: Workspace connections are not supported for registry: viennaglobal.azurecr.io.
    
        InnerException None
        ErrorResponse 
    {
        "error": {
            "message": "Package creation reached non-successful terminal state.\nState: Failed\nError:\nStatusCode: 400\nMessage: Failed to fetch details for Environment with Name: package Version: Autosave_2021-07-27T16:34:16Z_e22f01a7 Reason: Workspace connections are not supported for registry: viennaglobal.azurecr.io.\n"
        }
    }
    

    In the error it identifies that it can't find details for an Environment with Name == 'package' ?


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.