How to rerun successful AzureML Pipeline

Dance, Cody R. (ALT) 20 Reputation points

I have a successful pipeline that writes files to Blob storage. If I delete the files in blob storage and try to rerun the pipeline, the pipeline run is cached so the pipeline doesn't actually rerun. Is there a way to force rerun the pipeline?

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,655 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Sedat SALMAN 13,190 Reputation points

    If the inputs (data and parameters) of the step haven't changed, the system will just use the cached results, speeding up pipeline execution. However, you can force a re-run of the steps by using the allow_reuse parameter and setting it to False during the pipeline step definition.

    from azureml.pipeline.steps import PythonScriptStep
    step1 = PythonScriptStep(script_name="",
                             arguments=["--arg1", arg1],
                             allow_reuse=False)  # setting allow_reuse to False
    0 comments No comments