Specifying AzureML output destination in SDK v2

Question

Hi. I have set up an AzureML pipeline with YAML components using the Python SDK (v2) with an attached blob store. However, it appears that the output destination is handled automatically by AzureML and so I can't specify where on the blob the pipeline writes its output. I want to configure the AzureML pipeline run using ADF, which involves moving some data to the blob, running the AzureML pipeline, and then moving some data from the blob to somewhere else. The trouble is that ADF doesn't get access to the AzureML output directory, and so it won't know where to look for the output file.

I have tried to pass the output directory as an input rather than an output so that I can explicitly state where this should go. The directory, however, gets mounted as read only (quite sensibly by design, I trust) so that doesn't work. So I'm kind of running out of options.

Is there any way for me specify the output path for an Azure ML SDK v2 pipeline in a similar way to how I would specify an input path? Alternatively, is there another way of solving this particular predicament of mine?

I have looked through the notebooks (e.g. https://github.com/Azure/azureml-examples/blob/8a4070f55593c9641083784283b773f4f20955dd/sdk/jobs/pipelines/1a_pipeline_with_components_from_yaml/pipeline_with_components_from_yaml.ipynb) and I can't find an example where people explicitly control the output destination (which seems odd).

Thoughts?

Answer 1

Any updates on this?
I have the same problem, it seems you can specify the path in the output like
outputs={
"output_path": Output(type="uri_folder", mode="rw_mount", path=<path>),
}
It doesnt throw an error, however the path is ignored anyways...