How to utilize AutoML in an ML Designer pipeline?

kbennett 1 Reputation point
2023-02-24T21:15:45.0366667+00:00

Currently, I have an ML pipeline built in Azure ML Designer that effectively does the following basic steps:

  1. Trains a model across a full quantitative range of [0.0, 7.0] from a full baseline dataset.
  2. Scores model to generate predictions appended to actuals in a scored dataset.
  3. Passes the scored dataset to R-scripts that row filter the scored dataset based on criteria that reduce the full quantitative range of [0.0, 7.0] into 3 segmented quantitative ranges of range 1 = [0,1.0], range 2 = (1.0, 3.3], and range 3 = (3.3, 7.0].
  4. Trains 1 new model from each of 3 filtered datasets from step 3 across the corresponding segmented range using same predictors from step 1 plus 1 new predictor: the predictions generated in step 2.
  5. Scores each model from step 4 to generate predictions appended to actuals in scored datasets.

The idea here is that the predictions generated in step 2 are actually treated as an estimate, which is then used as an engineered feature in subsequent trainings to refine the predictions. I've provided the schematic of this pipeline in Designer.PNG.

Now, before I built this in Designer, I used Azure's AutoML service to do all this manually as sort of a POC that this idea would work. And those results were very promising. But passing, filtering, splitting, etc. all the data is difficult to do manually and I would like a pipeline to do it for me.

The problem? Ideally, I need ensemble models for each of the 4 trained models and I'm not seeing a way to get ensemble models out of Azure ML Designer.

Is there a way to utilize AutoML within designer? Or is there a way to eliminate the regression/train components from my Designer pipeline and replace with an already trained model stored in our workspace?

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,608 questions
{count} votes

1 answer

Sort by: Most helpful
  1. romungi-MSFT 42,761 Reputation points Microsoft Employee
    2023-02-28T07:09:42.4466667+00:00

    Update: kbennett There is a public preview feature on Azure ML studio that can help use components of various registries when turned on. This can list AutoML models for a custom pipeline i.e you can use the AutoML models from the azureml registry to build a pipeline on canvas.

    Turn on the preview feature from the announcements page.

    User's image

    Create a custom pipeline:

    User's image

    This should list the models, data and components.

    User's image

    With the custom pipeline, the pre-built components cannot be used currently.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    0 comments No comments