Trigger Azure ML Pipeline from Azure Data Factory

Question

Trigger Azure ML Pipeline from Azure Data Factory

Vinoth Kumar K 56

I have created and published a Azure ML pipeline. I want to trigger the ML pipeline from Azure Data Factory.

In ADF, i have chosen Machine learning execute pipeline and created the linked service to azure machine learning and able to choose the published pipeline endpoint. However while running, i am getting the below error. I couldn't find much information how to resolve the error.

"Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType"

Tka32 56 Reputation points

2022-02-17T10:22:38.593+00:00

I also have the same issue.
We have a pipeline triggering ML pipelines, which used to run successfully until yesterday, started failing this morning with the same error.
Razvan Vacaru 26 Reputation points

2022-02-17T10:25:37.343+00:00

We have ADF pipelines running in production already for sometime, we didn't update anything in relation to either the ADF pipeline or the AzureML pipeline.
Nevertheless from this morning at ~8 CET our scheduled pipelines failed with the same error. We've tried re creating the "Machine Learning Execute Pipeline" activity or using PipelineId instead of PipelineEndpointId but without any success so far. It seems to be un underlying issue in Azure/ADF with how ADF invokes the ML Pipeline

Please prioritize this.
Rodrigo Barraza 16 Reputation points

2022-02-17T12:35:24.377+00:00

I have exactly the same problem.
I have a pipeline in synapse which executes a pipeline in ML, I don't know any modifications have been made in two months in either of the two environments and it has always been executed correctly, but today it fails throwing the same message:

"errorCode": "2105",
"message": "Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType'"
Veselin Valchev 1 Reputation point

2022-02-17T15:15:11.35+00:00

Same problem here - anyone has any feedback from Microsoft re this?
Thomas Servotte 1 Reputation point

2022-02-17T15:20:59.05+00:00

Same problem here
Milen Chechev 1 Reputation point

2022-02-17T15:31:23.693+00:00

Exactly the same problem as well.

Someone from MS should investigate.
Kyle Musser 1 Reputation point

2022-02-17T19:23:41.913+00:00

Again same problem here, plz prioritize
Kyle Musser 1 Reputation point

2022-02-18T16:08:01.537+00:00

Seems like whatever fix they implemented worked, i was able to run my AML pipeline from inside synapse at 9am MDT
Camille Fougere (SHE/HER) 1 Reputation point

2022-02-18T18:21:36.577+00:00

Same problem here
Camille Fougere (SHE/HER) 1 Reputation point

2022-02-18T18:21:43.197+00:00

Same problem here

Accepted answer

1 additional answer

Your answer

Tka32 56 Reputation points

2022-02-17T10:22:38.593+00:00

I also have the same issue.
We have a pipeline triggering ML pipelines, which used to run successfully until yesterday, started failing this morning with the same error.
Razvan Vacaru 26 Reputation points

2022-02-17T10:25:37.343+00:00

We have ADF pipelines running in production already for sometime, we didn't update anything in relation to either the ADF pipeline or the AzureML pipeline.
Nevertheless from this morning at ~8 CET our scheduled pipelines failed with the same error. We've tried re creating the "Machine Learning Execute Pipeline" activity or using PipelineId instead of PipelineEndpointId but without any success so far. It seems to be un underlying issue in Azure/ADF with how ADF invokes the ML Pipeline

Please prioritize this.
Rodrigo Barraza 16 Reputation points

2022-02-17T12:35:24.377+00:00

I have exactly the same problem.
I have a pipeline in synapse which executes a pipeline in ML, I don't know any modifications have been made in two months in either of the two environments and it has always been executed correctly, but today it fails throwing the same message:

"errorCode": "2105",
"message": "Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType'"
Veselin Valchev 1 Reputation point

2022-02-17T15:15:11.35+00:00

Same problem here - anyone has any feedback from Microsoft re this?
Thomas Servotte 1 Reputation point

2022-02-17T15:20:59.05+00:00

Same problem here
Milen Chechev 1 Reputation point

2022-02-17T15:31:23.693+00:00

Exactly the same problem as well.

Someone from MS should investigate.
Kyle Musser 1 Reputation point

2022-02-17T19:23:41.913+00:00

Again same problem here, plz prioritize
Kyle Musser 1 Reputation point

2022-02-18T16:08:01.537+00:00

Seems like whatever fix they implemented worked, i was able to run my AML pipeline from inside synapse at 9am MDT
Camille Fougere (SHE/HER) 1 Reputation point

2022-02-18T18:21:36.577+00:00

Same problem here
Camille Fougere (SHE/HER) 1 Reputation point

2022-02-18T18:21:43.197+00:00

Same problem here

Answer 1

AnnuKumari-MSFT 34,556 Microsoft Employee Moderator

Hi @Vinoth Kumar K ,
Welcome to Microsoft Q&A platform and thankyou for posting your query.
As per the details you have shared in the query, it looks like a product bug. I have raised this issue with the internal Product team. Once I hear back from them, I will keep everyone posted on this. Thanks for your patience!

AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2022-02-17T16:55:08.917+00:00

Hi @Vinoth Kumar K ,
Product team has suggested to use SHIR instead of Azure IR as the current workaround which might help in mitigating the issue. Please do let us know if this solution works in your case. Although, the team is doing further investigation on the same.
Vinoth Kumar K 56 Reputation points

2022-02-18T06:18:09.517+00:00

Since my workload is still on the development stage, i would prefer to wait for the product team to fix the issue rather than setup the SHIR. Please do keep posted if the issue is resolved or any further developments on this.
Birsan, D.E. (Dumitru - Eugen) 1 Reputation point

2022-02-18T09:02:26.243+00:00

The SHIR solution is not working. I get the same error.
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2022-02-18T09:33:22.74+00:00

Hi ,
Apologies for the inconvenience . Product team is working on this issue taking it as priority. Hot fix for this issue is in progress. It should be resolved in next 12 hours.
Vinoth Kumar K 56 Reputation points

2022-02-21T04:09:50.617+00:00

Its working now without any issues. Thanks
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2022-02-21T05:22:57.103+00:00

Hi @Vinoth Kumar K ,
Thankyou for the update. Glad to know that the issue has been resolved . Please consider accepting the answer by clicking Accept Answer and please do Up-Vote for the same as accepted answer helps community as well. And, if you have any further query do let us know.
ELIO CECCHETTO 6 Reputation points

2022-02-21T08:18:26.837+00:00

I still do see the same error

Convert Failed. The value type 'System.String', in key 'azureCloudType' is not expected type 'Microsoft.DataTransfer.Common.Models.AzureCloudType'
Vinoth Kumar K 56 Reputation points

2022-02-21T08:42:03.647+00:00

What region you are using? May be the fix is not propagated to all regions yet. i am using East US
ELIO CECCHETTO 6 Reputation points

2022-02-21T08:50:39.677+00:00

West Europe
AnnuKumari-MSFT 34,556 Reputation points Microsoft Employee Moderator

2022-02-21T09:27:13.01+00:00

Hi @ELIO CECCHETTO ,
Could you please try refresh your login session , clear cache and try logging again and running the pipeline?
I checked with the product team and got the confirmation that the fix has been deployed to all the regions except Azure Sovereign Regions(government). One workaround they suggested is to use SPN authentication for running the pipeline. Please let us know if you still face the issue after implementing the suggested solutions. Thanks !
ELIO CECCHETTO 6 Reputation points

2022-02-21T15:42:21.86+00:00

Thanks Annu.
we do actually still have the problem. Though I have to add that because our AML is behind a private network, we used, for ADF, an IR hosted on Azure (public.). From that IR the connection is still giving the same error,
At the same time, we get no error if we use the native IR coming with ADF.
My conclusion is that the bug fix does not work for all sort of network configurations.
Birsan, D.E. (Dumitru - Eugen) 1 Reputation point

2022-02-21T15:50:37.753+00:00

The method that worked for me was the switch to SPN authentication regardless of the IR
Jose Manuel Espinoza 1 Reputation point

2022-02-24T22:54:34.117+00:00

Hi,

I'm facing the same issue posted here calling an ML pipeline through the endpoint from Synapse.
Region AU East.

Thanks for the help in advance.
Ahmed Mahmoud 81 Reputation points

2022-02-28T08:09:36.917+00:00

@Jose Manuel Espinoza Can you retry again and let us know if you still face the same issue.

Regards,
Ahmed

Answer 2

Jose Manuel Espinoza 1

Hi Ahmed,

Working now :)
Many thanks.

Regards,
José.

Ahmed Mahmoud 81 Reputation points

2022-02-28T10:15:22.403+00:00

Hi Jose Manuel,

Thankyou for the update. Glad to know that the issue has been resolved!

Regards,
Ahmed

Share via

Trigger Azure ML Pipeline from Azure Data Factory

1 additional answer

Your answer