How to disable server side validation when using web activity in DataFactory

Question

How to disable server side validation when using web activity in DataFactory

Nagarajan, Anand 26

We are planning to use data factory to fetch data from an API . The API provider has implemented a Client Credentials apporach where we will have to first call an API with a client secret and Client password to obtain a token . The obtained token will be used subsequently used with other Api calls to get data.

Our challenge here is that when we try to call the API to get the tokens , the web activity returns a 2108 error and it turns out the issue is due to the fact that server side validation certification fails and we are unable to figure out how to the disable server side validation in the web api.

Please note that the same call works in Postman when the server side validation is disabled.

Could you guide us how we can overcome this problem , is using webapi justified or is there any other activity that is more apt for this and means to use the same

Note that we have set the header to the correct Content type and have included the granttype , client_id and client_secret values without quotes in the body

HarithaMaddi-MSFT 10,146 Reputation points

2021-01-20T07:52:58.4+00:00

Hi @Nagarajan, Anand ,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Please share the pipeline run id that is throwing the error as that would help us to access logs for further investigation.

Looking forward for your response!
Nagarajan, Anand 26 Reputation points

2021-01-20T09:24:01.97+00:00

Hei @HarithaMaddi-MSFT
Here is the pipeline id : 569959e7-8442-489b-8267-8cc13009c710
I can add that the error code is : 2108
and that when i test the same url using postman with ssl disabled i see a warning " unable to get local issuer certificate" , although i get the results
and when i test the same using postman with ssl enabled , i get an error "unable to get local issuer certificate"
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-21T05:28:57.593+00:00

Hi @Nagarajan, Anand ,

Thanks for sharing the details. I have shared the information with product team and waiting to hear back from them. I will get back to you as soon as I get more insights.

Stay tuned!

Accepted answer

0 additional answers

Your answer

HarithaMaddi-MSFT 10,146 Reputation points

2021-01-20T07:52:58.4+00:00

Hi @Nagarajan, Anand ,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Please share the pipeline run id that is throwing the error as that would help us to access logs for further investigation.

Looking forward for your response!
Nagarajan, Anand 26 Reputation points

2021-01-20T09:24:01.97+00:00

Hei @HarithaMaddi-MSFT
Here is the pipeline id : 569959e7-8442-489b-8267-8cc13009c710
I can add that the error code is : 2108
and that when i test the same url using postman with ssl disabled i see a warning " unable to get local issuer certificate" , although i get the results
and when i test the same using postman with ssl enabled , i get an error "unable to get local issuer certificate"
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-21T05:28:57.593+00:00

Hi @Nagarajan, Anand ,

Thanks for sharing the details. I have shared the information with product team and waiting to hear back from them. I will get back to you as soon as I get more insights.

Stay tuned!

Answer 1

HarithaMaddi-MSFT 10,146

Hi @Nagarajan, Anand ,

Thanks for your patience. Product team mentioned it as a bug and they are working on fixing it by having an approach to add the following feature flag to the end of the data factory URL which will allow to select an option in the web activity to disable cert validation as below. This will be available in future releases of data factory and please follow the updates page to check this once available.

&feature.disableWebCertValidation=true

A new option will appear once the url is appended in web activity as below, please check the box, save and publish your pipeline, then, try executing it again.

Please let us know for further queries and we will be glad to assist.

--

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Nagarajan, Anand 26 Reputation points

2021-01-25T09:37:58.037+00:00

Hi @HarithaMaddi-MSFT
Thank you very much for your quick response. However the issue is still not solved.
As the authentication scheme that we use is basic authentication
Here is the pipeline ID :
25fdc616-3622-4ba3-bf54-a0e24dba7754
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-25T13:11:30.773+00:00

Hi @Nagarajan, Anand ,

Thanks for sharing the details. Is there any possibility to switch to this authentication mode for your requirement. I also reached out to product team to understand if there is any other workaround that can be used with Basic authentication and will update you once I hear from them.
Nagarajan, Anand 26 Reputation points

2021-01-25T13:15:58.133+00:00

hi @HarithaMaddi-MSFT Thank you very much for your quick response. Unfortunately we are not able to switch to anyother authentication other than basic authentication as the solution is delivered by a third party.
The third party solutioroviders have implemented oauth with a bearer token approach and the flow consists of authenticating first with a user name and password to get the bearer token and using the bearer token in subsequent calls.
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-27T06:58:28.037+00:00

Hi @Nagarajan, Anand ,

Thanks for sharing more details. Product team confirmed that we can use BASIC authentication by adding "&feature.disableWebCertValidation=true" to the URL and it should work well without server side validation. Please let us know how it goes and if issue persists, please share pipeline run id with us to share with internal team for additional investigation.
Nagarajan, Anand 26 Reputation points

2021-01-27T09:55:49.967+00:00

Hi @HarithaMaddi-MSFT
Unfortunately , this does not function even after adding the feature.disablewebcertvalidation = true on the url .
I have also tried adding the same to the headers and as well to the body but still the same error.

Here is the pipeline ID : 314693d9-e3bb-4280-adda-ea376eb9c498
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-27T10:04:33.58+00:00

Thanks @Nagarajan, Anand for confirming the workaround suggested. I shared the same to product team and I will get back to you as soon as I hear from them.

Appreciate your support!
HarithaMaddi-MSFT 10,146 Reputation points

2021-01-29T07:01:36.483+00:00

Thanks @Nagarajan, Anand for sharing the additional details. Product team mentioned that feature flag has to be appended to data factory URL and then these options appear where BASIC auth can be selected. From the shared pipeline run id, it is observed that the feature flag is appended to web activity URL but not MADRID UI.

Please check and let us know if this is working fine else share the pipeline run id with us again to look at the details.
Nagarajan, Anand 26 Reputation points

2021-01-29T08:14:49.473+00:00

hi @HarithaMaddi-MSFT
Thank you very much for your response.
Iam not able to figure out when you say : "feature flag has to be appended to data factory URL"
Is it possible to show an image that depicts this ?
Thanks in advance
HarithaMaddi-MSFT 10,146 Reputation points

2021-02-01T08:04:18.617+00:00
Hi @Nagarajan, Anand ,

As discussed, feature flag needs to be added to the URL of the data factory, sample one is as shown below and then it will work without server validation in web activity.

Please let us know for further queries and we will be glad to assist.

https://ms-adf.azure.com/en-us/authoring/pipeline/pipeline1?factory=%2Fsubscriptions%xxxx%2FresourceGroups%2Fxxx%2Fproviders%2FMicrosoft.DataFactory%2Ffactories%2Fharithaadf73&l=en-us&feature.disableWebCertValidation=true

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members
Nagarajan, Anand 26 Reputation points

2021-02-01T09:17:22.877+00:00

Hi @HarithaMaddi-MSFT
Thank you very much for your help , but it still does not work
ON appending the feature.disableWebCertValidation= true to the datafactory Url , I get a non trusted error and then iam redirected back to the select page for data factory

Here is a snapshot of the error that i get
HarithaMaddi-MSFT 10,146 Reputation points

2021-02-02T10:12:49.84+00:00

Thanks @Nagarajan, Anand for sharing. I reached out to product team with this error to know their thoughts, I will get back to you once I hear from them.
HarithaMaddi-MSFT 10,146 Reputation points

2021-02-03T07:29:50.513+00:00

Thanks @Nagarajan, Anand for sharing the details. Product team confirmed that it is a bug and they are working on it. The changes should be live in future releases. Apologize the inconvenience.

Please let us know for further queries and we will be glad to assist. Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members.
HarithaMaddi-MSFT 10,146 Reputation points

2021-02-04T05:29:58.273+00:00

Hi @Nagarajan, Anand ,

We have not received a response from you. Please suggest if above suggested details are helpful. Otherwise, let us know and we will continue to engage with you on the query.

Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members.
Nagarajan, Anand 26 Reputation points

2021-02-04T09:11:53.103+00:00

Hi @HarithaMaddi-MSFT
As it stands it is a bug at your end.
Once the bug is fixed , I can test the same and after it works can i accept the answer, isnt that correct.
Could you also let me know if there are any other work arounds, say use of another data factory task which perhaps iam not aware of to address the issue ?
HarithaMaddi-MSFT 10,146 Reputation points

2021-02-04T13:13:00.907+00:00

Hi @Nagarajan, Anand ,

Thanks for understanding. The bug deployment may take a month or more depending on the priority of the release items and it is recommended to accept answer as other customers knows that it is confirmed as a bug and awaiting deployment who are looking at community forum.

As per my understanding, there is no possible workaround for this scenario using data factory activities other than custom coding. Please suggest if this helps in clarifying the question.

Share via

How to disable server side validation when using web activity in DataFactory

0 additional answers

Your answer