Parse Data Flow -> External Call response using UTF-8 encoding?

Question

Parse Data Flow -> External Call response using UTF-8 encoding?

AshTT-7826 6

Hey all,

Having a little trouble getting ADF to correctly parse a UTF-8 XML response from an external call activity within a data flow. The API XML response doesn't have an encoding tag (limitation of the service we're using). Is there any way to force the encoding of the response in the activity.. have I missed a trick here?

Example data:

Directorâ s

Thanks, Ash.

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2022-04-15T15:33:10.43+00:00

Hello @AshTT-7826 ,

Thanks for the question and using MS Q&A platform.

As I understand the ask here that your External endpoint returns a response of XML type. Please correct me if I'm not correct.

As you are using External call transformation, you will be using REST connector. Please note that in ADF REST Connector, only response payload in JSON is supported.

You may have to try with HTTP connector in regular ADF pipeline and not Dataflows.

Please let us know if you have questions.
AshTT-7826 6 Reputation points

2022-04-18T21:58:07.973+00:00

Hey Kranthi,

Thanks for getting back to me! Ah okay, you are correct I am using External Call connector to consume an XML endpoint.

I didn't realise the connector only supported JSON, the UI itself within the External Call connector does seem to only allow REST inline connector but also does give the option to choose an XML format for the source data.

Not sure if this is a mistake in the UI or maybe I'm misunderstanding the use case? It's certainly working, parsing the XML into an object correctly just incorrect encoding when doing so..

Thanks, Ash.

1 answer

Your answer

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2022-04-15T15:33:10.43+00:00

Hello @AshTT-7826 ,

Thanks for the question and using MS Q&A platform.

As I understand the ask here that your External endpoint returns a response of XML type. Please correct me if I'm not correct.

As you are using External call transformation, you will be using REST connector. Please note that in ADF REST Connector, only response payload in JSON is supported.

You may have to try with HTTP connector in regular ADF pipeline and not Dataflows.

Please let us know if you have questions.
AshTT-7826 6 Reputation points

2022-04-18T21:58:07.973+00:00

Hey Kranthi,

Thanks for getting back to me! Ah okay, you are correct I am using External Call connector to consume an XML endpoint.

I didn't realise the connector only supported JSON, the UI itself within the External Call connector does seem to only allow REST inline connector but also does give the option to choose an XML format for the source data.

Not sure if this is a mistake in the UI or maybe I'm misunderstanding the use case? It's certainly working, parsing the XML into an object correctly just incorrect encoding when doing so..

Thanks, Ash.

Answer 1

AshTT-7826 6

Managed to find a workable solution, adjusting the Content-Type of my APIs response to contain the correct charset (e.g. Content-Type: text/xml; charset=utf-8) appears to work as expected and is parsed correctly by Data Factory. Thanks, Ash.

KranthiPakala-MSFT 46,642 Reputation points Microsoft Employee Moderator

2022-04-27T23:55:24.167+00:00

Hello @AshTT-7826 ,

Appreciate much for sharing your findings and glad to know that you were able to figure it out. Regarding the XML support, looks like product team may have added it recently but I have passed on the feedback to respective product owners to update the documentation with updated information. Good call out and thanks for helping us improve the Azure Product and our documentation.

Have a great day.

Share via

Parse Data Flow -> External Call response using UTF-8 encoding?

1 answer

Your answer