Calling Rest Api with Web Activity cannot parse the Response

Question

Calling Rest Api with Web Activity cannot parse the Response

Silvano P 91

Hello,

I have the following scenario need to call a Rest API to query a Database and save result to Azure Data Lake Gen 2.

I tried the REST API with PostMan and Soap UI and I have no problems, but when I try to use ADF I have issues. I managed to get the Token and used other Methods, but one is driving me crazy. I can get the Response through the Web Activity but no matter what "accept" I use in the header (CSV or JSON) I always get the following response:

"Response": "x��RMS�0\u0014�/\\u0015'�S{K�G�%\t��h��\u0007=y��}P�q\u0004d��f_�}&g\r%\u0010�d�� &��Tt揑L܉�OO�v΂�@��ں��U�x�a\u001ar@+�#p�)�)\u0004m\u0010\u0014YI!�j�d�ch=\u0012\u0004B�s�=pB��x�ܜ�茂\u000f�~K\u0019(�C�~&fY֩X�Xo��&kS<�H�O\u000f��#\u0016��ָ��\u0018OgQ5ѲU�_�d��C�!\u0016�ˈ�zhD�rj&\u0018�ʣ\u0006l#�~��U�}��L�\u0010B�8=�\u0016\"��0QZUJ�ە�^�:�:\u0002�w�\rr�\u0016��)��\u000f7�Eډ��\u0006\u000b��\u0014*PF��%�=R\�r�v�a�*�R��V6=F\u0004�a\"\u0016\u0019�\��\u0018�lIkMy\u0007�:W�~�O|�פWt6��\u0007�� J�(__�s1��(\u0011��KW_�!�\t�",
"ADFWebActivityResponseHeaders": {

Any idea?

Also is there a limitation for the Response Dimension?

I wanted to use the Copy activity but having similar issues.

Thanks

Vaibhav Chaudhari 37,881 Reputation points

2021-07-08T06:19:59.383+00:00

Are you using some encoding in headers when you are trying it in postman? Maybe try the same in Web activity too
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-09T10:22:46.48+00:00
Hi @Silvano P ,

Thank you for posting query in Microsoft Q&A Platform.

Could you please help on below clarifications to understand issue better and help on detailed resolution.

What is the expected response json from REST API call?

Are you using any encoding in header when making API call from postman?

You mentioned, API will query DB and save results. Will API response contains any json data too back from DB? If yes, Could you please help with sample data.

It would be great help if you can share post man request headers and body details.

Thank you.
Silvano P 91 Reputation points

2021-07-09T19:08:55.797+00:00

Hello Shaik,

thank you so much for your help.

The encoding is UTF-8 and the header's API as "accept" expects the following:

test/csv
application/x-ndjson

The API response can be CSV or Json.

Can you please provide a private email so I can send you the required data?

Thank you so much
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-12T04:15:44.987+00:00

Hi @Silvano P ,

Thank you for response.
You mentioned your API response can be csv or json. That means is it going to download a file when you may API call?
Silvano P 91 Reputation points

2021-07-12T04:22:48.227+00:00

Hello Shaik,

thanks for your answer.

I'm not sure how can I check if this is a file or a "string"?

Thanks
Silvano P 91 Reputation points

2021-07-13T16:32:53.287+00:00

Hello,

Yes it will download a file.

Can you please send me a private email so I can send you request and response?

Thank you so much
Silvano P 91 Reputation points

2021-07-15T16:58:25.223+00:00

Hello @ShaikMaheer-MSFT ,

can i send you Request and Response in your private email so you can give me a feedback?

Thank you so much
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-15T17:38:22.683+00:00

Hi @Silvano P ,

please do email us to AzCommunity[at]microsoft[dot]com with below details.
• Email subject: <Attn - Maheer: Microsoft Q&A Thread title>
• Thread URL: <Microsoft Q&A Thread>

Between, When you are trying to hit API which actually download file then you should use HTTP connector dataset as source in copy activity to copy that file your target. Kindly check provided answer below.

Please Accept Answer if that helps and feel free to share if any further queries. Thank you.
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-20T01:14:59.42+00:00

Hi @Silvano P ,

Following up to check is below provided answer helps you. If yes, then please Accept Answer. Thank you.

Vaibhav Chaudhari 37,881 Reputation points

2021-07-08T06:19:59.383+00:00

Are you using some encoding in headers when you are trying it in postman? Maybe try the same in Web activity too
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-09T10:22:46.48+00:00

Hi @Silvano P ,

Thank you for posting query in Microsoft Q&A Platform.

Could you please help on below clarifications to understand issue better and help on detailed resolution.

What is the expected response json from REST API call?

Are you using any encoding in header when making API call from postman?

You mentioned, API will query DB and save results. Will API response contains any json data too back from DB? If yes, Could you please help with sample data.

It would be great help if you can share post man request headers and body details.

Thank you.
Silvano P 91 Reputation points

2021-07-09T19:08:55.797+00:00

Hello Shaik,

thank you so much for your help.

The encoding is UTF-8 and the header's API as "accept" expects the following:

test/csv
application/x-ndjson

The API response can be CSV or Json.

Can you please provide a private email so I can send you the required data?

Thank you so much
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-12T04:15:44.987+00:00

Hi @Silvano P ,

Thank you for response.
You mentioned your API response can be csv or json. That means is it going to download a file when you may API call?
Silvano P 91 Reputation points

2021-07-12T04:22:48.227+00:00

Hello Shaik,

thanks for your answer.

I'm not sure how can I check if this is a file or a "string"?

Thanks
Silvano P 91 Reputation points

2021-07-13T16:32:53.287+00:00

Hello,

Yes it will download a file.

Can you please send me a private email so I can send you request and response?

Thank you so much
Silvano P 91 Reputation points

2021-07-15T16:58:25.223+00:00

Hello @ShaikMaheer-MSFT ,

can i send you Request and Response in your private email so you can give me a feedback?

Thank you so much
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-15T17:38:22.683+00:00

Hi @Silvano P ,

please do email us to AzCommunity[at]microsoft[dot]com with below details.
• Email subject: <Attn - Maheer: Microsoft Q&A Thread title>
• Thread URL: <Microsoft Q&A Thread>

Between, When you are trying to hit API which actually download file then you should use HTTP connector dataset as source in copy activity to copy that file your target. Kindly check provided answer below.

Please Accept Answer if that helps and feel free to share if any further queries. Thank you.
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-07-20T01:14:59.42+00:00

Hi @Silvano P ,

Following up to check is below provided answer helps you. If yes, then please Accept Answer. Thank you.

Answer 1

ShaikMaheer-MSFT 29,881 Microsoft Employee

Hi @Silvano P ,

Thank you using Microsoft Q&A Platform.

Thanks for sharing details on requested asks.
HTTP connector should be used to download files from API.
REST connector can be used if your response is json data(not json or csv file).

Based on your API response you can choose either of above connectors. Thank you.

Hope this will help.

--------------------------

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Aman Sharma 1 Reputation point

2021-11-30T07:42:10.49+00:00

Hi @Silvano P -

I ma also facing the same issue . Can you please help me with solution if you were able to solve this
ShaikMaheer-MSFT 29,881 Reputation points Microsoft Employee

2021-12-01T09:54:38.267+00:00

@Aman Sharma , Hope above comments and answer helps. Please feel free to open a new question in Microsoft Q&A with detailed steps of implementation and error details. Thank you.

Calling Rest Api with Web Activity cannot parse the Response

1 answer

Activity