400 Bad request error in Synapse using copy activity - download zip file from https

Question

Hi,

I built a pipeline in Azure Synapse to download zip files through API OAUTH 2.0. I used Copy activity in a For each loop. Each zip file has its own unique download link (an unique extract ID is included in the URL).

An sample URL is like this: https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/SchemaID/plugins/PluginId/extracts/ExtractID

The issue I have is, the copy activity works fine when the extractID is made up of numbers, it fails when the extractID consists of letter.

For example, this download will succeed:

https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/SchemaID/plugins/PluginId/extracts/1699858856

This will always fail:

https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/SchemaID/plugins/PluginId/extracts/MTY5OTg1OTEwMA

Any advice is appreciated.

A screenshot of the pipeline

The error message:

Thanks,

Vikki

Answer 1

Try to manually test the failing URL in a browser or in Postman

Check the Brightspace documentation to understand how it handles different formats of extractID. There could be specific guidelines or limitations when dealing with alphanumeric IDs.

Answer 2

Hi Cheng, Vikki ,

Welcome to Microsoft Q&A platform and thanks for posting your query here.

I understand that you are trying to download the zip file using ADF/Synapse pipelines using HTTP connector ( binary format) , however it's not working when the alphanumeric characters are present in URL but it's working fine when ID is numerical .

URL might have limitation to contain a specific set of characters, and if your ExtractID includes characters that are not allowed in a URL, you might need to encode them. Could you please try using header as Content-Type = application/x-www-form-urlencoded

and see if it works.

Please let us know how it goes. Thankyou