Invoking RESI API and Uploading File through API Management using ADF Data Pipeline

Question

Hello,

We have a requirement to Invoke a REST API call through API Management using ADF Data Pipeline. Data Pipeline would have the source as ADLS Gen2 where CSV files would be extracted and invoke External REST API call through API Management API, API Management would act as Façade for the external API call. While invoking the external REST API call File extracted from ADLS should be sent as parameter. Please let me know how we could achieve the requirement and suggest if there are alternative option to achieve the requirement.

Answer 1

Thanks,

So the external API is expecting the actual file content to be sent to it as part of the call? If this is the case then my initial idea won't work and I'm not sure how to proceed, but I provide my initial idea below in case it helps:

The call would be made to the APIM facade endpoint including a filepath as a parameter. APIM would then call another internal API (such as a function app) which would then use that filepath parameter (along with other information that it may have from config) to generate a user-delegation SAS (readonly, for that specific BLOB) via a system managed identity. This API would then pass back a full SAS based URL to APIM.. This SAS URL would be sent as a parameter to the external API call (by APIM), and the external API would use that SAS based URL to directly access the storage account to download the file.

There is example MS code for such a function app. The SAS would be given a short lifetime and could even be limited by source IP address (If that would be known in advance). Calling other APIs in APIM policy is done via the 'Send Request' policy. And receiving the returned value and then constructing the call to the down-stream API is straightforward in APIM policy.

I have implemented almost exactly this to allow end-user client to securely access and download a BLOB from storage, directly, without having to serialise/deserialise the file in the applications.

Good luck!