Struggling to create API in Azure to receive token from data source and extract data.

Question

So I'm trying to use 2 API's from our external data source, an organization that solely focuses on delivering data to customers (me).

They have 2 different API's:

• A login API: this API is called with basic authentication (username and password) and will provide a bearer token
• An API to extract data and manage filters (this API will require the bearer token acquired through the login API)

Important notes:

• The login API requires a self-signed certificate, I have created a certificate on my PC using openSSL and shared the public key with the external data source.
• I uploaded the .pfx file in Azure (however i cannot set the domain or port number, this seems to be an issue, I can do this in Postman but can't seem to find it in Azure)
• The data source confirmed that the certificate is valid and I am able to use the certificate in Postman but only if i set the correct hostname and also the correct port number (Else i get a TLS error in Postman)
• Everything is working as it should in Postman (I can call the Login API and i can use the bearer token to access and download the data using a json file made for postman that i received from the data source organization)

I uploaded the .pfx certificate in Azure using AppServices > TLS/SSL settings and upload the private certificate, however the hostname it says here is my own database name (I am not sure if this is correct, in Postman i set the hostname when i upload the certificate to be the hostname of our data source with the needed port number)

I tried making a web call in Azure Datafactory, here i've used the correct URL i also use in Postman to access the login API and selected basic authentication and filled in the credentials

However when i try to debug pipeline (only the login web call) it gives the following error:

Error code 2108
Troubleshooting guide
Failure type User configuration issue
DetailsError calling the endpoint '[LINK i put with portnumber, which is correct]'. Response status code: 'NA - Unknown'. More details: Exception message: 'NA - Unknown [ClientSideException] An error occurred while sending the request.'.

Request didn't reach the server from the client. This could happen because of an underlying issue such as network connectivity, a DNS failure, a server certificate validation or a timeout.
Source
Pipeline
Punctuality

Am i looking in the wrong direction for this kind of data retrieval? is something going wrong with the certificate? I can't seem to find where i can set the correct hostname and port number (Like i can in Postman when uploading the certificate in .pfx format) Should I use API Management Service rather than Data Factory to make the API call to the external data source?

Answer 1

Hello @Steff and welcome to Microsoft Q&A. I haven't touched API management before, but I will help as best I can.

In digest, I see several asks here:

• where do I put port number in Web activity in Data Factory?
• Should the requests be made through API Management, or Data Factory? (is this just the login, or also data?)

I am a little confused, but I think this is what you are sayings:

Data Factory ---(client cert)---> API Management ---(Basic auth)--->Login API > Bearer token  
Data Factory ---(Bearer token)---> Data source  

The web activity authentication allows you to choose Basic auth or client certificate, but not both at the same time.

Assuming you are going to use Data Factory's REST dataset and Copy Activity, there are a few things to check before investing more time.
The REST connector only works with JSON data. You mentioned your test data was JSON, so this sounds fine.
Will you be paginating? There are now more pagination options in the REST connector, but you still need to see if it is a compatible mechanism.

I have another possible cause for that error code. Is your API Management inside a private network? By default, Web activity does not support that. The workaround is to install a Self-Hosted Integration Runtime (SHIR) in that network and run the web activity through that.

I haven't found anything on specifying port specifically in Web activity, but I do have a suggestion. Try using it inside the URL like

http://example.com:80/path  

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Hey so I solved the issue by selecting the client certificate and redirect it to the certificate i uploaded in Azure Key Vault. Since the API required both a client certificate and a basic authorization i put the base64 encoded username and password in the header as: "Authorization" "Basic <encoded username:password>"

This way it seems to accept both the certificate and the basic authorization at the same time and it gives me the requested bearer token.

Thanks for the input and the problem is solved :)