Azure Data Factory Rest API - With AccountCode, APIKey & Token

Question

I have a use case wherein, I need to connect to an API for data request.

• The API requires a valid token to process the requests.
• To generate the token, I have a accountCode & secret key

Assume BaseURL as
BaseURL - http://api.xxxxx.com/{accountCode}/data (Value of account needs to be passed)

Below script in Python/Java needs to be run to fetch the dateToken & token

- If we use Python 3.6 or above. Below is the code -

--START-- {
import time

import requests

from hashlib import md5

account_code = "<account_name>"

key = "<api_key>"

actual_unix_time = int(time.time_ns() / 1000) # in milliseconds

TTL = 31536000000 # for 1 year

expiration_time = actual_unix_time + TTL

base_url = "https://api.xxxxx.com"

url = f"/{account_code}/data?fromDate=last6Hours&granularity=minute&type=ALL%2CVOD%2CLIVE&operation=reduceJoin&metrics=bufferratio"

pre_url = f"{url}&dateToken={expiration_time}"

token_generated = md5(f"{pre_url}{key}".encode('utf-8'))

token_value = token_generated.hexdigest()

request_url = f"{base_url}{pre_url}&token={token_value}"

response = requests.get(request_url)

print(response)

print(response.text)

} --END--

- If we use Java. Below is the code -

--START-- {

var key = pm.environment.get("NPAW-API-KEY");
var base_url = "https://api.xxxxx.com";

var url = pm.request.url.toString();
var path = url.replace(base_url, '');
var pre_url = pm.variables.replaceIn(path);

var moment = require('moment');
var actual_unix_time = moment().unix()*1000;

var TTL = 31536000000

var expiration_time = (actual_unix_time + TTL);
var pre_url = pre_url+"&dateToken="+expiration_time;

var token_generated = CryptoJS.MD5(pre_url + key).toString();

var token_value = token_generated;

var request_url = (base_url+pre_url+'&token='+token_value).toString();

}--END--

Example of how the final URL - https://api.xxxxx.com/kb-vivek/data?fromDate=today&granularity=hour&type=ALL,VOD,LIVE&operation=reduceJoin&metrics=views,playtime&dateToken=1699016056000&token=7a9c97a4d4f108d1d32be2f7f8d00731

I tried to use Postman, wherein, I could pass the above script in the Pre-Request script and set environment variables for accountCode & Secret Key and I was able to achieve the result as desired.

Question: How can I achieve this in Azure Data Factory?

Answer 1

Hi @Vivek Komarla Bhaskar ,

Thank you for posting query in Microsoft Q&A Platform.

If I understand correctly, you would like to make API call to which you need to pass token. But to fetch token you have a code in python or Java which you would like to execute too using Azure data factory. Please correct me if my understanding is wrong.

You can consider deploying your code to azure functions and then use Azure Function activity in Data factory Pipeline.

OR

Usually, to generate tokens APIs have some Token end points too. If you know them then you can consider using web activity in data factory and make API call to token end point to fetch same and use it along with base URL.

OR

You can also consider using Python activity in data factory to run Python code. Click here to know more about it.

Hope this helps. Please let me know if any further queries.

------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification