Azure Data Factory Web Activity Truncates Token

Sam Tran 1 Reputation point
2021-09-07T06:28:24.42+00:00

I've configured a web activity in Azure Data Factory to consume various REST services and specifically, get tokens from various Microsoft services such as Power BI and Azure Function (using AAD authentications) and the tokens are coming back. The problem seems the token has been truncated and can't be used in the next GET activity to actually get the data. The pipeline is definitely correct, just the tokens are all truncated. (If I substitute it with ones from Postman, it works). Anyone experience this before and know a workaround? This is the json that comes back with the call. The tokens I get in Postman or C# are all much longer

{ "token_type": "Bearer", "expires_in": "86399", "ext_expires_in": "86399", "expires_on": "1631080251", "not_before": "1630993551", "resource": "8256e7be-fcf0-4463-b7ec-a49300edc57a", "access_token": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCIsImtpZCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCJ9.eyJhdWQiOiI4MjU2ZTdiZS1mY2YwLTQ0NjMtYjdlYy1hNDkzMDBlZGM1N2EiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC83MmY5ODhiZi04NmYxLTQxYWYtOTFhYi0yZDdjZDAxMWRiNDcvIiwiaWF0IjoxNjMwOTkzNTUxLCJuYmYiOjE2MzA5OTM1NTEsImV4cCI6MTYzMTA4MDI1MSwiYWlvIjoiRTJaZ1lOZ3RQVEhIVkp6QjZScDc5aE9tcnA2SkFBPT0iLCJhcHBpZCI6IjgyNTZlN2JlLWZjZjAtNDQ2My1iN2VjLWE0OTMwMGVkYzU3YSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmsfRvd3MubmV0LzZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJyaCI6IjAuQVJvQXY0ajVjdkdHcjBHUnF5MTgwQkhiUjc3blZvTHdfR05FdC15a2t3RHR4WG9hQUFBLiIsInRpZCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsInV0aSI6InlQb2dzN3hYMTB5ZlkyNWF0bGZPQUEiLCJ2ZXIiOiIxLjAifQ.mKnct3f_UceaUOcXg_Oddd4nqU-Wmw1NHet3Cz_UuOA5HLgmWv0oGKNFTGfCmLX00zi1AgZ6WIl-9JIOssJCP3ha37SfJ1l2-Vm-XeU9kGqLd4EjGzJ6jXHI5bmzQs3quGMzjmIAUvfJymq-HSMNMOmnGNJPT5Exr1l-iFTX-dWWY8G2XQQjf-icq_cjod2dRSP5IK-b5_MprvzveibceKnCWOMEI5gWZwCZNh1TWPa6AAYZ0Yk1YexWG9shwqBPpUtHygt7LuO8zXzMIhpINyMuUp0NoDKSDhKgwmc_aRBWO7QSjsCBepc-2s0cmi8sIn-rEiJpuPed5qgyoVVwbA", "ADFWebActivityResponseHeaders": { "Pragma": "no-cache", "Strict-Transport-Security": "max-age=31536000; includeSubDomains", "X-Content-Type-Options": "nosniff", "x-ms-request-id": "b320fac8-57bc-4cd7-9f63-6e5ab657ce00", "x-ms-ests-server": "2.1.12025.12 - KRSLR1 ProdSlices", "Cache-Control": "no-store, no-cache", "P3P": "CP="DSP CUR OTPi IND OTRi ONL FIN"", "Set-Cookie": "fpc=AhFRmOZBxGtFgW2I5JvVnQI; expires=Thu, 07-Oct-2021 05:50:51 GMT; path=/; secure; HttpOnly; SameSite=None;x-ms-gateway-slice=estsfd; path=/; secure; httponly;stsservicecookie=estsfd; path=/; secure; httponly", "Date": "Tue, 07 Sep 2021 05:50:51 GMT", "Content-Length": "1319", "Content-Type": "application/json; charset=utf-8", "Expires": "-1" }, "effectiveIntegrationRuntime": "DefaultIntegrationRuntime (Southeast Asia)", "executionDuration": 0, "durationInQueue": { "integrationRuntimeQueue": 0 }, "billingReference": { "activityType": "ExternalActivity", "billableDuration": [ { "meterType": "AzureIR", "duration": 0.016666666666666666, "unit": "Hours" } ] } }

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,484 questions
0 comments

4 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2021-09-07T22:20:51.517+00:00

    Hello @Sam Tran ,
    Thanks for the ask and using the Microsoft Q&A platform .
    This is what i did , I created a web activity ( WA) and passed the required details and get the access token . Oncer I have the access token I pass the same to to the next web activity ( I am calling a ADF API ) . Please beware when you pass the access token to this WA , you will have to appened the "Bearer[Space]" before the token .

    This is dynamic expression which I used .

    @markus.bohland@hotmail.de ('Bearer ',activity('Web Access token').output.access_token)

    129994-image.png

    130063-image.png

    Please do let me know how it goes .
    Thanks
    Himanshu
    Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

    0 comments
  2. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2021-10-05T17:22:20.023+00:00

    Hello @Sam Tran ,
    Apolziges for the delay in response on my side . Unfortunately we are not getting the kind of response from the team here . if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.
    Subscription ID:
    Subject : Attn Himanshu
    Please let me know once you have done the same.
    Thanks
    Himanshu

    0 comments
  3. Sam Tran 1 Reputation point
    2021-09-07T23:05:32.993+00:00

    Hi Himanshu,
    Your suggestion is what I did but the problem is something else. As I said, the token that comes back is truncated. If you grab the same token (from the JSON) from the Output in the first step and use that in something like Postman, it fails too as it has been truncated.
    So as example, the token that comes back in ADF may have 100 characters but it should actually be 130

    So in ADF, the token that comes back is:
    eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCIsImtpZCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCJ9.eyJhdWQiOiI4MjU2ZTdiZS1mY2YwLTQ0NjMtYjdlYy1hNDkzMDBlZGM1N2EiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC83MmY5ODhiZi04NmYxLTQxYWYtOTFhYi0yZDdjZDAxMWRiNDcvIiwiaWF0IjoxNjMwOTkzNTUxLCJuYmYiOjE2MzA5OTM1NTEsImV4cCI6MTYzMTA4MDI1MSwiYWlvIjoiRTJaZ1lOZ3RQVEhIVkp6QjZScDc5aE9tcnA2SkFBPT0iLCJhcHBpZCI6IjgyNTZlN2JlLWZjZjAtNDQ2My1iN2VjLWE0OTMwMGVkYzU3YSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmsfRvd3MubmV0LzZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJyaCI6IjAuQVJvQXY0ajVjdkdHcjBHUnF5MTgwQkhiUjc3blZvTHdfR05FdC15a2t3RHR4WG9hQUFBLiIsInRpZCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsInV0aSI6InlQb2dzN3hYMTB5ZlkyNWF0bGZPQUEiLCJ2ZXIiOiIxLjAifQ.mKnct3f_UceaUOcXg_Oddd4nqU-Wmw1NHet3Cz_UuOA5HLgmWv0oGKNFTGfCmLX00zi1AgZ6WIl-9JIOssJCP3ha37SfJ1l2-Vm-XeU9kGqLd4EjGzJ6jXHI5bmzQs3quGMzjmIAUvfJymq-HSMNMOmnGNJPT5Exr1l-iFTX-dWWY8G2XQQjf-icq_cjod2dRSP5IK-b5_MprvzveibceKnCWOMEI5gWZwCZNh1TWPa6AAYZ0Yk1YexWG9shwqBPpUtHygt7LuO8zXzMIhpINyMuUp0NoDKSDhKgwmc_aRBWO7QSjsCBepc-2s0cmi8sIn-rEiJpuPed5qgyoVVwbA

    But if you run the same process in Postman, Python, C%, they all have more characters like this. The token that comes back outside ADF is good but the token that comes back in ADF has been truncated and is not accepted

    eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCIsImtpZCI6Imwzc1EtNTBjQ0g0eEJWWkxIVEd3blNSNzY4MCJ9.eyJhdWQiOiI4MjU2ZTdiZS1mY2YwLTQ0NjMtYjdlYy1hNDkzMDBlZGM1N2EiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC83MmY5ODhiZi04NmYxLTQxYWYtOTFhYi0yZDdjZDAxMWRiNDcvIiwiaWF0IjoxNjMwOTkzNTUxLCJuYmYiOjE2MzA5OTM1NTEsImV4cCI6MTYzMTA4MDI1MSwiYWlvIjoiRTJaZ1lOZ3RQVEhIVkp6QjZScDc5aE9tcnA2SkFBPT0iLCJhcHBpZCI6IjgyNTZlN2JlLWZjZjAtNDQ2My1iN2VjLWE0OTMwMGVkYzU3YSIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmsfRvd3MubmV0LzZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0Ny8iLCJyaCI6IjAuQVJvQXY0ajVjdkdHcjBHUnF5MTgwQkhiUjc3blZvTHdfR05FdC15a2t3RHR4WG9hQUFBLiIsInRpZCI6IjcyZjk4OGJmLTg2ZjEtNDFhZi05MWFiLTJkN2NkMDExZGI0NyIsInV0aSI6InlQb2dzN3hYMTB5ZlkyNWF0bGZPQUEiLCJ2ZXIiOiIxLjAifQ.mKnct3f_UceaUOcXg_Oddd4nqU-Wmw1NHet3Cz_UuOA5HLgmWv0oGKNFTGfCmLX00zi1AgZ6WIl-9JIOssJCP3ha37SfJ1l2-Vm-XeU9kGqLd4EjGzJ6jXHI5bmzQs3quGMzjmIAUvfJymq-HSMNMOmnGNJPT5Exr1l-iFTX-dWWY8G2XQQjf-icq_cjod2dRSP5IK-b5_MprvzveibceKnCWOMEI5gWZwCZNh1TWPa6AAYZ0Yk1YexWG9shwqBPpUtHygt7LuO8zXzMIhpINyMuUp0NoDKSDhKgwmc_aRBWO7QSjsCBepc-2s0cmi8sIn-rEiJpuPed5qgyoVVwbxAXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  4. Sam Tran 1 Reputation point
    2021-09-10T00:21:20.827+00:00

    Hi,
    I appreciate your help but the problem is the tokens gets truncated, even from other Microsoft services such as Power BI or Azure Active Directory for Function Apps.
    I am not trying to call Microsoft Graph. I'm trying to use Either Azure Functions or Power BI. I do not need to pass in a scope for these

    130848-capture.jpg

    0 comments