question

LOLALola-5682 avatar image
1 Vote"
LOLALola-5682 asked LOLALola-5682 commented

Copy Data Import Schemas not working for API call in ADF

In the last 2 weeks, the "Import Schemas" button in the Mapping tab of the Copy Data activity has quit working when the source dataset is an API. This obviously affects new development, but to test it, I attempted to import the schema of an existing working copy activity, and it gave the same error. How can we fix this, and is there a current workaround?

139964-error.jpg


azure-data-factory
error.jpg (42.7 KiB)
· 3
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @LOLALola-5682 ,

Thank you for posting query in Microsoft Q&A Platform.

I tried to use an dummy API as source and tapped on Import schema its working fine.

It would be great help you can share your implementation? So that we can follow along to repro scenario and help on resolution.

If you are using any dynamic expression using parameters or variable, may be make sure you are passing correct values there before importing schema.

0 Votes 0 ·

Hi @ShaikMaheer-MSFT,

Thank you for your reply.

The pipeline is very simple, here is an overview. It is working because I can route the output to json or a table I've manually mapped, and we receive data. It is only Import Schemas that is giving us trouble.
140293-pipeline.jpg

First we use OAuth2.0 to receive an access token:
140254-get-access-token.jpg

Then we save this access token to a variable, which is successful:
140285-save-access-token.jpg

Finally there's the Copy Data activity:
140264-copy-source.jpg

Here is the setup of the REST API:
140301-rest-api-access-token.jpg

And the linked service behind it:
140240-linked-service.jpg

Here's my sink, using a test json:
140265-copy-sink.jpg

When going to Import Schemas, I copy and paste in the access token from before, since it is good for an hour:
140230-import-schema.png

And here is the result:
140286-import-schema-fail.jpg

(continued below due to character limit)

0 Votes 0 ·
pipeline.jpg (16.5 KiB)
copy-source.jpg (49.7 KiB)
linked-service.jpg (61.1 KiB)
copy-sink.jpg (27.9 KiB)
import-schema.png (40.0 KiB)

Here is the json file showing a successful copy:
140275-json-data.jpg

Finally, here is the full text of the error:
Error code
FailToResolveParametersInExploratoryController

Details
The parameters and expression cannot be resolved for schema operations. Error Message: { "message": "ErrorCode=InvalidTemplate, ErrorMessage=The function 'variables' is not supported by schema version '<null>': the function is introduced starting with version '2016-04-01-preview'. Please see https://aka.ms/logicexpressions for more details." } Activity ID: 85dd82e6-f722-4f90-9358-63d34d2451f7

Thank you for your help.


0 Votes 0 ·
json-data.jpg (89.0 KiB)

1 Answer

ShaikMaheer-MSFT avatar image
0 Votes"
ShaikMaheer-MSFT answered LOLALola-5682 commented

Hi @LOLALola-5682 ,

Thank you for posting query in Microsoft Q&A Portal and thank you for sharing requested details.

While, we hit import schema under mappings then ADF actually tries to hit API and get schema of it. But as per your implementation, API hit will get successful only if you run entier pipeline, because you are getting token first and then using that token dynamically inside your copy activity.

So if you would like to avoid this error of import schema, then please try to follow below steps.

Step1: Run web activity alone and get token
Step2: Take that token value and hard code inside copy activity immediately and then try to perform import schema.
This way, while you perform import schema your copy activity holds correct token in it and API call will get success.

Please note, usually tokens may have some expiry time too. So make sure you try above steps within that time frame.

Hope this will help. Please let us know if any further queries.


  • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  • Want a reminder to come back and check responses? Here is how to subscribe to a notification

  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

I am having the same issue with getting the "The function 'variables' is not supported by schema version '<null>" error in ADF, except I am connecting to an Azure SQL DB, not an API. Sometimes the preview / test connection button works, but most of the time it does not. This has only started recently for me as well.

2 Votes 2 ·

@CarterWinston-0165

I have also experienced this. The import button started working for me intermittently. It's a roll of the dice whether it does or not. Manually plugging in the API key works 100% of the time however.

0 Votes 0 ·

@ShaikMaheer-MSFT

That worked!!!!!!!!!!! Thank you so much!!!!!!!!!
140331-success.jpg


0 Votes 0 ·
success.jpg (41.9 KiB)

Hi @LOLALola-5682 ,

Glad to know that your issue resolved. Thank you for marking as accepted answer.

0 Votes 0 ·