Error when I try to preview external call transformation

Diyaa, Shridi 25

When I try to preview data of an external call transformation, I get the following error:

Spark job failed: { "text/plain": "{"runId":"a07da92a-34f0-4ede-bdbe-8ce326b612c8","sessionId":"6d629a83-5f12-4905-bf18-50443171fc3a","status":"Failed","payload":{"statusCode":400,"shortMessage":"DF-DSL-001 DSL stream has parsing errors\nLine 103 Position 20: extraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}","detailedMessage":"Failure 2023-10-05 11:32:40.388 failed DebugManager.processJob, run=a07da92a-34f0-4ede-bdbe-8ce326b612c8, errorMessage=DF-DSL-001 DSL stream has parsing errors\nLine 103 Position 20: extraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}"}}\n" } - RunId: a07da92a-34f0-4ede-bdbe-8ce326b612c8

I tried navigating to line 103 position 20 but didn't find out what's wrong. Appreciate any help.

Diyaa, Shridi 25 Reputation points

2023-10-08T08:05:06.3266667+00:00

The thing is, when I run the whole pipeline that includes the same dataflow that has external call, I get no errors and the external call returns the expected result. I'm just unable to preview the data in the data flow debug and I don't know why. There seem no syntax error and validation does not return any errors.
QuantumCache 20,261 Reputation points

2023-10-08T16:20:35.84+00:00

@Diyaa, Shridi

when I run the whole pipeline that includes the same dataflow that has external call,

Yeah, this is strange as the whole pipeline is running fine. Can you please try re-creating the whole Data Flow ? just for the sake of removing the doubt of product error in the Dataflow jobs?

What is the region of the ADF?

Is this Production lead or Test Environment?

If this issue is persistent, we will make sure you get help in this matter from the support team.
Diyaa, Shridi 25 Reputation points

2023-10-09T05:55:42.7833333+00:00

I did try to create the whole data flow with only the external call to try and roll out errors that might be cause by other jobs. Got the same error.

The region is East US and this a UAT environment, but we are preparing for initial launch. Just wanted to mention that this did work a couple of weeks ago. Meaning, I was able to debug the data flow and see the body returned by the external call in debug mode. Don't know if it's related, but the API I'm calling is an endpoint of an Azure http trigger function that in turn calls a third party API. Doing a Rest call outside of data flow using the same function works with no issues.
QuantumCache 20,261 Reputation points

2023-10-11T02:02:38.2366667+00:00

Hello @Diyaa, Shridi

Please send an email to azcommunity@microsoft.com with the below details, so that we can work offline on this matter with one time support ticket.

Your Email Subject : Attn Satish Boddu

Your Email Body content must include the below:

Thread URL: Link to this thread.

Your Azure Subscription ID: <This is must to be provided to us>

Accepted answer

QuantumCache 20,261 Reputation points

2023-10-05T23:20:15.2033333+00:00

Hello @Diyaa, Shridi Welcome to QnA forum!

It is possible that there is a syntax error or a missing parameter in the DSL stream.

I would suggest reviewing the DSL stream carefully and checking for any syntax errors or missing parameters. You may also want to try removing the extraneous input ''' at line 103 position 20 and see if that resolves the issue.

You can review the input and output of the external call transformation to ensure that they are correctly defined and match the expected keywords.

10262023 Offline Support Summary: by Original Poster (Diyaa, Shridi )

I received and tested the following solution from Microsoft support and found it to resolve the issue.

preview request checkbox in the debug settings panel should be unchecked and the preview should start working.

Please let us know if this issue is still persistent? comment in the below section!
Please sign in to rate this answer.
Diyaa, Shridi 25 Reputation points

2023-10-11T14:21:54.04+00:00

Thanks. I sent the email as instructed.

QuantumCache 20,261 Reputation points

2023-10-11T20:52:45.0966667+00:00

@Diyaa, Shridi Thank you, we are working on it.

Brett VanderPlaats 5 Reputation points

2023-10-24T01:42:55.39+00:00

I receive a similar error (below), my inputs to the external call are simple, and it seems when ADF creates the meta-data in the background of the workflow at line 122 pos: 20 , it is "name": "select3"

, that appears to be false positive.

If I start counting lines in the "scriptLines": [ , and go to the 122 line it shows this " statusColumnName: 'status',",

which doesn't look wrong to me

error:
Spark job failed: { "text/plain": "{\"runId\":\"d3093a62-4800-4395-abd0-ddb1021dbf43\",\"sessionId\":\"15f5f2bf-fc6f-443d-9848-ecdace77f5ac\",\"status\":\"Failed\",\"payload\":{\"statusCode\":400,\"shortMessage\":\"DF-DSL-001 at : DSL stream has parsing errors\\nLine 122 Position 20: \\trequestColumnName: 'RequestPayload,\\nextraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}\",\"detailedMessage\":\"Failure 2023-10-23 22:41:22.634 failed DebugManager.processJob, run=d3093a62-4800-4395-abd0-ddb1021dbf43, errorMessage=DF-DSL-001 at : DSL stream has parsing errors\\nLine 122 Position 20: \\trequestColumnName: 'RequestPayload,\\nextraneous input ''' expecting {DECIMAL_LITERAL, HEX_LITERAL, OCT_LITERAL, BINARY_LITERAL, MAX_INT, MIN_INT, MAX_LONG, MIN_LONG, POSITIVE_INF, NEGATIVE_INF, '-', '!', '$', '~', ':', '(', '#', '[', '@(', '[]', FLOAT_LITERAL, HEX_FLOAT_LITERAL, STRING_LITERAL, REGEX_LITERAL, 'parameters', 'functions', 'stores', 'as', 'input', 'output', 'constant', 'expression', 'integer', 'short', 'long', 'double', 'float', 'decimal', 'boolean', 'timestamp', 'date', 'byte', 'binary', 'integral', 'number', 'fractional', 'any', IDENTIFIER, ANY_IDENTIFIER, META_MATCH, '$$', '$$$', '$#', OPEN_INTERPOLATE}\"}}\n" } - RunId: d3093a62-4800-4395-abd0-ddb1021dbf43

Diyaa, Shridi 25 Reputation points

2023-10-24T06:53:44.9366667+00:00

Exactly. The indication of the error place is completely useless because I also wasn't able to find anything wrong with Line 103 Position 20.

QuantumCache 20,261 Reputation points

2023-10-24T18:26:25.4+00:00

Hello @Diyaa, Shridi Thanks for adding more context in this matter.

AshTT-7826 6 Reputation points

2023-10-25T04:09:32.79+00:00

I have exactly the same issue without changes we're suddenly seeing this error in preview, any idea on when this might be fixed?

Thanks, Ash.

QuantumCache 20,261 Reputation points

2023-10-26T16:01:50.1466667+00:00

Hello @Diyaa, Shridi Thanks for giving us an opportunity to help you in this matter.

We have posted the support summary in the above Response as Answer.

Please "Accept" the answer so that we close this forum case.
Sign in to comment

1 additional answer

Diyaa, Shridi 25 Reputation points

2023-10-26T10:33:02.7766667+00:00

I received and tested the following solution from Microsoft support and found it to resolve the issue.

preview request checkbox in the debug settings panel should be unchecked and the preview should start working.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment

Share via

Error when I try to preview external call transformation

1 additional answer