BUG: ADF Dataflow descriptions cannot contain '@{'

Mike McLean (Shell) 1 Reputation point
2020-11-21T12:09:10.48+00:00

EDIT: Original post incorrectly asserted that this was an issue with Pipeline Descriptions. It's not. It's Dataflow Descriptions that cause the problem. Post edited accordingly.

The descriptions appear to attempt to parse the interpolation syntax.

If your description contains '@{' without a closing '}' then you'll get an error referencing the description and asserting that the closing brace is missing.
Worse... if you description does contain the closing brace around something that's not code then you get a pretty inpenetrable error message:

e.g. Description of Some Dataflow contains: "iden@{tifie}d"

{
"code":"BadRequest",
"message":"ErrorCode=InvalidTemplate,ErrorMessage=Unable to parse expression 'tifie'",
"target":"pipeline/Write Some Data/runid/783ec88c-76e5-4f3b-91f8-6a626be591bf",
"details":null,
"error":null
}

note that Write Some Data isn't even the right pipeline!

Even worse than THAT... if the reason that your description has '@{}' in it is that it's providing code explanation, and the contents of the braces IS valid code then you get this message which is COMPLETELY indecipherable:

e.g. Description of Some Dataflow contains: "array(@{join(variables('myVariable'), ',')})"

{
"code":"BadRequest",
"message":"ErrorCode=InvalidVariableOperation, ErrorMessage=Invalid Pipeline Definition",
"target":"pipeline/Write Some Data/runid/6e11505b-81d8-42a5-bdc2-f591017b75ca",
"details":null,
"error":null
}

No context whatsoever ... except for a pipeline reference ... which is WRONG!

I lost about an hour and a half git bisecting through commits, and then through lines of a commit, then through lines of the description on a single LINE of a commit to find this.

Note that neither the Description UI, nor the "Validate" action pick up on this. The error only occurs once you attempt to invoke the pipeline!
Please fix this!
Preferably the pipeline executer, but if not, then in the in the UI or the Validation task!

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,677 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.