Azure Data factory bug when creating dataset - error 800000 char limit exceeded in ARM template

Steve Waterfield 1 Reputation point
2021-12-08T10:36:43.617+00:00

i have a fairly complex data factory 18 pipelines, 120+ datasets 26 Dataflows. it has been working fine until last week when i tried adding a new dataset. 155943-alltogether.png
You can see that i am adding the dataset and spec the file path that it will point to in one operation.

it takes as much as 10seconds to execute this task

on publishing you get the following error155934-fail.png

i discovered after a week of tearing my hair out and learning about ARM templates
that you can get around this bug by

Adding a new dataset but not spec the file path leave as the root

publish the dataset ... all fine

Edit the newly created dataset, spec the file path you want the dataset to point to

publish, all is fine

Observations the first does not take 10s, it returns instantly

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
{count} votes

1 answer

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,236 Reputation points
    2021-12-15T19:49:52.163+00:00

    I have submitted your feedback @Steve Waterfield .

    FYI: This issue can happen when any parameter exceeds the limit.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.