Synapse Failed - Bad Request Error

Question

Synapse Failed - Bad Request Error

Rodney Arbon 0

I am getting a "Failed" Notification when modifying a pipeline in Synapse. The pipeline works fine until I add a parameter to the Sink dataset , to specify a dynamic folder path. As a test I also just specified just a default value for the parameter. Nothing fancy and replicates many ADF process designs.

Error messages as Follows:

User's image

The Pipeline

User's image

Sink Data Set

User's image

@concat('Rejected/ACCOUNT/SchemaRejected/',dataset().AccountLocation,'/',utcNow('yyyy') ,'/',utcNow('dd'),'/',utcNow('HH'))

thanks any advice on why this is failing, would be useful

MartinJaffer-MSFT 26,161 Reputation points

2023-01-31T18:53:04.0166667+00:00
@Rodney Arbon Hello and welcome to Microsoft Q&A.

I see you are getting a "Bad Request" Error. Just to verify, this is when submitting a debug run. Not when you save changes, publish changes, or submit a regular pipeline run?

The "Bad Request" class of errors are rather difficult to troubleshoot in my experience, as there is a very broad range of causes. This range includes a glitch in the browser. I'd like you to try restarting the browser, then try making changes again. This has helped me in the past.

Another thing to check is "does it let me publish and then do trigger now?" As a precaution, don't forget to duplicate your pipeline first.

Also, I would like to suggest an improvement to your code. This might coincidentally help your issue.

@concat('Rejected/ACCOUNT/SchemaRejected/',dataset().AccountLocation,'/',utcNow('yyyy') ,'/',utcNow('dd'),'/',utcNow('HH')) @concat('Rejected/ACCOUNT/SchemaRejected/',dataset().AccountLocation,'/',formatDateTime(UTCnow(),'yyyy/dd/HH'))

This change not only makes the code shorter, but it also improves the timing by only fetching the time once. Imagine the time is 23:59.59 , and the first utcNow fetches previous day and next utcNow gets the next day. Not likely to happen, but by only gettting the time once, we make it impossible.
Rodney Arbon 0 Reputation points

2023-02-01T08:54:14.7033333+00:00

Hi no this fails when we publish and run via a trigger also. We believe there might be an issue/bug with the naming of the dataset that for some reason throws an error. i.e. our dataset starts with a number such as "18_SinkDataSet". We have found that if we name the data set with a leading character rather than a number it starts to work. However we have also noticed that renaming of datasets itself is bugged. On publish its shows there is an edit - (due to name change) and this fails to publish, So this leads to creating a new data set and swapping then out. The CI/CD of synapse seems more buggy than ADF and leading numbers doesn't seem to be an issue in ADF etc.

Appreciate the suggestions above , but none of these work. Yes I take your point on using FormatDate on UTCNow.

Your answer

MartinJaffer-MSFT 26,161 Reputation points

2023-01-31T18:53:04.0166667+00:00

@Rodney Arbon Hello and welcome to Microsoft Q&A.

I see you are getting a "Bad Request" Error. Just to verify, this is when submitting a debug run. Not when you save changes, publish changes, or submit a regular pipeline run?

The "Bad Request" class of errors are rather difficult to troubleshoot in my experience, as there is a very broad range of causes. This range includes a glitch in the browser. I'd like you to try restarting the browser, then try making changes again. This has helped me in the past.

Another thing to check is "does it let me publish and then do trigger now?" As a precaution, don't forget to duplicate your pipeline first.

Also, I would like to suggest an improvement to your code. This might coincidentally help your issue.

@concat('Rejected/ACCOUNT/SchemaRejected/',dataset().AccountLocation,'/',utcNow('yyyy') ,'/',utcNow('dd'),'/',utcNow('HH')) @concat('Rejected/ACCOUNT/SchemaRejected/',dataset().AccountLocation,'/',formatDateTime(UTCnow(),'yyyy/dd/HH'))

This change not only makes the code shorter, but it also improves the timing by only fetching the time once. Imagine the time is 23:59.59 , and the first utcNow fetches previous day and next utcNow gets the next day. Not likely to happen, but by only gettting the time once, we make it impossible.
Rodney Arbon 0 Reputation points

2023-02-01T08:54:14.7033333+00:00

Hi no this fails when we publish and run via a trigger also. We believe there might be an issue/bug with the naming of the dataset that for some reason throws an error. i.e. our dataset starts with a number such as "18_SinkDataSet". We have found that if we name the data set with a leading character rather than a number it starts to work. However we have also noticed that renaming of datasets itself is bugged. On publish its shows there is an edit - (due to name change) and this fails to publish, So this leads to creating a new data set and swapping then out. The CI/CD of synapse seems more buggy than ADF and leading numbers doesn't seem to be an issue in ADF etc.

Appreciate the suggestions above , but none of these work. Yes I take your point on using FormatDate on UTCNow.

Share via

Synapse Failed - Bad Request Error

Your answer