I am getting a BadRequest error while running adf pipeline which contains mapping data flow activity

Question

Hi, I am getting BadRequest error while running adf pipeline which contains data flow activity. It was running fine previously.

Answer 1

Nikita Randive - Thanks for the question and using MS Q&A platform.

A BadRequest error in Azure Data Factory (ADF) can occur due to various reasons. Here are some steps you can try to troubleshoot and resolve the issue:

1. Check the input and output schema of the mapping data flow activity. Make sure that the schema matches the source and sink data stores. If there are any changes in the schema, update the mapping data flow activity accordingly.
2. Check the connection settings for the source and sink data stores. Make sure that the connection settings are correct and that the data stores are accessible.
3. Check the data flow transformation settings. Make sure that the transformation settings are correct and that they are not causing any data type or format issues.
4. Check the data flow debug logs. The debug logs can provide more information about the error and help you identify the root cause. You can enable debug logging for the data flow activity and check the logs in the ADF monitoring section.
5. Check the ADF pipeline settings. Make sure that the pipeline settings are correct and that they are not causing any issues. For example, check the trigger settings, concurrency settings, and timeout settings.
6. Check the ADF integration runtime settings. Make sure that the integration runtime settings are correct and that they are not causing any issues. For example, check the network settings, authentication settings, and performance settings.

If none of these steps work, you can try recreating the mapping data flow activity or the entire ADF pipeline from scratch. Sometimes, recreating the activity or pipeline can resolve any underlying issues.

I hope this helps you troubleshoot and resolve the BadRequest error in your ADF pipeline with a mapping data flow activity. If you have any further questions or concerns, please let me know.

Answer 2

Often caused by a trailing white space somewhere, including activity names. Check each activity especially any that have been changed recently.

Answer 3

It seems this can be caused by invalid activity parameters that don't fail the overall pipeline validation.

In my case, I was passing a parameter to a SQL stored procedure with the @ prefix instead of without it, i.e.:

• @[param name] - Debug failed with "BadRequest" message
• [param name] - Debug ran successfully

Answer 4

In my experience there is usually a variable or parameter (often under Settings for a dataflow) that has an empty value.