Assert error output not writing to blob

Question

Hello.

I have built a pipeline which gathers data from a .csv file and then goes through a couple of assert activities to check the validity of the data. Valid rows are supposed to be input into the table, while assert-failure rows are setup to be written in a data lake container.

I have built 6 of these pipelines, one for each file. Only one of these pipelines seems to actually write the assert failure output to the blob container, while the others succeed as an activity but do not create the folder or write any files, even though I have given the file rows which should very clearly not pass the assert.

I have checked the sink settings, multiple times for each one, and i cannot find any differences other than the obvious column mappings. I'm unsure as to why the one of the sinks, is actually writing to blob while the other ones are not.

All assert activities have an ID(containing no spaces) and description (description has an expression).
There are no filters for any of the individual asserts in any of the pipelines.

I would appreciate it if anyone has any specific suggestions or experience dealing with this in the past.

Answer 1

Hello Dejvis Leka,

Welcome to the Microsoft Q&A forum.

Since, I don't have access to your specific pipeline configurations or logs, Here are few things you can check from your end.

• Make sure the output settings in the failing pipelines are correctly configured to write to the blob container. This includes the path to the container, the filename, and any permissions required to write to the container.
• Verify if the issue might be with the data itself. Make sure that the data being processed by each pipeline is consistent and that the assert conditions are being met for each pipeline. If the data is not consistent or the assert conditions are not being met, it could cause the assert failure output to not be written correctly.
• You may also want to check the logs for each pipeline to see if there are any error messages or warnings that could provide more information about the issue.
• If you're using version control, compare the current version of the pipelines with a previous version that was working correctly.

I hope this helps. Please let me know if you have any further questions.

Answer 2

Hello. Sorry for the delay in my answer. I have checked the above, and they seemed to work fine. I can confirm this because i managed to manually output my asserts to a log table through derived columns and expressions to get the errors. I am guessing that there may be some sort of bug or possibly an inconsistency somewhere. Either way, thank you for the help and best of luck.