Dynamic Content in Assert Description giving error

Question

I have a flowlet which is calling an assert transformation. Let's say the assert expects true for col1 = col2 and fails the data flow if a record does not return true. In my assert description, I provided a concat function to pass the col1 value in the event of an error.

The validations are passing but when I preview the data from a data flow or call the data flow from a pipeline, I am getting the following error: "No code found for stream stream1 ~> stream2".

This is only happening when I am using expression for assert description. If I remove all column references from my assert description, the flowlet behaves as expected. Additionally, this is not limited to only col1 and col2. If I add col3 to my description, I get a similar error but with the appropriate streams for col3.

On further analysis, I found that the streams where it shows error are the input and output stream where the col1 is last modified. If col1 is never modified post source, the error would say: "No code found for stream ~> sourceName".

I also tried with Schema Drift enabled and with it disabled but no change to the error.

According to the video on the documentation page, this should be allowed. (doc link - Assert Transformation). Can anyone help with this?

Answer 1

The error you are encountering with the assert transformation in your Azure Data Factory flowlet appears to be related to how dynamic content expressions are processed in the assert description. Specifically, it seems that the pipeline is unable to resolve the column references within the assert transformation's dynamic expression, leading to the "No code found for stream" error.

Double-check the concat() function and ensure it is correctly formatted.

concat('Validation failed for record with col1: ', toString(col1), ' and col2: ', toString(col2))

If any column (col1, col2, etc.) can be null, use coalesce() or handle nulls explicitly to prevent runtime errors.

Verify that the columns (col1, col2...) used in the assert description are accessible in the assert transformation.

If a column is derived or transformed earlier in the data flow, make sure its lineage is intact. This could involve:

• Explicitly passing the column downstream.
• Checking for schema drift if the column originates from a schema-drift-enabled source.

Use a simpler expression in the assert description to isolate the issue. For example:

concat('Validation failed for col1 = ', toString(col1))