In Data Factory, what is the difference between Conditional Split and Assert? When to use what?

Question

In Data Factory, what is the difference between Conditional Split and Assert? When to use what?

Partha Das 276

Hi,

In Data Factory, what is the difference between Conditional Split and Assert? When to use what? If you can provide some use cases for that decision making.

ShaktiSingh-MSFT 11,136 Reputation points Microsoft Employee

2022-10-25T05:30:36.1+00:00

Hi @Partha Das ,

Just checking in to see if the below answers helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

ShaktiSingh-MSFT 11,136 Reputation points Microsoft Employee

2022-10-25T05:30:36.1+00:00

Hi @Partha Das ,

Just checking in to see if the below answers helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 1

MarkKromer-MSFT 5,176 Microsoft Employee

Conditional split is really just a case / switch / if-then-else construct whereas Assert is for data quality rules, with the built-in capability to set expectations, set quality rules, and set rows as error rows.

Answer 2

ShaktiSingh-MSFT 11,136 Microsoft Employee

Hi @Partha Das ,

Thanks for posting this question in Microsoft Q&A platform.

In addition to @MarkKromer-MSFT 's reply, you may refer to the official Microsoft documentation page for Conditional Split and Assert transformations as well as the videos demonstrating the same as below:

Assert in Data Flow

Conditional Split in Data Flow

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification

Partha Das 276 Reputation points

2022-10-26T09:18:24.197+00:00

My question is the real time scenarios or use cases when to choose Assert over Conditional split. Definitions I already read in MS documentations. So both the replies didn't answer my question.
Samy Abdul 3,361 Reputation points

2022-10-26T11:14:21.287+00:00

Hi @Partha Das , please go through below blog on Assert Transformations :

Assert Transformations in Azure Data Factory

https://www.sqlservercentral.com/articles/assert-transformations-in-azure-data-factory

Answer 3

ShaikMaheer-MSFT 35,796 Microsoft Employee

Hi @Partha Das ,

Conditional split transformation can be used when you want to split incoming rows as separate streams based on conditions and process them accordingly. Whereas Assert Transformation can be used when you want to perform data validation on the incoming rows.

Let's assume you have employee data coming in with different department Ids and want to split those rows based on departments and process each department related rows separately and load them into different tables. In this case you can use conditional split transformation. Please check below video where with an example I explained about conditional split transformation.
Conditional Split Transformation in Mapping Data Flow in Azure data factory

Let's assume if you want to validate incoming rows and check for data quality you can consider using assert transformation. In below video, I explain usage of assert transformation with some examples for better understanding. Kindly check same.
Assert Transformation in Mapping Data Flows in Azure Data Factory

Hope this helps. Please let me know if any further queries.

----------------

Please consider hitting Accept Answer and Up-Vote. Accepted answers help community as well.

Partha Das 276 Reputation points

2022-10-26T10:51:14.75+00:00

Thanks for this information. Again, I can use conditional split for quality rules like sending the error rows to csv file. Then what is the advantage of Assert over conditional split? Please correct me if my understanding is wrong.
ShaikMaheer-MSFT 35,796 Reputation points Microsoft Employee

2022-10-26T11:09:39.537+00:00

yes, you are right. You can use conditional split transformation as well for error rows to identify. In fact, when assert transformation was not there that time condition split transformation is the way to handle these. In below video of mine, I used conditional transformation only for checking error rows.
Handle Error Rows in Data Factory Mapping Data Flows

Assert transformation came later in game, and its more direct way of handling data validation and quality checks. Hence, when the requirement talks about validations and data quality checks its recommended to use assert transformation.

Let's take example from my above answer where I want to split employee rows for different departments to separate streams, here using condition split is the perfect fit. We cannot do that using assert. Hope this gives a better idea for you.

-----------------

Please consider hitting Accept Answer and Up-vote. Accepted answers help community as well.

In Data Factory, what is the difference between Conditional Split and Assert? When to use what?

3 answers

Activity