DataFactory Alter Row into Sink

Adrian 181 Reputation points
2021-08-31T11:11:40.06+00:00

Hello!

I am deleting some blank rows from my data, I am using the 'Alter Row' function to after which I am running a isNull(COLUMNNAME) to get rid of the data.

It is appearing fine in the Alter Row box during the data preview, however after I try to sink the data into a CSV - I am still getting all the columns that are being deleted beforehand? Not sure what I am doing wrong here, would appreciate any tips.

Here are the screenshots:
127899-image.png

This is the data preview of my Alter Row transformation. After is the data preview of my Sink transformation, as I can see all the rows that are supposed to be deleted, or have the red 'X' next to them, are still featured during the sink.

127923-image.png

Azure Database Migration service
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,566 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,030 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,933 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. HimanshuSinha-msft 19,386 Reputation points Microsoft Employee
    2021-08-31T22:27:04.857+00:00

    Hello @Adrian ,
    Thanks for the ask and using the Microsoft Q&A platform .
    I checked your logic and it looks fine to me , it should work . Did you tried to trigger the pipeline and test .

    Read more here . https://learn.microsoft.com/en-us/azure/data-factory/data-flow-alter-row

    "Alter Row transformations will only operate on database, REST, or CosmosDB sinks in your data flow. The actions that you assign to rows (insert, update, delete, upsert) won't occur during debug sessions. Run an Execute Data Flow activity in a pipeline to enact the alter row policies on your database tables."

    Please do let me know how it goes .
    Thanks
    Himanshu
    Please do consider clicking on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members