How to update data into Azure storage table from datafactory?

Question

1) Can i use storage table for capture log details of pipeline via datafactory. will it allow insert new values in
table?

2) I want to update specific column in storage table. how do i update the column which activity I should
use?

like I want to create table "control_table" with columns having "table_name" and "last_update" columns. every time I load data , i want to update the "last update column".

Answer 1

Hi @THIMMAIAH GARI,PRASHANTH,, ,

Welcome to Microsoft Q&A Platform. Thanks for posting the query.

Yes, Azure table storage can be used to log the details from Azure data factory. Both insert and update are possible.

• I have used copy activity to implement the above requirement with dummy source file whereas additional columns will be used for populating table name and date that can come from pipeline variables. This activity can be used after table is loaded in ADF pipeline to record the load into the Azure table.
• Replace will replace entire property of previous entity with new entity if the entity exists. If the new entity doesn't define property or having property value null then that property will be removed on updating whereas in merge, old properties will be retained even if the new entity didn't define new properties in new entity and it updates if new values are defined for old properties if entity exists. Hence, I have used merge in this scenario.
• Partition key + Row key combination has to be unique in a table and hence, partition key can be used appropriately from your scenario. I have used static value of 1 for partition key and used table name as row key in this scenario. So that whenever date property changes for same table name under 1 partition id.

Please find below GIF and JSON attached of the pipeline.

32392-tablestorageupsertjson.txt

Ref: difference-between-insert-or-merge-entity-and-insert-or-replace-entity

Hope this helps! Please let us know if it does not align with requirement or for further queries and we will be glad to assist.