Hi @Kudras, Stefan
Welcome to Microsoft Q&A platform and thanks for posting your query here.
Regarding your concern about using the "Truncate table" option in the Snowflake sink with Azure Data Factory Data Flows, here’s some clarification:
The "Truncate table" action works in two steps:
- It deletes all rows in the table (making it empty).
- It then inserts the new data.
This means there is a brief moment when the table is empty before the new data is added. So, if another query tries to access the table during this gap, it may see an empty table.
As for ACID compliance, while ADF Data Flows strive to maintain data consistency, "Truncate table" operations do not offer the same transactional guarantees as other ACID-compliant operations (e.g., ensuring no data loss or partial updates during failures).
To ensure data consistency and minimize any potential gaps:
- You can consider using a staging table to load data first, then swap or update the main table after validation.
- Alternatively, transactional controls available in Snowflake or batch strategies could further help maintain data availability.
I hope this helps. Please let me know if you have any questions.