A community member has associated this post with a similar question:
ADF incremental load error
Only moderators can edit this content.
ADF incremental load error
I am trying to perform an incremental load from a date in UNIX format (bigint) and for this, I am using a watermark. So I have the following fonts:
- Source DB (PostgreSQL)
- DB where the new data is copied to (Azure PostgreSQL)
- Watermarked table (Azure SQL Database) The copy process is as follows, with two searches to copy the data > to the watermark date and <= to the maximum date of the source DB. Once the data has been copied (THIS WORKS WELL FOR ME) the watermark date must be updated so that the next copy copies from there.
The problem I have is when updating the watermark. To do this, it looks for the maximum date in the source DB (bigint format) and must update it in the watermark table (configured as bigint).
However, the mapping is done as int64 and I get the following error when updating the watermark
"Cannot create Sql Source. Please double check the connection string, stored procedure are set with correct format. Error: The value of the property 'Value' is invalid for the stored procedure parameter 'watermark_value'."
Does anyone know how to fix it so that the watermark table is updated with the new value of the maximum date (UNIX)?
Thank you