This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
Hi,
I am lookingfor suggestions on different tools and technologies that can be used to validate destination data after ETL.
my source is CSV/Excel files in storage container/Local machine and destination is Azure SQL DB table.
I have more than 1500 source files for one target table, So I want to automate validation destination.
I want to test the total count, avg and sum and random check of a value in a column etc between source files and destination table.
Looking for a short time implementation tools for this.
you can use Greatexpectations : open source for data validations
https://greatexpectations.io/expectations
this can be triggered via databricks,azure functions etc
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Thank you Nandan Hegde
Hello Santhi Dhanuskodi
If the above answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.