Automated testing in azure datalake

Question

I want to automate the testing process of azure data lake resources, especially azure data factory and azure SQL database. I tried creating some test cases in azure devops test plans and build pipelines for testing SQL database schema, mappings etc. I ran some SQL scripts for validating the schema etc. through YAML pipelines. Now i want to automate this SQL testing part. Is there any feature, plugin or any utility for azure devops for automating the SQL database testing in data lake. I could not find any pipeline tasks for SQL, like there are lot of tasks for selenium, dotnet applications but not for data lake.

I would highly appreciate any help and suggestion for this.

Answer 1

Hello Smriti,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you would like to implement Automated testing in azure data lake.

Regarding your explanation, Azure DevOps doesn't have built-in tasks specifically for data lake testing, you can create custom tasks or scripts within your pipelines to handle data lake testing scenarios. Consider using Azure Data Factory or other specialized tools for comprehensive data lake testing. Also, you can achieve your goal by combining existing features and third-party tools. The following links are best options for your kind scenario.

• Azure SQL unit testing with tSQLt using Azure DevOps: https://nikolaithomassen.com/2019/02/26/azure-sql-unit-testing-with-tsqlt-using-azure-devops/.
• Run automated tests from test plans: https://learn.microsoft.com/en-us/azure/devops/test/run-automated-tests-from-test-hub?view=azure-devops.
• True DevOps: including databases in automated deployment: https://www.red-gate.com/hub/product-learning/sql-change-automation/moving-from-application-automation-to-true-devops-by-including-the-database.
• Automating SQL Server Database Testing with Azure DevOps: https://www.sqlops.com/automating-sql-server-database-testing-with-azure-devops/.

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam