Huge cross join in Azure

Question

Hi everyone,

In my project, I need to cross join two tables. One of these two tables includes 77m rows, and the other includes 5m rows. For the best run time and performance, which tool or method should I use to do this huge operation in Azure?

Best,

Answer 1

Hello Kerem Kabil

To cross join two tables in Azure SQL DB, you can use the CROSS JOIN operator. However, for such a large operation, you may want to consider using Azure Synapse Analytics instead of Azure SQL DB. Azure Synapse Analytics is a cloud-based analytics service that can handle large-scale data processing and analytics. It provides a distributed processing engine that can handle large data volumes and parallel processing. You can use Azure Synapse Analytics to perform the cross join operation on the two tables. Additionally, you can use the HASH distribution option in the CREATE TABLE statement to distribute the data across multiple nodes and optimize the performance of the cross join operation.

Doc: Design and performance for Teradata migrations - Azure Synapse Analytics | Microsoft Learn

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.