How can I create a temporary table in Spark ?

Question

I am using apache spark in Azure Synapse and I cannot find how to create a temporary table.

Is it possible to do that or there is a workaround?

Answer 1

In Spark SQL, I think the concept of sql temporary tables doesn't exist.

PySpark dataframes can provide similar functionality.

You can check this thread : https://stackoverflow.com/questions/77175754/how-to-create-a-temporary-table-in-spark-sql-not-a-temp-view