Delete folder if exists using Synapse notebook (Pyspark)

Alok Thampi 151 Reputation points
2023-04-17T14:01:45.9866667+00:00

Hi, I have an external delta table that I have to recreate every time when my Synapse PySpark notebook is executed. Since this is an external table, I need to clean up the folder also before dropping the table. I am able to delete the folder using the below code but this fails if the folder is not present.

mssparkutils.fs.rm(<folder>, True)

I am looking for a code snippet which would look for the existence of this folder and deletes it recursively if present. Could you please help? Thanks, Alok

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,373 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Alok Thampi 151 Reputation points
    2023-04-18T03:06:11.0066667+00:00

    Update : Got this one working with the below script

    if mssparkutils.fs.exists(folder_path):
        mssparkutils.fs.rm(folder_path, True)
    
    1 person found this answer helpful.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.