How to handling exceptions from mssparkutils.fs.head() in Azure Synapse notebooks?

Raghul Kannan 106 Reputation points


I would like to handle exceptions from mssparkutils.fs.head(), eg. like handling FileNotFoundException. However, I find traditional python exception handling doesn't work. How to do this?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
3,557 questions
0 comments No comments
{count} votes

Accepted answer
  1. Amira Bedhiafi 4,631 Reputation points

    You can still use traditional Python exception handling techniques with some modifications. If mssparkutils.fs.head() throws a FileNotFoundException or any other error, you can handle it using a try and except block.

        # Try reading the file
        result = mssparkutils.fs.head('/path/to/your/file.txt')
    except Exception as e:
        # Handle specific exceptions
        if 'FileNotFoundException' in str(e):
            print("File not found!")
            # Handle any other exceptions or print the error message
            print(f"An error occurred: {e}")

    The key is to capture the general Exception and then check the error message string for specific exception types or messages. This is because the exact exceptions thrown by Spark might not always map cleanly to Python's built-in exceptions.

    Ensure you replace '/path/to/your/file.txt' with your desired file path.

    0 comments No comments

0 additional answers

Sort by: Most helpful