How to handling exceptions from mssparkutils.fs.head() in Azure Synapse notebooks?

Question

Hi,

I would like to handle exceptions from mssparkutils.fs.head(), eg. like handling FileNotFoundException. However, I find traditional python exception handling doesn't work. How to do this?

Answer 1

You can still use traditional Python exception handling techniques with some modifications. If mssparkutils.fs.head() throws a FileNotFoundException or any other error, you can handle it using a try and except block.

try:

    # Try reading the file

    result = mssparkutils.fs.head('/path/to/your/file.txt')

    print(result)

except Exception as e:

    # Handle specific exceptions

    if 'FileNotFoundException' in str(e):

        print("File not found!")

    else:

        # Handle any other exceptions or print the error message

        print(f"An error occurred: {e}")

The key is to capture the general Exception and then check the error message string for specific exception types or messages. This is because the exact exceptions thrown by Spark might not always map cleanly to Python's built-in exceptions.

Ensure you replace '/path/to/your/file.txt' with your desired file path.