Error "[Errno 5] Input/output error" when opening ADLS Blob from mounted file system

David Leach 1 Reputation point
2022-10-19T06:24:55.107+00:00

Hi,

Using a Synapse Pyspark Notebook I have mounted an ADLS container and checked this is working ok with the following command:

"jobid=mssparkutils.env.getJobId()"
"mssparkutils.fs.ls(f"synfs:/{jobid}/synapse-ws01")"

This shows underlying folders and files as expected :

"[FileInfo(path=synfs:/1/synapse-ws01/app, name=app, size=0),
FileInfo(path=synfs:/1/synapse-ws01/client_secrets.json, name=client_secrets.json, size=441),
FileInfo(path=synfs:/1/synapse-ws01/mnt, name=mnt, size=0),
FileInfo(path=synfs:/1/synapse-ws01/synapse, name=synapse, size=0)]"

However when I attempt to open the file "client_secrets.json" using the command:

"f = os.open(f"/synfs/{jobid}/synapse-ws01/client_secrets.json",os.O_RDONLY)"

The execution hangs for about 6 minutes and finally returns:

"OSError: [Errno 5] Input/output error: '/synfs/1/synapse-ws01/client_secrets.json'"

The logs do not provide any clues as to what the issue is. The Synapse Workspace is a Storage Blob Data Contributor on the Storage Account. I have also repeated this test against a newly created test Storage Account with no firewall restrictions and the same result occurred.

Can anyone help or advise as to how I might get some more detail as to the error?

Thanks,

Dave

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,565 questions
{count} votes