Azure Synapse mount ADLS gen2

Indrajit Roy 1 Reputation point
2022-08-22T14:54:20.64+00:00

I am trying to access binary files of size >8 gb with mount point created on Synapse for ADLS gen2. When doing os.walk, the file paths are displayed fine. But when trying to read the binary files, it fails with no such dir or file after running for sometime.
Any file of size less than 8 gb are read fine. Only files > 8 gb give this issue.
Please suggest.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,422 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA-MSFT 78,576 Reputation points Microsoft Employee
    2022-09-05T06:58:26.917+00:00

    Hello @Indrajit Roy ,

    As per the internal team response - there is no such limiation on reading a file greater than 8 GB.

    0 comments No comments