This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
I am trying to access binary files of size >8 gb with mount point created on Synapse for ADLS gen2. When doing os.walk, the file paths are displayed fine. But when trying to read the binary files, it fails with no such dir or file after running for sometime.
Any file of size less than 8 gb are read fine. Only files > 8 gb give this issue.
Please suggest.
Hello @Indrajit Roy ,
Thanks for the question and using MS Q&A platform.
Can you confirm if you are able to list the file?
mssparkutils.fs.ls("synfs:/49/test/myFile.txt")
And, are you able to read the other files which are greater than 8 gb or only specific file?
Hello @Indrajit Roy ,
As per the internal team response - there is no such limiation on reading a file greater than 8 GB.
@PRADEEPCHEEKATLA-MSFT - yes, able to list the file as well as read any files <8gb.
only problem is with files >=8gb.
Hello @Indrajit Roy ,
I don't think there is any limitation as such.
We are reaching out to the internal team to get more details on this issue and will get back to you as soon as we have an update.