Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
934 questions
DataLakeFileSystemClient listPath, NumberFormatException when using String
AUDIER Maxime
0
Reputation points
Hello,
I am new to this and trying to list all files in a container recursively.
ListPathsOptions options = new ListPathsOptions();
options.setRecursive(true);
for (PathItem pathItem :fileSystemClient.listPaths(options, null)){ ... }
Doing so result in this error :
java.lang.NumberFormatException: For input string: "Thu, 14 Oct 2021 17:22:10 GMT"
I have tried to manually go through each directory and found the possible cause :
String dir = "my-directory-name/01/2021/10/14/17";
DataLakeDirectoryClient directory = fileSystemClient.getDirectoryClient(dir);
This code will generate the same error.
It seems like '2021/10/14/17' is interpreted as a date causing the method to fail.
The 'yyyy/mm/dd/hh' representation is by design so I cannot change the names easily for now,
it may be possible in the future or if the error cannot be resolved with other methods.
Edit / Update :
Got access to the container and did some test, the issue happen even if the 'blob object' is at the root.
The input string math the creation date of the blob object.
String dir = "test-directory";
DataLakeDirectoryClient directory = fileSystemClient.getDirectoryClient(dir);
for (PathItem pathItem : directory.listPaths()) {//Error happen here
System.out.println(pathItem.getName());
}
I tried to use https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-java code (List directory contents) -> Same error
Thank you in advance for your help.
{count} votes
-
AUDIER Maxime 0 Reputation points
2023-03-08T10:50:12.97+00:00 Hello,
I have tested your solution without success :
import java.net.URLEncoder; import java.nio.charset.StandardCharsets; try { String dir = "my-directory-name/01/2021/10/14/17/"; String encodedDir = URLEncoder.encode(dir,StandardCharsets.UTF_8.toString()); DataLakeDirectoryClient directory = fileSystemClient.getDirectoryClient(encodedDir); for (PathItem pathItem : directory.listPaths()) { System.out.println(pathItem.getName()); } }...The error is unchanged :
java.lang.NumberFormatException: For input string: "Thu, 14 Oct 2021 17:22:10 GMT"The value of encodedDir in debug : "my-directory-name%2F01%2F2021%2F10%2F14%2F17%2F"
Hello @AUDIER Maxime
Thanks for the question and using MS Q&A paltform.
The error you are encountering is due to the fact that the method
listPathsis interpreting the path as a date instead of a string. To resolve this issue, you can encode the path as a URL-encoded string before passing it to the method.Here is an example of how you can encode the path as a URL-encoded string:
This will encode the path as a URL-encoded string, which will prevent the method from interpreting it as a date.
Hope this helps. Do let us know if you any further queries.
@AUDIER Maxime Could you please share the code snippet which you are using along with the stack trace of the error message which you are experiencing?
Full stack :
@AUDIER Maxime Thanks for sharing the details. I'm checking with the internal team and I will get back to you as soon as I have an update.
@AUDIER Maxime Could you please use the code snippet which is available in the official document and see if that works: https://learn.microsoft.com/en-us/java/api/com.azure.storage.file.datalake.datalakefilesystemclient?view=azure-java-stable#com-azure-storage-file-datalake-datalakefilesystemclient-listpaths(com-azure-storage-file-datalake-models-listpathsoptions-java-time-duration)
Hope this helps. Do let us know if you any further queries.
Sign in to comment
1 answer
Sort by: Most helpful
Hello,
I have test the code without success (using a timeout of 1minutes).
I am not sure if you saw the edit to the original post, but I have found out that the date in the error match the creation date of the file (blob object) that should be printed.
I got access to the container and now working with a simple directory with file inside to test
Sign in to comment