Getting "put: Status code: -1 error code: null error message: InvalidAbfsRestOperationException" error

Naga Vinod K 0 Reputation points
2023-03-10T09:14:42.3266667+00:00

Team,

Getting below error while ingesting data from our VM to ADLS using below command and find below details.

Command:

curl -vkS sftp://user@domain.com:/FilePath/ingestion/file.eot | hadoop fs -Dhadoop.security.credential.provider.path=jceks://hdfs/file.jceks -Dhadoop.security.credstore.java-keystore-provider.password-file=file.ps -Dfs.azure.skipUserGroupMetadataDuringInitialization=true -put - abfss://datalake_environment@domain.dfs.net/path_of_file/account/file.eot

Error:

put: Status code: -1 error code: null error message: InvalidAbfsRestOperationException

Please help me on this and the thing is, When we run manually it is running fine but when we do this through scripts we are not able to do and getting above error messages and failing all the ingestion jobs.

Thanks in advance...!

Best Regards,

Vinod

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,338 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 25,881 Reputation points Microsoft Employee
    2023-03-18T02:14:28.2533333+00:00

    Hello Vinod K,

    Welcome to the MS Q&A platform.

    Since you were able to run the command manually, it could be an issue with credentials when running through scripts.

    Can you please ensure the credentials(username and password) you are using to access the ADLS account are correct and it has the necessary permissions to access the target directory?

    and can you try adding the "-v" flag to the "hadoop fs" command to increase the verbosity level and get more information about the error?

    0 comments