how setup dfs insted blob endpoint for BlobContainerClient

Question

Hi, In general I need to create app with java which will perform some operations on azure storage
like upload file, append to file, rename, check if exist and so on. And IMPORTANT It has to communicate with DFS endpoint https://xxxx.dfs.core.windows..

But I encounter some problems:

1: during using BlobContainerClient and uploading a file to azure storage an error appear: com.azure.storage.blob.models.BlobStorageException: Status code 400, "{"error":{"code":"MissingRequiredHeader","message":"An HTTP header that's mandatory for this request is not specified.\nRequestId:b225d695-201f-00ed-212e-c7c9e8000000\nTime:2021-10-22T10:23:12.4983407Z"}}"

How can I avoid this situation, what header is required and how to set up it?

2.afterward I have implemented something similar but using DataLakeFileSystemClient and this time uploading of file was totally fine.Unfortunately not all operations can be performed. e.g. exists() method internally uses BlobContainerClient
and perform call via blob endpoint https://xxxx.blob.core.windows.. what if forbidden in my case.
IMO It is caused because BlobContainerClientBuilder.endpoint(String endpoint) set up endpoint blobContainerClient
endpoint to blob, and dfs endpoint for DataLakeFileSystemClient.
source code:

public DataLakeFileSystemClientBuilder endpoint(String endpoint) { // Ensure endpoint provided is dfs endpoint endpoint = DataLakeImplUtils.endpointToDesiredEndpoint(endpoint, "dfs", "blob"); blobContainerClientBuilder.endpoint(DataLakeImplUtils.endpointToDesiredEndpoint(endpoint, "blob", "dfs"));

So questionis: is it a bug in BlobContainerClientBuilder.endpoint(String endpoint) ?
or how to fix this problem to use same endpoint for both clients.
Currently I have implemented wcomunicatend I'm using both clients: DataLakeFileSystemClient to perform actions
like upload, append etc. and BlobContainerClient to check if file exist. I would like to use only one of clients

Could you help me somehow, please ?

Answer 1

Hello @Miko ,

Welcome to the Microsoft Q&A platform.

Unfortunately, you cannot use BlobContainerClient Class with Azure Data Lake Gen2.

Reason: The BlobContainerClient allows you to manipulate Azure Storage containers and their blobs.

For Azure Data Lake Gen2 accounts: You can use the Azure identity client library for Java to authenticate your application with Azure AD.

For more details, refer to Connect by using Azure Active Directory (Azure AD).

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Hello :) @PRADEEPCHEEKATLA-MSFT

for upload():

2021-10-27 09:40:43.219 DEBUG 23648 --- [ctor-http-nio-3] r.n.http.client.HttpClientOperations     [] : [id:082a76c2-4, L:/171.26.90.201:57323 - R:account.dfs.core.windows.net/14.96.95.181:443] Received response (auto-read:false) : [Last-Modified=Wed, 27 Oct 2021 07:40:43 GMT, ETag="0x8D9991D14D08271", Server=Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0, x-ms-request-server-encrypted=false, x-ms-request-id=3cee61a0-d01f-0130-0205-cbdc13000123, x-ms-version=2020-08-04, x-ms-client-request-id=992234d0-e065-460e-b2b5-2150a96d9fax, Date=Wed, 27 Oct 2021 07:40:42 GMT, content-length=0] []  

for exist():

2021-10-27 09:42:43.915 DEBUG 23648 --- [ctor-http-nio-4] r.n.http.client.HttpClientOperations     [] : [id:b48125e3-1, L:/171.26.90.201:55181 - R:account.blob.core.windows.net/14.96.95.175:443] Received response (auto-read:false) : [Content-Type=application/octet-stream, Last-Modified=Wed, 27 Oct 2021 07:40:43 GMT, Accept-Ranges=bytes, ETag="0x8D9991D14D08271", Server=Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0, x-ms-request-id=48123186-b01e-0102-6306-cb84c3000200, x-ms-client-request-id=a8325b88-be58-453f-9696-7dbb7b96db0f, x-ms-version=2020-08-04, x-ms-creation-time=Wed, 27 Oct 2021 07:40:40 GMT, x-ms-lease-status=unlocked, x-ms-lease-state=available, x-ms-blob-type=BlockBlob, x-ms-server-encrypted=true, x-ms-access-tier=Hot, x-ms-access-tier-inferred=true, x-ms-owner=6c18430b-69f1-476e-81da-3959e8ddfae2, x-ms-group=7d964385-e564-4776-ae7b-5f1fbde42aa8, x-ms-permissions=rw-rw----+, Date=Wed, 27 Oct 2021 07:42:43 GMT, content-length=2215498] []  

For dataLakeFileSystemClient.getFileClient("someFile.txt").getProperties() response is almost the same like for calling exist() method.