DFExecutorUserError abfss://<data lake path>@<XXXXXX.dfs.core.windows.net>/ has invalid authority.

sachin gupta 376

Hello,

I have created a data transformation which picks the 2 set of files from Azure Data Lake directory and then do the join and sink the resultant file in Azure Data Lake directory.
For the above scenario I created a Dataset which is pointing to the desired location where I have the 2 set of file which are .csv format. My dataset connection is failing and giving me below error

Error code: DFExecutorUserError
Details: abfss://<data lake path>@<XXXXXX.dfs.core.windows.net>/ has invalid authority.

When I change the directory location to something else, the dataset connection works fine. I did the research on this error, but not getting any guidance on this.

Would really appreciate if someone can help on this issue ASAP.

Thanks,
Sachin

MartinJaffer-MSFT 26,021 Reputation points

2021-09-10T19:00:44.877+00:00

Hello @sachin gupta and welcome to Microsoft Q&A.

Could you please tell me what authentication method you are using? This sounds like a permissions issue that might happen with SAS authentication, as SAS can be specific to a directory location.
sachin gupta 376 Reputation points

2021-09-10T20:20:41.717+00:00

@MartinJaffer-MSFT ... Account SAS using account key we are using for authentication. I have couple of directories in the organization created storage container. I have issue only with one particular directory. Connecting to other directory using a dataset works fine.

Connecting to that particular directory from a pipeline works fine, issue is coming when I am connecting to that particular directory from a Data Flow.

Please provide some insights on it.
MartinJaffer-MSFT 26,021 Reputation points

2021-09-15T18:10:19.19+00:00

So you are saying it works fine from a copy activity, but not from data flow?

That sounds like it needs a deeper investigation than I can provide here. May I offer you a 1-time free support ticket, @sachin gupta ?

3 answers

sachin gupta 376 Reputation points

2021-09-15T18:26:19.397+00:00

@MartinJaffer-MSFT ...yes you got a right understanding. I spoke to the Admin of the Azure account who setup the blob storage and they mentioned that all the SAS account access is same for all directories in Storage container.

a support ticket would be great, but the only problem is my work is on work machine and I won't be able to share the screen from there. Please let me know how we can proceed
Please sign in to rate this answer.
Manisha Barnwal 1 Reputation point

2021-11-19T05:33:31.213+00:00

@MartinJaffer-MSFT I am facing a similar issue, I am trying to use runtime pipeline parameters to determine the location of the directory (blob store) and then process the data. The issue is when I hardcode the path, the data loads fine but when I try to use pipeline params I see this error. Could you help me out?

Manisha Barnwal 1 Reputation point

2021-11-23T16:35:39.61+00:00

I was able to fix that error after restarting the debug mode on the 2nd time, not sure why it worked
Sign in to comment
Nitika Rawat 1 Reputation point

2022-08-02T11:26:05.583+00:00

I have the same error is there any solution to that?
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
robin 0 Reputation points

2023-03-29T18:57:37.0166667+00:00

I got the same error with data flow. Any resolutions ?
Please sign in to rate this answer.
Lall, Reshey 0 Reputation points

2023-10-30T15:21:07.17+00:00

I am currently Facing the same Issue works Fine with Pipeline Copy but does not work in Dataflow when using File name as Parameters
Sign in to comment