LIST error

Gopinath Rajee 656 Reputation points
2023-06-01T02:30:04.6033333+00:00

All,

When I run "LIST 'abfss://****@sa.dfs.core.windows.net/'**;" against the EXTERNAL LOCATIONS I get the result in the first image.

If I append __unitystorage and run "LIST 'abfss://****@sa.dfs.core.windows.net/__unitystorage'**;" , I get an error. I ran the updated list to see how the folders are created. What does this error mean? The first LIST does not add any value to me and I was hoping that the second list would add value and below is the error:

Error in SQL statement: UnityCatalogServiceException: [RequestId=fa8a937d-d5ca-4020-9404-20412b7a39e2 ErrorClass=INVALID_PARAMETER_VALUE.LOCATION_OVERLAP] Input path url 'abfss://******@sa.dfs.core.windows.net/__unitystorage' overlaps with managed storage

What does this error mean? and how can I address this error?

Thanks,

grajee

User's image

User's image

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,514 questions
{count} votes

2 answers

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,546 Reputation points Microsoft Employee Moderator
    2023-06-01T12:53:43.4833333+00:00

    Hi Gopinath Rajee,

    Thank you for Posting query in Microsoft Q&A Platform.

    In Azure Databricks, the error message indicates that the input path URL 'abfss://******@sa.dfs.core.windows.net/__unitystorage' overlaps with managed storage. It suggests that the specified path conflicts with an existing managed storage location.

    To address this error in Azure Databricks, you can follow these steps:

    Verify the existing managed storage locations: Check your Databricks workspace configuration and ensure that there are no conflicting managed storage locations defined.

    Choose a different path or container: To avoid the overlap error, you need to provide a different path or container name for the external location. Instead of using the containertables container, try creating a new container specifically for your external storage and use that in your LIST command.

    Update the path in your code: Once you have a separate container or path for your external storage, modify your code to use the new path or container name. Make sure to remove the __unitystorage suffix.

    By addressing the overlap issue and using a distinct container or path, you should be able to resolve the error and successfully list the objects within your external storage in Azure Databricks.

    Please let me know how it goes.

    1 person found this answer helpful.

  2. Mingquan Liang [mliang-z] 0 Reputation points
    2024-04-29T22:27:04.44+00:00

    It is simple, UC blocks the access to the '__unitystorage/' folder. The error is misleading and so many false suggestions from Internet.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.