Azure databricks cannot create table in metastore.

최 규호 91 Reputation points
2023-02-07T08:53:19.2266667+00:00

I fallowed the docs https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/get-started but I can not make the table in workspace main catalog. why the error occur?

the error message is that

Error in SQL statement: UnityCatalogServiceException: [RequestId=1f3c07ef-0876-4d8c-a0ed-a3ba70842f95 ErrorClass=INVALID_STATE] Failed to access cloud storage: AbfsRestOperationException exceptionTraceId=e00354de-3968-4dd7-9bd1-9715e6d28264
com.databricks.backend.common.rpc.SparkDriverExceptions$SQLExecutionException: com.databricks.sql.managedcatalog.UnityCatalogServiceException: [RequestId=1f3c07ef-0876-4d8c-a0ed-a3ba70842f95 ErrorClass=INVALID_STATE] Failed to access cloud storage: AbfsRestOperationException exceptionTraceId=e00354de-3968-4dd7-9bd1-9715e6d28264
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException(ErrorDetailsHandler.scala:33)
	at com.databricks.managedcatalog.ErrorDetailsHandler.wrapServiceException$(ErrorDetailsHandler.scala:23)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.wrapServiceException(ManagedCatalogClientImpl.scala:91)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.recordAndWrapException(ManagedCatalogClientImpl.scala:3021)
	at com.databricks.managedcatalog.ManagedCatalogClientImpl.createStagingTable(ManagedCatalogClientImpl.scala:511)
	at com.databricks.sql.managedcatalog.ManagedCatalogCommon.defaultTablePath(ManagedCatalogCommon.scala:234)
	at com.databricks.sql.managedcatalog.ManagedCatalogSessionCatalog.defaultTablePath(ManagedCatalogSessionCatalog.scala:934)
	at com.databricks.sql.transaction.tahoe.catalog.DeltaCatalog.$anonfun$createDeltaTable$9(DeltaCatalog.scala:178)
	at scala.Option.getOrElse(Option.scala:189)
	at com.databricks.sql.transaction.tahoe.catalog.DeltaCatalog.$anonfun$createDeltaTable$1(DeltaCatalog.scala:178)
	at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80)
	at com.databricks.sql.transaction.tahoe.metering.DeltaLogging.recordFrameProfile(DeltaLogging.scala:265)
	at com.databricks.sql.transaction.tahoe.metering.DeltaLogging.recordFrameProfile$(DeltaLogging.scala:263)
	at com.databricks.sql.transaction.tahoe.catalog.DeltaCatalog.recordFrameProfile(DeltaCatalog.scala:85)
	at com.databricks.sql.transaction.tahoe.catalog.DeltaCatalog.com$databricks$sql$transaction$tahoe$catalog$DeltaCatalog$$createDeltaTable(DeltaCatalog.scala:113)
	at com.databricks.sql.transaction.tahoe.catalog.DeltaCatalog$StagedDeltaTableV2.$anonfun$commitStagedChanges$1(DeltaCatalog.scala:758)
	at com.databricks.spark.util.FrameProfiler$.record(FrameProfiler.scala:80)
	at com.databricks.sql.transaction.tahoe.metering.DeltaLogging.recordFrameProfile(DeltaLogging.scala:265)
	at com.databricks.sql.transaction.tahoe.metering.DeltaLogging.recordFrameProfile$(DeltaLogging.scala:263)

My Query is

Create table main.default.test as select * from samples.nyctaxi.trips

the table samples.nyctaxi.trips is exist.

User's image

and I attache the error capture.

User's image

please check about it.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,175 questions
{count} votes

Accepted answer
  1. Bhargava-MSFT 31,011 Reputation points Microsoft Employee
    2023-03-06T22:44:42.3+00:00

    Hello @최 규호 ,

    Sorry for the delayed response. Can you please follow the steps mentioned in this document and upgrade your existing unity catalog metastore?

    Once you upgrade the metastore it should fix the error.


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.