Share via

Cannot Create or List any spark Database

Alexander B. Hurtado 6 Reputation points
2021-05-26T17:20:38.233+00:00

Im trying to create a spark database either using spark.sql("CREATE DATABASE dbname") or df.write.mode("overwrite").saveAsTable(dbname.tablename) and is giving me this error:

AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;
Traceback (most recent call last):
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/session.py”, line 767, in sql
return DataFrame(self._jsparkSession.sql(sqlQuery), self._wrapped)
File “/opt/spark/python/lib/py4j-0.10.7-src.zip/py4j/java_gateway.py”, line 1257, in call
answer, self.gateway_client, self.target_id, self.name)
File “/opt/spark/python/lib/pyspark.zip/pyspark/sql/utils.py”, line 75, in deco
raise AnalysisException(s.split(‘: ’, 1)[1], stackTrace)
pyspark.sql.utils.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient;

It algo gives me the same error when I try to list the tables using: print(spark.catalog.listTables())

Azure Synapse Analytics
Azure Synapse Analytics

An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.


1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA 91,866 Reputation points
    2021-06-04T04:28:03.913+00:00

    Hello @Alexander B. Hurtado ,

    We had fixed the issue for your workspace. Could you please check and confirm on the same?

    This was one of the small set workspaces where we failed to enable SyMS. We created a hotfix from backend to resolve the issue.

    Hope this helps. Do let us know if you any further queries.

    ---------------------------------------------------------------------------

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

    Was this answer helpful?


Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.