Hello arkiboys,
from the error message, It seems like table access is not enabled on the cluster or Azure Databricks workspace administrators doesn't have access to file-level data.
Please confirm if you have acess at file-level? if you have access at file-level, then please check the following.
Table access control is available in two versions:
- SQL-only table access control, which restricts users to SQL commands.
- Python and SQL table access control, which allows users to run SQL, Python, and PySpark commands.
SQL-only table access control: set the following flag in the cluster’s Spark conf:
spark.databricks.acl.sqlOnly true
Python and SQL table access control
set the Spark config spark.databricks.pyspark.iptable.outbound.whitelisted.ports
to the ports you want to allow access.
To enable table access control:
from the documentation:
Before users can configure Python and SQL table access control, an Azure Databricks workspace must enable table access control for the Azure Databricks workspace and deny users access to clusters that are not enabled for table access control.
Go to admin settings page:
Workspace admin --> security--> click the table access control toggle and confirm
Once the table access control has been enabled on a cluster, see Data object privileges.
I hope this helps.