Hi,
Thanks for posting in Q&A platform.
I noticed that the issue is related to Azure Databricks which our forum doesn't focus on, I would suggest you could post a new thread with tag azure-databricks in the following forum where Azure Databrics support engineer will provide more professional help for you.
https://learn.microsoft.com/en-us/answers/topics/azure-databricks.html
Best Regards,
Sunny
----------
If the Answer is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.