An Apache Spark-based analytics platform optimized for Azure.
The HTTP 400 response is the most useful diagnostic signal here. It indicates that the request is reaching the Azure Databricks service but AI Runtime is not available to that workspace. Since serverless CPU works and the failure affects all users, this is unlikely to be caused by notebook permissions, compute permissions, or an individual user configuration.
Current azure databricks documentaton states that:
- AI Runtime availability depends on the workspace geography, and GPU availability can vary by workspace.
- For a workspace with the compliance security profile enabled, a workspace administrator must enable the AI Runtime preview.
- AI Runtime does not support compliance configurations using FedRAMP High/DoD IL5. [learn.microsoft.com]
Therefore, the workspace administrator should first verify whether the compliance security profile is enabled:
- Open the workspace and select the user profile menu.
- Select Previews.
- Check whether AI Runtime is listed and disabled.
- If it is not listed, also check the Previews page in the Azure Databricks account console, because previews can be controlled at either workspace or account scope. [learn.microsoft.com]
However, the absence of an AI Runtime toggle does not by itself prove a configuration problem. The documentation does not describe a self-service switch for every workspace. It explicitly notes that GPU availability varies by workspace and that capacity can be constrained or unavailable in a region. [learn.microsoft.com]
Given the explicit AI Runtime is not available response, the remaining likely causes are:
- The workspace is not currently eligible for AI Runtime.
- AI Runtime has not been provisioned for the account or workspace.
- The workspace security or compliance configuration is incompatible.
- GPU capacity or workspace-level availability is restricted.
A workspace or account administrator should use the Databricks sales/contact route referenced by the error and provide the Azure tenant ID, subscription ID, Databricks account ID, workspace ID, Azure region and the complete HTTP 400 response. Ask for confirmation of workspace eligibility and enablement rather than general notebook troubleshooting.
The documented AI Runtime prerequisites and limitations are available in the AI Runtime overview. AI Runtime is separate from Databricks Runtime ML on classic GPU compute, so successfully using a regular GPU cluster would not confirm serverless GPU eligibility. [learn.microsoft.com]
Help make this community better for everyone: if this answer resolved your issue, please accept it or leave an upvote. If not, share more details in a comment so we can continue the discussion and find the right solution