Databricks AI_QUERY fails with certain OpenAI models

Question

Databricks AI_QUERY fails with certain OpenAI models

James Bentley 10

We have been using successfully using the AI_QUERY function in Azure Databricks to interact with o1-mini and text-embedding-ada-002 models.

However in the past few days, we have started to encounter errors. Calling our o1-mini model as usual using:

spark.sql("""
SELECT AI_QUERY(
        'o1-mini', 
        request => 'What is the capital of United Kingdom?',
        failOnError => false,
        modelParameters => named_struct('temperature', 1)
    )
 """).display()

Sometimes, but not every time, results in the error:

"[REMOTE_FUNCTION_HTTP_FAILED_ERROR] The remote HTTP request failed with code 400, and error message 'HTTP request failed with status: {\"error_code\":\"BAD_REQUEST\",\"message\":\"{\\\\\"external_model_provider\\\\\":\\\\\"openai\\\\\",\\\\\"external_model_error\\\\\":{\\\\\"error\\\\\":{\\\\\"message\\\\\":\\\\\"Invalid prompt: your prompt was flagged as potentially violating our usage policy. Please try again with a different prompt: https://platform.openai.com/docs/guides/reasoning#advice-on-prompting\\\\\",\\\\\"type\\\\\":\\\\\"invalid_request_error\\\\\",\\\\\"param\\\\\":null,\\\\\"code\\\\\":\\\\\"invalid_prompt\\\\\"}}}\"}' SQLSTATE: 57012"

This can also be reliably replicated across different Databricks workspaces and different deployments of o1-mini.

We have started encountering issues with text-embedding-ada-002 as well.

These models work fine when being interacted with directly or through the model playground, and equally other models such as 4o, Llama 70B, and bge-large seem to be unaffected.

Has anyone else encountered these issues? Or does anyone else find this when running in their workspace?

Thanks :)

Ganesh Gurram 7,230 Reputation points Moderator

2025-03-13T11:35:37.17+00:00

Hi @James Bentley

We are reaching out to the internal team regarding this issue and will update you as soon as possible.
Ganesh Gurram 7,230 Reputation points Moderator

2025-03-18T04:17:24.1066667+00:00

@James Bentley - Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Anonymous

2025-03-25T08:59:39.0633333+00:00

@James Bentley looks like OpenAI is flagging your prompt as a potential policy violation, which is causing the request to fail with a 400 error. Basically, the system is rejecting the input as invalid, so you might need to tweak your prompt to get it working.
James Bentley 10 Reputation points

2025-03-25T10:54:10.01+00:00

@Anonymous thank you for your reply, however this does not explain why I can run the exact same prompt in the exact same cell a few times in a row (with a reasonable wait inbetween etc), and sometimes it works and sometimes it doesn’t?

Also if it were truly an issue of OpenAI policy, why does it only affect o1-mini models and not other OpenAI models?

i made another post which explains this better which was originally marked as spam, and has subsequently been marked as a duplicate, but it offers more insight in to the issue: https://learn.microsoft.com/en-us/answers/questions/2224387/databricks-ai-query-fails-with-o1-mini-and-text-em

1 answer

Your answer

Ganesh Gurram 7,230 Reputation points Moderator

2025-03-13T11:35:37.17+00:00

Hi @James Bentley

We are reaching out to the internal team regarding this issue and will update you as soon as possible.
Ganesh Gurram 7,230 Reputation points Moderator

2025-03-18T04:17:24.1066667+00:00

@James Bentley - Just checking in to see if the below answer helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Anonymous

2025-03-25T08:59:39.0633333+00:00

@James Bentley looks like OpenAI is flagging your prompt as a potential policy violation, which is causing the request to fail with a 400 error. Basically, the system is rejecting the input as invalid, so you might need to tweak your prompt to get it working.
James Bentley 10 Reputation points

2025-03-25T10:54:10.01+00:00

@Anonymous thank you for your reply, however this does not explain why I can run the exact same prompt in the exact same cell a few times in a row (with a reasonable wait inbetween etc), and sometimes it works and sometimes it doesn’t?

Also if it were truly an issue of OpenAI policy, why does it only affect o1-mini models and not other OpenAI models?

i made another post which explains this better which was originally marked as spam, and has subsequently been marked as a duplicate, but it offers more insight in to the issue: https://learn.microsoft.com/en-us/answers/questions/2224387/databricks-ai-query-fails-with-o1-mini-and-text-em

Answer 1

Ganesh Gurram 7,230 Moderator

@James Bentley - Apologies for the delayed response, and we appreciate your patience.

As per the recommendation from our internal team, you may try disabling or lowering the severity of the AOAI content filters to see if that helps resolve the issue.

Hope this helps. Do let us know if you have any further queries.

James Bentley 10 Reputation points

2025-03-20T08:47:14.33+00:00

<DUPLICATE COMMENT>
James Bentley 10 Reputation points

2025-03-20T08:48:34.72+00:00

Thank you for your suggestion! However it seems not to have resolved the issue, especially as the issue pertaining to the embedding model seems not to be related to content filters.
Ganesh Gurram 7,230 Reputation points Moderator

2025-03-24T03:35:56.9766667+00:00

@James Bentley -

The error you're encountering with the AI_QUERY function in Azure Databricks, specifically the [REMOTE_FUNCTION_HTTP_FAILED_ERROR] with SQLSTATE: 57012, indicates an issue that requires operator intervention. According to the Azure Databricks documentation on SQLSTATE error codes, the class 57 pertains to "operator intervention," suggesting that the operation cannot proceed without administrative action.

Recommended Actions:

Review Prompt Content - Examine the prompts you're sending to ensure they comply with OpenAI's usage policies. Avoid content that could be flagged as inappropriate or harmful.

Implement Content Moderation - Utilize OpenAI's moderation tools to pre-screen prompts before sending them to the model, helping to identify and filter out potentially problematic content.

Handle Errors Gracefully - Set the failOnError parameter to false in your AI_QUERY function to capture errors without interrupting the workflow. This allows you to log and analyze errors for further investigation.

Additionally, Verify the number of API requests sent to the server to ensure there is no rate limiting or throttling. Check the size of the data being transmitted. If a request fails, review the prompt size and break it into smaller chunks if needed.

By following these steps, you can address the 57012 error and ensure smoother operation of the AI_QUERY function in your Azure Databricks environment.

For more details refer: https://learn.microsoft.com/en-us/azure/databricks/error-messages/sqlstates#class-57-operator-intervention

Hope this helps. Do let us know if you any further queries.
James Bentley 10 Reputation points

2025-03-24T10:02:43.25+00:00

@Ganesh Guram thank you for your reply, however these solutions don’t seem to apply to this particular set of issues:

Review Prompt Content - we get the issue randomly when sending the exact same prompt as shown in my original post “what is the capital city of the Uk” does not usually trigger content violations, no should it?

Implement Content Moderation - As above, what moderation warnings would “what is the capital city of the UK?” cause? Also as previously mentioned, we can run the same cell 10 times in a row with a healthy time gap in-between and sometimes get the error and sometimes not (with o1-mini)

Handle Errors Gracefully - This was originally set to false and is set to true only for the above example. Our current option seems to be rerunning the query in a loop until it works which doesn’t seem very elegant and only addresses the symptom, not the issue

Additionally, Verify the number of API requests sent to the server to ensure there is no rate limiting or throttling - I can confirm in all Databricks workspaces we have this running in that there is definitely no rate limiting issues for our controlled test cases to get the error.

As previously mentioned as well, text-ada-embedding-002 also fails every time but with a different error as show in in this post: https://learn.microsoft.com/en-us/answers/questions/2224387/databricks-ai-query-fails-with-o1-mini-and-text-em

Share via

Databricks AI_QUERY fails with certain OpenAI models

1 answer

Your answer