Azure OpenAI Batch API: Receiving Empty Outputs and Errors

Question

Azure OpenAI Batch API: Receiving Empty Outputs and Errors

Richard E. Fan 10

The structured output format with the batch API has been used successfully for several weeks. Previously, it worked with both the 4o and o3-mini models.

Recently, after updating the prompts and attempting to re-run a task with a JSONL file containing 3002 lines (total size of 98 MB), the output returned 0 bytes with no error files generated. Despite reverting changes and re-uploading a successful job from 4 days ago (which completed 3002/3002 successfully), the current attempt using the same endpoint resulted in 0 completions, 240 errors, and 0 failures.

Is there a possibility that something has changed on the Batch API service side? An inquiry regarding a billing refund has been submitted due to extensive troubleshooting over the past 24 hours, which has consumed a significant number of tokens with minimal output generated.

The o3-mini model works as expected in typical chat-completion mode, and submitting a batch JSONL file of 1 or 10 lines also yields successful results. However, with an increase to 100 lines, consistent failures are encountered, with only 48 out of 100 lines completing, for example.

Are there any insights or suggestions regarding this issue?

Jack Boylan 5 Reputation points

2025-03-16T12:35:58.2166667+00:00

I am facing exactly the same issue
Vegas Reign 0 Reputation points

2025-03-16T14:18:17.4133333+00:00

Azure OpenAI Batch API: Receiving Empty Outputs and Errors

Richard E. Fan •

Follow
Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-19T07:36:38.8733333+00:00

Hi Richard E. Fan

Could you share the more details of errors and region for Repro.

Thank You.

Richard E. Fan 10

Region is East US 2. I am not getting any error files generated by the batch API, other than seeing only fewer tasks getting to the "completed" status. An example, this was a job that was successfully run on March 11th:

{
  "Batch ID": "original_batch_id",
  "Status": "completed",
  "Created At": "2025-03-11 19:39:17",
  "Completed At": "2025-03-11 20:50:18",
  "Output File ID": "original output file",
  "Error File ID": "original error file",
  "Request Counts": "BatchRequestCounts(completed=3002, failed=0, total=3002)",
  "Errors": null
}

when as part of trouble shooting, i resubmitted the exact input JSONL and got this result:

{
  "Batch ID": "second batch id",
  "Status": "completed",
  "Created At": "2025-03-14 11:47:28",
  "Completed At": "2025-03-14 11:59:07",
  "Output File ID": "second output file",
  "Error File ID": "second error file",
  "Request Counts": "BatchRequestCounts(completed=0, failed=290, total=3002)",
  "Errors": null
}

Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-20T10:07:19.9066667+00:00
Hi Richard E. Fan

It seems that you are experiencing issues with the Azure OpenAI Batch API where you are receiving empty outputs and errors after a recent change in your prompts and input file.

Here are some steps:

Make sure your JSONL file follows the correct format, as any improper lines can cause parsing errors. The error code invalid_json_line means there might be problems with how your file is structured.

The batch API allows up to 100,000 requests in one input file. If you're facing problems with larger batches, such as 100 lines, verify whether the total number of requests surpasses this limit.

If smaller batches (1 or 10 lines) work without issues, but larger batches consistently fail, consider breaking down your input into smaller chunks to identify if a specific line or request is causing the failure.

Kindly refer below link: Howto use global batch processing with Azure OpenAI Service

Thank You.
Richard E. Fan 10 Reputation points

2025-03-20T16:20:40.8766667+00:00

Hi Saideep Anchuri,

Yes you are correct, although i am not having the same issues you are describing (no invalid_json_line errors, in fact no errors at all).

I can try to break down my batch to smaller sizes, but i am using the same pipeline that has been successful for months. The JSONL files i am creating are 50-100 MB, with at most 3002 lines in this project, which is well under the 200MB and 100,000 lines.

Further, when i download a JSONL that was created on March 11th, that worked properly (3002/3002 lines complete), and i recreate a batch job using that exact file, i get a failure (0 complete, 290 failures) which leaves me confused.
Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-20T17:11:21.67+00:00
Hi Richard E. Fan

Verify if your subscription has exceeded usage limits (e.g., token quotas), which could cause unexpected failures in batch processing.

Attempt running smaller batches from the same file to see if there’s a specific line or set of lines causing the failure.

It's possible that a regression or limitation was introduced in the model you're using. Test with another model to see.

Thank You.

1 answer

Your answer

Jack Boylan 5 Reputation points

2025-03-16T12:35:58.2166667+00:00

I am facing exactly the same issue
Vegas Reign 0 Reputation points

2025-03-16T14:18:17.4133333+00:00

Azure OpenAI Batch API: Receiving Empty Outputs and Errors

Richard E. Fan •

Follow
Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-19T07:36:38.8733333+00:00

Hi Richard E. Fan

Could you share the more details of errors and region for Repro.

Thank You.
Richard E. Fan 10 Reputation points

2025-03-19T15:50:54.9833333+00:00

Region is East US 2. I am not getting any error files generated by the batch API, other than seeing only fewer tasks getting to the "completed" status. An example, this was a job that was successfully run on March 11th:

{ "Batch ID": "original_batch_id", "Status": "completed", "Created At": "2025-03-11 19:39:17", "Completed At": "2025-03-11 20:50:18", "Output File ID": "original output file", "Error File ID": "original error file", "Request Counts": "BatchRequestCounts(completed=3002, failed=0, total=3002)", "Errors": null }

when as part of trouble shooting, i resubmitted the exact input JSONL and got this result:

{ "Batch ID": "second batch id", "Status": "completed", "Created At": "2025-03-14 11:47:28", "Completed At": "2025-03-14 11:59:07", "Output File ID": "second output file", "Error File ID": "second error file", "Request Counts": "BatchRequestCounts(completed=0, failed=290, total=3002)", "Errors": null }
Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-20T10:07:19.9066667+00:00

Hi Richard E. Fan

It seems that you are experiencing issues with the Azure OpenAI Batch API where you are receiving empty outputs and errors after a recent change in your prompts and input file.

Here are some steps:

Make sure your JSONL file follows the correct format, as any improper lines can cause parsing errors. The error code invalid_json_line means there might be problems with how your file is structured.

The batch API allows up to 100,000 requests in one input file. If you're facing problems with larger batches, such as 100 lines, verify whether the total number of requests surpasses this limit.

If smaller batches (1 or 10 lines) work without issues, but larger batches consistently fail, consider breaking down your input into smaller chunks to identify if a specific line or request is causing the failure.

Kindly refer below link: Howto use global batch processing with Azure OpenAI Service

Thank You.
Richard E. Fan 10 Reputation points

2025-03-20T16:20:40.8766667+00:00

Hi Saideep Anchuri,

Yes you are correct, although i am not having the same issues you are describing (no invalid_json_line errors, in fact no errors at all).

I can try to break down my batch to smaller sizes, but i am using the same pipeline that has been successful for months. The JSONL files i am creating are 50-100 MB, with at most 3002 lines in this project, which is well under the 200MB and 100,000 lines.

Further, when i download a JSONL that was created on March 11th, that worked properly (3002/3002 lines complete), and i recreate a batch job using that exact file, i get a failure (0 complete, 290 failures) which leaves me confused.
Saideep Anchuri 9,425 Reputation points Microsoft External Staff Moderator

2025-03-20T17:11:21.67+00:00

Hi Richard E. Fan

Verify if your subscription has exceeded usage limits (e.g., token quotas), which could cause unexpected failures in batch processing.

Attempt running smaller batches from the same file to see if there’s a specific line or set of lines causing the failure.

It's possible that a regression or limitation was introduced in the model you're using. Test with another model to see.

Thank You.

Answer 1

I met the same issue: input/output tokens were consumed, but all requests failed. The response and error file I received was as follows.

response:

{
  "id": "XXX",
  "completion_window": "24h",
  "created_at": 1744292580,
  "endpoint": "/chat/completions",
  "input_file_id": "indow": "24h",
  "created_at": 1744292580,
  "endpoint": "/chat/completions",
  "input_file_id": "XXX",
  "object": "batch",
  "status": "completed",
  "cancelled_at": null,
  "cancelling_at": null,
  "completed_at": 1744293570,
  "error_file_id": "XXX",
  "errors": null,
  "expired_at": null,
  "expires_at": 1744378980,
  "failed_at": null,
  "finalizing_at": 1744293496,
  "in_progress_at": 1744292982,
  "metadata": null,
  "output_file_id": "XXX",
  "request_counts": {
    "completed": 0,
    "failed": 4,
    "total": 4
  }
}

error file:

{"custom_id": "2", "response": {"request_id": "", "status_code": null}, "error": {"code": null, "message": null}}
{"custom_id": "1", "response": {"request_id": "", "status_code": null}, "error": {"code": null, "message": null}}
{"custom_id": "3", "response": {"request_id": "", "status_code": null}, "error": {"code": null, "message": null}}
{"custom_id": "0", "response": {"request_id": "", "status_code": null}, "error": {"code": null, "message": null}}

Did you solve this issue?

Share via

Azure OpenAI Batch API: Receiving Empty Outputs and Errors

1 answer

Your answer