Azure Document Intelligence custom classifier training stuck at Running / 0%

Question

Azure Document Intelligence custom classifier training stuck at Running / 0%

Jason Garrido 0

Hello,

I am experiencing an issue with Azure Document Intelligence custom classification model training.

Resource:

jplaudit-shared-docintel

Region:

Southeast Asia

API version:

2024-11-30

Classifier:

jpl-wong-classifier-20260816-02

Operation ID:

31379484168_32e97360-3f16-4fab-9a70-f158e481d56e

The classifier has remained in the following state:

Status: running

Percent completed: 0

Created: 2026-08-16T12:37:12Z

Last updated: 2026-08-16T14:13:58Z

The lastUpdatedDateTime has not changed since then.

I verified the operation using both the List Operations API and the individual Get Operation API. Both return exactly the same state: running at 0%, with no error information.

I also tried retrieving the classifier directly. The service returned:

HTTP 400

ModelNotReady: Model is not ready for the requested operation. Wait for training to complete or check for operation errors.

The classifier therefore appears to exist as a training operation, but is not available as a completed classifier.

There was an earlier training attempt immediately before this one. That attempt failed within approximately four seconds. The training dataset was subsequently corrected, and the current classifier was trained against the corrected dataset. The current training attempt does not show a corresponding validation error or failure; it has simply remained at 0% with no further operation update.

The training dataset is within the documented service limits.

Could someone please investigate the server-side state of operation:

31379484168_32e97360-3f16-4fab-9a70-f158e481d56e

Specifically, I would like to know:

Is this operation actually still executing, or is it stuck/orphaned on the service side?
Is there a backend error or condition that is not being exposed through the operation API?
If the operation is orphaned/stuck, what is the recommended way to clear it so that a new training operation can be started?

I can provide API request IDs and additional diagnostic information if required.

Thank you.

0 comments

1 answer

Your answer

Answer 1

Jose Benjamin Solis Nolasco 12,026 Volunteer Moderator

Welcome to Microsoft Q&A!

Hello @Jason Garrido I hope you are doing well,

this does not appear to be a normal long-running training operation. T

he key indicators are:

Status remains running

Progress remains 0%

lastUpdatedDateTime has not changed

No error is returned by the Operations API

Get Classifier returns ModelNotReady

Microsoft documentation indicates that custom classifier training normally completes within a few minutes. A previous Microsoft Q&A case also addressed a classifier that remained in Running for an extended period and was considered potentially stuck. check this link (learn.microsoft.com)

You can also try submitting a new training operation with the corrected dataset. If the new operation completes successfully, that would strongly suggest the original operation was stuck.

If the new operation also remains at Running / 0%, Microsoft Support so they can investigate the backend processing.

For reference:

Build and train a custom classifier | Microsoft Learn

Document Classifiers REST API

If this answer helped clarify the platform capabilities and save you troubleshooting time, please consider marking it as Accepted. This helps others in the community find similar solutions.

Jason Garrido 0 Reputation points

2026-08-18T10:19:43.43+00:00
Update: Additional investigation completed — classifier build fails with InternalServerError after ~30.5 hours

Hi Microsoft,

We have completed the additional investigation requested/recommended in this thread and now have the final error response from the second classifier training attempt, as well as additional verification of the Azure resource and training data.

We would appreciate Microsoft's guidance on whether this indicates an Azure Document Intelligence backend issue or whether there is another configuration or dataset issue that we should investigate.

Environment:

Azure Document Intelligence resource: jplaudit-shared-docintel

Region: Southeast Asia

API version: 2024-11-30

Classifier: custom document classifier

Third training operation currently running: 31379382536_6dd7971d-3106-48d0-9d3f-b585cefcaeb9

Second training attempt eventually FAILED

We retrieved the complete Get Operation response for the second training attempt:

Operation ID:

31379484168_32e97360-3f16-4fab-9a70-f158e481d56e

The operation returned:

{

"operationId": "31379484168_32e97360-3f16-4fab-9a70-f158e481d56e",

"kind": "documentClassifierBuild",

"status": "failed",

"createdDateTime": "2026-08-16T12:37:12Z",

"lastUpdatedDateTime": "2026-08-17T19:06:35Z",

"percentCompleted": 100,

"error": {

"code": "InternalServerError", "message": "An unexpected error occurred.", "details": [ { "code": "InternalServerError", "message": "An unexpected error occurred." } ]

},

"apiVersion": "2024-11-30"

}

The operation therefore ran for approximately 30 hours and 29 minutes before reaching a terminal failed state.

Most importantly, the error contains no target, filename, class name, storage path, field name, or other information identifying a problem with a particular input document or classifier configuration.

The error.details array contains only the same generic InternalServerError and provides no additional diagnostic information.

This differs from the earlier training failure

An earlier classifier training attempt returned a specific validation-type error related to invalid content in the training source.

That was actionable because it identified a concrete problem with the input data.

The second attempt, however, did not return a validation or content-specific error. It completed its processing window and ultimately returned only:

InternalServerError

"An unexpected error occurred."

This distinction is why we would like Microsoft to determine whether the second failure represents an internal Document Intelligence service/backend problem or whether there is another condition that can produce this generic error.

Third training attempt is still running

The third training operation is:

31379382536_6dd7971d-3106-48d0-9d3f-b585cefcaeb9

Its current Get Operation response is:

{

"operationId": "31379382536_6dd7971d-3106-48d0-9d3f-b585cefcaeb9",

"kind": "documentClassifierBuild",

"status": "running",

"createdDateTime": "2026-08-17T16:51:03Z",

"lastUpdatedDateTime": "2026-08-17T17:08:15Z",

"percentCompleted": 0,

"apiVersion": "2024-11-30"

}

At the time of this investigation it remains at 0%.

We have deliberately not cancelled or modified this operation because the second attempt demonstrated that the service can take more than 24 hours before reaching a terminal state.

We also have not started a fourth training attempt.

Training data and container were independently verified

We inspected the actual training-samples container used by the Document Intelligence Studio project.

The container currently contains 193 objects.

The dataset structure was cross-checked against the expected training data and class manifests.

The training documents were also manually opened and verified through the Azure portal/Document Intelligence Studio preview.

We confirmed that:

The training documents are present in the configured container.

The documents can be accessed and viewed.

The expected OCR JSON artifacts are present.

The class manifest files are present.

The container/object counts are consistent with the expected dataset structure.

The Document Intelligence Studio project is configured against the intended storage container.

No obvious missing-file or inaccessible-file condition was identified.

We have also spot-checked the OCR JSON artifacts to ensure that they are not simply empty placeholder files.

Document Intelligence resource and authentication are working

We successfully authenticated against the Document Intelligence endpoint and were able to:

Query Document Intelligence information.

Retrieve long-running operation status.

Retrieve the classifier resource location.

Enumerate classifiers.

The Document Intelligence resource itself is responding normally to API requests.

The resource information endpoint also shows that the custom model/classifier quota is not exhausted.

Azure Service Health was checked

We checked Azure Portal:

Service Health → Health history

for the subscription and relevant period.

No service-health incident was found for the relevant period.

We then performed the more specific check recommended in this investigation:

Service Health → Resource health → Cognitive Services account → jplaudit-shared-docintel

The resource reports:

Status: Available

Azure states:

"There aren't any known Azure platform problems affecting this Cognitive Services resource."

The resource health history for the last four weeks shows no Degraded or Unavailable event covering the classifier failure window.

The relevant failure window for operation -02 was:

2026-08-16T12:37:12Z

through

2026-08-17T19:06:35Z

The only resource-health entry currently shown is:

2026-08-18 — Available

We understand that Resource Health represents the general health/availability of the resource and does not necessarily expose failures occurring inside a specific long-running backend training job. Therefore, we are not treating the Resource Health result as proof that no backend processing fault occurred. It only rules out a broader resource-level availability incident being recorded for this resource.

We also verified that both Microsoft.ResourceHealth and Microsoft.CognitiveServices are registered for the subscription.

Current assessment

At this point we have ruled out or substantially checked the following:

Authentication failure

Document Intelligence endpoint connectivity

Exhausted custom model quota

Missing training documents

Obvious storage/container accessibility problems

Obvious dataset/object-count mismatch

Obvious missing OCR artifacts

An active subscription-level Azure Service Health incident

A recorded resource-level availability/degradation incident

The strongest unresolved evidence is therefore the second classifier operation:

Approximately 30h29m runtime

Reached terminal status failed

percentCompleted = 100

error.code = InternalServerError

error.message = "An unexpected error occurred."

error.details contains only the same generic InternalServerError

No target, document, class, storage path, or configuration field is identified

We recognize that this does not by itself prove an Azure backend defect. It could potentially represent an internal service failure triggered by something specific in the dataset or classifier configuration that the service did not report correctly. We would therefore like Microsoft's interpretation rather than assuming the cause.

Questions for Microsoft:

What conditions in Document Intelligence classifier training can produce an InternalServerError with no target or diagnostic information?

Does this response indicate an internal backend/orchestration failure, or can a dataset/configuration problem legitimately result in this exact generic error?

Is there any server-side diagnostic information associated with operation 31379484168_32e97360-3f16-4fab-9a70-f158e481d56e that Microsoft Support can inspect?

Can Microsoft determine why the operation ran for approximately 30h29m before returning InternalServerError?

Is there anything specific we should verify or change before allowing the current third operation 31379382536_6dd7971d-3106-48d0-9d3f-b585cefcaeb9 to continue or before attempting another build?

We have intentionally not started another training attempt or changed the dataset while this investigation is ongoing, so that we do not introduce additional variables.

At this point we would appreciate guidance from Microsoft on the server-side meaning of this InternalServerError and whether the failed operation can be investigated from Microsoft's backend telemetry.

Thank you.