Creates a new evaluator version with an auto-incremented version identifier.

The API version to use for this operation.

The name of the resource

Display Name for evaluator. It helps to find the evaluator easily in AI Foundry. It does not need to be unique.

Metadata about the evaluator

The type of the evaluator

The categories of the evaluator

Evaluation levels this evaluator supports (e.g., turn, conversation). When omitted on create, the service defaults to ["turn"]. On update, omitting this field leaves it unchanged; an empty list is rejected. Custom code-based evaluators support only turn; custom prompt-based evaluators support exactly one level (turn or conversation).

Base evaluator configuration with discriminator

Code-based evaluator definition using python code

Inline code text for the evaluator

The entry point Python file name for the uploaded evaluator code (e.g. 'answer_length_evaluator.py')

The container image tag to use for evaluator code execution

The blob URI for the evaluator storage

Prompt-based evaluator

The prompt text used for evaluation

Rubric-based evaluator definition — stores dimensions produced by the generate API. Used for both quality and safety evaluators.

The set of dimensions — the scoring blueprint used by the LLM judge. Quality evaluators include a non-editable residual dimension with id 'general_quality' (always_applicable: true); safety evaluators include 'general_policy_compliance'. Both use the same Dimension structure.

Pass/fail threshold for the aggregate rubric score, on the same normalized 0.0-1.0 scale as the emitted score. When the runtime weighted average meets or exceeds this value, the result is pass. Defaults to 0.5 (equivalent to a raw 1-5 weighted average of 3.0). The 'any dimension scored 1 → fail' rule still applies regardless of this threshold.

Endpoint-based evaluator definition. The customer owns and hosts an HTTP endpoint that implements the evaluation contract. The evaluator references a Project Connection by name; the connection stores the endpoint URL and credentials (API Key or Entra ID). At execution time, the service resolves the connection to obtain the endpoint URL and authentication details, then calls the endpoint for each evaluation row.

Name of the Project Connection that stores the endpoint URL and credentials. The connection must exist on the project and have a non-empty target URL. Supported auth types: ApiKey (sends api-key header) and AAD/Entra ID (acquires a bearer token via the project's Managed Identity).

The asset description text.

Tag dictionary. Tags can be added, removed, and updated.

Display Name for evaluator. It helps to find the evaluator easily in AI Foundry. It does not need to be unique.

Metadata about the evaluator

The type of the evaluator

The categories of the evaluator

Evaluation levels this evaluator supports (e.g., turn, conversation). When omitted on create, the service defaults to ["turn"]. On update, omitting this field leaves it unchanged; an empty list is rejected. Custom code-based evaluators support only turn; custom prompt-based evaluators support exactly one level (turn or conversation).

Base evaluator configuration with discriminator

Code-based evaluator definition using python code

Inline code text for the evaluator

The entry point Python file name for the uploaded evaluator code (e.g. 'answer_length_evaluator.py')

The container image tag to use for evaluator code execution

The blob URI for the evaluator storage

Prompt-based evaluator

The prompt text used for evaluation

Rubric-based evaluator definition — stores dimensions produced by the generate API. Used for both quality and safety evaluators.

The set of dimensions — the scoring blueprint used by the LLM judge. Quality evaluators include a non-editable residual dimension with id 'general_quality' (always_applicable: true); safety evaluators include 'general_policy_compliance'. Both use the same Dimension structure.

Pass/fail threshold for the aggregate rubric score, on the same normalized 0.0-1.0 scale as the emitted score. When the runtime weighted average meets or exceeds this value, the result is pass. Defaults to 0.5 (equivalent to a raw 1-5 weighted average of 3.0). The 'any dimension scored 1 → fail' rule still applies regardless of this threshold.

Endpoint-based evaluator definition. The customer owns and hosts an HTTP endpoint that implements the evaluation contract. The evaluator references a Project Connection by name; the connection stores the endpoint URL and credentials (API Key or Entra ID). At execution time, the service resolves the connection to obtain the endpoint URL and authentication details, then calls the endpoint for each evaluation row.

Name of the Project Connection that stores the endpoint URL and credentials. The connection must exist on the project and have a non-empty target URL. Supported auth types: ApiKey (sends api-key header) and AAD/Entra ID (acquires a bearer token via the project's Managed Identity).

Service-managed provenance artifacts produced by an evaluator generation job. Present only on EvaluatorVersion resources created via the generation pipeline. The combined-JSONL Foundry Dataset is read-only and resolves to a versioned dataset in a service-reserved namespace.

Reference to a versioned Foundry Dataset.

Dataset name.

Dataset version.

The kinds of rows present in dataset. Always contains "spec" (the generated evaluation specification, a Markdown document describing what the evaluator measures). May additionally contain "tools" (when the generation pipeline produced or inferred OpenAI tool schemas) and/or "context" (when supplementary materials such as file uploads or trace samples were used during generation).

Read-only provenance link back to the EvaluatorGenerationJob that produced this version. Present only on evaluator versions created via the generation pipeline; absent for manually-created versions and unaffected by subsequent PATCH calls.

Categories of warnings surfaced on this generated evaluator version. Present only on versions created via an EvaluatorGenerationJob when the paired job produced non-empty warnings. Absent (treat as no warnings) when the version is not from generation, when the paired job was clean, or when a subsequent PATCH to definition cleared the paired job's advisories. Follow generation_job_id to fetch the detailed warning payloads.

Creator of the evaluator

Creation date/time of the evaluator

Last modified date/time of the evaluator

Asset ID, a unique identifier for the asset

The name of the resource

The version of the resource

String error code indicating what went wrong.

The error object.

One of a server-defined set of error codes.

A human-readable representation of the error.

The target of the error.

An array of details about specific errors that led to this reported error.

An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors.

One of a server-defined set of error codes.

String error code indicating what went wrong.

The error object.

One of a server-defined set of error codes.

A human-readable representation of the error.

The target of the error.

An array of details about specific errors that led to this reported error.

An object containing more specific information about the error. As per Azure REST API guidelines - https://aka.ms/AzureRestApiGuidelines#handling-errors.

One of a server-defined set of error codes.