ModelDeploymentScalingProfile Class

Definition

Namespace:

Azure.ResourceManager.ContainerServiceAIManager.Models

Assembly:

Azure.ResourceManager.ContainerServiceAIManager.dll

Package:

Azure.ResourceManager.ContainerServiceAIManager v1.0.0-beta.1

Source:

ModelDeploymentScalingProfile.cs

Scaling configuration for a model deployment. Exactly one of manual or autoscale must be set. This mutual-exclusion constraint is enforced by the service at request validation time, not by the schema. A PUT request that sets both manual and autoscale, or sets neither, is rejected with HTTP 400 (Bad Request) and an InvalidScalingProfile error code; Scale-to-zero semantics differ between the two modes:

• `manual` permits `replicas: 0`. This is an explicit operator action to

stop serving traffic while keeping the `ModelDeployment` resource (and its configuration) in place. While at zero replicas the endpoint returns errors for inference requests, and the deployment releases its GPU capacity.

• `autoscale` does not permit `minReplicas: 0`. Autoscaling decisions are

driven by serving-server runtime metrics (request rate, queue depth, GPU utilization); at zero replicas there is no signal for the autoscaler to scale back up from. Combined with GPU cold-start time (on the order of minutes) and constrained regional GPU capacity, a scale-from-zero event would produce unacceptable first-request latency and a high risk of capacity unavailability. Callers that want autoscaling with an idle state should delete the `ModelDeployment` instead.

public class ModelDeploymentScalingProfile : System.ClientModel.Primitives.IJsonModel<Azure.ResourceManager.ContainerServiceAIManager.Models.ModelDeploymentScalingProfile>, System.ClientModel.Primitives.IPersistableModel<Azure.ResourceManager.ContainerServiceAIManager.Models.ModelDeploymentScalingProfile>

type ModelDeploymentScalingProfile = class
    interface IJsonModel<ModelDeploymentScalingProfile>
    interface IPersistableModel<ModelDeploymentScalingProfile>

Public Class ModelDeploymentScalingProfile
Implements IJsonModel(Of ModelDeploymentScalingProfile), IPersistableModel(Of ModelDeploymentScalingProfile)

Inheritance

Object

ModelDeploymentScalingProfile

Implements

IJsonModel<ModelDeploymentScalingProfile> IPersistableModel<ModelDeploymentScalingProfile> IPersistableModel<T>

Constructors

Name	Description
ModelDeploymentScalingProfile()	Initializes a new instance of ModelDeploymentScalingProfile.

Properties

Name	Description
Autoscale	Autoscaling configuration. Mutually exclusive with manual.
ManualReplicas	Fixed number of replicas. May be 0 to stop serving traffic while keeping the deployment configuration (see ScalingProfile).

Methods

Name	Description
JsonModelCreateCore(Utf8JsonReader, ModelReaderWriterOptions)
JsonModelWriteCore(Utf8JsonWriter, ModelReaderWriterOptions)
PersistableModelCreateCore(BinaryData, ModelReaderWriterOptions)
PersistableModelWriteCore(ModelReaderWriterOptions)

Explicit Interface Implementations

Name	Description
IJsonModel<ModelDeploymentScalingProfile>.Create(Utf8JsonReader, ModelReaderWriterOptions)
IJsonModel<ModelDeploymentScalingProfile>.Write(Utf8JsonWriter, ModelReaderWriterOptions)
IPersistableModel<ModelDeploymentScalingProfile>.Create(BinaryData, ModelReaderWriterOptions)
IPersistableModel<ModelDeploymentScalingProfile>.GetFormatFromOptions(ModelReaderWriterOptions)
IPersistableModel<ModelDeploymentScalingProfile>.Write(ModelReaderWriterOptions)