CognitiveServicesArcDeploymentVllmProperties Class

Definition

Namespace:

Azure.ResourceManager.CognitiveServices.Models

Assembly:

Azure.ResourceManager.CognitiveServices.dll

Package:

Azure.ResourceManager.CognitiveServices v1.6.0-beta.4

Source:

CognitiveServicesArcDeploymentVllmProperties.cs

Effective vLLM runtime parameters for an Arc deployment.

public class CognitiveServicesArcDeploymentVllmProperties : System.ClientModel.Primitives.IJsonModel<Azure.ResourceManager.CognitiveServices.Models.CognitiveServicesArcDeploymentVllmProperties>, System.ClientModel.Primitives.IPersistableModel<Azure.ResourceManager.CognitiveServices.Models.CognitiveServicesArcDeploymentVllmProperties>

type CognitiveServicesArcDeploymentVllmProperties = class
    interface IJsonModel<CognitiveServicesArcDeploymentVllmProperties>
    interface IPersistableModel<CognitiveServicesArcDeploymentVllmProperties>

Public Class CognitiveServicesArcDeploymentVllmProperties
Implements IJsonModel(Of CognitiveServicesArcDeploymentVllmProperties), IPersistableModel(Of CognitiveServicesArcDeploymentVllmProperties)

Inheritance

Object

CognitiveServicesArcDeploymentVllmProperties

Implements

IJsonModel<CognitiveServicesArcDeploymentVllmProperties> IPersistableModel<CognitiveServicesArcDeploymentVllmProperties> IPersistableModel<T>

Properties

Name	Description
GpuMemoryUtilization	Fraction of GPU memory reserved for model execution.
IsEagerExecutionEnforced	Whether eager execution is enforced for the vLLM runtime.
MaxModelLen	Maximum model context length.
TensorParallelSize	Number of GPUs used for tensor parallelism.

Methods

Name	Description
JsonModelCreateCore(Utf8JsonReader, ModelReaderWriterOptions)
JsonModelWriteCore(Utf8JsonWriter, ModelReaderWriterOptions)
PersistableModelCreateCore(BinaryData, ModelReaderWriterOptions)
PersistableModelWriteCore(ModelReaderWriterOptions)

Explicit Interface Implementations

Name	Description
IJsonModel<CognitiveServicesArcDeploymentVllmProperties>.Create(Utf8JsonReader, ModelReaderWriterOptions)
IJsonModel<CognitiveServicesArcDeploymentVllmProperties>.Write(Utf8JsonWriter, ModelReaderWriterOptions)
IPersistableModel<CognitiveServicesArcDeploymentVllmProperties>.Create(BinaryData, ModelReaderWriterOptions)
IPersistableModel<CognitiveServicesArcDeploymentVllmProperties>.GetFormatFromOptions(ModelReaderWriterOptions)
IPersistableModel<CognitiveServicesArcDeploymentVllmProperties>.Write(ModelReaderWriterOptions)