RecommendVmSkuModel interface

Package:

@azure/arm-aiinfraoptimization

Model inputs used for metadata resolution and memory sizing.

Properties

headDim	The attention head dimension, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
huggingFaceModelId	The Hugging Face model identifier, for example meta-llama/Llama-3.1-70B-Instruct.
maxContextLength	The maximum context length the model supports, in tokens. Used only as an upper bound for the sizing context length when Hugging Face metadata provides no usable maximum.
numKeyValueHeads	The model key-value head count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
numLayers	The model layer count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
parameterCount	The logical total model parameter count, including all mixture-of-experts experts. May be supplied with or without a model identifier.
weightPrecision	The precision the model weights are served at. Matched case-insensitively.

Property Details

headDim

The attention head dimension, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

headDim?: number

Property Value

number

huggingFaceModelId

The Hugging Face model identifier, for example meta-llama/Llama-3.1-70B-Instruct.

huggingFaceModelId?: string

Property Value

string

maxContextLength

The maximum context length the model supports, in tokens. Used only as an upper bound for the sizing context length when Hugging Face metadata provides no usable maximum.

maxContextLength?: number

Property Value

number

numKeyValueHeads

The model key-value head count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

numKeyValueHeads?: number

Property Value

number

numLayers

The model layer count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

numLayers?: number

Property Value

number

parameterCount

The logical total model parameter count, including all mixture-of-experts experts. May be supplied with or without a model identifier.

parameterCount?: number

Property Value

number

weightPrecision

The precision the model weights are served at. Matched case-insensitively.

weightPrecision?: string

Property Value

string