Language

ArmAIInfraOptimizationModelFactory.RecommendVmSkuModel Method

Definition

Model inputs used for metadata resolution and memory sizing.

public static Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuModel RecommendVmSkuModel(string huggingFaceModelId = default, long? parameterCount = default, Azure.ResourceManager.AIInfraOptimization.Models.WeightPrecision? weightPrecision = default, int? numLayers = default, int? numKeyValueHeads = default, int? headDim = default, int? maxContextLength = default);
static member RecommendVmSkuModel : string * Nullable<int64> * Nullable<Azure.ResourceManager.AIInfraOptimization.Models.WeightPrecision> * Nullable<int> * Nullable<int> * Nullable<int> * Nullable<int> -> Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuModel
Public Shared Function RecommendVmSkuModel (Optional huggingFaceModelId As String = Nothing, Optional parameterCount As Nullable(Of Long) = Nothing, Optional weightPrecision As Nullable(Of WeightPrecision) = Nothing, Optional numLayers As Nullable(Of Integer) = Nothing, Optional numKeyValueHeads As Nullable(Of Integer) = Nothing, Optional headDim As Nullable(Of Integer) = Nothing, Optional maxContextLength As Nullable(Of Integer) = Nothing) As RecommendVmSkuModel

Parameters

huggingFaceModelId
String

The Hugging Face model identifier, for example meta-llama/Llama-3.1-70B-Instruct.

parameterCount
Nullable<Int64>

The logical total model parameter count, including all mixture-of-experts experts. May be supplied with or without a model identifier.

weightPrecision
Nullable<WeightPrecision>

The precision the model weights are served at. Matched case-insensitively.

numLayers
Nullable<Int32>

The model layer count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

numKeyValueHeads
Nullable<Int32>

The model key-value head count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

headDim
Nullable<Int32>

The attention head dimension, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.

maxContextLength
Nullable<Int32>

The maximum context length the model supports, in tokens. Used only as an upper bound for the sizing context length when Hugging Face metadata provides no usable maximum.

Returns

A new RecommendVmSkuModel instance for mocking.

Applies to