ArmAIInfraOptimizationModelFactory.RecommendVmSkuModel Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
Model inputs used for metadata resolution and memory sizing.
public static Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuModel RecommendVmSkuModel(string huggingFaceModelId = default, long? parameterCount = default, Azure.ResourceManager.AIInfraOptimization.Models.WeightPrecision? weightPrecision = default, int? numLayers = default, int? numKeyValueHeads = default, int? headDim = default, int? maxContextLength = default);
static member RecommendVmSkuModel : string * Nullable<int64> * Nullable<Azure.ResourceManager.AIInfraOptimization.Models.WeightPrecision> * Nullable<int> * Nullable<int> * Nullable<int> * Nullable<int> -> Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuModel
Public Shared Function RecommendVmSkuModel (Optional huggingFaceModelId As String = Nothing, Optional parameterCount As Nullable(Of Long) = Nothing, Optional weightPrecision As Nullable(Of WeightPrecision) = Nothing, Optional numLayers As Nullable(Of Integer) = Nothing, Optional numKeyValueHeads As Nullable(Of Integer) = Nothing, Optional headDim As Nullable(Of Integer) = Nothing, Optional maxContextLength As Nullable(Of Integer) = Nothing) As RecommendVmSkuModel
Parameters
- huggingFaceModelId
- String
The Hugging Face model identifier, for example meta-llama/Llama-3.1-70B-Instruct.
The logical total model parameter count, including all mixture-of-experts experts. May be supplied with or without a model identifier.
- weightPrecision
- Nullable<WeightPrecision>
The precision the model weights are served at. Matched case-insensitively.
The model layer count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
The model key-value head count, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
The attention head dimension, used for architecture-aware key-value cache sizing. Supply all architecture fields together with a parameter count.
The maximum context length the model supports, in tokens. Used only as an upper bound for the sizing context length when Hugging Face metadata provides no usable maximum.
Returns
A new RecommendVmSkuModel instance for mocking.