ArmAIInfraOptimizationModelFactory.RecommendVmSkuSizingWorkload Method
Definition
Important
Some information relates to prerelease product that may be substantially modified before it’s released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
The resolved per-replica workload inputs used for memory sizing.
public static Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuSizingWorkload RecommendVmSkuSizingWorkload(int maxConcurrentRequestsPerReplica = 0, int contextLength = 0);
static member RecommendVmSkuSizingWorkload : int * int -> Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuSizingWorkload
Public Shared Function RecommendVmSkuSizingWorkload (Optional maxConcurrentRequestsPerReplica As Integer = 0, Optional contextLength As Integer = 0) As RecommendVmSkuSizingWorkload
Parameters
- maxConcurrentRequestsPerReplica
- Int32
The maximum number of simultaneous inference requests handled by each replica.
- contextLength
- Int32
The context length, in tokens, used to size the key-value cache: the sum of promptTokens and maxOutputTokens when a token budget is supplied, otherwise the service default.
Returns
A new RecommendVmSkuSizingWorkload instance for mocking.