Language

ArmAIInfraOptimizationModelFactory.RecommendVmSkuSizingWorkload Method

Definition

The resolved per-replica workload inputs used for memory sizing.

public static Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuSizingWorkload RecommendVmSkuSizingWorkload(int maxConcurrentRequestsPerReplica = 0, int contextLength = 0);
static member RecommendVmSkuSizingWorkload : int * int -> Azure.ResourceManager.AIInfraOptimization.Models.RecommendVmSkuSizingWorkload
Public Shared Function RecommendVmSkuSizingWorkload (Optional maxConcurrentRequestsPerReplica As Integer = 0, Optional contextLength As Integer = 0) As RecommendVmSkuSizingWorkload

Parameters

maxConcurrentRequestsPerReplica
Int32

The maximum number of simultaneous inference requests handled by each replica.

contextLength
Int32

The context length, in tokens, used to size the key-value cache: the sum of promptTokens and maxOutputTokens when a token budget is supplied, otherwise the service default.

Returns

A new RecommendVmSkuSizingWorkload instance for mocking.

Applies to