RecommendVmSkuMemory interface
The memory calculation shared by every returned option.
Properties
| kv |
The key-value cache memory required for a single concurrent request, in gibibytes. |
| kv |
The key-value cache memory required for all concurrent requests on a replica, in gibibytes. |
| model |
The combined model-weight and key-value cache memory required per replica, in gibibytes. |
| model |
The memory required for the model weights, in gibibytes. |
Property Details
kvCacheGiBPerRequest
The key-value cache memory required for a single concurrent request, in gibibytes.
kvCacheGiBPerRequest: numberProperty Value
number
kvCacheGiBTotal
The key-value cache memory required for all concurrent requests on a replica, in gibibytes.
kvCacheGiBTotal: numberProperty Value
number
modelAndKvCacheGiB
The combined model-weight and key-value cache memory required per replica, in gibibytes.
modelAndKvCacheGiB: numberProperty Value
number
modelWeightsGiB
The memory required for the model weights, in gibibytes.
modelWeightsGiB: numberProperty Value
number
