ServingPerformanceEstimation interface

Package:

@azure/arm-containerserviceaimanager

Estimated relative inference performance of a single model replica on a given GPU SKU. Each metric is a scaling coefficient in the range [0, 1] relative to the best-performing SKU for this model, which scores 1.

Properties

relativeLatencyScore	Relative inference latency score in [0, 1]. Higher is better (1 matches the best-performing SKU's latency for this model). Note: this is a normalized score, not a raw latency ratio -- a larger value indicates lower latency.
relativeThroughputScore	Relative inference throughput score in [0, 1]. Higher is better (1 matches the best-performing SKU's throughput for this model).

Property Details

relativeLatencyScore

Relative inference latency score in [0, 1]. Higher is better (1 matches the best-performing SKU's latency for this model). Note: this is a normalized score, not a raw latency ratio -- a larger value indicates lower latency.

relativeLatencyScore: number

Property Value

number

relativeThroughputScore

Relative inference throughput score in [0, 1]. Higher is better (1 matches the best-performing SKU's throughput for this model).

relativeThroughputScore: number

Property Value

number