RecommendVmSkuBenchmark interface

Package:

@azure/arm-aiinfraoptimization

The external benchmark evidence selected for one virtual machine size option.

Properties

benchmarkModel	The model the benchmark was measured on.
level	How closely the benchmark evidence matches the recommended topology.
modelClass	The model size class the benchmark covers, for example 70b.
operatingSystem	The operating system the benchmark was measured on.
p99TpotMs	The 99th-percentile time per output token observed in the benchmark, in milliseconds.
p99TtftMs	The 99th-percentile time to first token observed in the benchmark, in milliseconds.
scenario	The benchmark scenario, for example Server or Offline.
software	The serving software the benchmark was measured with.
submissionId	The identifier of the selected benchmark submission.
submitter	The organization that published the benchmark submission.
submitterCount	The number of submitters that published results for this configuration.
system	The system the benchmark was measured on.
throughput	The throughput observed in the benchmark submission.
throughputUnit	The unit the observed throughput is expressed in, for example Tokens/s.
weightDataTypes	The weight data types used in the benchmark submission.

Property Details

benchmarkModel

The model the benchmark was measured on.

benchmarkModel?: string

Property Value

string

level

How closely the benchmark evidence matches the recommended topology.

level: string

Property Value

string

modelClass

The model size class the benchmark covers, for example 70b.

modelClass: string

Property Value

string

operatingSystem

The operating system the benchmark was measured on.

operatingSystem?: string

Property Value

string

p99TpotMs

The 99th-percentile time per output token observed in the benchmark, in milliseconds.

p99TpotMs?: number

Property Value

number

p99TtftMs

The 99th-percentile time to first token observed in the benchmark, in milliseconds.

p99TtftMs?: number

Property Value

number

scenario

The benchmark scenario, for example Server or Offline.

scenario: string

Property Value

string

software

The serving software the benchmark was measured with.

software?: string

Property Value

string

submissionId

The identifier of the selected benchmark submission.

submissionId?: string

Property Value

string

submitter

The organization that published the benchmark submission.

submitter?: string

Property Value

string

submitterCount

The number of submitters that published results for this configuration.

submitterCount: number

Property Value

number

system

The system the benchmark was measured on.

system?: string

Property Value

string

throughput

The throughput observed in the benchmark submission.

throughput?: number

Property Value

number

throughputUnit

The unit the observed throughput is expressed in, for example Tokens/s.

throughputUnit?: string

Property Value

string

weightDataTypes

The weight data types used in the benchmark submission.

weightDataTypes?: string

Property Value

string