KnownModelDeploymentPerformanceMode enum
The runtime performance mode of a model deployment.
Fields
| Balanced | A balanced trade-off between latency and throughput (default). |
| Latency | Optimize for low request latency. |
| Throughput | Optimize for high aggregate throughput. |
