az aiinfraoptimization

Experimental

Note

This reference is part of the aiinfraoptimization extension for the Azure CLI (version 2.75.0 or higher). The extension will automatically install the first time you run an az aiinfraoptimization command. Learn more about extensions.

This command group is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus

AI Infra Optimization.

Commands

Name	Description	Type	Status
az aiinfraoptimization recommendvmsku	Recommends Azure virtual machine sizes for an AI workload in the specified subscription and azure location.	Extension	Experimental

az aiinfraoptimization recommendvmsku

Experimental

Command group 'az aiinfraoptimization' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus

Recommends Azure virtual machine sizes for an AI workload in the specified subscription and azure location.

az aiinfraoptimization recommendvmsku [--acquire-policy-token]
                                      [--change-reference]
                                      [--deployment]
                                      [--ids]
                                      [--location]
                                      [--model]
                                      [--optimization]
                                      [--subscription]
                                      [--subscription-options]
                                      [--target-vm-sizes]
                                      [--workload]

Examples

Recommend virtual machine sizes for an AI inference workload

az aiinfraoptimization recommendvmsku --location eastus2 --model "{hugging-face-model-id:meta-llama/Llama-3.1-70B-Instruct,parameter-count:70553706496,weight-precision:FP16,num-layers:80,num-key-value-heads:8,head-dim:128,max-context-length:131072}" --workload "{max-concurrent-requests-per-replica:1,token-budget:{prompt-tokens:8000,max-output-tokens:1024}}" --deployment "{replica-count:1,procurement-option:OnDemand}" --optimization "{ranking-preference:Cost,max-deployment-cost-per-hour:{amount:50,currency-code:USD}}" --target-vm-sizes "[Standard_NC48ads_A100_v4,Standard_ND96asr_v4]" --subscription-options "{include-quota:True,include-placement:True,include-pricing:True}"

Optional Parameters

The following parameters are optional, but depending on the context, one or more might become required for the command to execute successfully.

--acquire-policy-token

Acquiring an Azure Policy token automatically for this resource operation.

Property	Value
Parameter group:	Global Policy Arguments

--change-reference

The related change reference ID for this resource operation.

Property	Value
Parameter group:	Global Policy Arguments

--deployment

Deployment scale and Azure virtual machine procurement option. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

--ids

One or more resource IDs (space-delimited). It should be a complete resource ID containing all information of 'Resource Id' arguments. You should provide either --ids or other 'Resource Id' arguments.

Property	Value
Parameter group:	Resource Id Arguments

--location -l

Location. Values from: az account list-locations. You can configure the default location using az configure --defaults location=<location>.

Property	Value
Parameter group:	Resource Id Arguments

--model

Model identity, sizing inputs, and optional maximum model context length. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

--optimization

Ranking preference and optional deployment-wide hourly cost limit. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

--subscription

Name or ID of subscription. You can configure the default subscription using az account set -s NAME_OR_ID.

Property	Value
Parameter group:	Resource Id Arguments

--subscription-options

The subscription-scoped lookups to include in the recommendation. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

--target-vm-sizes

The Azure virtual machine candidate set. When supplied, only these virtual machine sizes are ranked. Entries must be non-blank and unique, compared case-insensitively. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

--workload

Inference request concurrency and token shape assumptions. Support shorthand-syntax, json-file and yaml-file. Try "??" to show more.

Property	Value
Parameter group:	Body Arguments

Global Parameters

--debug

Increase logging verbosity to show all debug logs.

Property	Value
Default value:	False

--help -h

Show this help message and exit.

--only-show-errors

Only show errors, suppressing warnings.

Property	Value
Default value:	False

--output -o

Output format.

Property	Value
Default value:	json
Accepted values:	json, jsonc, none, table, tsv, yaml, yamlc

--query

JMESPath query string. See http://jmespath.org/ for more information and examples.

--verbose

Increase logging verbosity. Use --debug for full debug logs.

Property	Value
Default value:	False

Feedback

Was this page helpful?