Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Before you create a model deployment, choose a catalog model that's available to your subscription and location. Run az aimanager model list to find models, and run az aimanager model calculate-cost to compare feasible GPU plans and estimate VM cost.
Important
AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:
Before you begin
- An Azure subscription and a location supported during preview. Use the same location for model listing, cost calculation, and deployment. Before listing models, register the AI Manager preview feature.
- Azure CLI with access to the AI Manager CLI extension for the public preview.
- Permission to read model catalog entries and calculate costs. The Azure AIManager Model User role provides these permissions at the subscription scope or another applicable parent scope.
List available models
Sign in to Azure and select the subscription where you have the Azure AIManager Model User role.
az login az account set --subscription "<subscription-id>"List the catalog models available in the selected location.
az aimanager model list \ --subscription "<subscription-id>" \ --location "<supported-region>" \ --output tableThe table shows each model's
Name,ModelId, andDescription. If the table is empty, check the location and your Azure AIManager Model User role assignment.Choose a catalog model by its readable
ModelId. The matchingNamevalue is the model's ARM resource name. Use it for the cost calculator and for the ARM resource ID that you use to create a deployment. TheModelIdisn't the ARM resource name.
Compare GPU plans and estimates
Calculate plans for the selected model. Set
<resource-name>to theNamevalue that you chose in List available models.az aimanager model calculate-cost \ --subscription "<subscription-id>" \ --location "<supported-region>" \ --name "<resource-name>" \ --output tableThe table lists one plan per row. Feasible plans are listed first, from cheapest to most expensive.
Compare the plans. Select a plan where
FeasibleisTrue, and record itsVmSize. The table includes the following columns:Column Meaning VmSizeThe VM size for the plan. Record this value for a feasible plan. FeasibleWhether you can deploy the model with this plan. Select a plan where the value is True.VmsPerReplicaThe number of VMs that one model replica requires. VmHourlyPriceThe hourly retail price for one VM. A value of 0means that no price is available.TotalHourlyPriceThe hourly price for one model replica, which is VmHourlyPricemultiplied byVmsPerReplica.MaxAvailableReplicasThe number of replicas that your current quota allows for this VM size. QuantizationThe quantization applied to the model weights for this plan. InfeasibilityReasonFor an infeasible plan, the reason code. See the following table.
To estimate hourly VM cost for a fixed replica count, multiply TotalHourlyPrice by the number of replicas. With autoscaling, the actual hourly VM cost varies with the replica count. The calculator doesn't provide a complete estimate of every Azure charge associated with an AI Manager deployment. To see when prices were captured, run the command with --output json and check each plan's priceAsOf value.
The InfeasibilityReason column shows one of the following codes. For the full explanation, run the command with --output json and check each plan's infeasibilityReason.message value.
| Code | Meaning |
|---|---|
InsufficientQuota |
The subscription doesn't have enough quota for the plan. Request quota for the required VM family and location before you deploy. |
RegionUnavailable |
The VM size isn't available in the selected location. Choose another feasible plan or supported location. |
InefficientDeployment |
The plan's estimated serving performance is below the acceptable threshold for this model. Choose another plan or model. |
Use the selected plan
Use the model's ARM resource name and the selected vmSize when you create a model deployment. For an end-to-end example, see Deploy a model with AI Manager.