Creating Azure VMs with ARM PowerShell cmdlets

The Azure Resource Manager (ARM) is a new desired-state deployment manager for application resources deployed into Microsoft Azure. ARM is based on the concepts of resources and resource groups. A resource is the atomic unit of deployment in Azure. Storage accounts, VNETs, and VMs are examples of resources. A resource group is a deployed set of resources with a common management lifecycle – with each resource deployed to a single resource group. For example, all the VMs and storage accounts related to an application would be in a single resource group, while a VNET used by many applications would be in a “shared” resource group.

UPDATE 10/25/2015: This post has been updated to use the Azure PowerShell v1 preview cmdlets. The primary change is that cmdlet names have been changed from Verb-AzureNoun to Verb-AzureRmNoun.

Resource Management

The traditional way to manage services deployed into Azure is through the Azure Service Management REST API (ASM) or any method built on it such as the Production Portal and the (default mode of the) Azure PowerShell cmdlets. This method has two shortcomings: it does not support desired-state deployment and it does not support role-based access control (RBAC). These missing features are core requirements for a modern deployment manager.

ARM was developed to provide desired-state deployment and RBAC. There are various ways to manage resources using ARM:

ARM has been available since 2014 for many Azure services including Azure WebApps and Azure SQL Database. A public preview of the ARM mode of the Azure PowerShell cmdlets for core compute resources (Virtual Machines, Network and Storage) was announced at //Build2015. These core compute cmdlets are contained in v0.9.0 of the Azure PowerShell cmdlets.

ARM supports the imperative creation of core compute resources in Azure PowerShell – which is the focus of this post. It also supports desired-state deployment through the use of an ARM template file, which expresses the desired state of the resource group in a well-defined JSON schema. The template file can then be processed along with a parameter file, either in the preview Azure Portal or using Azure PowerShell, to deploy the specified resources into a resource group. This provides Azure with a modern deployment technique.

There were a number of ARM Templates presentations at //Build2015:

Introduction to What's New in Azure IaaS - Corey Sanders and Bryon Surace

Azure Resource Manager - Ryan Jones

ARM Mode of the Azure PowerShell cmdlets

Prior to v1, the Azure PowerShell cmdlets supported two modes: ASM for cmdlets using the original Azure Service Management control plance; and ARM for cmdlets using the new Azure Resource Manager control plane. In v1, these cmdlets are implemented in different modules that can be imported at the same time and there is no longer the need to switch between modes using Switch-AzureMode (as there was previously).

VM Images in ARM

ASM identifies the source image used to create a VM by the name of the VHD containing the image. ARM supports a new way to identify the source image for a VM, based on:

  • Publisher name
  • SKU
  • Offer
  • Version

ARM supports the following cmdlets for source images:

  • Save-AzureRmVMImage – save a custom image
  • Get-AzureRmVMImageSku – get the SKUs for a publisher and offer
  • Get-AzureRmVMImagePublisher – get the available publishers
  • Get-AzureRmVMImageOffer – get the avalailable offers from a publisher
  • Get-AzureRmVMImage – get the image for a specific SKU

The following ARM cmdlet get the details for a specific Windows Server source image:

Get-AzureRmVMImage -Location "westus" `
-PublisherName "MicrosoftWindowsServer" `
   -Offer "WindowsServer" -Skus "2012-R2-Datacenter" -Version "4.0.20150916"

It returns the following information:

 Id               : /Subscriptions/GUID/Providers/Microsoft.Compute/Locations/w
                   estus/Publishers/MicrosoftWindowsServer/ArtifactTypes/VMImage/Offers/WindowsServer/Skus/201
                   2-R2-Datacenter/Versions/4.0.20150916
Location         : westus
PublisherName    : MicrosoftWindowsServer
Offer            : WindowsServer
Skus             : 2012-R2-Datacenter
Version          : 4.0.20150916
FilterExpression : 
Name             : 4.0.20150916
OSDiskImage      : {
                     "OperatingSystem": "Windows"
                   }
PurchasePlan     : null
DataDiskImages   : []

Resource Providers

ARM uses resource providers to manage distinct types of resources. For example, the Microsoft.Compute provider is used to manage IaaS VMs while the Microsoft.Storage provider is used to manage storage accounts. Resource providers are an extensibility point allowing new resource providers to be added in a consistent manner as new services are added to Azure.

ARM already supports a variety of resource providers but the focus in this post is on the following core compute resource providers:

  • Microsoft.Compute
  • Microsoft.Network
  • Microsoft.Storage

The Microsoft.Compute provider supports the following resource types:

  • virtualMachines – IaaS VMs
  • virtualMachines/extensions – Extensions deployed into IaaS VMs
  • availabilitySets – Availability sets used to provide high availability (HA) for multiple VMs

The Microsoft.Network provider supports the following resource types:

  • publicIPAddresses – public IP address with an associated DNS name (VIP or PIP)
  • virtualNetworks – Azure VNET
  • loadBalancers – Azure Load Balancer providing load-balancing or NAT
  • networkInterfaces – virtual network interface card (NIC)

The Microsoft.Storage provider supports the following resource types:

  • storageAccounts – Azure Storage account

There are additional core compute resource types not considered in this post.

Versioning of resource providers is supported through an apiVersion – essentially a version date. The supported locations and api versions of the various resource types can be retrieved using the following Azure PowerShell cmdlet:

(Get-AzureRmResourceProvider).ResourceTypes

Note that ARM also brings new versions of core compute resource providers to Azure. The original resource providers are:

  • Microsoft.ClassicCompute
  • Microsoft.ClassicStorage
  • Microsoft.ClassicNetwork

The preview portal uses a v2 designation to distinguish resources created with the new resource providers.

Resource Types: Microsoft.Network/publicIPAddresses

A publicIPAddresses resource type represents a DNS name on the public internet. It may be used to provide a DNS name to either a virtual IP address (VIP) hosted on an Azure Load Balancer or a public instance IP address (PIP) hosted directly on a VM.

Historically, a cloud service name provided the prefix to a fully-qualified cloudapp.net DNS name for a collection of VMs. The cloud service concept has proven to be less useful in an IaaS environment so the concept of a cloud service is dropped from the ARM view of Virtual Machines resources. The DNS functionality of a cloud service is provided instead by the publicIPAddresses resource type which provides an optional way to specify a DNS name for either a VIP hosted on the Azure Load Balancer or a PIP hosted on a VM.

ARM supports the following cmdlets for the publicIpAddresses resource type:

  • Set-AzureRmPublicIpAddress – set the desired state for the resource
  • Remove-AzureRmPublicIpAddress – remove a public IP address
  • New-AzureRmPublicIpAddress – create a new public IP address
  • Get-AzureRmPublicIpAddress – get an existing public IP address.

For example, the following creates a public IP address with a dynamic (not reserved) VIP for a DNS name with a prefix of contoso.

$vip = New-AzureRmPublicIpAddress -ResourceGroupName "SomeResourceGroup" `
-Name "VIP1" -Location "westus" -AllocationMethod Dynamic `
-DomainNameLabel "contoso"

Historically, Azure has used cloudapp.net for the generic DNS suffix for cloud services. ARM instead uses a location specific suffix for VIPs and PIPs it creates for publicIPAddresses resource types. The above example would create a fully-qualified DNS name of contoso.uswest.cloudapp.azure.com.

Resource Types: Microsoft.Network/virtualNetworks

The virtualNetworks resource type represents a VNET and the subnets it contains. In practice, since a VNET is likely used by several applications a virtualNetworks resource would likely be deployed into a shared resource group rather than an application-specific resource group.

ARM supports the following cmdlets for the virtualNetworks resource type:

  • Set-AzureRmVirtualNetwork – specifies the desired state of a VNET
  • Remove-AzureRmVirtualNetwork – remove a VNET
  • New-AzureRmVirtualNetwork – create a new VNET
  • Get-AzureRmVirtualNetwork – retrieve a VNET configuration
  • Set-AzureRmVirtualNetworkSubnetConfig – specifies the desired state of a subnet configuration
  • Remove-AzureRmVirtualNetworkSubnetConfig – remove a subnet
  • New-AzureRmVirtualNetworkSubnetConfig – create a new subnet configuration
  • Get-AzureRmVirtualNetworkSubnetConfig – retrieve a subnet configuration
  • Add-AzureRmVirtualNetworkSubnetConfig – add a subnet configuration to a VNET

For example, the following creates a /24subnet configuration and uses it when creating a /16 VNET:

$subnet = New-AzureRmVirtualNetworkSubnetConfig -Name "Subnet-1" `
-AddressPrefix "10.0.64.0/24"

$vnet = New-AzureRmVirtualNetwork -Name "VNET" `
-ResourceGroupName "SomeResourceGroup" `
-Location "West US" -AddressPrefix "10.0.0.0/16" -Subnet $subnet

$subnet = Get-AzureRmVirtualNetworkSubnetConfig -Name $subnetName `
-VirtualNetwork $vnet

Resource Types: Microsoft.Network/loadBalancers

The loadBalancers resource type represents an Azure Load Balancer or Internal Load Balancer. It supports the configuration of load balancing rules that either port forward (NAT) or load balance traffic from a frontend IP address to one or more backend NICs. A loadBalancers resource thereby depends on a frontend publicIPAddresses resource and one or more backend networkInterfaces resources. It also supports the specification of health probes that can be used to control the removal of NICs from load-balancer rotation.

A loadBalancers resource has a complex configuration depending on which of load balanced and NATted traffic it supports. A load balancer is configured through:

  • Rule config – specifies the configuration for load-balanced traffic
  • Probe config – specifies the health probe for load-balanced traffic
  • Inbound NAT rule config – specifies the network-address translation rules
  • Backend address pool config – specifies the backend pool for load-balanced traffic
  • Frontend IP config – specifies the configuration of the front end (e.g., VIP)

These have cmdlets allowing the creation, removal and retrieval of the appropriate configuration. The load balancer resource is managed using the following cmdlets:

  • Set-AzureRmLoadBalancer – specifies the desired state of the load balancer resource
  • Remove-AzureRmLoadBalancer – removes the load balancer
  • New-AzureRmLoadBalancer – create a new load balancer
  • Get-AzureRmLoadBalancer – retrieves the load balancer resource

For example, the following creates:

  • Frontend IP Config
  • RDP NAT rules for two NICs
  • Backend address pool
  • Rule for load-balanced traffic on port 80
  • Health probe on port 80
  • Load balancer resource

$feIpConfig = New-AzureRmLoadBalancerFrontendIpConfig -Name "FEIP" `
-PublicIpAddress $vip

$inboundNATRule1 = New-AzureRmLoadBalancerInboundNatRuleConfig -Name "RDP1" `
-FrontendIpConfiguration $feIpConfig `
-Protocol TCP -FrontendPort 3441 -BackendPort 3389

$inboundNATRule2 = New-AzureRmLoadBalancerInboundNatRuleConfig -Name "RDP2"`
-FrontendIpConfiguration $feIpConfig `
-Protocol TCP -FrontendPort 3442 -BackendPort 3389

$beAddressPool = New-AzureRmLoadBalancerBackendAddressPoolConfig -Name "LBBE"

$healthProbe = New-AzureRmLoadBalancerProbeConfig -Name "HealthProbe" `
-RequestPath "HealthProbe.aspx" -Protocol http -Port 80 `
-IntervalInSeconds 15 -ProbeCount 2

 $lbrule = New-AzureRmLoadBalancerRuleConfig -Name "HTTP" `
   -FrontendIpConfiguration $feIpConfig1 -BackendAddressPool $beAddressPool `
   -Probe $healthProbe -Protocol Tcp -FrontendPort 80 -BackendPort 80

 

$alb = New-AzureRmLoadBalancer -ResourceGroupName "SomeResourceGroup" `
-Name "ALB" -Location "westus" -FrontendIpConfiguration $feIpConfig `
-InboundNatRule $inboundNATRule1,$inboundNatRule2 `
-LoadBalancingRule $lbrule -BackendAddressPool $beAddressPool `
-Probe $healthProbe

Resource Types: Microsoft.Network/networkInterfaces

The networkInterfaces resource type represents a virtual network interface card (NIC). This NIC is associated with a private DIP that is either dynamically or statically allocated in a subnet of a VNET specified by a virtualNetworks resource. It can also be associated with a publicIPAddresses resource (PIP), which exposes the NIC directly on the public internet (without going through the Azure Load Balancer). The NIC can also be associated with a loadBalancers resource allowing it to be the destination for load balanced or port-forwarded traffic coming from either the Azure Load Balancer or an Internal Load Balancer. A networkInterfaces resource depends on a virtualNetworks resource and optionally a publicIPAddresses resource and/or a loadBalancers resource.

ARM supports the following cmdlets for the networkInterfaces resource type:

  • Set-AzureRmNetworkInterface – specifies the desired state of the network interface
  • Remove-AzureRmVMNetworkInterface – removes a network interface from a VM configuration
  • Remove-AzureRmNetworkInterface – remove a network interface
  • New-AzureRmNetworkInterface – create a new network interface
  • Get-AzureRmNetworkInterface – get a network interface
  • Add-AzureRmVMNetworkInterface – add a network interface to a VM configuration

For example, the following creates two NICS in a subnet and associates it with a load balancer resource ($alb) for both port forwarding and load balancing:

$nic1 = New-AzureRmNetworkInterface -ResourceGroupName $resourceGroupName `
-Name "nic1" -Subnet "Subnet-1" -Location "westus" `
-LoadBalancerInboundNatRule $alb.InboundNatRules[0] `
-LoadBalancerBackendAddressPool $alb.BackendAddressPools[0]

$nic2 = New-AzureRmNetworkInterface -ResourceGroupName $resourceGroupName `
-Name "nic2" -Subnet "Subnet-1" -Location "westus" `
-LoadBalancerInboundNatRule $alb.InboundNatRules[1] `
-LoadBalancerBackendAddressPool $alb.BackendAddressPools[0]

Resource Types: Microsoft.Storage/storageAccounts

The storageAccounts resource type represents an Azure Storage account. ARM supports the following cmdlets for the storageAccounts resource type:

  • Set-AzureRmStorageAccount – specify the desired state of a storage account
  • Remove-AzureRmStorageAccount – remove a storage account
  • New-AzureRmStorageAccountKey – regenerate the account key of a storage account
  • New-AzureRmStorageAccount – create a new storage account
  • Get-AzureRmStorageAccountKey  - retrieve the account key of a storage account
  • Get-AzureRmStorageAccount – retrieve a storage account configuration

For example, the following creates a standard locally-redundant storage account named contoso:

New-AzureRmStorageAccount -ResourceGroupName "SomeResourceGroup" `
-Name "contoso" -Location "westus" -Type Standard_LRS

Resource Types: Microsoft.Compute/availabilitySets

The availabilitySets resource type specifies an availability set used to provide HA. The use of an availability set for two or more VMs ensures that they are deployed in a manner that avoids correlated failures – for example, the failure of a top-of-rack router or reboots during upgrades of physical hosts.

ARM supports the following cmdlets for the availabilitySets resource type:

  • Remove-AzureRmAvailabilitySet- remove an availability set
  • New-AzureRmAvailabilitySet – create a new availability set
  • Get-AzureRmAvailabilitySet – retrieve an availability set configuration

The following cmdlet creates an availability set:

New-AzureRmAvailabilitySet -ResourceGroupName "SomeResourceGroup" `
-Name "AVSet" -Location "westus"

Resource Types: Microsoft.Compute/virtualMachines

The virtualMachines resource type represents a virtual machine (VM). The virtualMachines resource is a complex resource that depends on a storageAccounts resource to host its OS Disk and a networkInterfaces resource to host its NIC. It may also depend on an availabilitySets resource when it is deployed into an availability set.

There are many ARM cmdlets to configure VMs. The general idea is to create a VM configuration, modify it as desired, and then create a new VM. The following are some of the cmdlets needed to create a simple VM:

  • New-AzureRmVMConfig – create a new VM configuration
  • Set-AzureRmVMOperatingSystem – specify the OS
  • Set-AzureRmVMSourceImage – specify the image used to create the VM
  • Set-AzureRmVMOSDisk – specify the location of the OS disk
  • Add-AzureRmVMNetworkInterface – specify the networkInterface resource for the VM
  • New-AzureRmVM – create the VM
  • Get-AzureRmVM – retrieve the configuration of a VM
  • Update-AzureRmVM – update a VM

For example, the following cmdlets:

  • Create the VM config, specifying the VM size and availability set name
  • Specify that the VM is running Windows, and provides the admin username and password
  • Specify which source image is used for the VM
  • Configure the OS Disk
  • Add the NIC
  • Create the new VM

$vmConfig = New-AzureRmVMConfig -VMName "contoso-w1" -VMSize "Standard_A1" `
   -AvailabilitySetId $avset.Id |

    Set-AzureRmVMOperatingSystem -Windows -ComputerName "contoso-w1" `
-Credential $cred -ProvisionVMAgent -EnableAutoUpdate |

    Set-AzureRmVMSourceImage -PublisherName $publisher -Offer $offer -Skus $sku `
-Version $version | 

    Set-AzureRmVMOSDisk -Name "ContosoDisk" -VhdUri "https://contoso.blob...." `
-Caching ReadWrite -CreateOption fromImage | 

    Add-AzureRmVMNetworkInterface -Id $nic.Id

 

New-AzureRmVM -ResourceGroupName $resourceGroupName -Location "westus" -VM $vmConfig

 

The following provide fully-worked examples:

Logging

The following cmdlets provide logging information about resource group operations, and can be used to debug errors in cmdlet syntax:

  • Get-AzureRmLog – retrieves logs for a specified resource provider, resource group or subscription                                                                                                      

By default, only the logs for the last hour are retrieve –but start and end times (in local time) can be provided. The following retrieves detailed logs for all failed operations against the Microsoft.Storage provider in the last hour:

Get-AzureRmLog -ResourceProvider "Microsoft.Storage" -DetailedOutput -Status Failed

Comments

  • Anonymous
    June 29, 2015
    Very nice.  This has become part of my Azure toolset and I reference it all of the time.

  • Anonymous
    July 14, 2015
    What about other cloud services like web role, worker role

  • Anonymous
    July 14, 2015
    Great resource!  Thanks Neil

  • Anonymous
    July 29, 2015
    Thanks for the great article Neil.  Would you be able to help with some more info about the concept of "shared" resource groups please?  Or, perhaps, point me towards any useful documentation. I like the idea of sharing a VNET between deployments in two different resource groups, which I think is what you were suggesting with a "shared" resource group, but I can't work out how to do it.  At first I thought there might be a new "shared" type of resource group, and then I thought maybe it meant having a single VNET be a member of more than one resource group but I can't work out how to do that. Appreciate any help or advice! Thanks, Rich

  • Anonymous
    August 16, 2015
    what if i want to use my own(custom) template/image. How can I get the sku, publisher, offer and version etc ? I don't want to go with images present by default rather I want to use my customised image.

  • Anonymous
    November 11, 2015
    Anyone can explain what this means (and how to use)? Ridiculous lack of documentation Set-AzureRmPublicIpAddress – set the desired state for the resource Trying to use it like 'Set-azurermpublicipaddress -publicipaddress 1.2.3.4' doesn't seem to work

  • Anonymous
    November 23, 2015
    Hi Similar question  to Aatif. I've uploaded a VHD file exported from Hyper-V into a Storage Account associated with a Resource Group a but I can't work out how to create an Image from it with the publisher/offer/sku information associated with it such that I can use it with the Get-AzureRmVMImage cmdlet. Is there a way of doing this with either PowerShell or the Portal? Thanks, David

  • Anonymous
    November 25, 2015
    I agree , incredibly twisted and difficult.

  • Anonymous
    March 23, 2016
    There is a typo in section 'Resource Types: Microsoft.Network/loadBalancers':$lbrule = New-AzureRmLoadBalancerRuleConfig -Name “HTTP” -FrontendIpConfiguration $feIpConfig1 -BackendAddressPool $beAddressPool -Probe $healthProbe -Protocol Tcp -FrontendPort 80 -BackendPort 80There is no $feIpConfig1. Use $feIpConfig instead.

  • Anonymous
    May 30, 2016
    Hello Neil , Thanks for excellent Reference Guide for the ARM PowerShell cmdlets.

  • Anonymous
    September 26, 2016
    I want to create a virtual machine using uploaded disk in ARM portal, is there any way to create a VM from uploaded VHD in new ARM portal.

  • Anonymous
    October 11, 2016
    To populate $cred use $cred = Get-Credential