Edit

Share via

AzureFileCopy@6 - Azure file copy v6 task

  • Article

Copy files to Azure Blob Storage or virtual machines.

This version of the task supports Workload Identity Federation and uses Azure RBAC to access Azure Storage. As a result of us using Azure RBAC, SAS tokens are no longer used. For more information, see the Remarks section.

Syntax

# Azure file copy v6
# Copy files to Azure Blob Storage or virtual machines.
- task: AzureFileCopy@6
  inputs:
    SourcePath: # string. Required. Source. 
    azureSubscription: # string. Alias: ConnectedServiceNameARM. Required. Azure Subscription. 
    Destination: # 'AzureBlob' | 'AzureVMs'. Required. Destination Type. 
    storage: # string. Alias: StorageAccountRM. Required. RM Storage Account. 
    #ContainerName: # string. Required when Destination = AzureBlob. Container Name. 
    #BlobPrefix: # string. Optional. Use when Destination = AzureBlob. Blob Prefix. 
    #resourceGroup: # string. Alias: EnvironmentNameRM. Required when Destination = AzureVMs. Resource Group. 
    #ResourceFilteringMethod: 'machineNames' # 'machineNames' | 'tags'. Optional. Use when Destination = AzureVMs. Select Machines By. Default: machineNames.
    #MachineNames: # string. Optional. Use when Destination = AzureVMs. Filter Criteria. 
    #vmsAdminUserName: # string. Required when Destination = AzureVMs. Admin Login. 
    #vmsAdminPassword: # string. Required when Destination = AzureVMs. Password. 
    #TargetPath: # string. Required when Destination = AzureVMs. Destination Folder. 
    #AdditionalArgumentsForBlobCopy: # string. Optional Arguments (for uploading files to blob). 
    #AdditionalArgumentsForVMCopy: # string. Optional. Use when Destination = AzureVMs. Optional Arguments (for downloading files to VM). 
    #enableCopyPrerequisites: false # boolean. Optional. Use when Destination = AzureVMs. Enable Copy Prerequisites. Default: false.
    #CopyFilesInParallel: true # boolean. Optional. Use when Destination = AzureVMs. Copy in Parallel. Default: true.
    #CleanTargetBeforeCopy: false # boolean. Clean Target. Default: false.
    #skipCACheck: true # boolean. Optional. Use when Destination = AzureVMs. Test Certificate. Default: true.

Inputs

SourcePath - Source
string. Required.

The location of source files. Supported values include YAML Pipelines and Classic Release support predefined system variables like Build.Repository.LocalPath.

Release variables are supported only in classic releases. The wild card symbol (*) is supported anywhere in the file path or file name.

Expression should return a single folder or a file.


azureSubscription - Azure Subscription
Input alias: ConnectedServiceNameARM. string. Required.

Specify the name of an Azure Resource Manager service connection configured for the subscription where the target Azure service, virtual machine, or storage account is located. See Azure Resource Manager overview for more details.


Destination - Destination Type
string. Required. Allowed values: AzureBlob (Azure Blob), AzureVMs (Azure VMs).

Specify the destination type.


storage - RM Storage Account
Input alias: StorageAccountRM. string. Required.

Specify a pre-existing ARM storage account. This is the storage account used as an intermediary for copying files to Azure VMs.


ContainerName - Container Name
string. Required when Destination = AzureBlob.

The name of the container into which files are copied. If the specified container does not exist in the storage account, it will be created.

To create a virtual directory inside the container, use the blob prefix input. For example, for the target location https://myaccount.blob.core.windows.net/mycontainer/vd1/vd2/, specify container name mycontainer and blob prefix: vd1/vd2.


BlobPrefix - Blob Prefix
string. Optional. Use when Destination = AzureBlob.

Specify a prefix for the destination virtual directory within the Azure Blob container. This applies when the SourcePath contains a wildcard that may match multiple items.

Example: You can append a build number to prefix the files from all blobs with the same build number.

Example: If you specify a blob prefix myvd1, a virtual directory is created inside the container. Files are copied from the source to https://myaccount.blob.core.windows.net/mycontainer/myvd1/.

In the case that the SourcePath is a single item with no wildcard, this blob prefix will function as the destination blob name.


resourceGroup - Resource Group
Input alias: EnvironmentNameRM. string. Required when Destination = AzureVMs.

Specify the name of the target Resource Group into which the files will be copied.


ResourceFilteringMethod - Select Machines By
string. Optional. Use when Destination = AzureVMs. Allowed values: machineNames (Machine Names), tags. Default value: machineNames.

Specify a VM host name or tag that identifies a subset of VMs in a resource group. Tags are supported for resources created via the Azure Resource Manager only.


MachineNames - Filter Criteria
string. Optional. Use when Destination = AzureVMs.

Provide a list of VM names or tag names that identify the VMs the task will target. Valid filter criteria includes:

  • The name of an Azure Resource Group.
  • An output variable from a previous task.
  • A comma-delimited list of tag names or VM names.
  • Format VM names using a comma-separated list of FQDNs or IP addresses.
  • Format tag names for a filter as {TagName}:{Value} Example: Role:DB;OS:Win8.1

vmsAdminUserName - Admin Login
string. Required when Destination = AzureVMs.

Provide the user name of an account with administrative permissions on all of the target VMs.

  • Supported formats include: username, domain\username, machine-name\username, and .\username.
  • UPN formats including username@domain.com and built-in system accounts such as NT Authority\System are not supported.

vmsAdminPassword - Password
string. Required when Destination = AzureVMs.

Provide the password for the Admin Login parameter.

To find the variable, locate the Admin Login parameter. Select the padlock icon for a variable defined in the Variables tab to protect the value and insert the variable name here.


TargetPath - Destination Folder
string. Required when Destination = AzureVMs.

Specify the path to the folder in the Azure VMs into which files will be copied.

Environment variables such as $env:windir and $env:systemroot are supported. Examples: $env:windir\FabrikamFiber\Web and c:\FabrikamFiber


AdditionalArgumentsForBlobCopy - Optional Arguments (for uploading files to blob)
string.

Provide additional arguments to AzCopy.exe for use when uploading to the Blob and downloading to the VMs. See Transfer data with the AzCopy Command-Line Utility for details.

For Premium storage accounts that support only Azure page Blobs use --blob-type=PageBlob as an additional argument.

Default arguments include --log-level=INFO (default) and --recursive (if the container name is not $root).


AdditionalArgumentsForVMCopy - Optional Arguments (for downloading files to VM)
string. Optional. Use when Destination = AzureVMs.

Provide additional arguments to AzCopy.exe that will be applied when downloading to VMs such as, --check-length=true.

If no optional arguments are specified, the following are added by default:

  • --log-level=INFO
  • --log-level=DEBUG (If the pipeline is running in debug mode set)
  • --recursive

enableCopyPrerequisites - Enable Copy Prerequisites
boolean. Optional. Use when Destination = AzureVMs. Default value: false.

When enabled, this option uses a self-signed certificate to configure the Windows Remote Management (WinRM) listener over the HTTPS protocol on port 5986. This configuration is required for performing copy operations on Azure VMs. Applicable only for ARM VMs.

  • If the target VMs are accessed through a load balancer, configure an inbound NAT rule to allow access on port 5986.
  • If the target VMs are associated with a Network Security Group (NSG), configure an inbound security rule to allow access on port 5986.

CopyFilesInParallel - Copy in Parallel
boolean. Optional. Use when Destination = AzureVMs. Default value: true.

Specify true to copy files in parallel to the target VMs.


CleanTargetBeforeCopy - Clean Target
boolean. Default value: false.

Specify true to clean-up the destination folder before copying files.


skipCACheck - Test Certificate
boolean. Optional. Use when Destination = AzureVMs. Default value: true.

WinRM requires a certificate for the HTTPS transfer when copying files from the intermediate storage Blob into the Azure VMs.

If you use a self-signed certificate, specify true to prevent the process from validating the certificate with a trusted CA.


Task control options

All tasks have control options in addition to their task inputs. For more information, see Control options and common task properties.

Output variables

This task defines the following output variables, which you can consume in downstream steps, jobs, and stages.

StorageContainerUri
Uri of the container where the files were copied to. Valid only when the selected destination is Azure Blob.

Remarks

AzureFileCopy@6 supports Workload Identity Federation and uses Azure RBAC to access Azure Storage. As a result of using Azure RBAC, SAS tokens are no longer used and the sasTokenTimeOutInMinutes task input is removed.

You might block the use of storage account keys and SAS tokens on your storage accounts. In these situations the AzureFileCopy@5 task, which relies on SAS tokens, can't be used.

The AzureFileCopy@6 task uses Azure RBAC to access blob storage instead. This requires the identity of the service connection used to have the appropriate RBAC role e.g. Storage Blob Data Contributor. See Assign an Azure role for access to blob data.

The AzureFileCopy@6 task also supports service connections that use workload identity federation.

Note

This task is written in PowerShell and works only when run on Windows agents. If your pipelines require Linux agents and need to copy files to an Azure Storage Account, consider running az storage blob commands in the Azure CLI task as an alternative.

The task is used to copy application files and other artifacts that are required in order to install the app; such as PowerShell scripts, PowerShell-DSC modules, and more.

When the target is Azure VMs, the files are first copied to an automatically generated Azure blob container and then downloaded into the VMs. The container is deleted after the files are successfully copied to the VMs.

The task uses AzCopy, the command-line utility built for fast copying data from and to Azure storage accounts. Version 6 of the Azure File Copy task uses AzCopy V10.

Azure File Copy version 6 require Azure Storage to be authorized via Microsoft Entra ID. Authentication using a service principal and managed identity are available. For managed identities, only system-wide managed identity is supported. The level of authorization required is shown in Option 1: Use Microsoft Entra ID.

To dynamically deploy Azure Resource Groups that contain virtual machines, use the Azure Resource Group Deployment task. This task has a sample template that can perform the required operations to set up the WinRM HTTPS protocol on the VMs, open port 5986 in the firewall, and install the test certificate.

Note

If you are deploying to Azure Static Websites as a container in Blob storage, use Version 2 or higher of the task in order to preserve the $web container name.

What are the Azure PowerShell prerequisites for using this task?

The task requires that Azure PowerShell is installed on the machine running the automation agent. The recommended version is 1.0.2, but the task will work with version 0.9.8 and higher. You can use the Azure PowerShell Installer v1.0.2 to obtain this.

What are the WinRM prerequisites for this task?

The task uses Windows Remote Management (WinRM) HTTPS protocol to copy the files from the storage Blob container to the Azure VMs. This requires that the WinRM HTTPS service is configured on the VMs, and a suitable certificate is installed.

Configure WinRM after virtual machine creation

If the VMs were created without opening the WinRM HTTPS ports, perform the following:

  1. Configure an inbound access rule to allow HTTPS on port 5986 of each VM.
  2. Disable UAC remote restrictions.
  3. Specify the credentials for the task to access the VMs using an administrator-level login in the simple form username without any domain part.
  4. Install a certificate on the machine that runs the automation agent.
  5. If you are using a self-signed certificate, set the Test Certificate parameter of the task.

What type of service connection should I choose?

  • For Azure Resource Manager storage accounts and Azure Resource Manager VMs, use an Azure Resource Manager service connection type. See Automating Azure Resource Group deployment using a Service Principal.

  • While using an Azure Resource Manager service connection type, the task automatically filters appropriate newer Azure Resource Manager storage accounts, and other fields. For example, the Resource Group or cloud service, and the VMs.

How do I create a school or work account for use with this task?

A suitable account can be created for use in a service connection:

  1. Use the Azure portal to create a new user account in Azure Active Directory.
  2. Add the Azure Active Directory user account to the co-administrators group in your Azure subscription.
  3. Sign into the Azure portal with this user account and change the password.
  4. Use the credentials of this account in the service connection. Deployments are then processed using this account.

If the task fails, will the copy resume?

Since AzCopy V10 does not support journal files, the task cannot resume the copy. You must run the task again to copy all the files.

Are the log files and plan files cleaned after the copy?

The log and plan files are not deleted by the task. To explicitly clean-up the files, add a CLI step in the workflow using azcopy jobs clean.

How do I use the Azure file copy task to copy a file to an Azure virtual machine that doesn't have a public IP address?

Make sure that you're using version 5 of the Azure file copy task. If the task fails, you can add a build step to run the command azcopy cp "source-file-path" "destination-file-path" to substitute the source and destination values.

Forbidden error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage' while using Azure File Copy task

The hosted agents are assigned randomly every time a build is triggered, the agent IP addresses will be different on every run. If these IP addresses are not in your allowed list of IPs, the communication between Azure DevOps and the storage account fails. In such scenarios, follow the steps outlined:

  1. Add a build step using Azure CLI to identify the IP address of the Microsoft Hosted Build agent at runtime. It will add the IP address to the Network rule on the Azure Storage Account.
  2. Run the build step for your Azure Storage Account.
  3. Add another build step using Azure CLI to remove the IP address of the build agent from the Azure Storage Account network rule.

Requirements

Requirement Description
Pipeline types YAML, Classic build, Classic release
Runs on Agent, DeploymentGroup
Demands Self-hosted agents must have capabilities that match the following demands to run jobs that use this task: azureps
Capabilities This task does not satisfy any demands for subsequent tasks in the job.
Command restrictions Any
Settable variables Any
Agent version 1.103.0 or greater
Task category Deploy