DevOps Pipeline Blob Storage

Bäcker 21 Reputation points
2022-01-12T15:25:35.693+00:00

What I want
When building my application in devops pipeline I create webpack assets. Then copy the static assets to azure storage blob so that I can use CDN to deliver my js and css files.

Problem
Doesn't seems that the permissions are there. I've tried during the build pipeline and release pipeline

TEST Build pipeline

Config 

- task: AzureFileCopy@4  
 condition: and(succeeded(), eq(variables.fs_mode, 'Testing'))  
 inputs:  
   SourcePath: '$(Build.ArtifactStagingDirectory)/web/Testing/Testing-assets/*'  
   azureSubscription: 'Microsoft Azure (xxxx-xxxx-xxx)'  
   Destination: 'AzureBlob'  
   storage: 'testcont'  
   ContainerName: 'static'  
   BlobPrefix: '$(fs_tag)/'  
   AdditionalArgumentsForBlobCopy: '--log-level=INFO --blob-type=BlockBlob --recursive'

Release Log 

Starting: AzureFileCopy  
==============================================================================  
Task         : Azure file copy  
Description  : Copy files to Azure Blob Storage or virtual machines  
Version      : 4.195.0  
Author       : Microsoft Corporation  
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-file-copy  
==============================================================================  
Import-Module -Name C:\Modules\az_6.5.0\Az.Accounts\2.7.0\Az.Accounts.psd1 -Global  
##[warning]Both Az and AzureRM modules were detected on this machine. Az and AzureRM modules cannot be imported in the same session or used in the same script or runbook. If you are running PowerShell in an environment you control you can use the 'Uninstall-AzureRm' cmdlet to remove all AzureRm modules from your machine. If you are running in Azure Automation, take care that none of your runbooks import both Az and AzureRM modules. More information can be found here: https://aka.ms/azps-migration-guide  
Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue  
Clear-AzContext -Scope Process  
Connect-AzAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential -Environment AzureCloud @processScope  
 Set-AzContext -SubscriptionId 47a1dd1b-10af-4f84-9550-4235a9ad2453 -TenantId ***  
Import-Module -Name C:\Modules\az_6.5.0\Az.Resources\4.4.0\Az.Resources.psd1 -Global  
Import-Module -Name C:\Modules\az_6.5.0\Az.Storage\3.12.0\Az.Storage.psd1 -Global  
Import-Module -Name C:\Modules\az_6.5.0\Az.Compute\4.17.1\Az.Compute.psd1 -Global  
Import-Module -Name C:\Modules\az_6.5.0\Az.Network\4.12.0\Az.Network.psd1 -Global  
 & "AzCopy\AzCopy.exe" login --service-principal --application-id "***" --tenant-id="***" --aad-endpoint "https://login.windows.net/"  
INFO: If you set an environment variable by using the command line, that variable will be readable in your command line history. Consider clearing variables that contain credentials from your command line history.  To keep variables from appearing in your history, you can use a script to prompt the user for their credentials, and to set the environment variable.  
INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
  
INFO: SPN Auth via secret succeeded.  
INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
  
Uploading files from source path: 'D:\a\1\a\web\Testing\Testing-assets\*' to storage account: 'testcont' in container: 'static' with blob prefix: 'testing-2022011226/'  
 & "AzCopy\AzCopy.exe" copy "D:\a\1\a\web\Testing\Testing-assets\*" "https://testcont.blob.core.windows.net/static/testing-2022011226"  --log-level=INFO --blob-type=BlockBlob --recursive  
INFO: Scanning...  
INFO: Authenticating to destination using Azure AD  
INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
  
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support  
  
Job 61c690e3-e7be-e14e-62e6-475cff2bfa6f has started  
Log file is located at: C:\Users\VssAdministrator\.azcopy\61c690e3-e7be-e14e-62e6-475cff2bfa6f.log  
  
  
0.0 %, 0 Done, 0 Failed, 258 Pending, 0 Skipped, 258 Total,   
INFO: Authentication failed, it is either not correct, or expired, or does not have the correct permission -> github.com/Azure/azure-storage-blob-go/azblob.newStorageError, /home/vsts/go/pkg/mod/github.com/!azure/azure-storage-blob-go@v0.10.1-0.20201022074806-8d8fc11be726/azblob/zc_storage_error.go:42  
===== RESPONSE ERROR (ServiceCode=AuthorizationPermissionMismatch) =====  
Description=This request is not authorized to perform this operation using this permission.

TEST Release pipeline 

- task: AzureFileCopy@4  
  displayName: 'AzureBlob File Copy'  
  inputs:  
    SourcePath: '$(System.DefaultWorkingDirectory)/_WebCont-Web/Testing-assets'  
    azureSubscription: 'Microsoft Azure (xxxx-xxxx-xxxx)'  
    Destination: AzureBlob  
    storage: testcont  
    ContainerName: static

Release log 

2022-01-12T14:36:35.7752846Z ##[section]Starting: AzureBlob File Copy  
2022-01-12T14:36:35.7897343Z ==============================================================================  
2022-01-12T14:36:35.7897697Z Task         : Azure file copy  
2022-01-12T14:36:35.7897972Z Description  : Copy files to Azure Blob Storage or virtual machines  
2022-01-12T14:36:35.7898243Z Version      : 4.195.0  
2022-01-12T14:36:35.7898452Z Author       : Microsoft Corporation  
2022-01-12T14:36:35.7898788Z Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-file-copy  
2022-01-12T14:36:35.7899167Z ==============================================================================  
2022-01-12T14:36:37.7620434Z Added TLS 1.2 in session.  
2022-01-12T14:36:38.4753880Z ##[command]Import-Module -Name C:\Modules\az_6.5.0\Az.Accounts\2.7.0\Az.Accounts.psd1 -Global  
2022-01-12T14:36:39.4521820Z ##[warning]Both Az and AzureRM modules were detected on this machine. Az and AzureRM modules cannot be imported in the same session or used in the same script or runbook. If you are running PowerShell in an environment you control you can use the 'Uninstall-AzureRm' cmdlet to remove all AzureRm modules from your machine. If you are running in Azure Automation, take care that none of your runbooks import both Az and AzureRM modules. More information can be found here: https://aka.ms/azps-migration-guide  
2022-01-12T14:36:42.5080898Z ##[command]Clear-AzContext -Scope CurrentUser -Force -ErrorAction SilentlyContinue  
2022-01-12T14:36:43.3336647Z ##[command]Clear-AzContext -Scope Process  
2022-01-12T14:36:43.9729508Z ##[command]Connect-AzAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential -Environment AzureCloud @processScope  
2022-01-12T14:36:46.5001096Z ##[command] Set-AzContext -SubscriptionId 47a1dd1b-10af-4f84-9550-4235a9ad2453 -TenantId ***  
2022-01-12T14:36:48.0579725Z ##[command]Import-Module -Name C:\Modules\az_6.5.0\Az.Resources\4.4.0\Az.Resources.psd1 -Global  
2022-01-12T14:36:48.5756691Z ##[command]Import-Module -Name C:\Modules\az_6.5.0\Az.Storage\3.12.0\Az.Storage.psd1 -Global  
2022-01-12T14:36:49.4031341Z ##[command]Import-Module -Name C:\Modules\az_6.5.0\Az.Compute\4.17.1\Az.Compute.psd1 -Global  
2022-01-12T14:36:50.3256324Z ##[command]Import-Module -Name C:\Modules\az_6.5.0\Az.Network\4.12.0\Az.Network.psd1 -Global  
2022-01-12T14:36:57.0622267Z ##[command] & "AzCopy\AzCopy.exe" login --service-principal --application-id "***" --tenant-id="***" --aad-endpoint "https://login.windows.net/"  
2022-01-12T14:36:57.1122621Z INFO: If you set an environment variable by using the command line, that variable will be readable in your command line history. Consider clearing variables that contain credentials from your command line history.  To keep variables from appearing in your history, you can use a script to prompt the user for their credentials, and to set the environment variable.  
2022-01-12T14:36:57.3762175Z INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
2022-01-12T14:36:57.3762729Z   
2022-01-12T14:36:57.4538926Z INFO: SPN Auth via secret succeeded.  
2022-01-12T14:36:57.6548934Z INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
2022-01-12T14:36:57.6549269Z   
2022-01-12T14:36:57.6698618Z Uploading files from source path: 'D:\a\r1\a\_WebCont-Web\Testing-assets' to storage account: 'testcont' in container: 'static' with blob prefix: ''  
2022-01-12T14:36:57.6790779Z ##[command] & "AzCopy\AzCopy.exe" copy "D:\a\r1\a\_WebCont-Web\Testing-assets" "https://testcont.blob.core.windows.net/static"  --log-level=INFO --blob-type=PageBlob --recursive  
2022-01-12T14:36:57.7169408Z INFO: Scanning...  
2022-01-12T14:36:57.7170597Z INFO: Authenticating to destination using Azure AD  
2022-01-12T14:36:57.9182451Z INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
2022-01-12T14:36:57.9182863Z   
2022-01-12T14:36:58.0376696Z INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support  
2022-01-12T14:36:58.0675997Z   
2022-01-12T14:36:58.0679291Z Job 5a4c2096-41ed-8146-6b0c-c115e0f6b22e has started  
2022-01-12T14:36:58.0680451Z Log file is located at: C:\Users\VssAdministrator\.azcopy\5a4c2096-41ed-8146-6b0c-c115e0f6b22e.log  
2022-01-12T14:36:58.0680886Z   
2022-01-12T14:36:58.0681107Z   
2022-01-12T14:37:00.0732041Z 0.0 %, 0 Done, 0 Failed, 258 Pending, 0 Skipped, 258 Total,   
2022-01-12T14:37:02.0749904Z 0.0 %, 0 Done, 225 Failed, 33 Pending, 0 Skipped, 258 Total,   
2022-01-12T14:37:02.0750860Z   
2022-01-12T14:37:02.0752901Z   
2022-01-12T14:37:02.0753728Z Job 5a4c2096-41ed-8146-6b0c-c115e0f6b22e summary  
2022-01-12T14:37:02.0758195Z Elapsed Time (Minutes): 0.0668  
2022-01-12T14:37:02.0758891Z Number of File Transfers: 258  
2022-01-12T14:37:02.0759389Z Number of Folder Property Transfers: 0  
2022-01-12T14:37:02.0759775Z Total Number of Transfers: 258  
2022-01-12T14:37:02.0760085Z Number of Transfers Completed: 0  
2022-01-12T14:37:02.0760375Z Number of Transfers Failed: 258  
2022-01-12T14:37:02.0760681Z Number of Transfers Skipped: 0  
2022-01-12T14:37:02.0760990Z TotalBytesTransferred: 0  
2022-01-12T14:37:02.0761251Z Final Job Status: Failed  
2022-01-12T14:37:02.0761399Z   
2022-01-12T14:37:02.3582449Z ##[command] & "AzCopy\AzCopy.exe" logout  
2022-01-12T14:37:02.4071466Z INFO: Logout succeeded.  
2022-01-12T14:37:02.6947399Z INFO: AzCopy.exe: A newer version 10.13.0 is available to download  
2022-01-12T14:37:02.6953531Z   
2022-01-12T14:37:02.8218096Z ##[warning]A parameter cannot be found that matches parameter name 'and'.  
2022-01-12T14:37:02.9768489Z ##[error]Upload to container: 'static' in storage account: 'testcont' with blob prefix: '' failed with error: 'AzCopy.exe exited with non-zero exit code while uploading files to blob storage.' For more info please refer to https://aka.ms/azurefilecopyreadme  
2022-01-12T14:37:03.0484649Z ##[section]Finishing: AzureBlob File Copy
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,689 questions
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
36,015 questions
0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest