Is it possible to populate data to Azure Storage Table from Azure Pipelines using powershell?

Question

Hi, I am trying to populate data to Storage Table from Powershell, using Yaml Pipeline from withing Azure Pipeline. I am using Managed Identity, so Service Principal. I am ale to create a table, bo unable to put a row.

My code:

param

(

[String]$env="myenv",

[String]$region="myregion",

[String]$rootDir=".",

[String]$subscriptionName="mysubscription"

)

$resourceGroupName = "myproject-$($env.ToLowerInvariant())-$($region.ToLowerInvariant())-rg"

$storageAccountName = "myproject$($env.ToLowerInvariant())$($region.ToLowerInvariant())storage"

Set-AzContext -SubscriptionId $subscriptionName

$context = Get-AzContext

$servicePrincipalId = $context.Account.Id

$principal = Get-AzADServicePrincipal -ApplicationId $servicePrincipalId

$principalRoles = Get-AzRoleAssignment -ObjectId $principal.Id -ResourceGroupName $resourceGroupName -ResourceName $storageAccountName -ResourceType "Microsoft.Storage/storageAccounts"

$principalRoles | ForEach-Object { Write-Host " - RoleDefinitionName: $($.RoleDefinitionName), Scope: $($.Scope)" }

$ctx = New-AzStorageContext -UseConnectedAccount -TableEndpoint "https://$($myproject)($myenv)($myregion)storage.table.core.windows.net/"

#create new table with random name to test

$random = Get-Random -Minimum 1000 -Maximum 9999

$testTableName = "testtable$($random)"

New-AzStorageTable -Name $testTableName -Context $ctx

Start-Sleep -Seconds 60

$testTable = Get-AzStorageTable -Name $testTableName -Context $ctx

$testTablePartitionKey = "testpartition"

$testTableRowKey = "testrow"

$testEntity = New-Object -TypeName Microsoft.Azure.Cosmos.Table.DynamicTableEntity -ArgumentList $testTablePartitionKey, $testTableRowKey

try {

$cloudTestTable = $testTable.CloudTable

Write-Host "Cloud table: $cloudTestTable"

$cloudTestTable.Execute([Microsoft.Azure.Cosmos.Table.TableOperation]::InsertOrReplace($testEntity))  

}

catch {

Write-Host "An error occurred:"

Write-Host $_

}

Creating test table: testtable6549

CloudTable : testtable6549

TableClient : Azure.Data.Tables.TableClient

Uri : https://myprojectmyenvmyregionstorage.table.core.windows.net/testtable6549

Context : Microsoft.WindowsAzure.Commands.Storage.Common.AzureStorageContex

          t 

Name : testtable6549

Retrieving test table: testtable6549

Created test table: testtable6549

Adding test entity to test table

An error occurred:

Exception calling "Execute" with "1" argument(s): "The specified resource does not exist."

What is wrong? Is it possible to do that?

I was looking into
https://learn.microsoft.com/en-us/azure/storage/tables/table-storage-how-to-use-powershell

so I learned how to add a table, but nothing changed in reagards to adding a rows.

Answer 1

It seems that you are encountering an issue when trying to insert an entity into your Azure Storage Table using PowerShell within Azure Pipelines. The error message "The specified resource does not exist" typically indicates that the table or the context you are trying to access may not be correctly set up or that there is a mismatch in the resource names.

Here are a few suggestions to troubleshoot the issue:

1. Check Table Existence: Ensure that the table you created (testtable6549) actually exists in your storage account. You can do this by listing all tables in your storage account using the Get-AzStorageTable cmdlet.

      Get-AzStorageTable -Context $ctx | Select Name
   

2. Verify Storage Context: Make sure that the $ctx variable is correctly initialized with the right storage account context. You can output the context to verify its properties.

      Write-Host "Storage Context: $($ctx.StorageAccountName)"
   

3. Use Correct Table Client: The error might also arise from using the wrong client or method for inserting the entity. Ensure that you are using the correct CloudTable reference. You might want to check if the CloudTable property is correctly assigned after retrieving the table.

      $cloudTestTable = $testTable.CloudTable
   

4. Check Managed Identity Permissions: Since you are using Managed Identity, ensure that the identity has the necessary permissions to perform operations on the Azure Storage Table. You may need to assign the appropriate role (like Storage Table Data Contributor) to the Managed Identity for the storage account.
5. Debugging Output: Add more debugging output to see the values of your partition key and row key before the insert operation. This can help confirm that they are being set correctly.

      Write-Host "Partition Key: $testTablePartitionKey, Row Key: $testTableRowKey"
   

By following these steps, you should be able to identify the cause of the error and successfully insert data into your Azure Storage Table.

---

References:

• Perform Azure Table storage operations with Azure PowerShell

Answer 2

Hi @ Mirko Boruc,

It looks like you're trying to populate an Azure Storage Table using PowerShell in an Azure YAML pipeline. You mentioned you can create a table, but you’re having trouble inserting a row, and you're getting an error that says "The specified resource does not exist."

Here are a few things you can check and try:

1. CloudTable Reference: Ensure that you're correctly referencing the CloudTable property of the table. You already have this lined up in your code, but it's crucial to verify that the table exists and is properly retrieved.
2. Use Correct Storage Context: In your New-AzStorageContext, ensure that your Storage Account name is correctly formatted in the endpoint URL, including the right environment variables.

   
      $ctx = New-AzStorageContext -UseConnectedAccount -TableEndpoint "https://$($storageAccountName).table.core.windows.net/"
   
   

3. Check Table Name: Make sure the testTableName you're using is correctly set and that the table actually exists before the row insertion.
4. Table Entity Class: Ensure that the DynamicTableEntity you are creating for the insert operation is set up properly. You should also double-check if the PartitionKey and RowKey you are using align with existing entries.
5. Try a Direct Insert: Sometimes, checking against the older methods can help. Use the Insert method directly instead of InsertOrReplace to see if that changes your outcome:

   
      $cloudTestTable.Execute([Microsoft.Azure.Cosmos.Table.TableOperation]::Insert($testEntity))
   
   

6. Permissions: Since you're using Managed Identity, ensure that the identity has the necessary permissions (e.g., 'Storage Table Data Contributor') for the Storage Account.

If you've verified all these aspects and still face issues, here are a few questions that might help narrow down the problem:

1. Can you confirm whether the table testtable6549 exists just before you try to insert the row?
2. Have you verified that the Managed Identity has the right permissions on the Storage Table?
3. What version of the Az PowerShell module are you using? It might help to update if it's not the latest.
4. Is the error consistent (always the same) or does it change based on different table names or structures?

Feel free to update me with any findings or further issues, and I’ll be happy to help!

References:

• How to develop with PowerShell
• Manage storage accounts with Azure PowerShell
• Azure Table Storage Operations

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.