Azcopy 7.3.0 to copy tables from table storage using GitHub Actions

Samael 1

I'm trying to copy tables from "table storage" using GitHub Actions , but with version 7.3.0 of azcopy shows the following error message :

[2022/09/27 17:13:45.525+00:00][VERBOSE] 7.3.0 : azcopy --source https://sourcestorage.table.core.windows.net/TestTable1 --destination /home/runner/tableexport/ --source-key ****** --verbose    

Error: 9/27 17:13:45][ERROR] The syntax of the command is incorrect. The source azure storage location type can not be inferred. Use --source-type <blob|file> to specify the location type explicitly.

The source type is not blob or file , and I think that using domain table.core.windows.net the command should detect that the source is table storage. Can you help me with this problem?. Thanks in advance.

Best Regards

ShaktiSingh-MSFT 15,421 Reputation points

2022-10-10T06:36:35.427+00:00

Hi @Samael ,

Following up to see if the below suggestion was helpful. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

2 answers

ShaktiSingh-MSFT 15,421 Reputation points

2022-10-04T10:06:50.277+00:00
Hi @Samael ,

Welcome to Microsoft Q&A portal and thanks for using Azure Services.

Regarding the syntax error received as mentioned in the question, could you please try below mentioned command:

AzCopy /Source:https://myaccount.table.core.windows.net/myTable/ /Dest:C:\myfolder\ /SourceKey:key

Reference link: Export data from Table storage-azcopy

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a [notification][40]
Please sign in to rate this answer.
Samael 1 Reputation point

2022-10-04T12:53:55.447+00:00

Hi @ShaktiSingh-MSFT ,

Sorry I forgot to say that I am using the ubuntu-latest image. When I run command in your previous post:

Error: 0/04 11:34:34][ERROR] The syntax of the command is incorrect. Unrecognized command or argument '/Source:https://sourcestorage.table.core.windows.net/TestTable1'
Error: Process completed with exit code 2.

because the syntax of command is :

azcopy 7.3.0-netcore Copyright (c) 2018 Microsoft Corp. All Rights Reserved.

azcopy <--source> <--destination> [--source-key] [--dest-key]
[--source-sas] [--dest-sas] [--verbose] [--resume] [--config-file]
[--quiet] [--parallel-level] [--source-type] [--dest-type]
[--recursive] [--include] [--check-md5] [--dry-run]
[--preserve-last-modified-time] [--exclude-newer] [--exclude-older]
[--sync-copy] [--set-content-type] [--blob-type] [--delimiter]
[--include-snapshot] [--block-size-in-mb]

The command in your previous post is similar to the command I'm trying to run on the GitHub Actions CI/CD:

azcopy --source https://sourcestorage.table.core.windows.net/TestTable1 --destination /home/runner/tableexport/ --source-key ****** --verbose

Best regards

ShaktiSingh-MSFT 15,421 Reputation points

2022-10-06T12:11:20.083+00:00

Hi @Samael ,

Thanks for the reply.

As you have mentioned, the syntax looks correct.
Could you please check if the AzCopy installation is done properly on Ubuntu?

Reference Link: storage-use-azcopy-linux

Samael 1 Reputation point

2022-10-11T08:43:35.89+00:00

Hi,

Could you please check if the AzCopy installation is done properly on Ubuntu?

I think that the installation is done properly on Ubuntu because shows me version and help without problems.

To solve the problem I split this part of my pipeline in other job using windows-latest image and I can export the table to local , but when importing using the same manifest ( without modifying manifest and json following the documentation https://learn.microsoft.com/en-us/previous-versions/azure/storage/storage-use-azcopy ) I get the below error .

Error: 0/11 08:08:27][ERROR] Failed to import Azure table. Detailed error: The manifest file name "D:\a\MDTest\MDTest\tableexport\testlakestorage_TestTable1_20221011T080748.7171627Z.manifest" contains invalid character ':'.

The content of the manifest is :

"Version":2,"PayloadFormat":"Json","Checksum":XXXXXXXXXXXXXXXXXXXX,"AccountName":"testlakestorage","TableName":"TestTable1","Timestamp":"2022-10-11T08:07:48.7171627Z","SplitSize":0,"TotalDataFiles":1

Can you help me again? . Thanks in advance.

Best Regards

ShaktiSingh-MSFT 15,421 Reputation points

2022-10-11T13:01:51.773+00:00

Hi @Samael ,

Thanks for replying back.

Regarding this error, it seems the data file's checksum in the title don't match the total checksum in the manifest file and contains invalid character.

Could you please refer to this post's discussions and see if it helps.

Please reply in case of further queries.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.
Samael 1 Reputation point

2022-10-13T05:58:09.603+00:00
Hi,

Could you please refer to this post's discussions and see if it helps.

I think that is not my problem because in my case I get only 2 files into directory .

Directory: D:\a\MDTest\MDTest\tableexport

Mode LastWriteTime Length Name

-a--- 10/12/2022 10:33 PM 330 testlakestorage_TestTable1_20221012T223342.3016262Z_0_0_D5681845FB3D
82AB.json
-a--- 10/12/2022 10:33 PM 202 testlakestorage_TestTable1_20221012T223342.3016262Z.manifest

Showing manifest with type command :

"Version":2,"PayloadFormat":"Json","Checksum":15377567616503481003,"AccountName":"testlakestorage","TableName":"TestTable1","Timestamp":"2022-10-12T22:33:42.3016262Z","SplitSize":0,"TotalDataFiles":1

I can't watch where is the ':' that is generating the problem. The document appears well formed.

Best Regards
Please sign in to rate this answer.
ShaktiSingh-MSFT 15,421 Reputation points

2022-10-13T06:08:08.537+00:00

Hi @Samael ,

This is strange and not expected. I would recommend you to please file a support ticket for deeper investigation and in case if you don't have a support plan, do let us know here so that we can check on other options to unblock you. Thanks
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Azcopy 7.3.0 to copy tables from table storage using GitHub Actions

2 answers

azcopy 7.3.0-netcore Copyright (c) 2018 Microsoft Corp. All Rights Reserved.

Your answer