azcopy sync .vhd (not virtual hard disk)

Question

Hello community!
I'm using, on linux, 'azcopy sync' to backup some project on an Azure storage;
the data for backup include also hardware description language with standard extensions as .vhd (vhdl coding language); this seems to trigger some exclusion related to Virtual Hard Disk (.vhd) extension.
The same happen for 'azcopy copy' command if --blob-type isn't specified.
Is there a way to do an 'azcopy sync' that could include these .vhd files ?
Thanks!

Answer 1

Hello Luca! It sounds like you're encountering a challenge with azcopy sync when trying to backup files with the .vhd extension, which are being excluded due to their association with Virtual Hard Disks, even though in your case, they represent VHDL coding language files. To include .vhd files in your azcopy sync operation, you can use the --include-pattern parameter to specifically include these files. While azcopy sync doesn't have a direct parameter to override the exclusion of .vhd files based on their extension alone, using include patterns can help ensure they're considered for synchronization.

Here's a general approach to include .vhd files in your azcopy operations: Using --include-pattern You can specify patterns for the files you want to include in your synchronization or copy operation. To include .vhd files, you can use the --include-pattern parameter followed by the pattern matching your VHDL files. For example: bash Copy code azcopy sync '<source-path>' '<destination-path>' --include-pattern "*.vhd" This command tells azcopy to sync all files that match the pattern .vhd from the source to the destination. It's important to ensure that your pattern accurately reflects the files you intend to include to avoid inadvertently excluding other necessary files. Note on --blob-type

For the azcopy copy command, when dealing with files that Azure might interpret differently based on their extension (like .vhd files for blobs), specifying the --blob-type parameter can be crucial. For non-disk-related files, you might use: bash Copy code azcopy copy '<source-path>' '<destination-path>' --blob-type BlockBlob --include-pattern ".vhd" This explicitly sets the blob type as BlockBlob and includes files matching the .vhd pattern, helping to ensure that your VHDL files are treated as regular blob files rather than virtual hard disks.

Additional Considerations Wildcards: The --include-pattern accepts wildcards, allowing for flexible file selection. Use this to your advantage for specifying multiple file types or naming conventions. Exclusion Rules: If there are other file types or specific files you wish to exclude, you can also use the --exclude-pattern parameter in conjunction with --include-pattern for finer control over what gets synchronized. Testing: Before running your final azcopy sync command on your entire dataset, consider testing with a smaller set of files or using the --dry-run option (if available) to review what actions azcopy would take without actually transferring any files. This can help ensure your command is correctly set up. Just be carefully crafting your azcopy command with the right parameters, you should be able to include your VHDL .vhd files in your backup or synchronization processes without them being mistakenly excluded.

I hope this helps? If you have any questions please let me know.

Answer 2

Hello Luca Cenzato,

Welcome to Microsoft Q&A Forum, thank you for posting your query here!

I understood that you’re encountering an issue where the .vhd extension, which is commonly associated with Virtual Hard Disk files, is being excluded due to its similarity to .vhdl (VHDL coding language) or .hvd extensions.

1. By default, AzCopy excludes certain file extensions during synchronization to prevent unintended transfers.
2. The .vhd extension is likely being excluded because it’s associated with virtual hard disks, which are typically not meant for direct synchronization.

To include .vhd files in your synchronization process, you can try the following approaches:

When using the azcopy copy command, you mentioned that specifying --blob-type resolves the issue.

• Similarly, when using azcopy sync, you can explicitly specify the blob type to ensure that .vhd files are included.
• Use the --blob-type BlockBlob flag to indicate that the files should be treated as block blobs (which is appropriate for .vhd files).

Alternatively, you can also try customizing the exclusion rules. Create a file named exclusion.txt (or any other name) and list the extensions you want to exclude (e.g., .vhdl, .hvd). Then use the --exclude-pattern flag to point to this file during synchronization:

./azcopy sync "/path/to/local/dir" https://[account].blob.core.windows.net/[container]/[path/to/virtual/dir]" --exclude-pattern="exclusion.txt"

Ensure to test the synchronization process in a safe environment before applying it to critical data and check the Azure Blob storage container settings allow the storage of .vhd files.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.