Is Azure Batch compatible with hierarchical namespaces?

Simon Hogg 0 Reputation points
2023-06-07T11:15:17.3833333+00:00

Hello,

We are trying to run NextFlow via Azure batch, and it is working well except it is failing to generate reports at the end due to a very unclear error:

Jun-06 15:53:35.187 [FileTransfer-2] WARN nextflow.processor.PublishDir - Failed to publish file: az://sftp/work/61/df5788e0ae539d01ccdf572c03e4eb/sample_1; to: az://sftp/test_Result/salmon/sample_1 [copy] -- See log file for details

com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidInput</Code><Message>One of the request inputs is not valid. RequestId:ecca5337-601e-0020-167e-985b64000000 Time:2023-06-06T13:53:35.1832956Z</Message></Error>"

We have this working in another storage account, which does NOT have hierarchical namespace enabled in...so we are wondering if something in Batch is not compatible with this feature?

Thanks in advance.

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,714 questions
Azure Batch
Azure Batch
An Azure service that provides cloud-scale job scheduling and compute management.
303 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

    Comments have been turned off. Learn more

  2. KarishmaTiwari-MSFT 18,522 Reputation points Microsoft Employee
    2023-06-14T01:31:16.43+00:00

    @Simon Hogg I'm glad that you were able to understand the cause of your issue and thank you for posting it here so that others experiencing the same thing can easily reference this!

    Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer. Accepted answers show up at the top, resulting in improved discoverability for others.

    Customer shared-

    Issue: We are trying to run NextFlow via Azure batch, and it is working well except it is failing to generate reports at the end due to a very unclear error:

    Jun-06 15:53:35.187 [FileTransfer-2] WARN nextflow.processor.PublishDir - Failed to publish file: az://sftp/work/61/df5788e0ae539d01ccdf572c03e4eb/sample_1; to: az://sftp/test_Result/salmon/sample_1 [copy] -- See log file for details

    Cause: Via the NextFlow Slack channel, I was able to get some more feedback and it seems there is an issue currently with Gen2 storage and NextFlow. We will open a ticket on the Nextflow site to try to resolve.

    If your issue remains unresolved or have further questions, please let us know in the comments how we can assist. We are here to help you and strive to make your experience better and greatly value your feedback.

    0 comments