Error while writing files to SFTP ADF Dataflow Target, Can't make directory for path "_temporary"

Jatinder Luthra 130 Reputation points
2023-07-17T21:23:40.1966667+00:00

Hello folks,

I am reading files from Unix VM in ADF dataflow using SFTP from one directory, transforming them and writing back to another directory.

While writing back the files, I am getting below error, Can't make directory for path "_temporary"

I tried creating directory using SFTP user manually under target location and it worked fine. Not sure why dataflow is throwing the error.

Thoughts?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,205 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,506 Reputation points Microsoft Employee
    2023-07-19T04:47:29.4+00:00

    Hi

    Thank you for posting query in Microsoft Q&A Platform.

    It seems like you are encountering an issue with creating a directory for the path "_temporary" when writing files back to a Unix VM in Azure Data Factory Data Flow. The error message indicates that the directory cannot be created.

    To troubleshoot this issue, you can try the following steps:

    Verify that the target location has the correct permissions. Make sure that the SFTP user has the correct permissions to create directories and write files to the target location. You can check the target location permissions to ensure that the SFTP user has the correct permissions.

    Check the network connectivity between the Azure Data Factory instance and the Unix VM. There may be network issues that are preventing the Azure Data Factory instance from connecting to the Unix VM. You can try pinging the Unix VM from the Azure Data Factory instance to verify the network connectivity.

    Check the configuration of the SFTP server**12**. The SFTP server may be misconfigured or experiencing issues. You can check the SFTP server documentation or contact the SFTP server administrator to verify the correct configuration.

    Check the file format path of the target location. The file format path may be incorrect or not supported by the SFTP server. You can check the SFTP server documentation or contact the SFTP server administrator to verify the correct file format path.

    Try creating the directory manually using the SFTP user. If you are able to create the directory manually using the SFTP user, it may indicate that there is an issue with the Data Flow. You can check the Data Flow configuration to ensure that it is set up correctly.

    Hope this helps. Please let us know how it goes. Thank you.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.