Copy Activity Sink to SFTP "useTempFileRename": false option not always working

Jay 561

Has anyone else experienced this issue, is it a bug?

When using Copy Activity and ftp to a site which does not allow file rename I set "useTempFileRename": false.

But even though ADF will say copy was successful the file sometimes is not there, does not arrive, vanished!

If I set "useTempFileRename": true and allow rename on server then the files always arrive OK.

KranthiPakala-MSFT 46,452 Reputation points Microsoft Employee

2020-09-15T20:50:03.86+00:00
Hi @Jay ,

Welcome to Microsoft Q&A platform and thanks for using this forum.

Could you please provide us pipeline, activity runID, and other details for the below scenarios so that I can escalate this to internal team for further investigation.

"useTempFileRename": true and file arrived - pipeline runID & Activity runID

"useTempFileRename": false and pipeline run successful and file didn't arrived - pipeline runID & Activity runID

What is the IR used for both Source and sink? If SHIR please share the version details?

What is the SFTP server your are connecting to ?

Looking forward to your response.

Thank you
KranthiPakala-MSFT 46,452 Reputation points Microsoft Employee

2020-09-16T22:13:40.367+00:00

Hi @Jay ,

Following up to see if you have got a chance to see my previous comment. If so, could you please share the details requested for further investigation.

Thank you
Jay 561 Reputation points

2020-09-17T11:37:42.153+00:00

Thanks for the reply, here is an example:

Triggered, master published runs this morning.

This files arrived
Pipeline run ID e85fc202-495a-44fc-90bb-6fcc696106da
Pipeline run ID 929174fd-4fb8-4dc1-b376-d417f3b52293
Pipeline run ID 231d9063-323b-41de-a6cc-60ec84a16df8

These files did not arrived
Pipeline run ID e9648d0f-d285-401c-932c-100da554b0e0
Pipeline run ID b69f701f-a0c4-404c-ba6e-4277278b74af
Jay 561 Reputation points

2020-09-17T11:40:06.35+00:00

My self hosted Integration Runtime is: 4.11.7521.1 but I'm in copy activity both sink and source are SFTP, I'll have to ask the third party about their SFTP server details.

1 answer

Jay 561 Reputation points

2020-09-18T08:09:53.447+00:00

Sorry KranthiPakala to maybe have messed you about here.

I've now been informed by the recipient that the files may indeed have made it to their site and they have an intermittent job moving them to a wrong folder they were not looking in, although I don't understand how they could see the files when I ticked to use temp unless was this was fluke timing and files not moved.

I'll come back if we do still have an issue.
Please sign in to rate this answer.

1 person found this answer helpful.
KranthiPakala-MSFT 46,452 Reputation points Microsoft Employee

2020-09-18T17:54:12.513+00:00

Hi @Jay ,

Not a problem. Thanks for the updating us back and glad you were able to figure it that there is no issue with ADF processing. Please feel free to reach out if you notice any issues.

Thank you
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Copy Activity Sink to SFTP "useTempFileRename": false option not always working

1 answer

Your answer