Streaming upserts constantly report mysterious log, are very slow

Alex Hoffer 1

Our streaming batch upserts are very slow and constantly report:

20/12/03 23:38:04 WARN RetryTolerableRenameFSDataOutputStream: Failed to rename temp file dbfs:/mnt/container/table/_delta_log/__tmp_path_dir/.00000000000000007431.json.e22ddeb8-a7f4-4f03-bef7-5f379a4428b6.tmp to dbfs:/mnt/container/table/_delta_log/00000000000000007431.json because file exists

in log4j. Any idea how to resolve this log?

HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2020-12-04T21:11:09.897+00:00

Hello @Alex Hoffer ,

Thanks for the ask and also for using the forum .

At this time we have reached out to the internal team, I will update you once I hear back from them .

Thanks
Himanshu
HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2020-12-08T19:56:08.283+00:00

Hello @Alex Hoffer ,

For a deeper investigation and immediate assistance on this issue,
if you have a support plan you may file a support ticket, else could you please send an email to AzCommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter.

Thread URL:
Subscription ID:
Subject:Attention : Himanshu

Please update this once you have done the same
Thanks Himanshu
HimanshuSinha-msft 19,476 Reputation points Microsoft Employee

2020-12-10T20:48:08.277+00:00

Hello ,
We have not heard back from you on this and was just following up .
Incase if you have resolution , request you to share the same here , so that other community members can benefit from that .

Thanks
Himanshu
Lakhs 0 Reputation points

2024-09-27T09:09:36.51+00:00

Is there a solution to this problem? Can anyone pls share?

Share via

Streaming upserts constantly report mysterious log, are very slow

Your answer