Although I detest Windows 11 and believe it's badly broken; I'm afraid this is a Microsoft Windows issue that had been going on long before Windows 11. It was just never fixed. The best thread concerning this issue that I've found, was never really answered. There were a lot of very knowledgeable people in that thread that went into real depth trying to solve the issue, to no avail. This included the radical and extreme measure of re-formatting the Windows drive and re-installing a fresh copy of Windows. Going to the hardware manufacturer and downloading current drivers also did not fix the issue. There was never an answer on that thread, and eventually people drifted away from it, so it was closed and locked with no answer. Responders in that thread also noted that transfers using a Mac or Linux OS from the same source responded perfectly.
I am experiencing the same issue. I have found that this issue gets worse when there are memory intensive background processes running. Although a long shot, when this happens, you can check task manager; look for Apps, programs or processes that are running, and shut down anything you don't actually need running during the copy process. Rebooting after the Copy process is complete, should restore everything except what was launched manually during the last session. While I don't believe that this is a total cure, I hope the advice helps someone. I have never hit on the "One" culprit that is a constant with this trouble. If someone else hits on an App, program or process that consistently helps I would sure like to know. :D