This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Hi!
Server 2025 with Hyper-V role and a couple of guests.
Got a 12TB volume with ReFS. 10TB allocated.
ReFS debup was enabled with this:
Enable-ReFSDedup -Volume L: -Type Dedup
But starting a job that do a dedup consumes all memory and ends up hanging the server.
This is after starting the job. And before it hangs the server I stopped it (a little before it exhaust the memory). But if I let it run everything comes to a halt and process start failing because of lack of memory.
Start-ReFSDedupJob -Volume L: -CpuPercentage 30
Stop-ReFSDedupJob -Volume L:
How can I limit the dedup process to work with "what it have" in memory and not taking down the server?
Get-Volume -DriveLetter L | fl
ObjectId : {1}\\HV\root/Microsoft/Windows/Storage/Providers_v2\WSP_Volume.ObjectId="{baef148e-8ffc-11eb-a21e-806e6f6e6963}:VO:\\?\Volume{4385d
0fb-a232-444c-a2e0-8a7f06507baa}\"
PassThroughClass :
PassThroughIds :
PassThroughNamespace :
PassThroughServer :
UniqueId : \\?\Volume{4385d0fb-a232-444c-a2e0-8a7f06507baa}\
AllocationUnitSize : 4096
DedupMode : Disabled
DriveLetter : L
DriveType : Fixed
FileSystem : ReFS
FileSystemLabel : p02v03sim-ded
FileSystemType : ReFS
HealthStatus : Healthy
OperationalStatus : OK
Path : \\?\Volume{4385d0fb-a232-444c-a2e0-8a7f06507baa}\
ReFSDedupMode : Dedup
Size : 13194072424448
SizeRemaining : 1460263964672
PSComputerName :
Get-ReFSDedupStatus L: | fl
Volume : L:
Enabled : True
Type : Dedup
Status : Dedup Cancelled
Used : 10,67 TiB
Deduped : 45,7 GiB
ScannedOnLastRun : 11,07 GiB
DedupedOnLastRun : 953,93 MiB
FullRun : False
LastRunTime : 2025-02-20 13:32:22
LastRunDuration : 01m:09.681s
NextRunTime : N/A
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 28.000000000000004%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
I'm running into this issue as well.
I use Windows 11 24H2 26000.3194.
I'm using a different command, refsutil dedup. Probably no different from PowerShell. However, I can't see the status even if Get-ReFSDedupStatus's Enable is false after dedup.
I have 2 TB of ReFS volumes on a portable hard drive and I find that when I do refsutil dedup, the memory fills up quickly. Then the hard drive where the operating system is located also starts reading and writing. Finally completed the dedup. (In fact, there are several times during the execution that the memory will be released suddenly, only to fill up again after a while.) ）
I still have some volumes to prepare for processing, and I'm worried that this operation will affect the life of the SSD, and the memory swap seems to cause my computer to freeze a lot when I'm using it normally, so I shut down the virtual memory and reboot and try again.
However, when you try again, the memory will also be full, but the system will be stuck for a few seconds and then an error will be reported. 0x80070015, the device is not ready. So the memory was quickly released. But the portable hard drive is still reading and writing. After a while, though, the order ended. The portable hard drive is also idle. Looks like dedup failed.
Later, I had to re-open the virtual memory and start again.
You can try to open the virtual memory and try again. However, if you are working with a 12 TB volume, it is best to reserve 0.6 TB (5% of the pending volume) for the volume where the virtual memory is located. And this operation may affect the performance of the computer, remember to close irrelevant programs. And only process one volume at a time, don't have multiple at the same time.
<Update at 2025/3/8>
Because it's so slow, (a 2 TB volume on a mechanical hard drive, it takes less than 5% to complete in a few hours.) I've put my plans for Dedup on a USB hard drive on hold for the time being.
I've switched to PowerShell's Enable-ReFSDedup and Start-ReFSDedupJob cmdlets and they're almost as slow.
One thing to clarify, I added the /SHA256Checksums parameter when formatting.
When I was running on a USB SSD, it was a lot faster. (But it's only 0.01% every 8 seconds, and the volume size is 1 TB.) However, the memory still fills up frequently.
But after a few hours, I noticed that the fan was much quieter. When I check Get-ReFSDedupStatus, the status shows 0x80070015.
I suspect it's the /SHA256Checksums parameter that is causing the memory usage to increase. Because I remember that when I used refsutil.exe to perform Dedup on ReFS 3.9 volumes, the speed was reasonable (even for HDDs) and there were no errors. A 2 TB volume took more than a dozen hours.
(The ReFS 3.9 volume was created by me in Windows 11 22H2.) This version of ReFS does not yet support SHA256 Checksum. Also, after I mount this volume on Windows 11 24H2, it seems to be automatically upgraded. ）
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 94%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIX%3C/text%3E%3C/svg%3E)
Hi micce,
Thanks for your post. Generally speaking, if you are using deduplication, there is a formula for how much RAM you need to handle it. If this server has been in place for a while, I’m wondering if you hit a limit of data to RAM ratio that it can no longer handle. You need 1GB of RAM for each 1TB of data for deduplication to run smoothly.
Best Regards,
Ian Xue
If the Answer is helpful, please click "Accept Answer" and upvote it.
In my case, the memory size of my computer is 16 GB.
As mentioned above, the size of the volume I dedup is 2 TB. But it doesn't seem to be like this reply, which only requires about 2 GB of memory.
When the virtual memory is set to system management, I can successfully dedup, but during this period, the physical memory is often full, and the virtual memory is constantly reading and writing, causing me to freeze often when using the computer.
When virtual memory is completely disabled, dedup fills up memory shortly after it starts and fails, 0x80070015, the device is not ready. (If you then open the virtual memory and try again before restarting, it will also fail.) ）
Thank you for your reply.
As mentioned it's a 12TB volume allocated with 10,5TB. I've got around 30GB of free memory starting the dedup process (the image in my post shows that). But still exhaust the memory and process starting to fail because of this, then the server hangs and becomes unresponsive, eventlog fills up with problem related to not enough memory for many process running.
Your link, it was purple - I've read that ☺️. Nowhere was any memory constrains or formula found. I've also search for this but found no precise answer. But if it is as you mention my 12TB volume should be fine with 30GB of free memory to do it "thing", shouldn't it?
RAM-handling. I've tested this right after a restart of the server so that memory isn't fragmented and messy.
Right now I stopped all essential VMs and the server got like 45GB+ available.
Started the job and it's at 30%...and this is what I at now. Having a stop command ready so I can stop it before it takes the server down ☺️
I didn't monitor it all the way. The dedup jobbed is no longer active, the server doesn't allow RDP and is acting real strange. So suspect that a low memory condition has been involved here.
The dedup job report strange values. (had to remote with KVM)
Info. I've started a case with Microsoft on this.