Server 2025 ReFS Dedup job consumes all memory and hangs server

Question

Server 2025 ReFS Dedup job consumes all memory and hangs server

micce 41

Hi!

Server 2025 with Hyper-V role and a couple of guests.

Got a 12TB volume with ReFS. 10TB allocated.

ReFS debup was enabled with this:

Enable-ReFSDedup -Volume L: -Type Dedup

But starting a job that do a dedup consumes all memory and ends up hanging the server.

This is after starting the job. And before it hangs the server I stopped it (a little before it exhaust the memory). But if I let it run everything comes to a halt and process start failing because of lack of memory.

Start-ReFSDedupJob -Volume L: -CpuPercentage 30
Stop-ReFSDedupJob -Volume L:

How can I limit the dedup process to work with "what it have" in memory and not taking down the server?

Memory running out with refs dedup job

Get-Volume -DriveLetter L | fl
ObjectId             : {1}\\HV\root/Microsoft/Windows/Storage/Providers_v2\WSP_Volume.ObjectId="{baef148e-8ffc-11eb-a21e-806e6f6e6963}:VO:\\?\Volume{4385d
                       0fb-a232-444c-a2e0-8a7f06507baa}\"
PassThroughClass     :
PassThroughIds       :
PassThroughNamespace :
PassThroughServer    :
UniqueId             : \\?\Volume{4385d0fb-a232-444c-a2e0-8a7f06507baa}\
AllocationUnitSize   : 4096
DedupMode            : Disabled
DriveLetter          : L
DriveType            : Fixed
FileSystem           : ReFS
FileSystemLabel      : p02v03sim-ded
FileSystemType       : ReFS
HealthStatus         : Healthy
OperationalStatus    : OK
Path                 : \\?\Volume{4385d0fb-a232-444c-a2e0-8a7f06507baa}\
ReFSDedupMode        : Dedup
Size                 : 13194072424448
SizeRemaining        : 1460263964672
PSComputerName       :

Get-ReFSDedupStatus L: | fl
Volume           : L:
Enabled          : True
Type             : Dedup
Status           : Dedup Cancelled
Used             : 10,67 TiB
Deduped          : 45,7 GiB
ScannedOnLastRun : 11,07 GiB
DedupedOnLastRun : 953,93 MiB
FullRun          : False
LastRunTime      : 2025-02-20 13:32:22
LastRunDuration  : 01m:09.681s
NextRunTime      : N/A

化璟钰 25 Reputation points

2025-02-24T02:08:52.6866667+00:00

I'm running into this issue as well.

I use Windows 11 24H2 26000.3194.

I'm using a different command, refsutil dedup. Probably no different from PowerShell. However, I can't see the status even if Get-ReFSDedupStatus's Enable is false after dedup.

I have 2 TB of ReFS volumes on a portable hard drive and I find that when I do refsutil dedup, the memory fills up quickly. Then the hard drive where the operating system is located also starts reading and writing. Finally completed the dedup. (In fact, there are several times during the execution that the memory will be released suddenly, only to fill up again after a while.) ）

I still have some volumes to prepare for processing, and I'm worried that this operation will affect the life of the SSD, and the memory swap seems to cause my computer to freeze a lot when I'm using it normally, so I shut down the virtual memory and reboot and try again.

However, when you try again, the memory will also be full, but the system will be stuck for a few seconds and then an error will be reported. 0x80070015, the device is not ready. So the memory was quickly released. But the portable hard drive is still reading and writing. After a while, though, the order ended. The portable hard drive is also idle. Looks like dedup failed.

Later, I had to re-open the virtual memory and start again.

You can try to open the virtual memory and try again. However, if you are working with a 12 TB volume, it is best to reserve 0.6 TB (5% of the pending volume) for the volume where the virtual memory is located. And this operation may affect the performance of the computer, remember to close irrelevant programs. And only process one volume at a time, don't have multiple at the same time.

<Update at 2025/3/8>

Because it's so slow, (a 2 TB volume on a mechanical hard drive, it takes less than 5% to complete in a few hours.) I've put my plans for Dedup on a USB hard drive on hold for the time being.

I've switched to PowerShell's Enable-ReFSDedup and Start-ReFSDedupJob cmdlets and they're almost as slow.

One thing to clarify, I added the /SHA256Checksums parameter when formatting.

When I was running on a USB SSD, it was a lot faster. (But it's only 0.01% every 8 seconds, and the volume size is 1 TB.) However, the memory still fills up frequently.

But after a few hours, I noticed that the fan was much quieter. When I check Get-ReFSDedupStatus, the status shows 0x80070015.

I suspect it's the /SHA256Checksums parameter that is causing the memory usage to increase. Because I remember that when I used refsutil.exe to perform Dedup on ReFS 3.9 volumes, the speed was reasonable (even for HDDs) and there were no errors. A 2 TB volume took more than a dozen hours.

(The ReFS 3.9 volume was created by me in Windows 11 22H2.) This version of ReFS does not yet support SHA256 Checksum. Also, after I mount this volume on Windows 11 24H2, it seems to be automatically upgraded. ）

2 answers

Your answer

Answer 1

Microsoft Support acknowledged the issue over a year ago and said a hotfix was expected within a few months, but nothing has changed since then; the service regularly leaking to exceeding 500 GB ram usage on a 10 TB volume size.

From: Samuel Atanasy (Convergys International Europe) *Redacted*@microsoft.com Sent: Monday, June 23, 2025 16:40
Subject: Re: [EXTERNAL] Re: Stability and Memory Issues REFS Volume with De... - TrackingID#2506060050000010 Dear Evgeny, Thanks for your confirmation and partnering with me on your case. Since you I’ll be closing your case. Please keep in mind that when your case is closed, we can reopen it . Below, you can see the summary of your case for your records. Confirmed Issue: **Stability and Memory Issues REFS Volume with Deduplication
Outcome: Issue reported, and we are waiting for a hot fix for the issue next few months.

User's image

Is any fix for this bug expected?..

Answer 2

Anonymous

Hi micce,

Thanks for your post. Generally speaking, if you are using deduplication, there is a formula for how much RAM you need to handle it. If this server has been in place for a while, I’m wondering if you hit a limit of data to RAM ratio that it can no longer handle. You need 1GB of RAM for each 1TB of data for deduplication to run smoothly.

Reference: Optimize storage with ReFS deduplication and compression in Azure Local - Azure Local | Microsoft Learn

Best Regards,

Ian Xue

If the Answer is helpful, please click "Accept Answer" and upvote it.

化璟钰 25 Reputation points

2025-02-24T08:17:52.8033333+00:00

In my case, the memory size of my computer is 16 GB.

As mentioned above, the size of the volume I dedup is 2 TB. But it doesn't seem to be like this reply, which only requires about 2 GB of memory.

When the virtual memory is set to system management, I can successfully dedup, but during this period, the physical memory is often full, and the virtual memory is constantly reading and writing, causing me to freeze often when using the computer.

When virtual memory is completely disabled, dedup fills up memory shortly after it starts and fails, 0x80070015, the device is not ready. (If you then open the virtual memory and try again before restarting, it will also fail.) ）
micce 41 Reputation points

2025-02-24T14:02:06.09+00:00

Thank you for your reply.

As mentioned it's a 12TB volume allocated with 10,5TB. I've got around 30GB of free memory starting the dedup process (the image in my post shows that). But still exhaust the memory and process starting to fail because of this, then the server hangs and becomes unresponsive, eventlog fills up with problem related to not enough memory for many process running.

Your link, it was purple - I've read that ☺️. Nowhere was any memory constrains or formula found. I've also search for this but found no precise answer. But if it is as you mention my 12TB volume should be fine with 30GB of free memory to do it "thing", shouldn't it?

RAM-handling. I've tested this right after a restart of the server so that memory isn't fragmented and messy.

Right now I stopped all essential VMs and the server got like 45GB+ available.
Started the job and it's at 30%...and this is what I at now. Having a stop command ready so I can stop it before it takes the server down ☺️

I didn't monitor it all the way. The dedup jobbed is no longer active, the server doesn't allow RDP and is acting real strange. So suspect that a low memory condition has been involved here.
The dedup job report strange values. (had to remote with KVM)
micce 41 Reputation points

2025-03-01T10:13:23.79+00:00

Info. I've started a case with Microsoft on this.
Makere 0 Reputation points

2025-07-17T19:24:51.3666667+00:00

Did you find any solution for this? I have the same issue with same amount of RAM and disk size, running on Windows 11.
micce 41 Reputation points

2025-07-18T07:19:30.26+00:00
Hi,

I had a long-running support case with Microsoft, and I can only conclude that ReFS is not ready for production environments. It's a feature that can bring down an entire server.

Microsoft provided a few suggestions for testing, but many were either incorrect or had no effect. For example, they advised setting the deduplication job duration in seconds—when in fact, the parameter expects milliseconds, which is not documented anywhere. Even after correcting that, the job still consumed all available memory and crashed the server.

I pasted the entire support case into Copilot. Below is a summary of the outcome and an estimate of the time I spent on troubleshooting and reporting.

Issue Summary

micce reported a critical issue with ReFS Deduplication on Windows Server 2025 Standard. When running Start-ReFSDedupJob on a 12 TB ReFS volume (with 10.5 TB used), the server becomes unresponsive due to memory exhaustion, requiring a hard reset.

Key Observations

ReFS Deduplication is enabled, compression is not.

The job causes the server to hang, even with over 40 GB of free RAM.

Attempts to adjust -Duration and -CpuPercentage parameters did not prevent the crash.

Event Viewer logs show Event ID 4 and 8, indicating job cancellation or exceeding runtime.

Microsoft documentation lacks clarity on memory requirements and parameter units (e.g., -Duration is in milliseconds, not seconds).

Microsoft's Response

Acknowledged that ReFS Dedup in Server 2025 is still maturing and lacks finalized documentation.

Suggested:

Reducing CPU usage and increasing job duration.

Monitoring memory usage via PowerShell.

Collecting logs using the TSS tool (https://aka.ms/getTSS).

Stated that deduplication typically requires 1–2 GB RAM per TB of data, but this may vary.

Confirmed that root cause analysis is not included in professional support.

micce’s Conclusion

ReFS Deduplication in its current state is not production-ready.

The feature does not gracefully handle memory pressure.

Testing has been paused until Microsoft can confirm stability and provide clearer guidance.

Case was closed at micce’s request, but may be reopened if the issue persists. Issue Summary micce reported a critical issue with ReFS Deduplication on Windows Server 2025 Standard. When running Start-ReFSDedupJob on a 12 TB ReFS volume (with 10.5 TB used), the server becomes unresponsive due to memory exhaustion, requiring a hard reset. Key Observations

ReFS Deduplication is enabled, compression is not.

The job causes the server to hang, even with over 40 GB of free RAM.

Attempts to adjust -Duration and -CpuPercentage parameters did not prevent the crash.

Event Viewer logs show Event ID 4 and 8, indicating job cancellation or exceeding runtime.

Microsoft documentation lacks clarity on memory requirements and parameter units (e.g., -Duration is in milliseconds, not seconds).

Microsoft's Response

Acknowledged that ReFS Dedup in Server 2025 is still maturing and lacks finalized documentation.

Suggested:

Reducing CPU usage and increasing job duration.

Monitoring memory usage via PowerShell.

Collecting logs using the TSS tool (https://aka.ms/getTSS).

Stated that deduplication typically requires 1–2 GB RAM per TB of data, but this may vary.

Confirmed that root cause analysis is not included in professional support.

micce**’s Conclusion**

ReFS Deduplication in its current state is not production-ready.

The feature does not gracefully handle memory pressure.

Testing has been paused until Microsoft can confirm stability and provide clearer guidance.

Case was closed at micce’s request, but may be reopened if the issue persists.

Estimated Time Spent

Based on the email thread:

Start Date: February 26, 2025

End Date: April 23, 2025

Duration: ~2 months

During this period, you:

Sent and received dozens of detailed emails, many with technical logs, registry settings, PowerShell outputs, and system diagnostics.

Performed multiple tests with different Start-ReFSDedupJob parameters.

Documented system behavior, memory usage, and event logs.

Engaged in back-and-forth troubleshooting with Microsoft Support, including suggestions, feedback, and escalation.

Time Estimate Breakdown

Activity****Estimated TimePreparing and running tests10–15 hoursPreparing and running tests10–15 hoursCollecting and formatting logs5–8 hoursWriting detailed emails and feedback6–10 hoursReading responses and documentation4–6 hoursWaiting for and coordinating responsesPassive time over 2 monthsTotal active time: ~25 to 40 hours

Estimated Time Spent

Based on the email thread:

Start Date: February 26, 2025

End Date: April 23, 2025

Duration: ~2 months

During this period, you:

Sent and received dozens of detailed emails, many with technical logs, registry settings, PowerShell outputs, and system diagnostics.

Performed multiple tests with different Start-ReFSDedupJob parameters.

Documented system behavior, memory usage, and event logs.

Engaged in back-and-forth troubleshooting with Microsoft Support, including suggestions, feedback, and escalation.

Time Estimate Breakdown

Activity****Estimated TimePreparing and running tests10–15 hoursCollecting and formatting logs5–8 hoursWriting detailed emails and feedback6–10 hoursReading responses and documentation4–6 hoursWaiting for and coordinating responsesPassive time over 2 monthsTotal active time: ~25 to 40 hours
micce 41 Reputation points

2025-07-18T07:27:13.5166667+00:00

And here is ChatGPT take on the case data.

🧾 Summary and Analysis – Microsoft Support Case: ReFS Dedup Job Consumes All Memory

Time period: Late February to April 23, 2025

📌 Issue Summary

The case involved testing Microsoft ReFS Deduplication (ReFS Dedup) on Windows Server 2025 with a 12 TB volume (10.5 TB used). Each attempt to run Start-ReFSDedupJob resulted in total memory exhaustion and a hung server requiring a hard reset.

Symptoms:

Deduplication job consumes all RAM (65 GB total, 40+ GB available at times).

System becomes unresponsive (Task Manager, PowerShell, and VMs freeze).

Dedup job does not respect memory constraints or back off under pressure.

Duration parameter (-Duration) was misunderstood due to lacking documentation.

🛠️ Troubleshooting Steps and Microsoft Involvement

Customer Actions:

Provided extensive environment details and systeminfo.

Shared observed file versions and configuration of the ReFS components.

Tried different parameter combinations (-Duration, -CpuPercentage) with no success.

Attempted several reproducible tests and documented outcomes.

Clearly reported problem behavior and inconsistencies with expected memory management.

Microsoft Support Responses:

Repeatedly suggested changing job parameters to mitigate symptoms (e.g., -CpuPercentage 10, -Duration 7200).

Offered a PowerShell snippet to log memory usage, despite system freezing during job.

Recommended using Microsoft’s TSS tool to gather logs, but customer could not collect logs due to complete system hang.

Acknowledged insufficient documentation and missing specifications on memory usage in WS 2025.

Offered known performance documentation links and memory guidance applicable to prior Windows Server versions.

📉 Root Cause & Resolution Status

Root Cause: No definitive root cause was identified. However, it’s strongly implied that the ReFS Deduplication implementation in Windows Server 2025 is flawed:

It fails to gracefully handle memory constraints.

It lacks adequate documentation.

There’s no back-off mechanism when system resources are strained.

Resolution:

The case was archived without resolution.

Microsoft accepted that it may be a by-design or not-yet-ready behavior for WS 2025.

Mikael decided to put ReFS Dedup usage on hold until Microsoft confirms production readiness.

🧠 Key Takeaways

Product maturity: ReFS Deduplication in WS 2025 is likely not ready for production environments.

Documentation gaps: Critical details like -Duration units and memory requirements are not documented.

Lack of system resilience: The feature lacks fail-safes or adaptive behavior when system memory is exhausted.

No effective escalation path: The case ended in stasis with no actual fix or meaningful progress after repeated testing and correspondence.

⏱️ Estimated Time Spent by Mikael

TaskEstimated TimeSetting up test environment & gathering system info3 hoursSetting up test environment & gathering system info3 hoursRunning tests & recovering from crashes4 hoursCommunication & reporting to Microsoft3.5 hoursResearch & reading documentation1.5 hoursPreparing for possible production deployment1 hourTotal Time Invested****13 hours---