Windows Server 2019 dedup compatibility issues after 2021-06 Cumulative Update (KB5003646)

Question

I have discovered a compatibility issue with the Windows Deduplication feature on Windows Server 2019 after the 2021-06 Cumulative Update (KB5003646) is applied.

If the Windows Server 2019 machine has a volume attached which was originally deduped on a Windows Server 2016 machine then it will BSOD relating to dedup.sys.

The dedup.sys driver for Windows Server 2019 was updated between 2021-05 CU (file version 10.0.17763.1554) and 2021-06 CU (file version 10.0.17763.1971).

I have reproduced this issue with a clean install of Windows Server 2019 from the RTM ISO with no additional software or configuration changes. Up to and including OS build 17763.1935 (2021-05 CU) there are no issues accessing data on a volume originally deduped on Windows Server 2016. Once the system is updated to OS builds 17763.1999 (2021-06 CU) then accessing data on that same volume will cause a BSOD relating to dedup.sys.

The issue is not present for volumes which were first deduped on Windows Server 2019 only when the volume was first deduped on Windows Server 2016.

This was originally discovered when performing a rolling OS upgrade of a failover file server cluster running Windows Server 2016 nodes which use deduped volumes for file shares. Once the roles with the deduped volumes were moved to the first Windows Server 2019 node it caused that node to BSOD.

Answer 1

I agree, unoptimizing the volumes and starting over with 2019 June driver would be a secure workaround. Unfortunately this might be almost impossible for us. We have multiple volumes, which would all hit their boundries unoptimized. This would imply a lot of time-consuming copy jobs and disk space in dimensions we just don't have lying around. For folders that mostly have to be available 24/7.

Also the changed driver had another serious impact for us:
Our backup software Veeam relies on it. And I guess Veeam is not the only backup solution with similiar mechanisms. Currently I cannot restore any single file originally stored on a deduped volume of the file server. I hope this will work again as soon as I uninstall KB5003646 from the backup machine, too.

Nevertheless this is mere a slight change in the driver. The above point shows it made it literally incompatible within the same server version! You expect that when you step up from 2016 to 2019 and it's well-known and described for Veeam. But this broke so many things at once, we feel at risk right now.

Oh and btw. shadow copies are broken right now, too. They are disabled after the rollback to the may update and cannot be activated again. Guess something has changed with the way of scheduled tasks implementation in the June CU.

Answer 2

Have The exact issue ourselves.

2016 with an In-place upgrade a few years ago to 2019.

everything was working perfectly until a month ago, which would coincide with the last patch tuesday.

if we disable dedup on our storage volumes then it drops our reboots down to 1 or two after hours, when backups to this file server commence.

we can uninstall the update via DISM from Windows RE/PE

dism.exe /Image:D:\ /get-packages /format:list

>

6. At the "Package Identity" column, find out the Package Name of the update that you want to remove.
   • e.g. "Package_for_KB4058702~31bf3856ad364e35~amd64~~16299.188.1.0"
7. Then give the following command to remove the problematic update package:

dism /image:D:\ /Remove-Package /PackageName:PackageName
e.g. To remove the "Package_for_KB4025376~31bf3856ad364e35~amd64~~10.0.1.0 ", give the following command:

dism /image:D:\ /Remove-Package /PackageName:Package_for_KB4025376~31bf3856ad364e35~amd64~~10.0.1.0

excerpt from here: https://www.repairwin.com/how-to-remove-updates-from-windows-recovery-environment-winre/

Answer 3

We had the same problem on several Servers. It seems that the July CU fixed the issue. At least we had no more BSODs with July CU installed.
Can anyone confirm this? Is there an official statement from microsoft?

We are still scared because we had several crashes a day on more than five file-servers affecting several hundred users.