Share via


Best practice - disable or remove deduplication on a volume

Question

Monday, February 26, 2018 8:06 AM

Hello everyone,

My colleague asked me for turn off data deduplication on my file server. He cares about backup and every time when it is needed to run some data deduplication operation h needs to backup huge changes.

I wonder what is best solution. disable or remove completely deduplication on a volume? I gues that if I would like to Unoptimization data, I will need more space, so I have to enlarge partitition.

FreeSpace    SavedSpace   OptimizedFiles     InPolicyFiles      Volume
                 
89.65 GB     193.26 GB    513569             513514             E:
26.31 GB     242.23 GB    154093             153992             F:

Is there some reason why I have unoptimization, I wonder if it is possible and good just to only disable data deduplication and let nwe files in normal form.

And could anyone tell me how big space should I add for successful data deduplication?

Thank you
Tomas

All replies (6)

Tuesday, February 27, 2018 3:15 AM ✅Answered | 2 votes

Hi Thomas,

Based on my knowledge, Disabling Data De-duplication doesn't undupe the data already duped. After you directly disable data deduplication, the volume remains in a deduplicated state and the existing deduplicated data is accessible. So to undo data deduplication on a volume, use the Start-DedupJob cmdlet and specify Unoptimization for the Type parameter first. Then you could disable it.

And after you disable data deduplication on a volume, you can perform all read-only deduplication cmdlet operations on the volume.

/en-us/powershell/module/deduplication/disable-dedupvolume?view=winserver2012-ps

For details about disabling Deduplication, please refer to the article below,

https://social.technet.microsoft.com/Forums/azure/en-US/b69a89bc-fb65-436e-86cb-6e8f4dc9d61b/remove-data-deduplication-role?forum=winserverfileshttp://nickwhittome.com/2014/10/01/disabling-data-deduplication-on-windows-server-2012r2/

Please Note: Since the web site is not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.

And if you need to run data Deduplication, please also check the blogs about Sizing Volumes for Data Deduplication in Windows Server

https://blogs.technet.microsoft.com/filecab/2014/12/04/sizing-volumes-for-data-deduplication-in-windows-server/

Best Regards,

Mary

Please remember to mark the replies as answers if they help.
If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


Tuesday, February 27, 2018 9:07 AM

Hi Mary

Thank you for your reply, I don´t plan to remove deduplication role now. I just wonder what is the best. If I will run Unoptimization, I will need more space. I can ask my colleague for extend partition so it is not problem. My main question is if it is better to do Unoptimization  or if it is ok to let it be as it is and just disable next optimization tasks. I just want to prevent future problems with this state I would create :-) What you would do?

PS C:\Windows\system32> get-dedupstatus | fl

Volume                             : E:
Capacity                           : 660 GB
FreeSpace                          : 88.98 GB
UsedSpace                          : 571.01 GB
UnoptimizedSize                    : 764.27 GB
SavedSpace                         : 193.26 GB
SavingsRate                        : 25 %
OptimizedFilesCount                : 513569
OptimizedFilesSize                 : 672.34 GB
OptimizedFilesSavingsRate          : 28 %
InPolicyFilesCount                 : 513514
InPolicyFilesSize                  : 671.79 GB
LastOptimizationTime               : 2/6/2018 4:38:06 PM
LastOptimizationResult             : 0x00000000
LastOptimizationResultMessage      : The operation completed successfully.
LastGarbageCollectionTime          : 2/6/2018 9:54:44 PM
LastGarbageCollectionResult        : 0x00000000
LastGarbageCollectionResultMessage : The operation completed successfully.
LastScrubbingTime                  : 1/10/2018 12:19:47 PM
LastScrubbingResult                : 0x00000000
LastScrubbingResultMessage         : The operation completed successfully.

Volume                             : F:
Capacity                           : 500 GB
FreeSpace                          : 26.25 GB
UsedSpace                          : 473.74 GB
UnoptimizedSize                    : 715.97 GB
SavedSpace                         : 242.23 GB
SavingsRate                        : 33 %
OptimizedFilesCount                : 154093
OptimizedFilesSize                 : 699.83 GB
OptimizedFilesSavingsRate          : 34 %
InPolicyFilesCount                 : 153992
InPolicyFilesSize                  : 699.8 GB
LastOptimizationTime               : 2/6/2018 4:16:50 PM
LastOptimizationResult             : 0x00000000
LastOptimizationResultMessage      : The operation completed successfully.
LastGarbageCollectionTime          : 2/6/2018 5:43:43 PM
LastGarbageCollectionResult        : 0x00000000
LastGarbageCollectionResultMessage : The operation completed successfully.
LastScrubbingTime                  : 1/9/2018 5:58:35 PM
LastScrubbingResult                : 0x00000000
LastScrubbingResultMessage         : The operation completed successfully.


Tuesday, February 27, 2018 9:22 AM

Hi,

>My main question is if it is better to do Unoptimization  or if it is ok to let it be as it is and just disable next optimization tasks

Based on my understanding, to turn off Data Deduplication for the volume, we need to run the Unoptimization job first. Then you could disbale  DedupVolume

Here is the article for more details.

/en-us/powershell/module/deduplication/disable-dedupvolume?view=win10-ps

Best Regards,

Mary

Please remember to mark the replies as answers if they help.
If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.


Tuesday, February 27, 2018 2:43 PM

I decided to make a complete return to the unoptimized state. I write the result here.


Wednesday, February 28, 2018 8:47 AM

Hello everyone,

so results are good, but proces takes 10 hours, in one moment I was worry about disk space, because it wanted more and more space.

Volume                             : E:
Capacity                           : 810 GB
FreeSpace                          : 24.07 GB
UsedSpace                          : 785.93 GB
**UnoptimizedSize                    : 979.19 GB   ** (first calculation was 715.97 GB)
SavedSpace                         : 193.26 GB
SavingsRate                        : 19 %

So now I have ove volume in normal - non deduplication state. it is not possible to run cleanup process ( GarbageCollection )

PS C:\Windows\system32> start-dedupjob -Volume E: GarbageCollection
start-dedupjob : MSFT_DedupVolume.Volume='E:' - HRESULT 0x80565323, The specified volume is not enabled for deduplication.
At line:1 char:1

  • start-dedupjob -Volume E: GarbageCollection

    + CategoryInfo          : NotSpecified: (MSFT_DedupJob:ROOT/Microsoft/...n/MSFT_DedupJob) [Start-DedupJob], CimException
    + FullyQualifiedErrorId : HRESULT 0x80565323,Start-DedupJob

I found that this command can only be run when dedup is enabled, so I re-enabled dedup, but excluded all folders (root) on the drive, then re-ran the command.

My last results are :

PS C:\Windows\system32> Get-DedupStatus  | fl

Volume                             : E:
Capacity                           : 810 GB
FreeSpace                          : 98.51 GB
UsedSpace                          : 711.49 GB
UnoptimizedSize                    : 711.49 GB
SavedSpace                         : 0 B
SavingsRate                        : 0 %
OptimizedFilesCount                : 0
OptimizedFilesSize                 : 0 B
OptimizedFilesSavingsRate          : 0 %
InPolicyFilesCount                 : 0
InPolicyFilesSize                  : 0 B
LastOptimizationTime               :
LastOptimizationResult             :
LastOptimizationResultMessage      :
LastGarbageCollectionTime          : 2/28/2018 9:41:25 AM
LastGarbageCollectionResult        : 0x00565302
LastGarbageCollectionResultMessage : There are no actions associated with this job.
LastScrubbingTime                  :
LastScrubbingResult                :
LastScrubbingResultMessage         :

Best Regards,

Tomas


Thursday, March 1, 2018 1:19 AM

Hi,

Glad that works and thanks for sharing the results.

Best Regards,

Mary

Please remember to mark the replies as answers if they help.
If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.