Share via

How to disable CBT (Change Block Tracking) on a specific Hyper-V VM

Anonymous
2021-03-10T17:04:54.153+00:00

Hello!

We started backing up a SQL server with Veeam backup and have had performance problems with the SQL server since that point. The performance/latency issue isn't while the backup is running - it's all of the time.

Since Veeam uses the CBT built into Windows Server 2019 Hyper-V, I was thinking that CBT might be the cause of the poor performance.

I couldn't find anything on how to "officially" turn off CBT for a specific VM, so after experimenting, I found that you could shutdown a VM, remove the *.rct and *.mrt files and they wouldn't be re-created once the VM was turned back on. (At least until the next backup which would re-enable CBT).

The problem is that after I deleted the *.rct and *.mrt files from the SQL server, I didn't see any performance improvement. This could be because the problem lies somewhere else (not related to CBT) or it could be that I didn't disable CBT correctly and there is still a performance hit because CBT isn't fully disabled.

Can anyone shed any light on this?

Thanks!

Windows for business | Windows Client for IT Pros | Storage high availability | Virtualization and Hyper-V
Windows for business | Windows Server | Storage high availability | Clustering and high availability
Windows for business | Windows Server | User experience | Other
0 comments

Answer accepted by question author

Anonymous
2021-03-23T18:54:18.007+00:00

It turns out that the CBT/RCT wasn't affecting the performance of my VM. I'm sure that there is some performance impact, but it wasn't the cause of my specific issue.

At least reading the posts here, there is no "official" way of disabling CBT/RCT for a specific VM. But it seems to work to shutdown the VM and delete the .rct and .mrt manually.

Was this answer helpful?

6 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Anonymous
    2023-12-14T22:13:40.97+00:00

    We have been using veeam backup for some while now - no issues was detected during this period. even using CBT in HyperV to backup our ordinary VM's.

    We also have a large (20Tb) Virtual HyperV machine - and we only took backup of C: drive (with CBT) - and until now we have, had no performance issues with this - c: drive is only a 100G drive..

    However a few days a ago we started to take backup of the large data drives as well - the same minute the backup was initiated the complete server started to have performance issues, and the issues persisted afte the backup completed.

    Before we initiated this backup of the large drive, performance was as said fine, but now it sucks.
    I have ongoing WMI performance monitoring of all my VMs every 5 min - so I can see/document the behavior and go back the last 30 days to inspec he WMI counters..

    or physical disk que size, read transfere, write transferes og both my physical server runnig the VM and the VM it self.

    it is clear that the disk que used to be aprox 5000/sec, now they are at 150.000/sec
    leading to extremly slow disk IO.
    I can see the physical server is not transfering same amount of data, as before - mainly becouse the VM can not fetch the data due to a disk que.

    if I move other VM's to the same physical server, these VM's can make normal disk IO -basically telleing me the issues is not the physical server. reather the VHDX files belonging to the bad behaiving VM

    all in all, I can confirm that enabling CBT is a bad idea on large/heavy disk IO VM's running under HyperV.

    My plan is now to move this vm to vmware, and start using CBT there - hopefully this may solve this issue that MS has had for years and not solved.

    Im in the happy situation that I can migrate to vmware cluster, and HyperV/failovercluster will be shurtdown permanently in our production enviroment within a few months :) - I have tested this beta software (HyperV) all to may times, and it keeps having issues not seen in ESXi spending to much time and downtime for by applications.

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  2. Pascal Slijkerman 91 Reputation points
    2021-03-18T10:59:03.393+00:00

    Hi Dave,

    CBT or in Hyper-V known as RTC (Resilient Change Tracker) is a mecanisme to track all changes on block level. This way Veeam knows exactly what to backup instead of scanning/looking for the changes which takes way more time. So during backup you should see a performance improvement.

    Seems like you can disable CBT in Veeam. CBT is enabled in the Veeam filter driver on the host. So disabling that might not disable RCT or because RTC is on the host it's already bypasses CBT....But I am no Veeam Expert... For what I know there is no way to disable RTC on Hyper-V. It's enabled by default if you meet the criteria.
    https://helpcenter.veeam.com/docs/backup/hyperv/changed_block_tracking.html?ver=110

    I think however that your probleem lies somewere else. Do other VMs on the host expierence performance issue's? Are the performance issue's on cpu/mem/disk level in the VM or maybe on the host itself?

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments
  3. Anonymous
    2021-03-11T15:59:58.387+00:00

    Actually, I wasn't looking for help in troubleshooting the performance issues. I would like like someone to comment on how to remove CBT (Change Block Tracking) for a specific VM.

    Does anyone know how to do this? Other than just to remove the .rtc and .mrt files?

    Thanks!

    Was this answer helpful?

  4. Xiaowei He 9,946 Reputation points
    2021-03-11T06:56:16.367+00:00

    Hi,

    Thanks for your post, based on my understanding, the issue is that Hyper V VM has performance issue when use Veeam backup. Since performance issue need to analyze performance logs, we are limited to collect the performance logs via forum and the issue is also related with third-party application, which will be complex to troubleshoot, I would suggest you open a case with MS for better help.

    Below is the link to open a case with MS:

    https://support.microsoft.com/en-us/gp/customer-service-phone-numbers

    Thanks for your time!
    Best Regards,
    Anne

    If the Answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.