Full Database backups are taking long time after upgrading SQL Server 2008 R2 to 2016 (SP2)

Udham Singh 96 Reputation points
2020-09-09T08:24:18.953+00:00

Hi,

Recently we have upgraded (in-place) our on prem SQL Server 2008 R2 Standard Edition to 2016 (Sp2) Standard. After the upgrade we are seeing Full database backups are taking longer time to complete. We have a db of 1.5TB size, to backup this db before upgrade it use to take around 65 minutes, but now its taking around 9-10 hours.

We have notice same thing with other dbs (same instance) also but their sizes are small so not much worried.

Any idea what's wrong here. How can we resolve this issue. We have changed the compatibility level of DB to 130. And OS is 2012 R2 standard edition.

Thank you,
Udham Singh

SQL Server | Other
{count} votes

Accepted answer
  1. Udham Singh 96 Reputation points
    2020-10-30T07:26:21.057+00:00

    After troubleshooting the issue it turn out to be storage performance issue.

    "Avg Disk Sec\Write" remains > 100 ms, while monitoring in Resource manager -> Disk -> sqlserver.exe we were able to see that Response Time were around 400ms. Which was quite high. This should not go beyond 15ms.


9 additional answers

Sort by: Most helpful
  1. tibor_karaszi@hotmail.com 4,316 Reputation points
    2020-09-09T13:27:23.353+00:00

    Did you change compression settings for your backups?

    1 person found this answer helpful.

  2. m 4,276 Reputation points
    2020-09-10T03:42:26.017+00:00

    Hi @Udham Singh ,

    You can reference @Shashank Singh ’s suggestion and @tibor_karaszi@hotmail.com ’s suggestion to have a test, install the latest CU or change the compression.

    You can also check by next script :

    while(1=1)  
    begin  
    select CURRENT_TIMESTAMP  
    select * from sys.sysprocesses where spid>50 and cmd like '%backup%'  
    select * from sys.dm_exec_requests where command like  '%backup%'  
    if exists(select * from sys.sysprocesses where spid>50 and blocked<>0 and waittime > 10000)    
       begin  
        print '*************blocking detect******************'  
    select * from sys.sysprocesses where spid>50  
    end  
    waitfor delay '00:01:00'  
    end  
    

    Troubleshooting steps:

    1.Enable trace flag for SQL server.

    In management studio, please help run below command to enable trace flag, so that we can get more clues about the backup performance:

    DBCC TRACEON(3004, 3051, 3212,3014, 3605, 1816, -1)

    After test is done, please turn them off this way:

    DBCC TRACEOFF(3004, 3051, 3212,3014, 3605, 1816, -1)

    2.Check whether have one IO bottleneck;

    More information: slow-backup-or-restore-operations, full-database-backup-takes-so-long?forum=sqldatabaseengine

    BR,
    Mia


    If the answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    0 comments No comments

  3. m 4,276 Reputation points
    2020-09-11T01:31:30.433+00:00

    Hi @Udham Singh ,

    Is the reply helpful?

    BR,
    Mia


    If the answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.


  4. tibor_karaszi@hotmail.com 4,316 Reputation points
    2020-09-11T07:00:32.267+00:00

    OK, so we now know that you are waiting for I/O during the backup operation, considering your wait stats. First, I suggest you read up on this wait stats at: https://www.sqlskills.com/help/waits/async_io_completion/. Make sure you read the full article, especially the "known occurrences" section.

    Now enable the trace flags suggested my Mia, and you can after doing a backup study in the errorlog where the time is spent.

    You now have some knowledge exactly where you are spending all this time. Perhaps the checkpoint takes a very long time now? Etc. My guess is that something else happened with this upgrade that affected the I/O throughput. But that is for you to find out with the info you'll get from those trace flags.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.