6 days of backups are gone with no errors in the event logs

Question

6 days of backups are gone with no errors in the event logs

Jim 306

Server 2022 all updates current.

For some reason, after vacation, I came in and checked the backups. We have a local (first image) and cloud backup (second image)

Looks like backups stopped on the 13th and started again on the 20th. I find nothing in the event logs relevant to this.

I tried to recover a folder from the cloud folder and got to mounting the drive to recover from (F:). This is where it stopped. The drive got mapped, the process seemed to just stop. I tried opening the F: drive and it just hung up Windows explorer and I had to restart that process.

The local backup recovery is running now and appears to be fine.

Guess I need to skip the vacations....

Any idea how I can debug this? I have not tried rebooting the server yet as it is business hours.

I only know about net stop/start wbengine to restart backup. But I was wondering if there was a process I could restart to do the same thing.

Thanks for any info.

User's image

Jim 306 Reputation points

2024-10-29T18:07:57.8566667+00:00

Nothing??
Jim 306 Reputation points

2024-10-31T14:43:57.58+00:00

Same thing happened again, backups stopped with no errors. Only common factor is both times it started on a Sunday.
Jim 306 Reputation points

2024-11-01T15:25:29.0366667+00:00

Now it looks like they are not running at all. I can run them manually fine. I set VSS service from manual to automatic but no joy.

Anybody have a clue?
Darrell Gorter 2,731 Reputation points

2024-11-03T16:02:49.03+00:00

Anything in the Backup Event log? Is it enabled?
Jim 306 Reputation points

2024-11-11T13:48:27.7933333+00:00
Thanks, I had rebooted the server and the issue went away for a while. But now it's back. The backups stopped in both cloud and local on the 7th. They work manually. The only thing in the event log for local are informational messages:

Basically job start job complete, job status change. The messages just stop on the 7th.

The cloud backup looks similar but there is one error on the 8th. Event ID 11 The backup operation has completed with errors:

Friendly view

System

Provider

[ Name] CloudBackup [ Guid] {abb3d988-13d1-4d34-808e-e53a2a84d97f} EventID 11 Version 1 Level 2 Task 0 Opcode 1 Keywords 0x4000000000000000

TimeCreated

[ SystemTime] 2024-11-08T06:01:02.0253514Z EventRecordID 8615 Correlation

Execution

[ ProcessID] 7236 [ ThreadID] 14816 Channel CloudBackup Computer server.domain.local

Security

[ UserID] S-1-5-18

EventData StopInfo <?xml version="1.0"?> <CBJob><JobType>Backup</JobType><JobStatus><JobState>Aborted</JobState><StartFileTime>133755192012588230</StartFileTime><EndFileTime>133755192620157303</EndFileTime><RecoveryPointFileTime>0</RecoveryPointFileTime><ErrorInfo><ErrorCode>1073841973</ErrorCode><DetailedErrorCode>-2147467259</DetailedErrorCode><ErrorParamList/></ErrorInfo><DatasourceStatus><CBDatasourceStatus><JobState>Aborted</JobState><LastCompletedJobState>NotStarted</LastCompletedJobState><ErrorInfo><ErrorCode>1073841973</ErrorCode><DetailedErrorCode>-2147467259</DetailedErrorCode><ErrorParamList/></ErrorInfo><Datasource><DataSourceId>933230287271316019</DataSourceId><DataSourceName>C:\Program Files\Microsoft Azure Recovery Services Agent\Scratch\SSBV</DataSourceName><WriterId>da57a531-e7e7-4346-9a68-b511f551deb6</WriterId><ApplicationId>8c3d00f9-3ce9-4563-b373-19837bc2835e</ApplicationId></Datasource><ByteProgress><Total>0</Total><Changed>0</Changed><Progress>0</Progress><Failed>0</Failed></ByteProgress><FileProgress><CurrentFile></CurrentFile><Total>0</Total><Changed>0</Changed><Progress>0</Progress><Failed>0</Failed></FileProgress><PartialIcRun>false</PartialIcRun></CBDatasourceStatus></DatasourceStatus></JobStatus><RecoveryPointID>0</RecoveryPointID><RecoveryType>Undefined</RecoveryType><RecoveryRegionType>Undefined</RecoveryRegionType><IsRetried>false</IsRetried></CBJob>

Thanks for any help.
Darrell Gorter 2,731 Reputation points

2024-11-11T16:04:56.32+00:00

Hello,

I would look at the System and App Event logs, starting around an hour before the backup failure occurs.

Compare to the same timeframe when the back succeeds.

There may be some thing triggering the reason the backup do not happen, so Iwould start by investigating what is happening just prior. If you make the event logs available, for download I will take a look at them as well.

Darrell
Jim 306 Reputation points

2024-11-12T11:59:33.67+00:00

I have looked at them but didn't see anything. I will check again though. One more data point. I rebooted the system around 22:00 last night and this morning I looked and the backups went as scheduled. The last time I did that they went for about a week before stopping.

I'll revisit the logs today...
Jim 306 Reputation points

2024-11-12T17:05:55.76+00:00

I checked the logs, all of them, for any issues in the hour before the failed backups. Nothing.

I had another thought. Keep in mind both the local backup AND the cloud backup fail to run. After a reboot, they both run for several days but then stop. My question is, what are the things/files/processes that both have in common? This might give me a lead as to where to look.
Jim 306 Reputation points

2024-11-12T21:28:33.8366667+00:00

I began to consider that the online backups are conflicting with the local backups and perhaps sort of cancelling each other out. This is some of the history for the online backup:

This is some of the history for the local backup:

Here is what I can't figure out. This is the settings for the online backup:

This is the setting for the local backup:

Now, looking at 11/6 as a typical example, why do I show a system state backup to Azure at 1:09 under the local backup as well as one at 1:06 am under the cloud backup? I have nothing scheduled for the 1am time slot under local.

Something doesn't make sense here.

Also, keep in mind that this has all worked fine for over a year with no changes at all on my part, excluding patching.
Darrell Gorter 2,731 Reputation points

2024-11-12T22:40:52.15+00:00

Hello.

Just randomly trying to think of things that could affect this.

Anything scheduled around the same times. check task scheduler. ( schtasks /query)

Any time changes, check the logs for time events and look for jumps forward or backward. ( Event log security , Microsoft Windows Security auditing.

Also check the security log for any logons around the time of scheduled backups.

Any Virus scans occurring around the same times? Unlikely but at this points grasping.

Any new apps running or new services?

Darrell
Pranay Reddy Madireddy 6,180 Reputation points Microsoft External Staff Moderator

2024-11-25T20:21:30.47+00:00

Hi Jim - Just checking in to see if the below answer provided by @Darrell Gorter helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Pranay Reddy Madireddy 6,180 Reputation points Microsoft External Staff Moderator

2024-11-26T17:28:32.2233333+00:00

Hi Jim - Just checking in to see if the below answer provided by @Darrell Gorter helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Jim 306 Reputation points

2024-11-26T19:07:30+00:00

@Darrell Gorter Sorry for not getting back sooner. This is a 2 day a week gig so I can be a bit slow to respond

Anyway, all of those were good suggestions but alas, did not bear fruit. For the time being have set up a task to reboot the system at 3am on Sunday mornings which appears to have solved it for now so I can take time to look further into it.

Your answer

Jim 306 Reputation points

2024-10-29T18:07:57.8566667+00:00

Nothing??
Jim 306 Reputation points

2024-10-31T14:43:57.58+00:00

Same thing happened again, backups stopped with no errors. Only common factor is both times it started on a Sunday.
Jim 306 Reputation points

2024-11-01T15:25:29.0366667+00:00

Now it looks like they are not running at all. I can run them manually fine. I set VSS service from manual to automatic but no joy.

Anybody have a clue?
Darrell Gorter 2,731 Reputation points

2024-11-03T16:02:49.03+00:00

Anything in the Backup Event log? Is it enabled?
Darrell Gorter 2,731 Reputation points

2024-11-11T16:04:56.32+00:00

Hello,

I would look at the System and App Event logs, starting around an hour before the backup failure occurs.

Compare to the same timeframe when the back succeeds.

There may be some thing triggering the reason the backup do not happen, so Iwould start by investigating what is happening just prior. If you make the event logs available, for download I will take a look at them as well.

Darrell
Jim 306 Reputation points

2024-11-12T11:59:33.67+00:00

I have looked at them but didn't see anything. I will check again though. One more data point. I rebooted the system around 22:00 last night and this morning I looked and the backups went as scheduled. The last time I did that they went for about a week before stopping.

I'll revisit the logs today...
Jim 306 Reputation points

2024-11-12T17:05:55.76+00:00

I checked the logs, all of them, for any issues in the hour before the failed backups. Nothing.

I had another thought. Keep in mind both the local backup AND the cloud backup fail to run. After a reboot, they both run for several days but then stop. My question is, what are the things/files/processes that both have in common? This might give me a lead as to where to look.
Jim 306 Reputation points

2024-11-12T21:28:33.8366667+00:00

I began to consider that the online backups are conflicting with the local backups and perhaps sort of cancelling each other out. This is some of the history for the online backup:

This is some of the history for the local backup:

Here is what I can't figure out. This is the settings for the online backup:

This is the setting for the local backup:

Now, looking at 11/6 as a typical example, why do I show a system state backup to Azure at 1:09 under the local backup as well as one at 1:06 am under the cloud backup? I have nothing scheduled for the 1am time slot under local.

Something doesn't make sense here.

Also, keep in mind that this has all worked fine for over a year with no changes at all on my part, excluding patching.
Darrell Gorter 2,731 Reputation points

2024-11-12T22:40:52.15+00:00

Hello.

Just randomly trying to think of things that could affect this.

Anything scheduled around the same times. check task scheduler. ( schtasks /query)

Any time changes, check the logs for time events and look for jumps forward or backward. ( Event log security , Microsoft Windows Security auditing.

Also check the security log for any logons around the time of scheduled backups.

Any Virus scans occurring around the same times? Unlikely but at this points grasping.

Any new apps running or new services?

Darrell
Pranay Reddy Madireddy 6,180 Reputation points Microsoft External Staff Moderator

2024-11-25T20:21:30.47+00:00

Hi Jim - Just checking in to see if the below answer provided by @Darrell Gorter helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Pranay Reddy Madireddy 6,180 Reputation points Microsoft External Staff Moderator

2024-11-26T17:28:32.2233333+00:00

Hi Jim - Just checking in to see if the below answer provided by @Darrell Gorter helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Jim 306 Reputation points

2024-11-26T19:07:30+00:00

@Darrell Gorter Sorry for not getting back sooner. This is a 2 day a week gig so I can be a bit slow to respond

Anyway, all of those were good suggestions but alas, did not bear fruit. For the time being have set up a task to reboot the system at 3am on Sunday mornings which appears to have solved it for now so I can take time to look further into it.

Share via

6 days of backups are gone with no errors in the event logs

Your answer