Azure Storage Sync Error [Querying for new jobs failed]

Kerim Gules 1 Reputation point
2022-05-06T12:48:03.153+00:00

Hello,

Azure Storage Sync my service has been down for about a week. When I control the logs from the server, the following warning caught my attention. Shortly before this warning, syncing stopped.

I using Windows Server 2012 R2 and all updates installed. AFS agent v15 installed ( It was running version v14.1 before the problem. I updated to version v15 after having problems ).

Querying for new jobs failed.
HttpErrorCode: 0x80C87014
InternalErrorCode: 0x80C87014

The errors on the portal are:

Download
Sync failed
Error Code
0x80071128
199659-image.png

Upload
Sync failed
Error Code
0x80C80248
199665-image.png

Cloud Tiering
The file failed to tier because the current sync mode is initial upload or reconciliation.
Error Code
0x80C86050
199676-image.png

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,189 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Sumarigo-MSFT 44,251 Reputation points Microsoft Employee
    2022-05-09T09:30:03.383+00:00

    @Kerim Gules Firstly, apologies for the delay in responding here and any inconvenience this issue may have caused.
    Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

    Troubleshooting step1: Can you restart the server and also , install KB2919355 If this update will not install because a later update is already installed, go to Windows Update, install the latest updates for Windows Server 2012 R2 and restart the server.

    Uninstall the Azure File Sync agent
    The last step is an optional task, since we removed the server from the sync group and unregistered it, those are the more important tasks. If you would like to uninstall the agent, it is 4 simple steps as outlined below Click Start > Control Panel > Add/Remove Programs (Programs and Features).
    Click on Storage Sync Agent > Click Uninstall.

    Troubleshoot Azure File Sync: https://learn.microsoft.com/en-us/azure/storage/file-sync/file-sync-troubleshoot?tabs=portal1%2Cazure-portal

    Sync is not adhering to the tiering policy because it is in “Initial Download Sync” mode. Since this is an initial download, tiering will not work. Tiering will work once the initial download completes and sync enters a “Regular Sync” state. For this initial download for this customer, sync should be running in “Fully Ghosted Sync” mode. The difference between Initial Download and Fully Ghosted is that in Fully Ghosted mode, only the namespace is downloaded from the cloud in the form of ghosted files which are files that occupy a location in the namespace but have no data. In contrast, Initial Download Sync downloads the namespace with full data files, meaning that much more disk space is taken up by the files in the Initial Download Sync compared to Fully Ghosted Sync. In this case, the server kept downloading full data files until it filled up the entire volume, since the size of the cloud share is far greater than the size of that volume.

    Debug-StorageSyncServer -FileSyncErrorsReport

    To collect logs and traces on the server, run the following PowerShell commands:

    Import-Module "C:\Program Files\Azure\StorageSyncAgent\StorageSync.Management.ServerCmdlets.dll"

    Debug-StorageSyncServer -AFSDiag -OutputDirectory C:\output -KernelModeTraceLevel Verbose -UserModeTraceLevel Verbose

    Based on this error message: 0x80C86050 No action required. The file will be tiered once sync completes initial upload or reconciliation.

    Troubleshooting steps 2:

    1. Disable and Stop FileSyncSvc service
      . sc config "filesyncsvc" start=disabled
      a. net stop filesyncsvc
      b. taskkill /f /im filesyncsvc.exe /t
    2. Extend the volume with the sync root *** GB. This will ensure that you have enough space to create the sync DB and all of the tiered files.
    3. Launch a 'CMD.exe' as Local System using the SysInternals tool 'psexec.exe' to access the files/directories under "System Volume Information". Here are instructions for running psexec.exe:
      . Download psexec.exe from https://learn.microsoft.com/en-us/sysinternals/downloads/psexec. Note that psexec.exe is part of larger set of tools you can download in a zip file "PSTools.zip" from this web page.
      a. Copy psexec.exe to some directory on your server. For the sake of discussion, assume you copy it to "C:\AFS\psexec.exe".
      b. Open an elevated command prompt.
      c. Run: c:\AFS\psexec.exe -i -s -d cmd
      d. This will open another command prompt running as "Local System" (you see "nt authority\system" if you run "whoami").
    4. The database directory is <DriveLetter>:\SystemVolumeInformation\HFS\SyncShareState\<afs service name><SyncGroupName><endpoint name>{Guid}\Metadata
    5. Delete the database directory. This is mainly done to free up space on the disk, since the server is experiencing problems with not enough space on the disk.
    6. Modify the MaxFastDrSessions registry key to force a FastDr run when you start FileSyncSvc again:
      . reg ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Azure\StorageSync" /v MaxFastDrSessions /t REG_DWORD /d 200 /f
    7. Ensure that you don’t have 200 share snapshots on the file share. To be safe, leave room for at least 2 share snapshots to be created per-server.
    8. Enable and start FileSyncSvc
      . sc config "filesyncsvc" start=auto
      a. net start filesyncsvc
    9. Make sure the new DB directory is recreated.

    If the issue still persists, I would like to work closer on this issue.

    Please let us know if you have any further queries. I’m happy to assist you further.


    Please do not forget to 200207-screenshot-2021-12-10-121802.png and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.