Edit

On-demand backup in Azure Service Fabric

  • Article

You can back up data of Reliable Stateful services and Reliable Actors to address disaster or data loss scenarios.

Azure Service Fabric has features for the periodic backup of data and the backup of data on a need basis. On-demand backup is useful because it guards against data loss/data corruption because of planned changes in the underlying service or its environment.

The on-demand backup features are helpful for capturing the state of the services before you manually trigger a service or service environment operation. For example, if you make a change in service binaries when upgrading or downgrading the service. In such a case, on-demand backup can help guard the data against corruption by application code bugs.

Prerequisites

  • Install Microsoft.ServiceFabric.Powershell.Http Module (Preview) for making configuration calls.
    Install-Module -Name Microsoft.ServiceFabric.Powershell.Http -AllowPrerelease

Note

If your PowerShellGet version is less than 1.6.0, you'll need to update to add support for the -AllowPrerelease flag:

Install-Module -Name PowerShellGet -Force

  • Make sure that Cluster is connected using the Connect-SFCluster command before making any configuration request using Microsoft.ServiceFabric.Powershell.Http Module.

    Connect-SFCluster -ConnectionEndpoint 'https://mysfcluster.southcentralus.cloudapp.azure.com:19080'   -X509Credential -FindType FindByThumbprint -FindValue '1b7ebe2174649c45474a4819dafae956712c31d3' -StoreLocation 'CurrentUser' -StoreName 'My' -ServerCertThumbprint '1b7ebe2174649c45474a4819dafae956712c31d3'

Triggering on-demand backup

On-demand backup requires storage details for uploading backup files. You specify the on-demand backup location, either in the periodic backup policy or in an on-demand backup request.

On-demand backup to storage specified by a periodic backup policy

You can configure the periodic backup policy to use a partition of a Reliable Stateful service or Reliable Actor for extra on-demand backup to storage.

The following case is the continuation of the scenario in Enabling periodic backup for Reliable Stateful service and Reliable Actors. In this case, you enable a backup policy to use a partition and a backup occurs at a set frequency in Azure Storage.

Powershell using Microsoft.ServiceFabric.Powershell.Http Module


Backup-SFPartition -PartitionId '974bd92a-b395-4631-8a7f-53bd4ae9cf22'

Rest Call using Powershell

Use the BackupPartition API to set up triggering for the on-demand backup for partition ID 974bd92a-b395-4631-8a7f-53bd4ae9cf22.

$url = "https://mysfcluster.southcentralus.cloudapp.azure.com:19080/Partitions/974bd92a-b395-4631-8a7f-53bd4ae9cf22/$/Backup?api-version=6.4"

Invoke-WebRequest -Uri $url -Method Post -ContentType 'application/json' -CertificateThumbprint '1b7ebe2174649c45474a4819dafae956712c31d3'

Use the GetBackupProgress API to enable tracking for the on-demand backup progress.

On-demand backup to specified storage

You can request on-demand backup for a partition of a Reliable Stateful service or Reliable Actor. Provide the storage information as a part of the on-demand backup request.

Powershell using Microsoft.ServiceFabric.Powershell.Http Module


Backup-SFPartition -PartitionId '974bd92a-b395-4631-8a7f-53bd4ae9cf22' -AzureBlobStore -ConnectionString  'DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=<account-key>;EndpointSuffix=core.windows.net' -ContainerName 'backup-container'

Rest Call using Powershell

Use the BackupPartition API to set up triggering for the on-demand backup for partition ID 974bd92a-b395-4631-8a7f-53bd4ae9cf22. Include the following Azure Storage information:

$StorageInfo = @{
    ConnectionString = 'DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=<account-key>;EndpointSuffix=core.windows.net'
    ContainerName = 'backup-container'
    StorageKind = 'AzureBlobStore'
}

$OnDemandBackupRequest = @{
    BackupStorage = $StorageInfo
}

$body = (ConvertTo-Json $OnDemandBackupRequest)
$url = "https://mysfcluster.southcentralus.cloudapp.azure.com:19080/Partitions/974bd92a-b395-4631-8a7f-53bd4ae9cf22/$/Backup?api-version=6.4"

Invoke-WebRequest -Uri $url -Method Post -Body $body -ContentType 'application/json' -CertificateThumbprint '1b7ebe2174649c45474a4819dafae956712c31d3'

You can use the GetBackupProgress API to set up tracking for the on-demand backup progress.

Using Service Fabric Explorer

Make sure Advanced Mode has been enabled in Service Fabric Explorer settings.

  1. Select the desired partitions and click on Actions.

  2. Select Trigger Partition Backup, and fill in information for Azure:

    Trigger Partition Backup

    or FileShare:

    Trigger Partition Backup FileShare

Tracking on-demand backup progress

A partition of a Reliable Stateful service or Reliable Actor accepts only one on-demand backup request at a time. Another request can be accepted only after the current on-demand backup request has completed.

Different partitions can trigger on-demand backup requests at a same time.

Powershell using Microsoft.ServiceFabric.Powershell.Http Module


Get-SFPartitionBackupProgress -PartitionId '974bd92a-b395-4631-8a7f-53bd4ae9cf22'

Rest Call using Powershell

$url = "https://mysfcluster-backup.southcentralus.cloudapp.azure.com:19080/Partitions/974bd92a-b395-4631-8a7f-53bd4ae9cf22/$/GetBackupProgress?api-version=6.4"

$response = Invoke-WebRequest -Uri $url -Method Get -CertificateThumbprint '1b7ebe2174649c45474a4819dafae956712c31d3' 
$backupResponse = (ConvertFrom-Json $response.Content) 
$backupResponse

On-demand backup requests can be in the following states:

  • Accepted: The backup has started on the partition and is in progress. 
    BackupState             : Accepted
TimeStampUtc            : 0001-01-01T00:00:00Z
BackupId                : 00000000-0000-0000-0000-000000000000
BackupLocation          :
EpochOfLastBackupRecord :
LsnOfLastBackupRecord   : 0
FailureError            :
  • Success, Failure, or Timeout: A requested on-demand backup can be completed in any of the following states:
    • Success: A Success backup state indicates that the partition state has backed up successfully. The response provides BackupEpoch and BackupLSN for the partition along with the time in UTC. 
      BackupState             : Success
TimeStampUtc            : 2018-11-21T20:00:01Z
BackupId                : 5d64b697-6acd-45a4-adbd-3d75e0078081
BackupLocation          : SampleApp\MyStatefulService\974bd92a-b395-4631-8a7f-53bd4ae9cf22\2018-11-21 20.00.01.zip
EpochOfLastBackupRecord : @{DataLossNumber=131873018908156893; ConfigurationNumber=8589934592}
LsnOfLastBackupRecord   : 36
FailureError            :
    • Failure: A Failure backup state indicates that a failure occurred during backup of the partition's state. The cause of the failure is stated in response. 
      BackupState             : Failure
TimeStampUtc            : 0001-01-01T00:00:00Z
BackupId                : 00000000-0000-0000-0000-000000000000
BackupLocation          :
EpochOfLastBackupRecord :
LsnOfLastBackupRecord   : 0
FailureError            : @{Code=FABRIC_E_BACKUPCOPIER_UNEXPECTED_ERROR; Message=An error occurred during this operation.  Please check the trace logs for more details.}
    • Timeout: A Timeout backup state indicates that the partition state backup couldn't be created in a given amount of time. The default timeout value is 10 minutes. Initiate a new on-demand backup request with greater BackupTimeout in this scenario. 
      BackupState             : Timeout
TimeStampUtc            : 0001-01-01T00:00:00Z
BackupId                : 00000000-0000-0000-0000-000000000000
BackupLocation          :
EpochOfLastBackupRecord :
LsnOfLastBackupRecord   : 0
FailureError            : @{Code=FABRIC_E_TIMEOUT; Message=The request of backup has timed out.}

Next steps