Server to Server Storage Replication

  • Article

 

Updated: November 19, 2015

You will configure these computers and storage in a server-to-server configuration, where one node replicates its own set of storage with another node and its set of storage. These nodes and their storage should be located in separate physical sites, although it is not required.

There are no graphical tools in Windows Server 2016 Technical Preview that can configure Storage Replica for standalone replication.

Important

In this test, one node will need to be in a physical or logical site, and the other node in a different physical or logical site. Each server must be able to communicate with the other via a network.

Terms

This walkthrough uses the following environment as an example: .

  • Four servers, named SR-SRV01 and SR-SRV03.

  • A pair of logical “sites” that represent two different data centers, with one called Redmond and one called Bellevue.

Server to Server Storage Replication

FIGURE: Server to Server Replication

Prerequisites

  • Two sets of storage, using SAS JBODs, Fibre Channel SAN, iSCSI Target, or local fixed disks (DAS). The storage should contain a mix of HDD and SSD media. You will make each storage set available only to each of the servers, with no shared access.

  • At least one 1GbE connection on each file server, preferably 10GbE, iWARP, or InfiniBand.

  • Appropriate firewall and router rules to allow ICMP, SMB (port 445, plus 5445 for SMB Direct) and WS-MAN (port 5985) bi-directional traffic between all nodes.

  • A network between the two sets of servers with at least 1Gbps throughput (preferably 8Gbps or higher) and average of ≤5ms round trip latency.

  • Membership in the built-in Administrators group on all server nodes.

Many of these requirements can be determined by using the Test-SRTopology cmdlet. You get access to this tool if you install Storage Replica or the Storage Replica Management Tools features on at least one server. There is no need to configure Storage Replica to use this tool, only to install the cmdlet. More information is included in the steps below.

Provision operating system, features, roles, storage, and network

Warning

Windows Server 2016 Technical Preview does not support Storage Replica on production servers.

  1. Install Windows Server 2016 Technical Preview on both server nodes with an installation type of Windows Server 2016 Technical Preview (Server with Desktop Experience). Do not choose Standard Edition if it is available, as it does not contain Storage Replica.

  2. Add network information and join them to the domain, then restart them.

    Note

    From this point on, always logon as a domain user who is a member of the built-in administrator group on all servers. Always remember to elevate your Windows PowerShell and CMD prompts going forward when running on a Full graphical server installation or a Windows 10 computer.

  3. Connect first set of JBOD storage enclosure, iSCSI target, FC SAN, or local fixed disk (DAS) storage to the server in site Redmond.

  4. Connect second set of storage to the server in site Bellevue.

  5. As appropriate, install latest vendor storage and enclosure firmware and drivers, latest vendor HBA drivers, latest vendor BIOS/UEFI firmware, latest vendor network drivers, and latest motherboard chipset drivers on all four nodes. Restart nodes as needed.

    Note

    Consult your hardware vendor documentation for configuring shared storage and networking hardware.

  6. Ensure that BIOS/UEFI settings for servers enable high performance, such as disabling C-State, setting QPI speed, enabling NUMA, and setting highest memory frequency. Ensure power management in Windows Server is set to high performance. Restart as required.

  7. Configure roles as follows:

    • Graphical method

      1. Run ServerManager.exe and create a Server Group, adding all server nodes.

      2. Install the File Server and Storage Replica roles and features on each of the nodes and restart them.

    • Windows PowerShell method

      On SR-SRV03 or a remote management computer, run the following command in a Windows PowerShell console to install the required features and roles and restart them:

      $Servers = 'SR-SRV01','SR-SRV03'

$Servers | ForEach { Install-WindowsFeature –ComputerName $_ –Name Storage-Replica,FS-FileServer –IncludeManagementTools -restart }

      For more information on these steps, see Install or Uninstall Roles, Role Services, or Features

  8. Configure storage as follows:

    Important

    • You must create two volumes on each enclosure: one for data and one for logs.

    • Log and data disks must be initialized as GPT, not MBR.

    • The two data volumes must be of identical size.

    • The two log volumes should be of identical size.

    • All replicated data disks must have the same sector sizes.

    • All log disks must have the same sector sizes.

    • The log volumes should use flash-based storage, such as SSD.

    • The data disks can use HDD, SSD, or a tiered combination and can use either mirrored or parity spaces or RAID 1 or 10, or RAID 5 or RAID 50.

    • The data volume should be no larger than 10TB (for a first test, we recommend no more than 1TB, in order to lower initial replication sync times).

    • The log volume must be at least 8GB and may need to be larger based on log requirements.

    • For JBOD enclosures:

      1. Ensure that each server can see that site’s storage enclosures only and that the SAS connections are correctly configured.

      2. Provision the storage using Storage Spaces by following Steps 1-3 provided in the Deploy Storage Spaces on a Stand-Alone Server using Windows PowerShell or Server Manager.

    • For iSCSI storage:

      1. Ensure that each server can see that site’s storage enclosures only. You should use more than one single network adapter if using iSCSI.

      2. Provision the storage using your vendor documentation. If using Windows-based iSCSI Targeting, consult iSCSI Target Block Storage, How To.

    • For FC SAN storage:

      1. Ensure that each set server can see that site’s storage enclosures only and that you have properly zoned the hosts.

      2. Provision the storage using your vendor documentation.

    • For local fixed disk (DAS) storage:

      • Ensure the storage does not contain a system volume, page file, or dump files.

      • Provision the storage using your vendor documentation.

  9. Start Windows PowerShell and use the Test-SRTopology cmdlet to determine if you meet all the Storage Replica requirements.

    1. For example, to validate the proposed nodes that each have a F: and G: volume and run the test for 30 minutes:

      MD c:\temp

Test-SRTopology -SourceComputerName SR-SRV01 -SourceVolumeNames f: -SourceLogVolumeName g: -DestinationComputerName SR-SRV03 -DestinationVolumeNames f: -DestinationLogVolumeName g: -DurationInMinutes 30 -ResultPath c:\temp

      Important

      When using a test server with no write IO load on the specified source volume during the evaluation period, consider adding a workload or it will not generate a useful report. You should test with production-like workloads in order to see real numbers and recommended log sizes. Alternatively, simply copy some files into the source volume during the test or download and run DISKSPD to generate write IOs. For instance, a sample with a low write IO workload for five minutes to the D: volume:

      Diskspd.exe -c1g –d300 -W5 -C5 -b8k -t2 -o2 -r –w5 –h d:\test.

  10. Examine the TestSrTopologyReport.html report to ensure that you meet the Storage Replica requirements.

Configure Server to Server Replication using Windows PowerShell

Now you will configure server-to-server replication using Windows PowerShell. You must perform all of the steps below on the nodes directly or from a remote management computer that contains the Windows Server 2016 Technical Preview RSAT management tools.

Warning

The Windows PowerShell cmdlets are not complete for the server-to-server scenario in Windows Server 2016 Technical Preview. The cmdlets do not work properly when used remotely and you should not attempt to make them work through Invoke-Command, Enter-PSSession, or -CimSession.

  1. Configure the server-to-server replication, specifying the source and destination disks, the source and destination logs, the source and destination nodes, and the log size.

    New-SRPartnership -SourceComputerName sr-srv0 -SourceRGName rg01 -SourceVolumeName f: -SourceLogVolumeName g: -DestinationComputerName sr-srv03 -DestinationRGName rg02 -DestinationVolumeName f: -DestinationLogVolumeName g: -LogSizeInBytes 8GB

    Storage_SR_PS_NewSRPartnership

    Important

    The default log size is 8GB. Depending on the results of the Test-SRTopology cmdlet, you may decide to use –LogSizeInBytes with a higher or lower value.

  2. To get replication source and destination state, use Get-SRGroup and Get-SRPartnership as follows:

    Get-SRGroup
Get-SRPartnership
(Get-SRGroup).replicas

    Storage_SR_PS_GetSRGroup

  3. Determine the replication progress as follows:

    1. On the source server, run the following command and examine events 5015, 5002, 5004, 1237, 5001, and 2200:

      Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica –max 20

    2. On the destination server, run the following command to see the Storage Replica events that show creation of the partnership. This event states the number of copied bytes and the time taken. Example:

      Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica –max 1 | Where-Object {$_.ID -eq "1215"} | fl

Log Name:      Microsoft-Windows-StorageReplica/Operational
Source:        Microsoft-Windows-StorageReplica
Date:          4/13/2015 6:00:13 PM
Event ID:      1215
Task Category: (1)
Level:         Information
Keywords:      (1)
User:          SYSTEM
Computer:      sr-srv03.corp.contoso.com
Description:
Bitmap recovery completed successfully for replica.

ReplicationGroupName: Replication 2
ReplicationGroupId: {9d4a9a2a-747a-487f-a68b-5be13c6d7542}
ReplicaName: \\?\Volume{67d83739-b33e-46bd-8007-516804946d09}\
ReplicaId: {d9f2f467-e7e5-401f-8892-6f4d04cfbdbf}
End LSN in bitmap: 
LogGeneration: {00000000-0000-0000-0000-000000000000}
LogFileId: 0
CLSFLsn: 0xFFFFFFFF
Number of Bytes Recovered: 10701766656
Elapsed Time (ms): 32306

      Note

      Storage Replica dismounts the destination volumes and their drive letters or mount points. This is by design.

    3. Alternatively, the destination server group for the replica states the number of byte remaining to copy at all times, and can be queried through PowerShell. For example:

      (Get-SRGroup).Replicas | Select-Object numofbytesremaining

      As a progress sample (that will not terminate):

      while($true) {

 $v = (Get-SRGroup -Name "Replication 2").replicas | Select-Object numofbytesremaining
 [System.Console]::Write("Number of bytes remaining: {0}`r", $v.numofbytesremaining)
 Start-Sleep -s 5
}

    4. On the destination server, run the following command and examine events 5009, 1237, 5001, 5015, 5005, and 2200 to understand the processing progress. There should be no warnings of errors in this sequence. There will be many 1237 events; these indicate progress.

      Get-WinEvent -ProviderName Microsoft-Windows-StorageReplica | FL

Manage replication

Now you will manage and operate your server-to-server replicated infrastructure. You can perform all of the steps below on the nodes directly or from a remote management computer that contains the Windows Server 2016 Technical Preview RSAT management tools.

  1. Use Get-SRPartnership and Get-SRGroup to determine the current source and destination of replication and their status.

  2. To measure replication performance, use the Get-Counter cmdlet on both the source and destination nodes. The counter names are:

    • \Storage Replica Partition I/O Statistics(*)\Number of times flush paused

    • \Storage Replica Partition I/O Statistics(*)\Number of pending flush I/O

    • \Storage Replica Partition I/O Statistics(*)\Number of requests for last log write

    • \Storage Replica Partition I/O Statistics(*)\Avg. Flush Queue Length

    • \Storage Replica Partition I/O Statistics(*)\Current Flush Queue Length

    • \Storage Replica Partition I/O Statistics(*)\Number of Application Write Requests

    • \Storage Replica Partition I/O Statistics(*)\Avg. Number of requests per log write

    • \Storage Replica Partition I/O Statistics(*)\Avg. App Write Latency

    • \Storage Replica Partition I/O Statistics(*)\Avg. App Read Latency

    • \Storage Replica Statistics(*)\Target RPO

    • \Storage Replica Statistics(*)\Current RPO

    • \Storage Replica Statistics(*)\Avg. Log Queue Length

    • \Storage Replica Statistics(*)\Current Log Queue Length

    • \Storage Replica Statistics(*)\Total Bytes Received

    • \Storage Replica Statistics(*)\Total Bytes Sent

    • \Storage Replica Statistics(*)\Avg. Network Send Latency

    • \Storage Replica Statistics(*)\Replication State

    • \Storage Replica Statistics(*)\Avg. Message Round Trip Latency

    • \Storage Replica Statistics(*)\Last Recovery Elapsed Time

    • \Storage Replica Statistics(*)\Number of Flushed Recovery Transactions

    • \Storage Replica Statistics(*)\Number of Recovery Transactions

    • \Storage Replica Statistics(*)\Number of Flushed Replication Transactions

    • \Storage Replica Statistics(*)\Number of Replication Transactions

    • \Storage Replica Statistics(*)\Max Log Sequence Number

    • \Storage Replica Statistics(*)\Number of Messages Received

    • \Storage Replica Statistics(*)\Number of Messages Sent

    For more information on performance counters in Windows PowerShell, see Get-Counter.

  3. To move the replication direction from one site, use the Set-SRPartnership cmdlet.

    Set-SRPartnership -NewSourceComputerName sr-srv03 -SourceRGName rg02 -DestinationComputerName sr-srv01 -DestinationRGName rg01

    Warning

    Windows Server 2016 Technical Preview does not prevent role switching when the initial sync is ongoing, which can lead to data loss if you attempt to switch before allowing initial replication to complete. Do not switch directions until the initial sync is complete.

    Check the event logs to see the direction of replication change and recovery mode occur, and then reconcile. Write IOs can then write to the storage owned by the new source server. Changing the replication direction will block write IOs on the previous source computer.

  4. To remove replication, use Get-SRGroup, Get-SRPartnership, Remove-SRGroup, and Remove-SRPartnership on each node. Ensure you run the Remove-SRPartnership cmdlet on the current source of replication only, not on the destination server. Run Remove-Group on both servers.

    Get-SRPartnership
Get-SRPartnership | Remove-SRPartnership
Get-SRGroup | Remove-SRGroup

See Also