Share via

Storage Spaces - Two Way Mirror / Three way Mirror / Quorum and metadata question

WineIsFine 6 Reputation points
2021-02-09T11:22:16.627+00:00

Hi all,

I've been using Storage Spaces since almost first days. Always in Two-Way mirror setups with good/best quality and low latency SSD's to have low maintenance cost and good low queue depth performance and easy setup. First just for VDI, since some years for everything. (spoiler: I'm somewhat unhappy, that I need two DC licenses instead of STD for small clients for SSP direct).

As Space is increasing and quality of SSD's is +- falling with the raise of layers I'm thinking about Three-Way-Mirrors but also thinking about two-way-mirror other ways.

It is really hard to find information how Storage Spcades is acting in reality but some time ago I've found a good explanation for the Three-Way-Mirrors and why it needs five discs as a requirement. It makes sense to me that metadata is looked separately.
--> https://arstechnica.com/civis/viewtopic.php?t=1203237 from PreventRage

As I have to Upgrade one of my first installations, i ask me some questions:

  • Does Storage Spaces act the same in *two-way-mirror" setups when usingen more than two disks? So does it use automatically metadata copies on separate discs than the 256MB chunks? For example a one column two-way-mirror with three or four drives. So Data on D1, D2, changes/metadata/quorum (how ever it works exactly, I could not find really good answers) for example on D1, D3
  • If yes: Is more than two discs in a one column two-way-mirror better than the same thing with two discs even if the potential of one disc going offline is raised the more discs you are using together just because metadata is stored on different location?

What I don't like about most actual SSD's is the fact that IOPS raise but latency and specially average latency is going worse. For Storage Spaces latency as low and stable as possible is crucial. The more discs are involved, the longer is the time for the ACK. Especially when latency is not stable.

thanks a lot

Windows for business | Windows Server | Storage high availability | Other
0 comments No comments

2 answers

Sort by: Most helpful
  1. WineIsFine 6 Reputation points
    2021-02-10T08:54:53.77+00:00

    Hi,

    Thanks for your reply. But I think you did missunderstood a bit how mirroring works in SP. ;)

    Well it does never become more efficient in Mirror-Setups as every data-chunk (slab) is stored in two different locations without calculated parity in "Two-Mirror-Setup" and three times the Space in "Three-Way-Mirror". If the data-chunk (slab) is striped (more than one column) it does just mean, that reads and writes are probably faster because more discs are involved for each read/write. So each slab is striped. The better the discs are - so the better the average latency is - the better works the scaling. Thats why consumer grade ssd work often very bad with storage spaces. Average latency is normally not stable.

    The only thing why just one or two discs can fail in the hole disc set (like with Raid 5/6) is the fact that every disc does not have an exact mirror partner because a chunk and its mirrors can be on every disc (1column) or on every disc-set (more than 1 column). This makes Storage Spaces special and different from traditional RAID. You have the mirror on a chunk level not disc level. But this has nothing to do with the overall capacity you can use from the discs. Two Datasets means half space, three datasets means a third of the space. By using parity volumes, space increases with every disc involved in a n+1 or n+2 setup. But brings also load and complexity.
    --> Thats why storage spaces is so flexible in question of different disc size compared to traditional raid. On the other hand, in traditional raid you can loose two disc on every physiacl disc set (one or two exactly defined partners discs when mirroring). So theoretically traditional raid is more save. In Theory because it was often the case that when you rebuild a mirror, that the load on the mirror partner was 100% over a long time, performance degraded and more than likely that the partner will die too because of the load and same age of the disc. In Storage Spaces data chunks (slabs) are spread over all discs. This brings a theoretically worse secureness because less discs can effectively fail in a perfect world, but in practice it is more likely, that this brings security for rebuilding because load is not on one single disc but also spread over a lot of discs. --> If there are more than just two discs in a set, otherwise load is also 100%

    My question aims in the direction that I would like to know if two-way-mirror acts the same than a three-way-mirror if more discs are involved and what would be the best number of discs for mirroring. Also if this brings really more security in a small setup because of the presence of more discs in a disc set.
    If I understand it right, two-way-mirror with two discs or two disc per stripe (more than one column) is not like storage spaces "likes" to work but it is possible to doo so. If it works like three-way-mirror and handles old and changed data separately, it should always have different disc-sets for old and changed data / old and changed metadata to always have a reliable majority for correct data and metadata when a disc fails. So in my understanding of a perfect world for Storage Spaces that would mean minium 4 discs for a two way mirror with 1 column to have fully separated discs for new and old metadata/pointers completely separated from the old/new data. So changed data and specially metadata/pointers are always written on minimum one completely new disc and just one old disc or minimal two in three way with two discs of 5 failing.

    So is the perfect number of discs for best security: (Total Copiex x n-columns) + (n of mirror partners x n-columns)
    Two-way: (2x n columns) + (1 x n of columns)
    Three-way (3x n columns) + (2x n of columns)

    Or do i missunderstand?

    Was this answer helpful?

    1 person found this answer helpful.

  2. Mico Mi 1,941 Reputation points
    2021-02-10T06:53:20.98+00:00

    Hi,
    Thanks you for posting such interesting questions. It's worth researching on it.
    Storage Spaces does not keep whole copies of volumes – rather, it divides them into tiny 'slabs' which are distributed evenly across all drives in all servers. This has some practical consequences. For example, using two-way mirroring with three servers does not leave one server empty. Likewise, when drives fail, all volumes are affected for the very short time it takes to repair them.
    Consider the scenario where some drive fails. We do lose all the slabs stored on that drive. And we do need to read from each slab's surviving copy in order to repair. But, these surviving copies are evenly distributed across almost every other drive in the pool! One lost slab might have its other copy on Drive 15; another lost slab might have its other copy on Drive 03; another lost slab might have its other copy on Drive 07; and so on. So, almost every other drive in the pool has something to contribute to the repair!
    But the difference is that Storage Spaces can tolerate 2 disks failed if you got 3-way replication configured or double parity! If you have a two way mirror storage space with more than two disks, it can only tolerate one disk failed. For example, 4-disk 2-way in 2-column mode is essentially a striped mirror, but without being able to withstand a 2-disk failure.

    Thanks for your time!
    Best Regards,
    Mico Mi

    -----------------------------

    If the Answer is helpful, please click "Accept Answer" and upvote it.
    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    Was this answer helpful?

    1 person found this answer helpful.
    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.