Windows 8 Storage Spaces

[octoberasian]

I posted this link in the Frontpage news, but looks like it should be here:
http://blogs.msdn.com/b/b8/archive/...rage-for-scale-resiliency-and-efficiency.aspx

It's called Windows Storage Spaces. It's a lot like an enhanced version of Windows Drive Extender from Windows Home Server, now available in Windows 8.

From Engadget:

"This time it's "Storage Spaces," which will let you pool both virtual and physical drives (regardless of how they're connected) in a variety of ways to keep your data as safe as possible. That includes the ability to expand individual storage pools on an as-needed basis, and various ways to mirror your data for some added redundancy -- as you'd expect, the "spaces" themselves simply behave as a regular disk."​

 

[lordsegan]

Wow..

Also, what is this new REFS file system? Is that going to be anywhere near as good as ZFS?

 

[Ryan711]

Very glad to see that this is for home users as well as enterprise. It said in the blog post that they've scaled it with hundreds of disks and there is no limit. Better than WHS DE too because it is more closely integrated with the FS and has parity built in instead of just mirroring.

 

[dustNbone]

Sounds a whole lot like ZFS. Someone should slap a frivolous patent suit on their asses for a change.

 

[devil22]

Sounds a whole lot like ZFS. Someone should slap a frivolous patent suit on their asses for a change.

For software RAID? And might I remind you that Sun got their asses slapped with a suit for ripping stuff in ZFS off other companies.

 

[dastral]

Resiliency through parity

There’s another resiliency attribute, called parity, which directs Storage Spaces to store some redundancy information alongside user data contained within the space, thereby enabling automatic data reconstruction in the event of physical disk failure. While conceptually similar to mirroring, parity-based resiliency utilizes capacity more efficiently than mirrored spaces do, but with higher random I/O overhead. Parity spaces are well suited for storing data such as large home videos, which have large capacity requirements, large sequential (predominantly append) write requests, and an infrequent-to-minimal need to update existing content.

Akin to mirrored spaces, slabs for parity spaces are strewn across available disks (with capacity utilized for parity information) as shown below for a parity space contained within a six-disk pool:

Parity space in a six-disk pool

When a disk fails, the parity space recovers equally transparently and automatically as does the mirrored space. For parity spaces, Storage Spaces utilizes the parity information to reconstruct affected slabs for all affected spaces, and then automatically reallocates the slab to utilize any available hot-spare disk or any other suitable disk within the pool (just as it does for mirrored spaces)

The illustration below shows two spaces – one with mirrored resiliency and the other with parity resiliency – carved out from the same pool:

Q) Are Storage Spaces some kind of RAID? If it is, what RAID versions do you implement?

Fundamentally, Storage Spaces virtualizes storage in order to be able to deliver a multitude of capabilities in a cost-effective and easy-to-use manner. Storage Spaces delivers resiliency to physical disk (and other similar) failures by maintaining multiple copies of data. To maximize performance, Storage Spaces always stripes data across multiple physical disks. While the RAID concepts of mirroring and striping are used within Storage Spaces, the implementation is optimized for minimized user complexity, maximized flexibility in physical disk utilization and allocation, and fast recovery from physical disk failures. Given these significant differences in objectives and implementation between Storage Spaces and traditional inflexible RAID implementations, the RAID nomenclature is not used by Storage Spaces.

So they will be doing a RAID5/6 (stripped parity) but not calling it a Raid5/6.

Man i was SOOOOOOO hoping see a FlexRaid/Unraid native in windows.
X+1 Drives, with the +1 used as parity, and the X "pooled" together. Halas fail

 

[Valaire]

Yeah, I was hoping they'd be doing a native Unraid too (w/o striping).

 

[Trepidati0n]

Decent information on ReFS. Yes...it looks like ZFS. However, the big thing here is that it is integrated into a Windowss OS. This will save a lot of admins time/money.

http://blogs.msdn.com/b/b8/archive/...-generation-file-system-for-windows-refs.aspx

 

[Xon]

ReFS does look interesting, but it is missing a number of features which is why it isn't a bootable filesystem yet. Storage spaces's Parity Spaces is software raid5 on a thin provisioned volumn, great that it doesn't need to rebuild empty space and uses 'trim' commands to free diskspace back to the pool. There is also information that it has an on-disk journal to handle the raid 5 write hole. There is also information which suggests you can define where the journal is, and you can manually pick which disks the files are actually stored on.

While ReFS does do checksumming, and when using mirrors is able todo automatic recovery, with parity spaces ReFS & Storage Spaces does not appear able to correctly determine if the data or parity has been corrupted by a soft error (aka bit flip).

Keep in mind this is a v1.0 release fro Microsoft, and while they are promising it will be production ready it isn't going to be utterly feature complete.

 

[brutalizer]

It seems ReFS has basic checksum functionality of metadata, but the data itself is not checksummed. Thus, ReFS is not that safe. But way safer than NTFS.

It seems that ReFS is a ZFS wannabe, just as BTRFS is.

 

[devil22]

It seems ReFS has basic checksum functionality of metadata, but the data itself is not checksummed. Thus, ReFS is not that safe. But way safer than NTFS.

It seems that ReFS is a ZFS wannabe, just as BTRFS is.

Says right on the blog:

In addition, we have added an option where the contents of a file are check-summed as well. When this option, known as “integrity streams,” is enabled, ReFS always writes the file changes to a location different from the original one. This allocate-on-write technique ensures that pre-existing data is not lost due to the new write. The checksum update is done atomically with the data write, so that if power is lost during the write, we always have a consistently verifiable version of the file available whereby corruptions can be detected authoritatively.

 

[Xon]

ReFS has checksumming on metadata and files.

The problem is it can only actually correct these reliably when using Mirrors (aka Raid1),

Q) What about parity spaces and ReFS?
ReFS is supported on the fault resiliency options provided by Storage Spaces. In Windows Server 8, automatic data correction is implemented for mirrored spaces only.

It would be great to be proven wrong, but the initial release can not reliably correct soft errors on Parity Spaces (aka raid5) despite having checksums protecting metadata and being able to protect filedata with checksums.

At least, for what it is worth, it will be able to tell you when it messes up and the parity was wrong and trashes the actual data during a scrub.

 

[Trepidati0n]

Right now, even if ReFS doesn't do "RAID5", it does not bother me at all as a home user nor would it matter to a small business. I've been effectively running RAID1 via WHS v1 for years now. The cost NEVER bothered me. What did bother me was corruption during write and bit-rot. This was thee and only reason I was consideirng moving to ZFS. However, this means I would need two PC's, one running ZFS and the other running WHS v1/v2 for machine backups. I know I could go with a bare-metal virtual setup..but that means not only would I have to become very comfortasble with *nix, but also dealing with the virtualizaiton and the increased cost of HW.

Having something like ReFS running on WHS v2 would pretty much make my life SIMPLE.

---

I think a lot of people here are right...ReFS is not mature; ZFS is also superior. However, in this case for me if I had to choose between dealing with *nix and windows..I know which one I will take at this point in my life.

 

[mikesm]

The 2 big things that ZFS is missing compared to (I think) what Windows 8 server will have:

1) Easy expansion of storage - I can't take an 8 disk raidz2 array and just add one more disk to add space . I have to add disks in chunks of new arrays, and when they are created, I can't change the number of drives in that array without destroying it and recreating it.

This is a big problem for simple administration, and even my adaptec controller can do this kind of expansion (though slowly). There is no reason this should still be an issue with ZFS - the physics of the system does not prevent this feature from being added. It's just that no one has done it.

2) Management and setup experience. This still has a lot of sharp edges on it, and even harder if you have a lot of disks in JBOD enclosures, where I can't light lights on the enclosure to find a bad disk, etc... Napp-it is nice, but it doesn't deal with the labeling issues of the disk setup, enclosure management, and system install and startup, etc...

To see where MSFT is going here, look at this video starting at around 20 mins in: http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-449T

Oracle has been sitting on their ass for a long time, and MSFT sees an opening and is going for it. Even if you don't run windows, this is a good thing as it forces others to keep up. And MSFT will do wonders in getting vendors to proliferate JBOD's and expanders as a normal config because they make it a lot easier to manage.

Competition is a good thing.