Contemplating using ReFS + Storage Spaces ( 24 x 4TB Drives )

G

Guldan

Weaksauce
Joined
Jan 16, 2013
Messages
113
I happen to have a 2012 R2 server attached to a couple MD1000 arrays. I dumped 24 x 4 TB drives in there with the intention of creating a large file server for my work ( we already have a few of these running RAID60 / NTFS Volumes )

I figured now that we are on R2 I would try Storage Spaces, also why not give ReFS a shot? I did some performance testing and ReFS is a performance pig however resiliency is more important for us so not a deal breaker.

So all day i've been playing with storage spaces and I was thinking of actually splitting it up between two servers. My new plan is 2 x 2012 R2 Servers with data mirrored over two physical locations (over a dark fibre 10 gig link). I am not sure however how I want to setup the storage spaces.

Right now we have a couple hardware raid60 file servers using NTFS, for our office shares I'm using DFS which I hate with a passion. If I can replace DFS with a mirrored storage space over two servers I would be extremely happy.

Anyone have advice / suggestions? Sorry for the wall of text
 
I have no advice, but I would sure love to hear your results. I am getting ready to convert my old ZFS server to a 2012 R2 using ReFS and storage spaces on my little HP Micro Server holding 4x1TB drives :)
 
What replication mode to you plan to use when you configure the 24 drives in to vDisks?

- Performance is good with "simple" spaces, but then you only have Raid0 resiliency (i.e., none at all)

- Performance is also good with "mirror" spaces (raid1).

- But write performance just stinks with Parity spaces (Raid 5) or double-partiy (raid 6)

You can make the write performance of parity spaces acceptable by adding SSDs to implement a write journal (similar to a ZFS ZiL). But unlike ZFS you need to add multiple Journal SSDs at a time - two SSDs for Parity Spaces or three SSDs for dual-parity.

Also - with Simple or Mirror spaces - you can also use SSD to implement tiered storage. The implementation in 2012 R2 is a bit primitive but it is interesting.
 
I'm not looking to buy SSDs, would probably do a mirror between the two servers.

Do you have to actually setup a replication? I figured it would do it automatically.

I've done testing and I know the write performance sucks, with NTFS raid 60 I was doing 1200MB read and 550MB write. With REFS/Storage Spaces it's 500MB / 200MB

But If I can get Self Healing / Redundant storage spaces that automatically mirror over multiple servers I might hop on board. I'll be doing testing next week with this.

Aren't Vdisks in ZFS not ReFS/NTFS?
 
Last edited:
Stay away from storage spaces. It's a nice idea, but it's way too easy to lose your data with it. There really is not enough fine-grained control available, and it just isn't widely enough documented/supported compared with existing solutions. Maybe things are better on R2, but then you're still limited by available documentation and support. I'd be very uncomfortable with trusting anything mission critical to it.
 
I think you might have some terminology and technologies mixed up a bit. Storage Spaces does not do any kind of replication on its own. It is a storage sub system that allows you to aggregate a bunch of disks into a pool and then provision volumes from the pool. In your current scenario I see the following to easily accomplish your goal . . .

1. Each site would contain a single server and an MD1000 array attached to each server.
2. A single storage pool would be created on each server with identical volumes created on each. The volume type (1, 5, 6) is up to you based on your capacity on performance needs, as is the file system used (NTFS, ReFS).
3. Each volume would then be present using SMB shares.
4. A DFS namespace would be created to represent the data structure and enabling replication on each volume to allow propagation of changes.

Being that there is a 10GB link between the two sites, I would assign one of the servers as the primary in DFS for all clients to use, regardless of which site they are at. This will all but eliminate conflicts to deal with and your link should be plenty to handle it (depending on your usage of course). This is all with the assumption that this is a basic user file share. No extra software or hardware needed as its all built into Windows Server 2012.

Should take less than a day to rack, stack, setup and provision everything. Then just robocopy everything to the primary node and let DFS-R work its magic. This is all very basic and straight-forward to setup and maintain.

Jake
 
One quick note, DFS-R is not supported when using ReFS at this time. So in this scenario you would be limited to NTFS on the volumes.
 
snakethejake said:
I think you might have some terminology and technologies mixed up a bit. Storage Spaces does not do any kind of replication on its own. It is a storage sub system that allows you to aggregate a bunch of disks into a pool and then provision volumes from the pool. In your current scenario I see the following to easily accomplish your goal . . .

1. Each site would contain a single server and an MD1000 array attached to each server.
2. A single storage pool would be created on each server with identical volumes created on each. The volume type (1, 5, 6) is up to you based on your capacity on performance needs, as is the file system used (NTFS, ReFS).
3. Each volume would then be present using SMB shares.
4. A DFS namespace would be created to represent the data structure and enabling replication on each volume to allow propagation of changes.

Being that there is a 10GB link between the two sites, I would assign one of the servers as the primary in DFS for all clients to use, regardless of which site they are at. This will all but eliminate conflicts to deal with and your link should be plenty to handle it (depending on your usage of course). This is all with the assumption that this is a basic user file share. No extra software or hardware needed as its all built into Windows Server 2012.

Should take less than a day to rack, stack, setup and provision everything. Then just robocopy everything to the primary node and let DFS-R work its magic. This is all very basic and straight-forward to setup and maintain.

Jake
Click to expand...

I'm aware Storage Spaces doesn't do replication, someone else mentioned that I think earlier.

I actually am currently using NTFS/DFS for our company file share between FS01/02 however I hate DFS and was looking for an easier solution. It's implemented exactly how you've laid it out.

I thought there was a way to possibly raid between sites, so create a pool / space between two physical locations ie FS01/FS02 12 disks each set to 2 hot spares, mirror the 10 remaining disks on each.
 
OK spent a day on this, the problem is all my file servers are direct attached storage (md1000's) and Storage Spaces / Clustering requires iSCSI / FC. So none of this fancy new technology can help my situation.

Also here are my disk latencies transferring over some giant SQL backup files over the network. Storage Spaces beat the piss out of my disks, I should note I was using a dell H810 for "hardware raid". SS Parity was with RAID0's disks (to pass through the h810)

ReFS - Storage Spaces Parity - 2000+ms
NTFS - Storage Spaces Parity - 2000+ms
NTFS - Windows Software RAID5 ~ 500ms
NTFS - Hardware RAID5 ~ 25ms
NTFS - Hardware RAID5 + SS ~ 25ms
NTFS - Hardware RAID0 (1 disk) ~ 200ms

Back to good ol' HW Raid and NTFS
 
Guldan said:
OK spent a day on this, the problem is all my file servers are direct attached storage (md1000's) and Storage Spaces / Clustering requires iSCSI / FC. So none of this fancy new technology can help my situation.

Also here are my disk latencies transferring over some giant SQL backup files over the network. Storage Spaces beat the piss out of my disks, I should note I was using a dell H810 for "hardware raid". SS Parity was with RAID0's disks (to pass through the h810)

ReFS - Storage Spaces Parity - 2000+ms
NTFS - Storage Spaces Parity - 2000+ms
NTFS - Windows Software RAID5 ~ 500ms
NTFS - Hardware RAID5 ~ 25ms
NTFS - Hardware RAID5 + SS ~ 25ms
NTFS - Hardware RAID0 (1 disk) ~ 200ms

Back to good ol' HW Raid and NTFS
Click to expand...

In light of the response to this post below, I wanted to assure you that although I'm talking about the same author as who made FlexRAID, I'm talking about a totally different produce of his called Transparent RAID.

Hi! I apologize for "necroing" this thread, but I wanted to make you all aware of something that's got all the strengths of Storage Spaces but it's major weaknesses.

To cut to the chase, and no this isn't SPAM because I have no investment into this product other than the fact that I bought it, I use it very heavily on a Server 2012 R2 environment, and I've been using it since it was beta but it's been final for about a couple of years now.

Transparent RAID, which is a subset of a more corporate-aimed future product the author is working called NZFS (Next-Generation Zion File System), available for both Windows and Linux is a very robust solution.

It's available at http://www.flexraid.com/download-try-buy-traid/.

Here's a screenshot of how I'm currently using it:

219xau9.jpg


One thing this screenshot doesn't show is an optional landing disk I have configured. You can configure either a folder on a hard drive used for other things, or dedicate an entire hard drive for doing so (it still uses a folder on it) as temporary faster space for caching any new write operations. Even if it were to fill up before emptying, it automatically spills over to the storage pool. I have my landing disk on a RAID 0 setup of two old 160 GB 7200 RPM SATA laptop hard drives.

I'll probably add a second parity drive soon. You can add parity or data drives at any time, plus they can be absolutely any size at all, with the one usual caveat that the size of the parity drive determines the maximum size used of data drives. Said the other way, the parity drive(s) must be the same size or larger than any of the data drives.

Other pros are that even, for example in my configuration, if I lose any two drives, I would still retain the information on all of the other drives and it would be usable. Unlike standard RAID, each drive can be accessed individually if so wishes, although one must be careful to do it correctly if doing so while it's also a part of the storage pool/Transparent RAID configuration.

The author designed it to be robust first. It was excruciating at first because in beta testing it was extremely slow, but he wanted reliability first since his main aim was and still is to eventually release an even heavier capability corporate product. He later added and had tested optimizations and options to speed things up. I stream video from it all the time, and also have done so while downloading to it at the same time, so it can handle quite a bit, and I haven't taken taken advantage of every speed optimization available yet.

I had used Windows 8's Storage Spaces for maybe a year or less, but once it got near full, it didn't matter how many old (read: not necessarily massive, but certainly large enough) hard drives I added, I couldn't create enough free space to use it any more, which makes sense if it's similar to normal RAID. It ended up taking close to 20 hard drives with 8 TB of combined space that only contained a little less than 4 TB of actual data, in order to make it usable enough to move all the data to (at the time) only three 3 TB hard drives.

I hope I kept this short enough and if you have any questions, don't hesitate to ask.

Cheers and good luck!
 
Last edited:
roirraWedorehT said:
Hi! I apologize for "necroing" this thread, but I wanted to make you all aware of something that's got all the strengths of Storage Spaces but it's major weaknesses.
Click to expand...

I can't believe I just read that entire post.

Everyone here surely knows about flexraid. There is no way this would be even slightly useful for the intended application. There is next to no support. You can't migrate the license from one system to another. The pooling is broken at best. It's generally just messy and certainly not for a business environment. It can work in a home environment though, where performance and reliability aren't as important.
 
S-F said:
I can't believe I just read that entire post.

Everyone here surely knows about flexraid. There is no way this would be even slightly useful for the intended application. There is next to no support. You can't migrate the license from one system to another. The pooling is broken at best. It's generally just messy and certainly not for a business environment. It can work in a home environment though, where performance and reliability aren't as important.
Click to expand...

Hello. We seem to be talking about two different things. The same author does make a product called FlexRAID, which is useful for some people, but certainly not the product I'm talking about, and FlexRAID wouldn't even be close to be a solution for me. NZFS/Transparent RAID is a completely different product than FlexRAID. What you know about FlexRAID isn't applicable to Transparent RAID.

I've been using Transparent RAID, as I detailed in my post, for three years including the time I was beta testing it and I haven't had one problem with pooling or the RAID system in general. Any minor irritation I've had was either my own fault or easily solved.

I don't want you to think Windows Storage Spaces was my first introduction to "RAID" or pooling solutions. Been using RAID 0/1/5/6 for years through other methods; hence my dissatisfaction with them as a long term solution, and how happy I was when I found Transparent RAID.

FlexRAID is the name of his "company", website and his first product. Those are the only things in common with Transparent RAID, same company and website.
 
When I said flexraid I was talking about all of his products as a whole not just his snapshot product. I've tried them all.
 
S-F said:
When I said flexraid I was talking about all of his products as a whole not just his snapshot product. I've tried them all.
Click to expand...

I won't speculate why it specifically didn't work for you. It's not the product as far as Transparent RAID hasn't gone down on me once in the last three years.

Have a good one.
 
If flexraid is so good then why aren't fortune 500 companies using it?

Would you care to address my criticisms clearly? Feel free to itemize your response.
 
It seems that Transparent RAID has some issues?
http://www.wegotserved.com/2014/08/18/review-flexraid-transparent-raid/3/
To test the robustness of the RAID recovery process I disconnected a drive while the machine was running to simulate a sudden and total failure of a drive. Transparent RAID’s response was … nothing. The web interface insisted the drive was still there, and the volume continued to operate as normal. Windows Disk Management acknowledged that the drive had disappeared, but Transparent RAID didn’t seem to get it even after I restarted the web interface.

After restarting the volume it did finally notice that the drive wasn’t available and showed an informative message letting me know about the failure. However, I was concerned that the software had nothing to say about the loss of a drive in an active, working volume.
Click to expand...
 
You must log in or register to reply here.
Back
Top