Parity Storage: Storage Spaces vs FlexRAID?

[kac77]

I have nearly 7 TB of data so 10 hours is reasonable I guess. By transfer speeds I mean over the network between my server and desktop. Those speeds were before the Snapshot RAID was set up.

Unfortunately my server crashed half way through the parity calculation so I'll have to start it again. Server Manager lists no events before Microsoft-Windows-Kernel-Power (unclean shutdown) so I have no idea what caused the crash.

This is getting kind of crazy. You've spent so much time trying to just come up with a system that's mediocre at best. I would seriously think about other solutions that just plain do this better.

 

[DragonQ]

10 hours for 7TB isn't very reasonable if it is taking 3 hours or 42.86% longer; I run mostly WD Green's, arguably the slowest setup, and it doesn't take that long. As for transfer speeds, I highly doubt it has anything to do with FlexRAID, coincidence maybe, because nothing is being written or updated in regards to parity nor does it require any overhead. There could be various reasons why it runs slower - hard drive, network, computer issues.

I never said the slow transfer speeds had anything to do with FlexRAID. There must be a reason though...

This is getting kind of crazy. You've spent so much time trying to just come up with a system that's mediocre at best. I would seriously think about other solutions that just plain do this better.

Like what?

 

[kac77]

Like what?

Well there's ZFS and there's Linux both of those are faster and not too difficult to setup. Definitely easier than what you've been through thus far.

 

[JJ Johnson]

Wouldn't the storage pooling potentially affect transfer speed? When using pooling, don't all file operations have to go through the pooling service?

 

[FrostBite]

Wouldn't the storage pooling potentially affect transfer speed? When using pooling, don't all file operations have to go through the pooling service?

Not really because data isn't striped (which would actually speed it up) nor is FlexRAID doing anything beyond choosing the drive with space and directing it there. You can choose to have FlexRAID auto balance your hard drives, but even so, there is little to no overhead. I haven't seen any decrease in transfer speeds when switching over, and I've increased it by adjusting network settings.

As for ZFS, not an ideal choice in this situation if you are thinking lower end. Beyond hardware, a low amount of drives and adding single drives is not going to work. If you were going with 5+ drives and were looking to expand at a good pace, ZFS is a good choice. Just not in this case. Also, if you don't have Unix (ZFS hasn't been fully ported yet?), experience difficulty goes through the roof; good reason to learn *nix but will make a new install much harder. _Gea does a great job here with help though.

 

[kac77]

As for ZFS, not an ideal choice in this situation if you are thinking lower end. Beyond hardware, a low amount of drives and adding single drives is not going to work. If you were going with 5+ drives and were looking to expand at a good pace, ZFS is a good choice. Just not in this case. Also, if you don't have Unix (ZFS hasn't been fully ported yet?), experience difficulty goes through the roof; good reason to learn *nix but will make a new install much harder. _Gea does a great job here with help though.

There's still the Linux option and here you can add 1 drive at a time and it doesn't cost anything to do. In 2 weeks the average person could have stood up their file server and moved on to something else.

 

[DragonQ]

My file server used to use Linux, I saw no advantages over Windows and it pissed me off how difficult some things were to do. On top of that, Greyhole (used for duplicating files in certain shares) wasn't as efficient as parity, the power savings weren't that good because the drives never seemed to be spun down even though I set them to (probably to do with Greyhole), plus the Linux version of TeamViewer runs through Wine, meaning it ate up ~50% of the server's CPU when just having a connection open (compared to ~5% now with Windows). I can also run uTorrent on Windows, which is better than Transmission IMO.

I could be totally wrong but it is my understanding that ZFS is more designed for NAS devices? In addition, it doesn't run on Linux or Windows, AFAIK, which rules it out since I need to run OSCam and be able to actually know how to do stuff.

Anyway, parity calculation took 8h40m at 55 MB/s and everything seems to be up and running now. Will test transfer speeds again tonight to see if they've improved now the server has "calmed down" p) and also will set "Quick Validate" to run every night during a few days of normal use to see if there are any often-changing files that I've forgotten to exclude.

 

[kac77]

My file server used to use Linux, I saw no advantages over Windows and it pissed me off how difficult some things were to do. On top of that, Greyhole (used for duplicating files in certain shares) wasn't as efficient as parity, the power savings weren't that good because the drives never seemed to be spun down even though I set them to (probably to do with Greyhole), plus the Linux version of TeamViewer runs through Wine, meaning it ate up ~50% of the server's CPU when just having a connection open (compared to ~5% now with Windows). I can also run uTorrent on Windows, which is better than Transmission IMO.

I could be totally wrong but it is my understanding that ZFS is more designed for NAS devices? In addition, it doesn't run on Linux or Windows, AFAIK, which rules it out since I need to run OSCam and be able to actually know how to do stuff.

Anyway, parity calculation took 8h40m at 55 MB/s and everything seems to be up and running now. Will test transfer speeds again tonight to see if they've improved now the server has "calmed down" p) and also will set "Quick Validate" to run every night during a few days of normal use to see if there are any often-changing files that I've forgotten to exclude.

I don't think I've ever seen so many excuses in my life. MDADM is quicker than Windows and offers more in the way of parity than Windows does out of the box. Why use Greyhole? It's not needed. There's a million and one alternatives to Team Viewer also not needed. And uTorrent? I can't remember the last time a torrent manager made me spend $50 and 2 weeks + 8 hours calculating parity. Giving that as a reason is equally lame (aside from the point there's more than just Transmission to choose from).

I get the whole different strokes for different folks bit. I really do. But 2 weeks to get parity and drive pooling is ridiculous.

 

[DragonQ]

I don't think I've ever seen so many excuses in my life. MDADM is quicker than Windows and offers more in the way of parity than Windows does out of the box.

Never heard of it. Does it offer RAID-over-file-system like FlexRAID and SnapRAID do? I quite like the idea of being able to read the data drives in any PC.

Why use Greyhole? It's not needed.

I know it's not needed, it's what Amahi uses and Amahi (Fedora-based) was what my file server used to run.

There's a million and one alternatives to Team Viewer also not needed.

I've tried a few (years ago) and TeamViewer was my choice. Should I change that on all of my other machines and all of my family members' machines so I can stick with Linux (which I don't really like)?

And uTorrent? I can't remember the last time a torrent manager made me spend $50 and 2 weeks + 8 hours calculating parity. Giving that as a reason is equally lame (aside from the point there's more than just Transmission to choose from).

Didn't realise 8 hours calculating parity was a big deal. I also haven't fully committed to FlexRAID yet, I can pretty easily switch to SnapRAID if I want to. Storage pooling is easy and fast, as are shares. It's just the parity calculation that takes a while.

I get the whole different strokes for different folks bit. I really do. But 2 weeks to get parity and drive pooling is ridiculous.

Two weeks?? Where are you getting that from. I started inquiring 2 weeks ago cos I was planing ahead. It took 2 days to do all the hardware and software setup, whilst doing various other things.

 

[odditory]

I get the whole different strokes for different folks bit. I really do. But 2 weeks to get parity and drive pooling is ridiculous.

It takes 15 seconds to pool and parity a set of JBODs with FlexRAID or DriveBender+SnapRAID, so obviously the guy was having some unrelated complications.

 

[kac77]

Never heard of it. Does it offer RAID-over-file-system like FlexRAID and SnapRAID do? I quite like the idea of being able to read the data drives in any PC.

I think you have. If you were using Fedora I'm pretty sure you know the answer to this.

I've tried a few (years ago) and TeamViewer was my choice. Should I change that on all of my other machines and all of my family members' machines so I can stick with Linux (which I don't really like)?

It's obvious now that you don't like Linux. However, coming up with a remote access scheme isn't laborious.

Didn't realise 8 hours calculating parity was a big deal. I also haven't fully committed to FlexRAID yet, I can pretty easily switch to SnapRAID if I want to. Storage pooling is easy and fast, as are shares. It's just the parity calculation that takes a while.

Yes 8 hours is a long time when anything else would have done it quicker. I think you also spent more than 8 hours because of the crash earlier.

Two weeks?? Where are you getting that from. I started inquiring 2 weeks ago cos I was planing ahead. It took 2 days to do all the hardware and software setup, whilst doing various other things.

The first post was 2 weeks ago. I'm not saying that FlexRaid or even Snap is bad. But the crash you experienced yesterday is more time then I've ever spent standing up pooling and parity.

 

[kac77]

It takes 15 seconds to pool and parity a set of JBODs with FlexRAID or DriveBender+SnapRAID, so obviously the guy was having some unrelated complications.

That's precisely the point.

 

[DragonQ]

Yes 8 hours is a long time when anything else would have done it quicker. I think you also spent more than 8 hours because of the crash earlier.

Yes, it took 8 hours...whilst I was asleep. It's not like I'm calculating the parity by hand, lol. I don't understand the problem here, and I don't understand how you can calculate parity for 7 TiB of data in 15 seconds.

However, coming up with a remote access scheme isn't laborious.

Sounds like you don't have distant family members who are novices at PCs and wouldn't even know how to install programs. Yes, I could spend time on all of my PCs changing the remote access software I use but that's the easy part. Plus, like I said, TeamViewer was the best that I found at the time.

The first post was 2 weeks ago.

Yes, and it's obvious you haven't been reading them if you think it took me 2 weeks to do this.

I also like how, having used Fedora for a bit, I must automatically know about MDADM.

Sorry but you sound like the stereotypical Linux user that puts me off the OS.

 

[odditory]

@DragonQ: ZFS is great in many respects but its more "enthusiastic" fans have a bad habit of swooping into threads claiming it as the be-all, end-all yet almost always leave out the downsides and gotchas, of which there are considerable (example, no one-disk-at-a-time capacity expansion without a reformat). There are other important factors to consider when choosing a filesystem and data protection strategy: #1, Usage scenario - in your case, like most people on this forum, storing home media. #2, comfort level - in your case you're familiar with Windows and may not want to deal with the learning curve of a brand new operating system just to store some files. Granted, there are excellent free WebGUI's like Gea's NAPP-IT that allow you not to have to deal with command line, but if and when problems arise, you'll be sweating at the console in unfamiliar territory.

Striping systems like ZFS or hardware RAID aren't where things are headed for home media storage because they add an abstraction layer and unnecessary layer of complexity and risk versus pooled JBOD with parity or duplication which in my mind is really where things are headed for home storage in the next few years, and take that from someone standardized on hardware RAID almost exclusively for the past 15 years storing home media. Striping is mostly a performance multiplier, but goes to waste in the average home media storage scenario since a single disk already saturates a GigE link. RAID is an uptime multiplier, which is also less important in a home setting since time to recovery typically isn't as much of an issue as say a business setting. With few exceptions, the upsides of striped RAID in the context of home storage and streaming movies to a few TV's in the house don't outweigh the downsides anymore as software based JBOD pooling/parity/duplication solutions continue to mature.

Its good to learn about all these systems so you can make the best most informed decision for your storage strategy going forward, learning the strengths and weaknesses of each. Unfortunately its more than I can detail here but plenty of info to be found on this forum and elsewhere. And important to remember: THERE IS NO PERFECT SYSTEM. I've tested them all extensively, and nothing right now is without drawbacks or compromise.

One last bit of advice: Stay away from Windows 8/Server 2012 storage spaces for the next 6-12 months if not SP1, Microsoft's got a long road ahead with this disaster of a system and they'll either fix it or remove it as they have with their previous attempts at native pooling. This was a clearcut case of not wanting a broken feature holding up general release of the O/S, and deciding to "fix it in post" rather than remove since it was a feature already advertised as included. Use with test data ONLY.

 

[DragonQ]

One last bit of advice: Stay away from Windows 8/Server 2012 storage spaces for the next 6-12 months if not SP1, Microsoft's got a long road ahead with this disaster of a system and they'll either fix it or remove it as they have with their previous attempts at native pooling. This was a clearcut case of not wanting a broken feature holding up general release of the O/S, and deciding to "fix it in post" rather than remove since it was a feature already advertised as included. Use with test data ONLY.

There were a few reasons I decided against it:

- As you said, it's new and could have problems.
- Real-time parity is probably overkill for my data (mostly music and video) and would likely slow down file transfers since the server's AMD E-350 isn't that powerful (chosen a year ago for low power usage).
- Striped data basically ties me into the system, just like hardware RAID (if Microsoft decides to remove the feature in future versions of Windows then I'm stuck and can't access the data in other PCs).

 

[kac77]

I don't understand how you can calculate parity for 7 TiB of data in 15 seconds.

I can tell you ZFS took about 45 minutes to do half that.

Sounds like you don't have distant family members who are novices at PCs and wouldn't even know how to install programs. Yes, I could spend time on all of my PCs changing the remote access software I use but that's the easy part. Plus, like I said, TeamViewer was the best that I found at the time.

I actually do and I've been using any one of the cloud based solutions that don't require anything more than java or .net.

Yes, and it's obvious you haven't been reading them if you think it took me 2 weeks to do this.

Oh I read what you had to say. I didn't say anything until the first attempt crashed out.

I also like how, having used Fedora for a bit, I must automatically know about MDADM.
Sorry but you sound like the stereotypical Linux user that puts me off the OS.

Because if you got Fedora installed (a Red Hat distro) Amahi installed, Greyhole installed, Wine installed along with Team Viewer at some point the inability to do find out what mdadm was seems highly unlikely. You would almost have to skip the links on Amahi's own site that talks about it.

If you don't like Linux then say so. But don't create excuses that have solutions.

 

[DragonQ]

I can tell you ZFS took about 45 minutes to do half that.

Really? Surely that'd require average reads of 1.3 GB/s. How could I achieve such speeds with 6 "green" HDDs? In any case, 45 minutes is not 15 seconds.

I actually do and I've been using any one of the cloud based solutions that don't require anything more than java or .net.

Examples? I'd like to know if any of these offer TeamViewer-like functionality.

Oh I read what you had to say. I didn't say anything until the first attempt crashed out.

So how did you come to the conclusion that it took me two weeks to create a storage pool and calculate parity?

Because if you got Fedora installed (a Red Hat distro) Amahi installed, Greyhole installed, Wine installed along with Team Viewer at some point the inability to do find out what mdadm was seems highly unlikely. You would almost have to skip the links on Amahi's own site that talks about it.

Oh right so if I set up my operating system, I should then magically come up with a 5 letter acronym and google it to find out what it is? Why would I have a desire to find out about something that I don't even know exists?

I got Amahi installed with Greyhole functionality using their installation guide, which does not mention RAID or mdadm. Even the more detailed version doesn't mention these.

I also don't like the fact that the only stable build of Amahi for months was one that ran on an unsupported OS that no longer received security patches. The guides still recommend that people use this OS . I like what Amahi is trying to do but it seems like they don't have enough developers to keep things moving forward.

If you don't like Linux then say so. But don't create excuses that have solutions.

I did say so and gave some of my reasoning. If they amount to "excuses" for you (which assumes that everyone should be using Linux unless they can come up with a good reason not to), then so be it. Again, the "haha noob you don't use Linux??" vibe is pretty strong here.

 

[SirMaster]

Yeah, your performance isn't quite up to what it should be. I'm using older 2TB 5400RPM Samsung drives and parity calculation runs at ~80MB/s.

Reads and writes to the array via the network are about ~100MB/s and moves between drives on the array are about ~120MB/s

ZFS is not always the better solution for everyone. I had specific requirements that ZFS couldn't meet, so I went with a product that could meet them. That was FlexRAID. I couldn't be happier with how it's been going for these few months so far. It does everything I want it to do and it does it well, fast and easily.

 

[ziggo0]

I've been using MDADM for years. It's fast and effective. You can set up ssmtp or some other mailer and have mdadm when a drive has failed/dropped from the array. You can expand raid5 and raid6 arrays to be larger, you can convert a raid5 array into raid6. You can shrink arrays (currently experimental/not recommended) - arrays can take quite a bit of abuse. Lots of options/choices to choose from when creating the array, recovery and the array is moveable from comp to another comp, you just need mdadm. I currently have 6x2TB WD Green drives in raid5. They put out 450MB/s ~ 500MB/s in read speed and 350MB/s of write speed (generally large bulk files, I don't use the array for small stuff). The rig it's on is a cheap AMD A4-3300 dual core, AM3 mobo, cheap gigabit intel nic - I easily have multiple users reading files and writing with no issue.

Cons as you would like, it can be annoying/aggravating. There is lots of information available on the web but it's quite scattered, sometimes what you would think is a small fix can be very complicated. It's come a long way since then, but a recommendation would be to stay on stable versions of the software...an unstable version caused me to lose a whole raid6 array once, live/learn I guess.

Recovery times are pretty nice...if a drive craps, put the new one in, add it - takes about 300 minutes/6 hours to replace a 2tb drive in my array.

 

[DragonQ]

Yeah, your performance isn't quite up to what it should be. I'm using older 2TB 5400RPM Samsung drives and parity calculation runs at ~80MB/s.

Reads and writes to the array via the network are about ~100MB/s and moves between drives on the array are about ~120MB/s

ZFS is not always the better solution for everyone. I had specific requirements that ZFS couldn't meet, so I went with a product that could meet them. That was FlexRAID. I couldn't be happier with how it's been going for these few months so far. It does everything I want it to do and it does it well, fast and easily.

Just ran some more tests and got ~75 MB/s writes to the server and ~55 MB/s reads, which is better than before.

Moving files on the server approaches 100 MB/s, which is far better than the 5-10 MB/s I got with Amahi.

 

[odditory]

Realtek or Intel NICs at either the server or client end? Realtek is not your friend when it comes to throughput.

 

[DragonQ]

Marvell on my desktop, Realtek on server. Both have latest drivers from manufacturer.

 

[kac77]

I did say so and gave some of my reasoning. If they amount to "excuses" for you (which assumes that everyone should be using Linux unless they can come up with a good reason not to), then so be it. Again, the "haha noob you don't use Linux??" vibe is pretty strong here.

I don't think I'm making any kind of recommendation that everyone should use anything. I said there were alternatives. You can either try them or not. No one has a gun to your head.

 

[ziggo0]

Realtek or Intel NICs at either the server or client end? Realtek is not your friend when it comes to throughput.

With as cheap as Intel NICs are I don't see why not. I have an Intel in my file server, all the Realtek sets are limited right around 940mbps.

 

[DragonQ]

Took a bit of research but finally got a MPPE 128-bit VPN set up (with my Dynamic DNS host) and I get 1.5 MB/s transfers over it. Probably as high as can be expected with an upload speed of ~15 Mbps at home.

I also did a bit of rebalancing on the storage pool for greater response and energy efficiency (all shares, except the huge Video one, are on a single drive now). I think that's probably everything now....although I'm sure I'll think of something else to set up when I find myself with some more free time.

 

[DragonQ]

Bit of an update: things are working well so far. I tried Liquesce today but it didn't seem to work properly. I could create the pool but renaming a file would produce an error saying that the parent directory didn't exist and would I like to create it. If I said "yes", the error would simply repeat. My programs also had trouble accessing the data on the pool and none of the shares were accessible, even though the Liquesce management window seemed to show them correctly.

Dokan, which Liquesce uses, doesn't officially support Windows 8 / Windows Server 2012 yet so that might have something to do with it.

 

[JJ Johnson]

Using FlexRAID's storage pooling with snapshot RAID, if you were to pool several SSDs, is there any reason why the parity disk couldn't be a hard drive?