Parity Storage: Storage Spaces vs FlexRAID?

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Hi guys, I currently have an Amahi server using Greyhole for duplication of important files. I'd like to replace this with parity redundancy (still Just a Bunch Of Disks), which Amahi doesn't offer (also Amahi is running on the unsupported Fedora 14, which probably has all sorts of vulnerabilities now).

I've been looking into Windows Server 2012 and its Storage Spaces feature. Looks pretty good having tested it in a virtual machine but I can't test performance this way. I've heard horror stories of terrible write speeds when using parity (30 MB/s), plus a couple of benchmarks showing it's fine (80 MB/s). I've also heard of FlexRAID but haven't got any experience with that.

Basically, I'd like to know what the pros and cons are of each solution? Bare in mind that I can get Windows Server 2012 for free (student) and that my hardware is low-end (an AMD E-350, mainly for low power usage) but I do have an SSD to run the OS on.


Thanks! :D
 

kac77

2[H]4U
Joined
Dec 13, 2008
Messages
2,787
Basically, I'd like to know what the pros and cons are of each solution? Bare in mind that I can get Windows Server 2012 for free (student) and that my hardware is low-end (an AMD E-350, mainly for low power usage) but I do have an SSD to run the OS on.
Thanks! :D

If it were me I would either go the Linux or Solaris route both of them are fast and cheap to run. If you need Win 2012 for school purposes then go with that. But if not go with the aforementioned alternatives.
 

Forceman

[H]F Junkie
Joined
Aug 7, 2005
Messages
9,243
I use FlexRAID and it works well for what I use it for - bulk storage of media files and backups - and it was easy to set up. I don't use the drive pooling though, so I can't speak to that. The developer is moving on to a new project though, so I don't know how much support it'll get in the future. I had it running on an old Sempron 2800, so it doesn't take a lot of power to run.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
If it were me I would either go the Linux or Solaris route both of them are fast and cheap to run. If you need Win 2012 for school purposes then go with that. But if not go with the aforementioned alternatives.
I'd prefer Windows mainly because I am far more experienced with it and haven't had great times with Linux in the past. Also, Fedora 14 doesn't run amazingly well on the hardware either so I'm unsure whether Linux would give a speed advantage (mainly because TeamViewer runs on Wine and together they use ~50% CPU when in use - this isn't the case on Windows).

I use FlexRAID and it works well for what I use it for - bulk storage of media files and backups - and it was easy to set up. I don't use the drive pooling though, so I can't speak to that. The developer is moving on to a new project though, so I don't know how much support it'll get in the future. I had it running on an old Sempron 2800, so it doesn't take a lot of power to run.
Yeah bulk storage is what I need. The only other things the server does are downloading and running OSCam (to decrypt TV channels - yes I have a subscription ;)). My original plan was to get 2 SSDs and use one for Windows and one as a journal disk for Storage Spaces' parity mode (I doubt you can use a disk for OS and journalling). I'm just interested to see if FlexRAID would provide a better solution for a similar thing (JBOD, drive pooling, parity).
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
The biggest difference is that Storage Spaces stripes the data across the drives while FlexRAID does not. If you want that bump in read speed (ala RAID 0 or 10), you may consider Storage Spaces but run the risks of possible complete data loss if the array degrades beyond parity vs. FlexRAID's lack of speed/performance but only losing data on the individual drives if the array degrades beyond parity. Beyond that, parity is mainly designed for data that is very sporadically updated so you may not want to include OSCam recordings into your array if you are recording/deleting shows.

Will FlexRAID provide a better solution for a similar thing? No. For Windows? Yes. FlexRAID can run fine on low end hardware, and takes roughly 1 hour per TB to start the parity/validation process.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Hmm. So if FlexRAID doesn't stripe data but still does parity...does that mean that the parity drive will just be nonsense by itself but the other drives would all be readable by another PC? If so, that's a decent advantage over Storage Spaces. Are there any good benchmarks of FlexRAID working on Windows (preferably 8/2012 but 7/2008 R2 would do)? I think I'm gonna have to try both solutions to see what the performance is like. Might need to take a few days off work to do all of this testing though, lol.

By the way, OSCam would be run on the OS SSD and the recordings are currently stored on my HTPC's HDD, not on the server's storage drives. I would be looking to shift my tuners and MediaPortal TV-Server to the server eventually but the recording/timeshift folders would be on a drive that isn't in the storage pool.
 

dreamnid

Limp Gawd
Joined
Dec 15, 2002
Messages
143
Yes, you can pull the drives and reuse in another machine. Even if you lose more hard drives than what's covered by the parity (ie. lose more than 1 HD in a raid 5 setup), you can still view the files on the good hard drives.
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
Speed depends on how you set RAID via snapshot or real time (not sure if this is still in beta?). Snapshot is manually run (or set via scheduler) to see if new data was stored and to update the parity data. This does not affect performance in that parity calculations are looked at later so you it is pretty much the same as single drive performance. You do want to validate the data every so often which essentially checks every single file against parity and that takes roughly 1 hour a TB.

Real time does parity calculations in real time, but I don't use it since my data is pretty static.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
That's an interesting feature, I wasn't aware of that. I could have parity updates running each night and validation running once a week or something. Sounds like something to look into!
 

Forceman

[H]F Junkie
Joined
Aug 7, 2005
Messages
9,243
That's an interesting feature, I wasn't aware of that. I could have parity updates running each night and validation running once a week or something. Sounds like something to look into!

That's how I have FlexRAID set up - it updates parity late every night, validates it every week, and verifies it every month (or maybe validate and verify are switched, can't remember which is which).
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Is the parity feature under "Data Protection (RAID-F™)"?

Basically trying to find out whether I'd need the "Storage Pooling" or "Combo" edition to set this up.
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
There are two "different" components of FlexRAID - storage pooling and RAID. If you only purchase storage pooling, you can "pool" your drives together to show as one big drive, but there is no RAID/redundancy. If you only purchase RAID, you have parity redundancy, but you cannot pool your drives together. Buying the combo allows you to pool and use RAID.
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
+1 for FlexRAID. I used to use Linux MD RAID 5 but wanted a non-striped solution because I didn't want array complications taking down my whole array. I also really wanted drive pooling and I want to be able to easily expand my array in the future and I love that FlexRAID supports N-parity drives.

There is a cool feature that you can use in FlexRAID if you use both parity and storage pooling. His proprietary recycle bin option.

What that does is when you delete a file, it disappears, but you don't actually reclaim the space until you do an update to the array. This is important because deleting a file compromises the parity integrity. So if you delete a file and then lose a drive you would have corruption for part of the rebuild that was dependent on that missing file.

With the special recycle bin enabled and the file not actually being deleted until you perform an update this problem is no longer a problem. Your array will never be compromised then. You only stand to lose new data that you have added since the last update should you lose a drive and should that drive be the one that had the new data on it.
 

Cool1Net6

Limp Gawd
Joined
Dec 21, 2008
Messages
299
Can you run FlexRAID on a networked computer and still get the benefits of the special recycle bin? Does it handle network deletes the same way?
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Can you run FlexRAID on a networked computer and still get the benefits of the special recycle bin? Does it handle network deletes the same way?
Good question. 99% of my file operations on my file server are over my network (or VPN).
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
Yes, FlexRAID can include external hard drives, network attached storage, whatever. It essentially takes over the drive and shows the drive(s) as virtual images to the OS with FlexRAID as the front per say. Honestly, you are probably going to be better off taking this to the FlexRAID forums as there is a lot more information there.
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
Good question. 99% of my file operations on my file server are over my network (or VPN).

I am going to say yes.

Because I've taken a look into the recycle bin feature on my own server and this is how it seems to work.

When you delete a file through the storage pool, the pool service actually just moves it to .../_flxr_/d/"pathsofdeletedfile" on the drive that the file was originally on. Since file moves within the same drive do not compromise parity integrity.

Then when the update runs, it must just ignore the /d/ folder's contents and after the update was successful it then actually deletes /d/

UPDATE
I just checked this on my own sever for you guys. It definitely moves the files to _flxr_/d even when you delete it via a network share/path.
 
Last edited:

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
UPDATE
I just checked this on my own sever for you guys. It definitely moves the files to _flxr_/d even when you delete it via a network share/path.

Thanks for testing this. Are you running Linux or Windows on your server and client?
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
Windows for both. But accessing network shares is CIFS protocol either way so the client shouldn't matter.

When you share a folder you are sharing a path that is on the pool, so when the CIFS service accesses the files it's accessing them through the pool which does it's recycle bin thing.

Using linux as the server may lead to different results but I highly doubt it. I'm sure Brahim designed it to work the same way on both platforms.
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135

So you are saying I should have used MDADM with LVM?

MDADM is nice, but it only supports up to 2 parity drives in RAID 6. 2 is what I use now in FlexRAID, but I will eventually want to move to 3 and so on. Also, I like the ability to add drives of any size to my array. I don't think that's really possible with parity-based mdadm volumes.

I'm also not really a fan of striping, so going forward i'd rather try to avoid it. I like that I can use each disk independently if something happens to my array or to my server or during a rebuild. So for me it was a choice between things like unRAID, SnapRaid, and FlexRAID. I only have 1 server box and I also wanted a Windows server so rather than messing with virtual machines to run both Linux and Windows on my one box I opted for the solution that can keep me always in Windows, so FlexRAID.
 

JJ Johnson

Gawd
Joined
Jun 26, 2008
Messages
802
A couple of questions about FlexRAID with both parity and storage pooling:

- Are files ever spanned across disks, or is that explicitly avoided?

- Can parity be written on the fly, or is it strictly a process that is run manually or scheduled?
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
Files are never spanned across disks. Data disks aren't even normally written to by FlexRAID. On Windows they remain normal NTFS disks that can still work like normal disks outside of the array.

The main version of FlexRAID is snapshot so parity data is calculated and written only whenever the update process runs, either automatically or manually.

There is a real-time RAID mode but it's experimental and only supports one parity drive. It's not ready for normal use.
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
Does FlexRAID support ReFS?

No, just NTFS on Windows.

It does have some interesting features though.

For example, to combat bit-rot.

Lets say a file becomes bit-rotted. First off this would not affect the parity nor would it trigger a parity update process. This is because bit-rot would not change the file's metadata in any meaningful way (last modified date, file size, etc)

Also, FlexRAID stores checksums of every file on the array. It also has an operation called Validate. What Validate does is run through every file and verify the checksum. This Validate operation is designed to detect bit-rot because bit-rot will certainly change the checksum. Now that the Validate process will have detected the corrupted files, we can use the restore process to restore the uncorrupted version from parity since the corresponding parity data was not compromised.

There is also an operation called Verify which goes through the entire array and checks whether the parity data is bit-for-bit accurate. Neither of these operations writes to the array. They merely notify the user of a problem and where the problem is which allows the user to perform a restore to that data to correct it.
 

kac77

2[H]4U
Joined
Dec 13, 2008
Messages
2,787
So you are saying I should have used MDADM with LVM?

MDADM is nice, but it only supports up to 2 parity drives in RAID 6. 2 is what I use now in FlexRAID, but I will eventually want to move to 3 and so on. Also, I like the ability to add drives of any size to my array. I don't think that's really possible with parity-based mdadm volumes.

LVM is how you do pools. It doesn't really have anything to do with mdadm. If you need your pool to be backed up with multiple volumes you can. So the RAID 6 problem is hardly a problem because similarly to ZFS you can add many volumes to make up a pool. They can be partitions, whole disks or raid collections you choose.

So if I wanted the pool to be backed up by 4 disks of parity you would just configure the collection of disks and add them to the pool.
 

zgamer

n00b
Joined
Dec 14, 2005
Messages
7
For humor did some testing this afternoon with 2012 Essentials, all patches were downloaded included the quasi service-pack update.

esxi 5.1 host
Dual Xeon 5160
24GB ram

hardware mirror for the datastore (7200RPM 1.5TB seagate)

Test included no other VM's running

VM Specs
VM hardware version: 9
vCPU: 2
Ram: 8GB
HD: 3x RDM (vmkfstools -z) 1.5TB Seagate 7200RPM 7200.11 drives
NIC: vmxnet3

Created one storage space of 3TB with a pair of volumes attached
1st volume 2TB parity volume
2nd volume 1TB mirror volume

Upload large 2x4GB files over 1GB link via SSD on the client side

Parity upload:
25-85meg start with 90% link utilization
after about 20 seconds drops to 35meg with link utilization mostly under 25%

Mirror upload:
85meg start with 90% utilization
after 30 seconds
60-80% link utilization

--------------------

downloading:

parity
100meg after 10 seconds drops between 90 after another 10 drops to 80 and continues down ending at 50meg on a single 4gig file
climbs to

mirror
starts at 100+meg but quickly drops to 50meg and slowly declines from there
nic utilization gets a short burst of 90% utilization but stabilizes at 25%...after around 40 seconds goes back to 80% utilization


-----------

while uploading both vCPU's are pegged hard but while downloading usually they are sitting around 25%, idle is between 0 and 5%.

Overall definitely not pretty. If you don't care about performance it should be ok, but when migrating from the existing whs v1 box 10TB+, I don't want it to take a month copy. FreeNAS 8.3 looks promising but I still wish ZFS had RLM (online raid level migration) and the vmxnet3 driver for FreeNAS worked well.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Alright guys I've ordered an SSD and additional SATA controller for my file server (all 6 SATA ports full already).

Been looking into FlexRAID but was wondering...does SnapRAID work with Windows Server 2012? If so, does it work with Windows Server 2012's built-in storage pooling? What about ReFS?

Also, I assume that both with SnapRAID and FlexRAID the parity disk has to be the largest disk and outside of the storage pool?
 

JJ Johnson

Gawd
Joined
Jun 26, 2008
Messages
802
Also, I assume that both with SnapRAID and FlexRAID the parity disk has to be the largest disk and outside of the storage pool?

That's a good question. I would think it only needs to be large enough to store the parity data, which would depend on the amount of data in the array, not the disk sizes or even the potential maximum size of the array.
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
Alright guys I've ordered an SSD and additional SATA controller for my file server (all 6 SATA ports full already).

Been looking into FlexRAID but was wondering...does SnapRAID work with Windows Server 2012? If so, does it work with Windows Server 2012's built-in storage pooling? What about ReFS?

SnapRAID should work on 2012, I doubt it would work with any other storage pooling or ReFS as it takes over the drives. The purpose of keeping the drives in the NTFS format is that if the array fails, you can still access the drives data. With that being said, you really need to start going to their respective forums for this information as they are focused on their own specific products.

Also, I assume that both with SnapRAID and FlexRAID the parity disk has to be the largest disk and outside of the storage pool?[/QUOTE]

Yes, it needs to be the largest disk, e.g., 5x 1TB, 3x 2TB, 1x 4TB, parity drive needs to be the 4TB. For FlexRAID, you would need to "span" the data/DDU drives.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
SnapRAID should work on 2012, I doubt it would work with any other storage pooling or ReFS as it takes over the drives. The purpose of keeping the drives in the NTFS format is that if the array fails, you can still access the drives data.
ReFS can be read by Windows 8 or Windows Server 2012 machines, which is fine. What do you mean by "any other storage pooling"? Other than what? I thought SnapRAID only did parity snapshots and didn't have anything to do with storage pooling?

With that being said, you really need to start going to their respective forums for this information as they are focused on their own specific products.
SnapRAID doesn't really have its own forum, just a SourceForge one. :/

Yes, it needs to be the largest disk, e.g., 5x 1TB, 3x 2TB, 1x 4TB, parity drive needs to be the 4TB. For FlexRAID, you would need to "span" the data/DDU drives.
Thanks, I have 3x2 TB + 3x1.5 TB so I'll use a 2 TB. My next HDD will be a 3 TB so I'll have to use that for parity when I get it.
 

SirMaster

2[H]4U
Joined
Nov 8, 2010
Messages
2,135
You can use third party storage pooling software with SnapRAID and even with FlexRAID if you only purchase the basic version without the built-in pooling.

However note that none of the storage pooling solutions have the nice proprietary recycle bin functionality that you get when you use FlexRAID + storage pool.

See my post on page 1 explaining this feature.

All of this stuff is also on the FlexRAID wiki, but I don't mind answering any questions about FlexRAID,
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Alright guys, gonna try setting up a FlexRAID storage pool tonight and then adding a parity disk tomorrow (once my first 3 TB HDD arrives :D).

Is there a list somewhere of commonly found files that should be excluded from the parity calculations? I know for Snapshot RAID you're not supposed to include files that change regularly, so I'll exclude thumbs.db. My housemate uses a Mac and whenever he opens a folder on the server it always makes bloody .DS-Store files, so I guess I need to exclude those too. Any more?

Also, I've asked this on the official forum but haven't received a reply yet: does anyone know if FlexRAID will play nicely with Windows Server 2012's data deduplication feature?
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
Thanks for that link, I'll get him to run that on his Mac.

I've got my storage pool done but for some reason I can't get write access to one of the shares. Permissions are set correctly as far as I can see - I can write to all the shares except "Video" and I can't figure out why!

Permissions%20Issue.png


The FlexRAID forums are kinda useless, no-one's answered any of my questions yet.
 

benogil

Limp Gawd
Joined
Sep 28, 2006
Messages
143
I guess you can try 2 things, write NLS a PM on the Flex forum and ask for help, or uninstall/reinstall. Seems Brahim is MIA, probably due to writing a new driver for Win8.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
I fixed that problem by removing the shares in the Windows Management Console and starting again. Not sure why it happened or why it happened only with that share.

Most things seem to be working, creating the parity drive now, which'll take ~10 hours. I'm disappointed by the transfer speeds so far - getting ~40 MB/s reads and ~30 MB/s writes (when my server was running Fedora, I could get 50-60 MB/s reads and writes). Googled for some tips for speeding up SMB access but haven't found anything useful so far. Could try to upgrade to Windows 8 (since that should utilise SMB 3.0) but that's a bit drastic for now.
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
That doesn't seem right as it should only take 1 hour per 1TB for initial parity calculations, and when you mean transfer speeds, do you mean parity write speeds? Transfer speeds shouldn't change as it's not real-time RAID, only the update/verify/validate will check/write to the parity drive.
 

DragonQ

Limp Gawd
Joined
Mar 3, 2007
Messages
351
That doesn't seem right as it should only take 1 hour per 1TB for initial parity calculations, and when you mean transfer speeds, do you mean parity write speeds? Transfer speeds shouldn't change as it's not real-time RAID, only the update/verify/validate will check/write to the parity drive.

I have nearly 7 TB of data so 10 hours is reasonable I guess. By transfer speeds I mean over the network between my server and desktop. Those speeds were before the Snapshot RAID was set up.

Unfortunately my server crashed half way through the parity calculation so I'll have to start it again. Server Manager lists no events before Microsoft-Windows-Kernel-Power (unclean shutdown) so I have no idea what caused the crash. :mad:
 

FrostBite

[H]ard|Gawd
Joined
Jun 2, 2004
Messages
1,086
10 hours for 7TB isn't very reasonable if it is taking 3 hours or 42.86% longer; I run mostly WD Green's, arguably the slowest setup, and it doesn't take that long. As for transfer speeds, I highly doubt it has anything to do with FlexRAID, coincidence maybe, because nothing is being written or updated in regards to parity nor does it require any overhead. There could be various reasons why it runs slower - hard drive, network, computer issues.

There should be logs that you can cut and paste onto the FlexRAID forums. Brahim is still posting as I recently asked a question there and he replied.
 
Top