• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS vs Raid 10

M

muad_dib

n00b
Joined
Jul 6, 2012
Messages
9
My company host a web app which is, in this order: CPU-bound, Memory-bound, Disk-Bound. Disk has never been a problem, since CPU was the main factor.

We used 4 disks in RAID 10, and everything was fine. Eventually some customers requiring higher speeds were hosted on SSDs.

Recently we upgraded our servers:

i7-2600, 32 gb ram,
gigabit fiber uplink,
4 x SATA III 3 TB 7.2 rpm disks,
software raid 10
CentOS 6

Now that the cpu is much faster, the disk subsystem started to become a problem.

I thought about installing 3 SATA disk in RaidZ with an SSD cache, using ZFS on CentOS, but I'm worried that the disk gain (if any) could be offset by additional CPU consumption.

My questions are:

1) How are the performance of ZFS on linux? I heard that there was a perfomance penality compared to FreeBSD

2) Is raidz cpu intensive? Can a RAID card help me?

3) Will my performance be closer to RAID10 or native SSD?

Consider that each VM has no more than 15 gb of space, and that we host no more than 15 VMs per server, so most of the data can be held within the SSD, but the additional space of SATA drives is very welcome for local backups.

The alternative to this configuration would be:

2 x SATA in Raid 1
2 x SSDs JBOD, with periodic (12h) synch with the disk (we can loose up to 24 hours of data without a problem)

with each machine with a partition for the data, and one for the backups. Anyhow this would complicate everything, by not being transparent, by being more expensive, by having a lower perfomance overall.
 
Last edited:
You can do a setup that is almost identical to raid 10 with ZFS, you don't NEED to use ZRAID. You can setup two mirrored groups and they would then be striped. Then you can do an SSD for cache and if you want to speed up async writes a dedicated drive for intent log too.
 
extide said:
You can do a setup that is almost identical to raid 10 with ZFS, you don't NEED to use ZRAID. You can setup two mirrored groups and they would then be striped. Then you can do an SSD for cache and if you want to speed up async writes a dedicated drive for intent log too.
Click to expand...

I can have up to 4 disks per server, I guess that your suggestion requires at least 5 disks.
 
The goal is to have the redundancy of mirroring without using more than 4 drives, the speed of SSDs using only 1 SSD.

Am I dreaming? :D
 
Just mirror 2 SSD's. That will be really fast and have redundancy.
 
extide said:
Just mirror 2 SSD's. That will be really fast and have redundancy.
Click to expand...

yes, that's the fallback program.

Anyhow we can rent the disks from our DC, for a very low monthly fee, while we have to bring our own SSD.

The more SSD we have, the more likely one will fail, which is a huge cost for us in manpower.

Instead the HDDs can fail as much as they want, since the DC will replace them in less than 30 minutes (we have hot swap bays).

1 SSD + 3 ZRAID would be awesome.
 
It hugely depends on *how* and *what* your application is doing.

An SSD can only speed up, what it has cached. If you never access the same thing twice, it will never help.

If you always access the same things, it will help a lot.

If your mainly writing, it's not going to help, unless you ONLY use an ssd.

The speed of a raidz, is the same speed as using a single disk, EXCEPT when your transfering large sequential loads on it (backup datasets, iso's, movies).
 
muad_dib said:
the cool thing of ZFS is that you can use the very efficient raid 5, without its downsides.
Click to expand...

depends. if you're doing anything over the network (iscsi, nfs, cifs) then the only way you're making use of ZFS' sequential writing is by way of a screaming fast ZIL drive. without the zil you're writes over the wire will suck unless you disable all forms of sync.
 
patrickdk said:
It hugely depends on *how* and *what* your application is doing.
Click to expand...

Absolutely right.

That's why I said each VM use at most 15 gb, but usually around 8, with 15 VMs per node, thus we have 120 gb, the exact size of the SSD :D

An SSD can only speed up, what it has cached. If you never access the same thing twice, it will never help.

If you always access the same things, it will help a lot.

If your mainly writing, it's not going to help, unless you ONLY use an ssd.
Click to expand...

not even if what I do are mostly small writes?

The speed of a raidz, is the same speed as using a single disk, EXCEPT when your transfering large sequential loads on it (backup datasets, iso's, movies).
Click to expand...

exactly what I'm looking for. Speed for moving backups.

Now what drives I/O to the stars is when, in addition to usual I/O, we add backups.

patrickdk said:
It sounds like he has drive limitations, likely a 1u server with only 4 3.5" slots.
Click to expand...

Exactly. Only 4 drives have hot swap.

We can't afford to have disks on non hot-swappable bays.

madrebel said:
depends. if you're doing anything over the network (iscsi, nfs, cifs) then the only way you're making use of ZFS' sequential writing is by way of a screaming fast ZIL drive. without the zil you're writes over the wire will suck unless you disable all forms of sync.
Click to expand...

The only thing I do over the network is moving large backups, back and forward.

Should I split my SSD between ZIL and L2ARC?
 
extide said:
You can do a setup that is almost identical to raid 10 with ZFS, you don't NEED to use ZRAID. You can setup two mirrored groups and they would then be striped. Then you can do an SSD for cache and if you want to speed up async writes a dedicated drive for intent log too.
Click to expand...

Are you sure you could do that? In the ZFS Best Practices Guide (http://www.solarisinternals.com/wik...s_Guide#Additional_Cautions_for_Storage_Pools) there is a Caution that states "Do not create a storage pool that contains components from another storage pool. Deadlocks can occur in this unsupported configuration"
As far as I know, the way to create a RAID10 with ZFS is to first create 2 "mirrored" storage pools and then create a "stripe" of the 2 mirrors.
Am I missing something? I'm fairly new with this RAID stuff, so don't take my word for it.
 
kzrussian said:
Are you sure you could do that? In the ZFS Best Practices Guide (http://www.solarisinternals.com/wik...s_Guide#Additional_Cautions_for_Storage_Pools) there is a Caution that states "Do not create a storage pool that contains components from another storage pool. Deadlocks can occur in this unsupported configuration"
As far as I know, the way to create a RAID10 with ZFS is to first create 2 "mirrored" storage pools and then create a "stripe" of the 2 mirrors.
Am I missing something? I'm fairly new with this RAID stuff, so don't take my word for it.
Click to expand...

You create a new pool from one mirror (Raid-1). If you add another mirror to your pool you have a Raid 10. You can add more mirrors to increase space and performance.
In this case you have one pool build from several vdevs/ Raids . ZFS stripes automatically over all vdevs.

Problems can occur for example if you slice/partition a disk/SSD and adds the slices to several pools example to add a cache to several pools or to use a slice of the boot disc as cache for a data pool. This is not recommended
 
Last edited:
You must log in or register to reply here.
Back
Top