Planning my ZFS storage pool setup

M

m1abram

2[H]4U
Joined
Mar 15, 2002
Messages
3,175
So I have been researching ZFS lately and am in the planning stages of building an ESXi server with a file server host and am trying to determine the best pool setup for my file server prior to order the drives. This is primarily going to support a media server that will have large concurrent reads and writes happening. Concurrent writes come from SageTV recording up 6 HD streams at a time.
I may use the file server for storing the VM images but am not completely sold on that yet.

My capacity requirements right now are not very high, 6TB as a minimum should be enough, however I would like to be able to expand incrementally.

Even though I have offsite backups, I do want good protection from drive failure. I have worked with enough enterprise RAID 5 arrays to not trust RAID 5. More than once had a second drive failure occur during rebuild. For production servers I tend to push for RAID 10 in most cases or use RAID 6 and keep the number of devices low per RAID 6.

After reading best practices for ZFS I think I am going to go with a strip of mirrors. Now my question is should I just use 2-way mirrors or move to 3-way mirrors? If I did 2-way mirrors I would have at least 1 drive as a hotspare. My initial budget for drives is ~12 2TB drives, so if went with 2-way mirrors I would have 10TB of storage with 2 hotspares. If I went with 3-way mirrors I would have 8TB of storage.

I am kinda leaning towards 3-way mirrors, the cost per GB does not bother me as much as the risk of data loss and with 3-way mirrors I would not need the hotspares.

Does this sound like a good plan, or am I missing something?
 
Not unreasonable. Note also that 3-way mirrors will give you excellent random read performance.
 
danswartz said:
Not unreasonable. Note also that 3-way mirrors will give you excellent random read performance.
Click to expand...

Yep I did notice that, however given the SAME number of total drives would it be that much difference in random read and then also reduce the write performance since with the same number of drives you would have fewer striped sets.

12 drives 2-way with 2 hotspares = 5 - 2way mirrors
12 drives 3-way = 4 - 3way mirrors.

so if we say a 2-way mirror gives you 2x read then you would have a 10x read (note this is way simplified but using it for example) and 5x write
then 3-way with 3x read over single drive we would have a 12x read and 4x write.

Mind you still leaning towards 3-way.

Also if you drop the 2 hotspares from the 2-way, you read would be 12x. Course you would have much lower protection from data loss.
 
Well, it's up to you to decided the tradeoff of usable space vs paranoia :) I'd lean toward the 5x2 pool. Slightly worse random reads vs slightly better random writes. One thought: if you can pick up one more drive, you could go with 6x2 mirror with one hot spare.
 
danswartz said:
Well, it's up to you to decided the tradeoff of usable space vs paranoia :) I'd lean toward the 5x2 pool. Slightly worse random reads vs slightly better random writes. One thought: if you can pick up one more drive, you could go with 6x2 mirror with one hot spare.
Click to expand...

Actually I was thinking the other way, going 1 drive less and having just a single hotspare for a 5x2 mirror that has 1 drive more of storage of the 4x3 but costs 1 drive less.
But you are right go with a 6x2 with hotspare and I have 2 drives more of storage and better overall performance of a 4x3 at just the cost of 1 more drive. Course at the cost of some protection, but really not sure if that reduction in protection is that much higher if I keep a hot spare.
 
Unless this is a mission critical setup, I tend to think 2 hotspares is overkill, but that's just me.
 
danswartz said:
Unless this is a mission critical setup, I tend to think 2 hotspares is overkill, but that's just me.
Click to expand...

Well my wife uses this server and its contents a lot, and I have learned when it is not running she is not happy. Happy Wife == Happy Life :)
 
extide said:
What about RaidZ2 ?
Click to expand...

Besides the performance issue, that is mentioned above.

RAID-z is not as easy to expand the pool as mirrored is. I.E. it is highly recommended that each stripe member be the same type and size. So if you have a RAID-Z2 of 6 drives then to grow the pool you would need 6 more drives. I know technically you can mix and match but it is not recommended.
 
You must log in or register to reply here.
Back
Top