• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

All In One Storage Configuration

A

awesome_kyl3

n00b
Joined
Jan 21, 2014
Messages
9
So I'm in the process of building an ESXi all-in-one so I can teach myself about virtualization and various networking technologies. To save space/money for now, I'm starting with an all-in-one centered around NAS4Free with a ZFS pool. You can read about my build here: http://www.reddit.com/r/homelab/comments/1vbuzj/new_server_build/ One change I think I'm going to make is to go with napp-it instead of NAS4Free It looks like a good platform that encourages the all-in-one design.

My question is regarding my storage configuration. I can't seem to decide on how I want to split my zpool/vdevs. I have 10x1TB hard drives, and the plan right now is to make an 10 disk RAIDZ2 vdev(8 data, 2 parity). I've read about keeping the number of data drives at a power of two and to not create too large of a vddv, so it's either 4 or 8 data disks. I'm going with 8 because that'll give me 10 disks total, and my enclosure conveniently holds 20. When I want to expand, I can either replace each drive with a 2TB and/or add another 10 disk RAIDZ2 vdev.

However, I'm not sure if that's the way to go. One advantage of going with a smaller vdev is that it all fits on one HBA card, it's more secure (less disks to fail), and it's cheaper. But I'm not sure if I'm just overthinking it. Would it be better to get 2 more disks and do a 2x(4+2) setup? Are there any BKMs I should know about when deciding how to partition my storage? In the past I've never used ZFS for my primary storage, and now that I am, it's a bit daunting that once my decision is made it's "final" unless I want to start a new zpool with another configuration. I'm also not sure if it's necessary to set up a ZIL. I have 4 SSDs availible, 2x SATAII Intel X-25M 160GB and 2x Intel 250GB SSD 510 Series. Based on what I've read, for the ZIL, I want fast reliable SSDs. Would these consumer SSDs be any good at that or would I be better off skipping the ZIL until I get a dedicated SSD? Any help would be appreciated. Thanks
 
A couple of thoughts: you don't just go with napp-it. That is a gui for solaris-based platforms. You still need to decide *which* solaris derivative to use. Gea recommends (and I agree) omnios. That said, if you are going to serve up storage to virtualization guests, do not go with raidz*, as it has only 1 IOPS per vdev. I would recommend a 5x2 mirror, which will have much better random IOPS (as much as 5 for writes and 10 for reads.) Yes, you can lose two disks and lose your pool, but the odds of both drives being in the same pool is low. I would not bother with a ZIL for this, just set sync=disabled. Since it is an all-in-one, the chances of getting corruption is minimal. I would make sure to run a daily snapshot cronjob, so if you do get some corruption of a guest, you can restore it from the previous day. And, of course, make sure you are backing up unless it doesn't matter to you. If you have another couple of disks for a backup pool, you can run the zrep ksh script (just google for it.)
 
Right, I meant napp-it with OmniOS (using his preconfigured VM).

RAIDZ is really that slow? I'd considered just a bunch of mirrors, but I'm not sure it's worth losing half my storage capacity. I have a fair amount of data (mostly media) to store and the reason I was leaning towards RAIDZ2 is because I would still have protection without losing too much storage. Would it be a good idea to use RAIDZ2 with my hard drives, use the small, slow SSDs for my host storage (just holding the OmniOS/napp-it VM) and then mirroring my 250GB SSDs for VM storage?
 
No, it isn't slow for sequential throughput. The problem is that in general, a block is striped over all N drives (data not parity), so it needs to read all N for a read. This kills random I/O. Mirrors are even better for reads, since they load balance over both halves. Hell, if you want, you can have 3-way mirrors and really rock (but lose 2/3 of your space...)
 
250GB won't give you a lot for VMs, but if you don't do that much, it's probably okay. Yeah, if most of it is media stuff (particularly streaming), raidz2 is fine...
 
I think for now, 250GB should be enough. I don't have very intense VM needs (for now) and many of the things I will use them for I can use an external share to store the data on (like Windows Updates, etc.). Once I have more VMs and a need for more space, I can then upgrade my VM storage. I'm not as concerned about that because it should be simple enough to replace assuming I set up the SSDs in their own zpool.
 
Sounds cool, then. Given the large raidz2 pool, I would recommend running something like zrep hourly to replicate your VM datastore dataset to the big pool. Once the initial replication is done, it shouldn't use much if any resources after that (unless of course, you are pounding the snot out of the VM datastore with writes...)
 
250Gb shold be plenty for a home user. What large VMs would you really have, other than something like a WHS which you could just run off your ZFS array?
 
You must log in or register to reply here.
Back
Top