Help Planning a New ZFS Build

Jesse B

[H]ard|Gawd
Joined
May 30, 2010
Messages
1,631
I'm working on slowly building up a storage server, and planning on running ZFSGuru. I have some pre-exisiting hardware that I'd like to use, and just make a few changes. My current hardware is as follows:

AMD Athlon II X2 245 @ 2.9GHz
MSI 785GM-E65
CORSAIR CMPSU-400CX
OCZ 2GB DDR3 1800 (PC3 14400)

That's all thrown in a simple tower at the time being. Eventually I plan on putting it in a rackmount enclosure, and putting it in a cabinet along with a couple other things.

First things first, I'm going to throw in either 4GB or 8GB of RAM (Opinions on how much is really necessary?), as I've been told numerous times that ZFS likes its RAM. I plan on dropping the voltage a tad, and maybe underclocking the CPU a bit, just to cut down on power usage. I'm also going to throw in an Intel Gbit NIC.

Next, I'll have to do something about the storage. I'm kind of torn between a few options at this point in time, so I'm looking for some input. I'm looking for decent performance (enough to hopefully saturate my Gbit network, ideally, but it's not a huge deal as long as I still have good performance), while also having some redundancy. RAID-Z seems like a good option. The setups I'm looking at are as follows:


Samsung F4 320GB x 4 in RAID-Z
Total Usable Space: 960GB
Cost: $175.96
(Score: 953)

Samsung F3 500GB x 3 in RAID-Z
Total Usable Space: 1000GB
Cost: $164.97
(Score: 887)

Sam. F4 2TB x2 in MIRROR
Total Usable Space: 2000GB
Cost: $199.98
(Score: 881)

The scores are just benchmark ratings off some random website, just to help me compare performance between drives. The first option is the fastest drive of the 3, but the smallest, forcing me to use more disks. The second option is a good middle ground, giving good performance and a good amount of storage. The last option gives lots of storage, but less performance. I don't really need a whole lot of storage, but I'd like enough that I don't need to upgrade in the near future. At the time being, my data collection is only about 300GB, but as mentioned, I'd like to leave some room to grow (which it will with time).

Another thing is what's happening with the ZFSGuru OS itself. I want to do a proper install rather than running a LiveCD. As far as I know I'm able to run it off of a USB drive, or install it onto the ZFS pool itself. I'm not sure if this is the case, so correct me if I'm wrong. Is there an advantage to one or the other? Also, I've seen people add SSD's as a cache to improve performance. What kind of storage and performance does the SSD need to have? Am I able to add this after the fact, or does it need to be done during the initial setup?

So that's pretty much all I have at this point in time, although I'm sure I'll come up with more questions. If anybody has any advice pertaining to this build in general, I'd love to hear it. I'm looking forward to getting this thing up and running.

Thanks,


- Jesse
 
Personally I wouldn't consider using anything less than 1tb drives if you're buying new, even then it makes more sense to go with 2tb considering the prices of each. Why have more drives spinning than necessary when 2tb units are so cheap?

As soon as I can bring myself to spend some money I plan on moving my nas box over to a zfs setup. I already have a 1tb green but I plan on moving to 2tb drives. 15 bucks for twice the space seems like a no brainer to me. If you have the space you can always find something to fill it up with. ;)
 
Ram: Given your build 4GB will be plenty.
Raid type: Generally the raid type is dictated by your needs.
Cache device: Yes, you can add this after the fact. You can also remove it at any time as well.
 
I have to agree with PointandClick. Use the two 2TB F4's for the build. You can mirror them to start or just grab another and RAIDZ it. Your performance will be far above the other two configurations you have (in RAIDZ). Or if you don't want to buy another F4 then you should just mirror them for now and then RAIDZ them when you get a chance.
 
Indeed - do buy the F4 2TB; being 5400rpm they give you high throughput and alot of storage at a very low price and has a low idle power consumption. These are the drives you would want for a new ZFS system i think! You do want to flash them with new firmware to fix a corruption issue, though.

As for ZFSguru; the LiveCD is just a temporary medium; you should perform a Root-on-ZFS installation; either on a USB pendrive or to the pool itself. Many people ask about performance impacts; there are NONE! So probably, there is no real reason to dedicate one HDD to the system disk, as conventionally would be the preferred choice, when using ZFS.

The only downside of installing to your pool is that you cannot use the 4K sector override feature. So if you want to use that feature (for example when doing a 4-disk RAID-Z) then install Root-on-ZFS to a USB pendrive instead.


Losing your system disk
While many people feel an urge to protect the system disk against crash/failure, i tried to design my distro so that losing your system disk shouldn't be bad! Just grab your livecd and reinstall, import your pool and voila! The only thing missing to make that work is that currently you can't export the Samba configuration and other configuration files; a feature that i will add in next release.

So with that feature, if your USB stick dies, you would have an easy time:
- grab your livecd, boot your system with it
- put in a new pendrive, format it GPT, create a pool on it and install Root-on-ZFS to it
- boot from the new pendrive
- now you can import configuration settings like Samba configuration via the newly added System->Import; this configuration file is an archive of many config files, packed together in a single file that you can keep on your desktop computer as backup, in case you need it again.
- now restart the services and you should be up and running as per before the 'crash'

Your data on the ZFS pool is never harmed as a result of losing your system disk. You need nothing from the system disk to access that data.

Also note that underclocking AMD means CnQ won't work, resulting in much higher idle power consumption; not what you want on a NAS! This method only reduces stressed power consumption a bit; which is not useful at all. Let CnQ do its job! And RAM... well i wouldn't buy anything beyond 1.5V; and 1.25V and 1.35V are also available (Kingston LoVo). Always buy sticks of 4GiB; not 2 or 1, this would allow you to upgrade your RAM should you need to. Note that RAM quantity is far more important than RAM speed.

4 or 8GiB? I would prefer 8 and this would help on writing particularly. Since DDR3 RAM is very cheap now, i recommend to buy two sticks of 4GiB each; giving you 8GiB total. Perhaps you want to scout for ECC pricing as well; currently the ECC memory is 30% more expensive than normal DDR3 in my locale. Since you use AMD this gives you cheap access to ECC, if your motherboard supports it (most AMD boards do actually including cheap Micro-ATX boards). But if you want to cut down on cost, 4GiB would work as well for just a few disks; but if prices rise again in Q2/Q3 then you could lose the best moment to buy DDR3 RAM. Due to oversupply and low Q1 demands, the DRAM prices have fallen considerably, making this the ideal moment to buy alot of memory.
 
Losing your system disk
While many people feel an urge to protect the system disk against crash/failure, i tried to design my distro so that losing your system disk shouldn't be bad! Just grab your livecd and reinstall, import your pool and voila! The only thing missing to make that work is that currently you can't export the Samba configuration and other configuration files; a feature that i will add in next release.

So with that feature, if your USB stick dies, you would have an easy time:
- grab your livecd, boot your system with it
- put in a new pendrive, format it GPT, create a pool on it and install Root-on-ZFS to it
- boot from the new pendrive
- now you can import configuration settings like Samba configuration via the newly added System->Import; this configuration file is an archive of many config files, packed together in a single file that you can keep on your desktop computer as backup, in case you need it again.
- now restart the services and you should be up and running as per before the 'crash'

Your data on the ZFS pool is never harmed as a result of losing your system disk. You need nothing from the system disk to access that data.

Dear Sub.Mesa,

I feel this is very good thinking and one of the useful ways of looking at the overall situation. Honest, though I understand some schemes, I belong to the old ways and never put much thought on it until I read the quoted part.

I downloaded and tested ZFSGuru and found it full of potential. As seems many on the forum are accumulating data on their NAS, Recovery should be a consideration. Once the distribution is stable, I recommend you to put the excellent part above into a FAQ or Recovery Document for quick reference to users.

Cheers
 
Awesome, thanks for the input everybody.

At this point in time, I'm pretty much set on grabbing a pair of the 2TB disks. As funds permit, I'll add an SSD as well. I'll have to watch some sales and hope I can snag 8GB for a bit cheaper than it already is :)

Thanks again,


- Jesse
 
Alright, so I've just had a family friend ask me to build a computer for them. This works out nicely, as I'll just charge them for building/configuring everything by ordering some RAM from Newegg with their hardware :D I think I'm just gonna grab a single 4GB stick for now, and then I can order another stick if need be. Once I'm back working I'll order the drives as well, and then this thing should be up and running!

I think I'm going to install the OS on a flash drive, for the sake of simplicity. I'll order one of those when I get my memory as well. I'm not gonna worry about putting it in a rackmount case until I start buying some more rackmount equipment. It'll be fine in the tower it's in for now.
 
Back
Top