Compact 20TB: planning.

R

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
Ok, i need to replace my old 3U NAS soon - can't keep it anymore after move and i need some suggestions about new one i planning to build...
My current NAS can hold ~8TB of data and it almost filled up. So i want at least 2x space.
Budget is ~2k$ with HDDs (and 8xHDDs takes around half from that budget)
Purpose: home media, iSCSI for ESXi lab. There will be no torrents/encoders/etc on that machine, only storage and storage services (NFS, AFP with TimeMachine, iSCSI, etc)

Chassis: NSC-800 8 bay hot-swap, 4 internal 2.5, already with all (SFF, power) cables and stuff. 275$. Unbeatable.
HDDs: 8x3000Gb Seagate (ST3000DM001) 64Mb 7200rpm SATA3 Barracuda 7200.14 - pretty generic choice. Can yield ~20TB space in RaidZ.
I will also put some generic SSD for system and some cache.
PSU: Seasonic SS-350M1U 350W 80 Plus. I have two of them in my lab and i completely satisfied in terms of noise level (zero) and stability.
OS: nas4free or freenas, i plan to use ZFS.
And here is where i need your help...
Due to format (Mini-ITX) and amount of HDDs (12 max) i have small amount of choices in terms of motherboard. Especially when it comes to IPMI.
What i found: Asrock with that new 8-core Atom. 12(!) SATA ports. I personally think this is ideal solution for that build, except i need to rewire SATA cables (chassis comes with SFF-8087). Also only 6 ports is SATA3. Not sure if i will notice any performance difference in array, thought (4xSATA3, 4xSATA2, 2xSATA3 for caches)

Another option: buy normal mobo (ivy/haswell) put some i3 + RAID card.
Cons: no ECC in that config, price (600+ USD). Not sure if separate PCI HBA will make any difference in performance.

Your opinion?
 
B

bds1904

Gawd
Joined
Aug 10, 2011
Messages
1,007
use the asrock you have quoted as is REQUIRES ECC MEMORY

Other option would be a supermicro c2750 based mobo.
 
R

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
bds1904 said:
Other option would be a supermicro c2750 based mobo.
Click to expand...
Sadly only 6 SATA ports and 400$ price. Probably possible to find low-end xeon e3 + mobo for that price, since it still requires external SATA HBA.
 
P

PigLover

[H]ard|Gawd
Joined
Jul 11, 2009
Messages
1,188
The ASRock board is interesting, but the 12 on-board SATA ports are provided by Marvell controllers with some known support issues (which will certainly be overcome in newer BIOS & drivers) as well as internal PCIe x1 interconnects per chip - that's 4-drives on an internal PCIe x1. Not ideal for a file server.

Getting the SuperMicro C2750 board and a cheap LSI-based HBA is really the way to go here. It will cost you a total of about $100 more - and it will be worth every penny spent. Plus you won't have to re-wire the 8087 cables.
 
R

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
PigLover said:
The ASRock board is interesting, but the 12 on-board SATA ports are provided by Marvell controllers with some known support issues (which will certainly be overcome in newer BIOS & drivers) as well as internal PCIe x1 interconnects per chip - that's 4-drives on an internal PCIe x1. Not ideal for a file server.

Getting the SuperMicro C2750 board and a cheap LSI-based HBA is really the way to go here. It will cost you a total of about $100 more - and it will be worth every penny spent. Plus you won't have to re-wire the 8087 cables.
Click to expand...

>x1
oh. So that's how they done it.
Yep, in that case it will be better to go with HBA. Something like classic M1015 then, maybe? Not sure if it supports 6GB/s on SATA ports.
 
J

Jim G

Limp Gawd
Joined
Jun 2, 2011
Messages
221
Rivera said:
>x1
oh. So that's how they done it.
Yep, in that case it will be better to go with HBA. Something like classic M1015 then, maybe? Not sure if it supports 6GB/s on SATA ports.
Click to expand...

It does.
 
R

raiderj

Limp Gawd
Joined
Jun 21, 2011
Messages
340
That's a really slick case, definitely curious how it'll work with the SuperMicro + HBA setup if you go that route.

Are there any good mini-itx boards that would host an i7 or Xeon? That'd make a great ESXi all-in-one box.
 
B

bloodypulp

Gawd
Joined
May 17, 2010
Messages
903
you can go smaller with higher density storage(5TB Toshiba drives) and need less SATA ports
 
S

stm

Limp Gawd
Joined
Jan 2, 2013
Messages
172
bloodypulp said:
you can go smaller with higher density storage(5TB Toshiba drives) and need less SATA ports
Click to expand...

Didn't realize such large capacities were available but potential data loss from a single 5TB HD going bad makes my stomach churn....
 
F

FnordMan

[H]ard|Gawd
Joined
Apr 22, 2011
Messages
1,727
stm said:
Didn't realize such large capacities were available but potential data loss from a single 5TB HD going bad makes my stomach churn....
Click to expand...

so is this where I not mention the new 6TB Seagate "enterprise" drives :) (couldn't resist)

Tight now i've got a RAID-Z on 2TB drives.. methinks any new array will be either Z2 or 6. Probably shouldn't even have Z on the 2TB drives there.
 
B

Bobalias_LeShay

Limp Gawd
Joined
Sep 27, 2013
Messages
287
I'm going to disagree with the consensus so far in this thread and recommend AGAINST the Supermicro board. (Disclaimer - I have the ASRock board)
1) It doesn't use full size DIMMS.
2) It needs a SATA card in its only PCI-E slot, so you have no expandability.
3) It's more expensive.

SATA 2 vs SATA 3 doesn't matter for spinning platter drives, because you're not going to saturate either interface. Obviously you want SATA 3 for any cache/L2ARC/OS SSDs, but the ASRock board has enough ports for that.

I like that case. I almost got one but couldn't justify the cost being twice other options, plus the increased cost of a 1U PSU instead of a normal ATX. Definitely the ideal case for a build like this I think.

I'd go with 4TB drives. You don't really save much cost/GB with 3TB drives, and you seriously hamper your maximum capacity in the 8 bay.

Finally, the newer i3 processors support ECC RAM, so that option is viable. The only real reason to go Avoton over an i3 or E3 Xeon is if you want low power consumption, particularly at load.

If the only reason you're worried about the Avoton is performance, don't be. My array scrubs at around 400MB/s for a 5 drive RAIDz1. It's definitely not a fast machine, but it's more than adequate for ZFS and some daemons/torrents/light hosting.

PigLover said:
The ASRock board is interesting, but the 12 on-board SATA ports are provided by Marvell controllers with some known support issues (which will certainly be overcome in newer BIOS & drivers) as well as internal PCIe x1 interconnects per chip - that's 4-drives on an internal PCIe x1. Not ideal for a file server.
Click to expand...
So that's (at PCIe 2 speeds) 4 drives that share 500MB/s. That doesn't leave much overhead, but I haven't seen spinning platter drives get much above 80-100MB/s each. Am I missing something?
 
Last edited:
K

kac77

2[H]4U
Joined
Dec 13, 2008
Messages
2,913
Bobalias_LeShay said:
I'm going to disagree with the consensus so far in this thread and recommend AGAINST the Supermicro board. (Disclaimer - I have the ASRock board)
Click to expand...

i agree that the HBA not might be the way to go. However, it's because of different reasons.

With such a small form factor you are limited in just about every way. If you should ever need the HBA one of the first things that will go is the case. Therefore I would assess the priority of what you want most. If a small footprint is the primary concern then the HBA is a waste. You'll need every penny you have for the storage. High density drives aren't cheap and 8 of them isn't going to be either. Plus I think they will be mandatory with this build. I'm assuming you want backup and maybe some redundancy. That's going to eat into your drive count and the HBA isn't going to be able to do anything to mitigate that.

The second problem the HBA brings with it is additional heat (fanless CPU + fanless enterprise HBA + 8 drives at 95F) for no good reason other than an increase in performance that's going to be mitigated by the onboard 1GbE nics. You'll get more performance out of moving to 10 Gbe (which would be doable without the HBA).
 
L

levak

Limp Gawd
Joined
Mar 27, 2011
Messages
386
Pick a Supermicro X10SL7-F + ECC memory + cheap Pentium or low power i3 to go along.

Matej
 
R

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
levak said:
Pick a Supermicro X10SL7-F + ECC memory + cheap Pentium or low power i3 to go along.

Matej
Click to expand...

Well, ZFS was designed with ECC in mind, so i think atom or low-level E3 will be better way.
Anyway, i need IPMI, given mITX + IPMI combo i'm very limited in choices. Mobo price starts from 200$, CPU ~100$ (that's not counting good low-profile fan), and supermicro atom board costs 355. Not much difference.

>I'd go with 4TB drives.
It will cost me 600$ more :/
 
R

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
I got chassis (sadly our retarded post broke it, so it required some hammer work...) and happy to answer questions if anyone interested. Photos

That's how it looked straight from box...

Motherboard + RAM + PSU still in transit.

I decided to go with supermicro board, so i got one of that SAS2008-based LSI HBAs and reflashed it to IT mode. Was a bit tricky on UEFI board...
 
IceDigger

IceDigger

[H]F Junkie
Joined
Feb 22, 2001
Messages
11,905
Why not some SAS drives. They usually last a lot longer than SATA drives.
 
You must log in or register to reply here.
Top