The perfect ZFS home server motherboard.

okashira said:
ASROCK C2550D4I

Here's a teaser:
-Quad Core
-Mini-ITX
-12x SATA onboard (not a typo)
-Dual Intel GbE LAN (with Teaming)
-ECC low power DDR3
-Something like 12 w TDP
-Should make a possible a 20+TB server that idles at 25W with drives spun down.
Click to expand...
Nice, so it's almost as good as the SuperMicro X10SL7-F, which has 14 drive ports ;)

edit: Wait, mITX, not mATX! Kudos ASRock!
 
TeeJayHoward said:
Nice, so it's almost as good as the SuperMicro X10SL7-F, which has 14 drive ports ;)

edit: Wait, mITX, not mATX! Kudos ASRock!
Click to expand...

when looking for power consumption figures of the LSI SAS 2308, all I could find was this anecdote posted by omniscence:

Which CPU cooler do you use on the X10SL7-F? The heatsink of the LSI controller gets extremely hot, I almost burned my fingers on it.

Yeah, not only mATX, but that board will use 3x+ power this atom board uses with only two less SATA ports. Note I said HOME SERVER motherboard; transfers of 2 GB/s are overkill, even for enthusiasts.
 
okashira said:
transfers of 2 GB/s are overkill, even for enthusiasts.
Click to expand...
I do hope you're joking. I'm at 4Gb/s (ish, teamed I350-T4), and looking to upgrade to 40Gb/s Infiniband because it's just too slow (And 10Gb/s equipment's too expensive). It took me almost a full day to back up my NAS the last time I tried, and it's not even full. My biggest bottleneck is the network. I'm sure there's a couple of guys in this subforum who experience similar issues.
 
(many of) You appear to be content with a max of ~100 MB/s (ie, Gbit) of data "bandwidth" to/from your array, as is suggested by an Atom processor being sufficient.

Question: Doesn't that imply a drive rebuild time of 30-40+ hours when you have a drive failure (based on a 6-7 [x2TB] drive array)? Isn't that a bit risky?

(Not to mention the extended time for scheduled maintenance/data-verification runs.)

It is important to keep the quest for low power consumption in proper perspective. One watt of 24/7 electric usage will cost most of us US$1-2 per year. Extrapolate that, in the context of your total server investment, and decide whether severely limiting your CPU power available is truly wise.
 
spazoid said:
It's not uncommon for SATA/SAS controllers to consume 10-15 watts - even dumb ones without onboard processor and RAM.

And you can, of course, spin down drives in ZFS systems.
Click to expand...

Right, no arguement here. But if (in your 10 HDDs system) you want to read a little file on a specific drive, the whole 10 drives will spin up right away. Is that correct?
 
Elpee said:
Right, no arguement here. But if (in your 10 HDDs system) you want to read a little file on a specific drive, the whole 10 drives will spin up right away. Is that correct?
Click to expand...
The little file is probably spread across all 10 drives, not located on a single one, so yes, all ten will spin up.
 
TeeJayHoward said:
I do hope you're joking. I'm at 4Gb/s (ish, teamed I350-T4), and looking to upgrade to 40Gb/s Infiniband because it's just too slow (And 10Gb/s equipment's too expensive). It took me almost a full day to back up my NAS the last time I tried, and it's not even full. My biggest bottleneck is the network. I'm sure there's a couple of guys in this subforum who experience similar issues.
Click to expand...

2GB/s is more then 4Gb/s. I am talking about performance unlocked by a high end SAS controller on a Xeon platform.

Yeah your setup is overkill unless it's making you money.
 
Elpee said:
Right, no arguement here. But if (in your 10 HDDs system) you want to read a little file on a specific drive, the whole 10 drives will spin up right away. Is that correct?
Click to expand...

Correct, unless it happens to be located on a SSD serving as L2ARC .


Elpee said:
Hey, wait... I was told mITX is less power consumption then mATX, is that correct? Thanks.
Click to expand...

There is no connection between form factor and power. ITX boards will typically be lower power because they can't support higher power allowed by physically larger boards.
 
UhClem said:
(many of) You appear to be content with a max of ~100 MB/s (ie, Gbit) of data "bandwidth" to/from your array, as is suggested by an Atom processor being sufficient.

Question: Doesn't that imply a drive rebuild time of 30-40+ hours when you have a drive failure (based on a 6-7 [x2TB] drive array)? Isn't that a bit risky?

rebuild speed is not limited by the network speed, but by processor speed, configuration, drive speed and port speed.
Click to expand...
 
okashira said:
rebuild speed is not limited by the network speed, but by processor speed, configuration, drive speed and port speed.
Click to expand...
I believe that the implication was that the Atom processor lacks the power to rebuild an array at greater than 100MB/s. At least, that's how I read it.

And I misread your 2GB/s comment. I interpreted it as the network teaming aspect of the motherboard rather than the controller capabilities.
 
TeeJayHoward said:
I believe that the implication was that the Atom processor lacks the power to rebuild an array at greater than 100MB/s. At least, that's how I read it.
Click to expand...

If that's the case I will change the thread to reflect some haswell supermicro setup, lol.

My cheap underclocked dual core Pentium ivy bridge chip does 400MB/s + on a raidz1 rebuild, so I hope this new atom can at least handle 200.

are zfs parity calcs multithreaded?
 
okashira said:
My cheap underclocked dual core Pentium ivy bridge chip does 400MB/s + on a raidz1 rebuild, so I hope this new atom can at least handle 200.
Click to expand...
According to the intarwebs, a Sandy Bridge chip is 5x faster than an Atom. I'm comparing apples to moon rocks here, and using a WTF review to do it, but based on that evidence, I could see the CPU being a limiting factor for a ZFS home server. Barely. I do recall that the Atom D525s can't reach gigabit line speeds for disk transfers, making them a poor choice for a NAS. Of course, how old are those?
 
TeeJayHoward said:
According to the intarwebs, a Sandy Bridge chip is 5x faster than an Atom. I'm comparing apples to moon rocks here, and using a WTF review to do it, but based on that evidence, I could see the CPU being a limiting factor for a ZFS home server. Barely. I do recall that the Atom D525s can't reach gigabit line speeds for disk transfers, making them a poor choice for a NAS. Of course, how old are those?
Click to expand...

This atom (Avoton) has much faster IPC then the old atoms (it's based on Silvermont) (like more then double) and is available quad and octo core, hence my confidence it will be OK. I bet you can get 500MB/s with the octocore if not more.
 
TeeJayHoward said:
I believe that the implication was that the Atom processor lacks the power to rebuild an array at greater than 100MB/s. At least, that's how I read it.
Click to expand...
Yes, that was the (rough) gist of my point ... that an Atom might be a significant constraint when it really mattered (a rebuild).

I don't use ZFS, but, in addition to the RAID calc, aren't the data checksums also in play during verifications, and a rebuild?

The overall point is that capacity planning should not be limited to the best-case scenario (serving data over Gbit), especially when much of the original intent (for a ZFS server) is data integrity and recoverability.
 
UhClem said:
Yes, that was the (rough) gist of my point ... that an Atom might be a significant constraint when it really mattered (a rebuild).

I don't use ZFS, but, in addition to the RAID calc, aren't the data checksums also in play during verifications, and a rebuild?

The overall point is that capacity planning should not be limited to the best-case scenario (serving data over Gbit), especially when much of the original intent (for a ZFS server) is data integrity and recoverability.
Click to expand...

You have also implied the rebuild failures are completely time based, which I would disagree with, at least from a mechanical HDD failure standpoint.

But anyway, my hope and expectation that this atom will be much faster then you guys are expecting. hopefully we can get some real world benches soon.
 
Avoton is vastly different from previous Atoms. It got out-of-order execution back, the most important thing Intel killed from Pentium M to create the Atom. I estimate it would have no problem saturating gigabit ethernet as a fileserver, or else it would have made no sense to outfit it with 4x 2.5 gbit interfaces. It also has two memory channels.
 
I want to build a mini nas server with this board.
Here is my planed hardware, only the case is a problem.
Does any body know a case thats around 4l size with 20x20x10 cm dimensions
with a 5.25" slot?
I want it to be as small as possible.

current plan:
- mini-itx case m5x
- 6x WD RED 2.5" 1.4W on usage
- Thermaltake MAX-1562 6xbay 2.5" fits in a 5.25 slot

I couldn't find a case that has a 5.25 slot and is small.
Because of this, I think I will mount the Thermaltake MAX-1562 on top of the m5x.

Whats a reasonable power adapter for this config?
80W enough?
Never done a mini-itx build. Are there any problems with my hardware selection?

This baby will rock with nas4free ;)
 
You may want at least 100W to 120W to cover start up load of the drives?

I will be going with a 225W power supply. Dell RM112 that I will be modifying.

It's not easy to find a P/S that's efficient at 15W - 30W load, but dell has some good options. go for used and modffy if you have the ability. (add atx connector, lengthen/shorten wires, etc)


EDIT: never mind didn't see the 2.5" drives. Neat setup, but not sure it will be cost effective. I Went for the cheapest 3.5" 3TB drives I could find, and plan to use low power board + spin down to manage power.
 
IMO this avoton chip will be very similar in performance to the chips zfs was designed on. I wouldn't be worried about it
 
A 5,25" bay is about the size of a complete ITX box (with crappy external PSU) so a box including one is poised to be difficult to find.

I'm all for reducing power usage but only if it makes economic sense, apparently this new Atom is EXTREMELY expensive. Maybe good for embedded usage, but not the home.
 
Aesma said:
A 5,25" bay is about the size of a complete ITX box (with crappy external PSU) so a box including one is poised to be difficult to find.

I'm all for reducing power usage but only if it makes economic sense, apparently this new Atom is EXTREMELY expensive. Maybe good for embedded usage, but not the home.
Click to expand...

Maybe expensive. But maybe not when you look at total system cost. ASROCK Avoton board is expected to come in $350-400 street, which is high compared to $100 MBs. Agreed. But...

Those $100 MBs don't include the CPU. Add $70-150 for comparable.
Those $100 MBs don't support ECC. Add $50 to get into server class boards.
Those $100 MBs don't have 12 SATA. Add $70-200 for HBA.
Those $100 MBs probably use cheap Realtek NICs that won't perform well for NAS. Add $20-50 for Intel NICs.

At the end of the day the ASROCK/Avoton could end up being a $bargain.
 
PigLover said:
Maybe expensive. But maybe not when you look at total system cost. ASROCK Avoton board is expected to come in $350-400 street, which is high compared to $100 MBs. Agreed. But...

Those $100 MBs don't include the CPU. Add $70-150 for comparable.
Those $100 MBs don't support ECC. Add $50 to get into server class boards.
Those $100 MBs don't have 12 SATA. Add $70-200 for HBA.
Those $100 MBs probably use cheap Realtek NICs that won't perform well for NAS. Add $20-50 for Intel NICs.

At the end of the day the ASROCK/Avoton could end up being a $bargain.
Click to expand...
$399 for the comparable mATX mobo + CPU. (Has Intel NICs and SAS HBA).

$$$/performance isn't there if you're looking for something powerful. On the other hand, if you care about saving 20W over a normal system build, or really need the space, you can't go wrong with the ASRock board.

edit: Actually, that 5.25" bay comment got me thinking. It'd be fun to build a mITX computer in a full-sized case. Stick the motherboard in a 5.25" bay, print out a picture of a motherboard, and put that in the normal spot. Have a window case to show it all off. Might make a few folks go, "Eh?" at a LAN party.
 
Last edited:
TeeJayHoward said:
$399 for the comparable mATX mobo + CPU. (Has Intel NICs and SAS HBA).

$$$/performance isn't there if you're looking for something powerful. On the other hand, if you care about saving 20W over a normal system build, or really need the space, you can't go wrong with the ASRock board.

edit: Actually, that 5.25" bay comment got me thinking. It'd be fun to build a mITX computer in a full-sized case. Stick the motherboard in a 5.25" bay, print out a picture of a motherboard, and put that in the normal spot. Have a window case to show it all off. Might make a few folks go, "Eh?" at a LAN party.
Click to expand...

Fair play. But your comparing an on-sale, best ever, combo price of a MB/CPU that has been out for over 6 months vs list price of a not-yet-released brand new technology. Not really apples/apples. 6-8 months forward pricing and add NewEgg Supersaver treatment probably gets the Avoton down well below $300. And yes, that MB/CPu does give you more horsepower...but its horsepower that's completely wasted on a NAS so its pretty meaningless in the application at hand.
 
PigLover said:
Not really apples/apples.
Click to expand...
Yeah, none of my comparisons are. I'm just a huge SuperMicro fanboy. :D For what it's worth, the combos on SuperMicro motherboards + E3 CPUs are fairly constant. MicroCenter generally has something on par with that as well.

The price on the motherboard will probably go down. I'm actually expecting it to come out closer to $325 than $400. At $400, it wouldn't make sense to go with the more limiting motherboard. At $350, it might. Around the $300 mark, it definitely does. 12 drive bays take up a lot of space. Even if they're 2.5" drives, you're still looking at a case that would fit a mATX motherboard just as easily as an mITX one. If they're 3.5" bays, even an ATX motherboard would be dwarfed by the case size.
 
I'm generally a SM fan too. My only purpose for posting that was to counter some "oh my goodness - that MB is expensive" stuff earlier in the thread. People are so quick to count how much one part of a build "costs" but ignore total system cost...and end up spending just as much as what they complained about.

So yes - the ASROCK/Avoton board is expensive. But the system you would build with it isn't any more expensive than you'd spend building it another way, and might be le$$. Just trying to provoke some thought. Same argument, BTW, why in some larger servers the "massively expensive" $500-800 SM Xeon boards are actually quite cost effective.
 
Though you may be able to buy a Supermicro/E3 1220 for only a but more than this board up front, you have to consider the power bill. 10W = ~$9-10/year in most markets in electricity cost. That 80W TDP processor only is going to consume far more power than this board with its 14 or 20 Watt SoC, and it only gets worse when you amortize it over the number of years you'll own the server. It'll also put out more heat, which is bothersome if you keep your NAS in your living space, as I imagine most people building mini-ITX systems would. Hotter room and more work for your A/C.

Naturally, if you actually need more processing speed for some reason, it's a different story, but if it's solely a NAS, minimizing power use, and thus cost, is huge.
 
all this talk is funny, i'm thinking of moving my esx+zfs server to a socket 2011 setup...

I do like the board in teh op tho
 
From a "power need" only aspect, the board may be fine at home especially for a small NAS only box.

What I would miss at first place:
- vt-d with the option of a fast All-In-One with ESXi 5.x as base and different VMs on it (storage and others)
I would never buy a board, that lacks this feature. Even when starting with a small NAS only use, you may
use BSD, OSX, Linux, Solaris or Windows some day together with a working pass-through of a storage controller,
a GPU or a mediaserver with a sat adapter.

see my All-In-One idea with a preconfigured "ESXi 5.5 and ready to use ZFS storage appliance"

With the new ESXi 5.5 where there is no 32 GB Ram limit left, such an All-In-One is even usable
for a lot of VMs together with a highspeed NAS/SAN in one box, especially with a more Pro like
mainboard with a 2011 chipset where you can add up to 768 GB Ram.

With RAM now cheaper and cheaper, this RAM option can be important.
With Xeon-L you start with under 17W max TDP so the CPU difference is not to large.
If you add up to twelve disks, forget the difference based on the CPU.

Last:
If you need miniITX, your choices are very limited. I would use a uATX as base
with much more options at a lower price.
 
I gave up trying to cram everything into a small space. What for? Just buy that spacey 24-bay rack case, load it with a dual-socket E-ATX board and never have to upgrade anything again except for stuffing in more HDDs.

My dual-S2011 board draws less than 40W idle with one CPU and 32GB RAM and I have options to go to 48 threads and 1TB of RAM.
 
Bobalias_LeShay said:
Though you may be able to buy a Supermicro/E3 1220 for only a but more than this board up front, you have to consider the power bill. 10W = ~$9-10/year in most markets in electricity cost. That 80W TDP processor only is going to consume far more power than this board with its 14 or 20 Watt SoC, and it only gets worse when you amortize it over the number of years you'll own the server. It'll also put out more heat, which is bothersome if you keep your NAS in your living space, as I imagine most people building mini-ITX systems would. Hotter room and more work for your A/C.

Naturally, if you actually need more processing speed for some reason, it's a different story, but if it's solely a NAS, minimizing power use, and thus cost, is huge.
Click to expand...

If $9-10/year is a concern, maybe you should rethink building a NAS...
 
PigLover said:
Maybe expensive. But maybe not when you look at total system cost. ASROCK Avoton board is expected to come in $350-400 street, which is high compared to $100 MBs. Agreed. But...

Those $100 MBs don't include the CPU. Add $70-150 for comparable.
Those $100 MBs don't support ECC. Add $50 to get into server class boards.
Those $100 MBs don't have 12 SATA. Add $70-200 for HBA.
Those $100 MBs probably use cheap Realtek NICs that won't perform well for NAS. Add $20-50 for Intel NICs.

At the end of the day the ASROCK/Avoton could end up being a $bargain.
Click to expand...

If its under $300 then sure. But at $350+ I don't see it as a bargain. I can't justify the cost unless I NEEDED itx. Then yes sure it's a fantastic option, but looking at benchmarks at servethehome it's not a grandslam. It is much better than an Atom, but not enough to justify the cost.

SUPERMICRO MBD-X10SL7-F-O

Which has :
  • 1x PCI-E 3.0 x8 (in x16) slot
  • Supports ECC
  • 1x PCI-E 2.0 x4 (in x8) slot
  • 4 x SATA 3.0Gb/s + 2 x SATA 6.0Gb/s
  • 8x SAS2 (6Gbps) ports via LSI 2308
  • Dual Intel I210-AT LAN

So by adding that $140 to the mobo you meet all the requirements but just lack CPU. Grab an $100 i3 and you're at the same price but have the ability to grow.
Plenty of Supermicro boards support ECC, have dual intel nics for not much more than that $100.

But I am biased as I bought a $100 board, a used Supermicro X9SCM-F :)
 
Last edited:
Seems like Asrock got a lot of things right there. Too bad they didn't equip it with a high quality Intel onboard NIC too.
 
Maximus825 said:
If $9-10/year is a concern, maybe you should rethink building a NAS...
Click to expand...
That's for 10W. Now obviously you're not running the machine at full tilt all the time, but if you were 80W TDP - 20W TDP = 60W ==> ~$60/year * 3 years = $180. Power savings are serious business.

The above analysis isn't terribly realistic since it assumes running flat out all the time (and ignores the power consumption of the motherboard/chipsets along with inefficiencies), which obviously isn't going to happen, but running them both at partial load still would give a substantial power savings to the Avoton over the life of the server. Which makes the cost equation much different than "Well if I can get an E3 for only a little more initial cost why wouldn't I?" The answer is because with power costs factored in, it's not a little more cost, it's a lot more. If you don't actually need that power, you're wasting money. If you do need the computational power, obviously the E3 is a better call.

Surly73 said:
Seems like Asrock got a lot of things right there. Too bad they didn't equip it with a high quality Intel onboard NIC too.
Click to expand...
Unless I'm misunderstanding, the dual GigE is from an Intel i210 on the ASRock, isn't it?
 
Last edited:
Bobalias_LeShay said:
"Well if I can get an E3 for only a little more initial cost why wouldn't I?" The answer is because with power costs factored in, it's not a little more cost, it's a lot more.
Click to expand...
What about in a more realistic scenario? When your Avoton is running at 25-50% utilization, is it drawing more power than an E3 would performing the same tasks? Seems to me that the E3 would be idling while the SoC would be hard at work. Since an E3 idles at around 20W, and the Avoton runs full-bore at the same, is it safe to say that there's about a 10W difference between the two?
 
You must log in or register to reply here.
Back
Top