Become a Patron!

Need MB rec for Low Power, ZFS, FreeBSD, Home NAS

Discussion in 'SSDs & Data Storage' started by learnedbyerror, Sep 28, 2010.

  1. learnedbyerror

    learnedbyerror n00bie

    Messages:
    18
    Joined:
    Sep 20, 2010
    A year or so ago, I decided to re-enter the FreeBSD world at the allure of ZFS. Things have worked OK, but not stellar (see - http://learnedbyerror.blogspot.com/ if you are morbidly curious :mad:) I have learned a lot through my trials and travails. Two of the most important things are: 1 - ZFS takes 8GB of RAM, 2 - FreeBSD is still picky about hardware. While both of these can be debated, I'm not really interested in doing so in this thread. Please accept these two assertions as givens.

    Ideally, I would like to achieve the following:

    1. Re-use my existing Chenbro Hotswap Mini-ITX Case and 4 - 1.5 WD 1.5TB EADS Drives
    2. 64-bit Processor
    3. 8 GB of RAM
    4. Low Power Consumption < 45W at full speed, <<20W idle
    5. Use L2ARC & SLOG (?? dual-IDE using IDE/CF Adapter and UDMA CF Cards ??)
    6. Full SATA Support by FreeBSD
    7. APIC Support by FreeBSD, no confusion on FreeBSD's part with shutdown -h rebooting
    8. Reliable and performant USB support for FreeBSD
    9. And did I mention, cheap? :p

    I am in the US so any recommendations need to be accessible here.

    Thanks in advance for your help!!

    lbe
     
  2. sub.mesa

    sub.mesa 2[H]4U

    Messages:
    2,508
    Joined:
    Feb 16, 2010
    Hey there!

    You are replacing your Atom-330 setup, right?

    Let me go over it point by point:

    1. So you want Mini-ITX form factor motherboard; not too many of those but some interesting models are available!
    2. Atom D510 would be 64-bit and able to run ZFS. But for some more juice a Core i3 would be better. Both should be around the same idle power consumption, it won't matter all that much. Especially the newer NM-10 chipset is very power efficient.
    3. Either that means 2x 4GiB or 4x 2GiB. 4GiB has my preference, but DDR3 4GiB modules are kind of expensive.
    4. <20W would be very hard if you include the disks and power supply. Any reason you need <45W at full load? A NAS would benefit from very low power consumption but still the ability of high performance when you need it. Thus a powerful CPU that has 65W+ TDP could be almost as efficient when idling as a core i3/atom setup. You don't HAVE to go atom, is what i'm saying. And in half a year, AMD's Ontario might be very worthwhile to look at.
    5. L2ARC would benefit from CompactFlash/USB stick but more so from real SSD. SLOG is not really recommended at this time as few SSDs can write safely due to having a supercapacitor onboard.
    6. I think all normal AHCI SATA is supported? You would want to use the ahci driver in FreeBSD 8.1. You have to enable it in the loader.conf. This will causes disks to be named "ada" instead of "ad".
    7. I think you mean ACPI? This is a hard one, as many BIOS come with flawed ACPI tables that may work in Windows but other OS would have problems with it. The result is that ACPI may not work and powering down does not happen automatically; you would have to press the power button. Best advice here is to buy a motherboard which is known to work with FreeBSD. Updating BIOS may sometimes also help. Some BIOS have options like "ACPI tables for Linux", enable those.
    8. USB 'just works' for me. Never had any issues with it really, at least not for simple USB devices.
    9. No you didn't. :p

    Some options:

    Zotac H55-ITX (for Core i3)
    [​IMG]

    SuperMicro X7SPA-H (integrated Atom D510)
    http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPA.cfm?typ=H

    [​IMG]
     
  3. ghost6303

    ghost6303 2[H]4U

    Messages:
    2,292
    Joined:
    Jul 24, 2004
    the 4 hard drives alone will consume almost 45 watts, so that is probably not a realistic system requirement.

    #5- i probably wouldnt do that with a IDE/CF adapter; eventhough you may buy a real nice and fast CF card, the cheap IDE>CF interface chips in the cheap adapters are slowwww. i use this setup to load freebsd itself, but if you want to put l2arc / slog on something else, put it on a SSD.

    #9- what is your definition of this... 'cheap'?

    depending on what this is to be used for, your best bet will probably be to look at intel atom cpu/mobos, or build a low power i3 or i5 setup. ive been told early Atom chipsets were not very good for power consumption. im sure someone can recomend you a good power efficient motherboard.

    you want your power supply to be as low-wattage as you can while still being able to power everything at full load. so if your system pulls 160w at the wall at full tilt, you should be using a 180 to 200w high efficiency PS. (a $19.95 200w PS is not usually the most efficient remember).
     
  4. learnedbyerror

    learnedbyerror n00bie

    Messages:
    18
    Joined:
    Sep 20, 2010
    Responding to Sub-Mesa First,
    1 - Yes, I want to stick with the Mini-ITX. I realize that this is limiting. But we need challenges, right? :)

    3 - The driver for me on power is that this box sits in the walk in closet in my bed room. The closet has minimal air conditioning but can get warm. So I want something that runs cools. <20W is a goal, but I realize it will be a challenge. The drives idle at 0.8W so 3.2W for them. The trick is getting an efficient chipset where I can disable everything that I don't need and having an efficient PSU. My current box idles at 45W where the chipset pulls at least 25W. So I'm hoping that the newer technology than existed when I built the last box a year ago will get me there or close.

    6 - I cannot get my current FreeBSD 8.1 to recognize my current AHCI SATA controller. I have had this problem with FreeBSD 7.x on other hardware.

    7 - Yes, ACPI, sorry fat fingers. I realize that this can be a pain but if there is a MB that is known to work, I would like to get one that does.

    8 - USB works but is neither fast nor reliable. When I connect external drives to import photographs, I seldom get the read rates that I get on Linux or Windows. If I try to export anything of size, say greater than 50G, the transfer will fail for IO errors. I have tried rsync, cp as well as a program I personally wrote to limited memory and IO consumption. The problem is a combination of not so solid USB drivers and FreeBSD's agressive memory allocation with its less than aggressive release. If you are running on fast enough hardware with a large amount of memory and not doing large transfers, you won't see problems.

    9 - I have Scottish blood in me, so less is best! I don't have a hardline here other than doing it as cost effectively as I can with maintaining reliability and a modicum of performance.


    Ghost,

    5 - I have had very good performance with Ultra CF-IDEA adapters (http://www.amazon.com/gp/product/B001TJCO50/ref=oss_product). I come very close to the theoretical read and write speeds of the media in them until I bump in the IDE interface transfer speed limit (100 MB/s).

    9 - see above

    My heaviest usage pattern is two DIVX streams either via UPNP or CIFS while importing/exporting 100 GB of photos. So 30-40 MB/s sustained should take care of my needs.

    Thanks to both of you guys for your input!!

    lbe
     
  5. sub.mesa

    sub.mesa 2[H]4U

    Messages:
    2,508
    Joined:
    Feb 16, 2010
    1) What do you think about the motherboards i posted? Those are Mini-ITX and should work fine for ZFS on FreeBSD. I can't vouch for them personally, though. I do have Zotac Atom510 Mini-ITX board, you may want to look at that one as well.

    3) WD 3,5" Green drive uses between 3.5 and 4.5W of idle power. Only 2,5" disks use below 1W of idle power. SSDs with DIPM can use as low as 75mW or 0,0075W. So with 4 green disks you would do about 16W of AC idle power consumption if we take 4.0W per drive. I think that is reasonable.

    6+7) Interesting. Did you try changing BIOS options like "Plug and Play OS" or updating the BIOS to latest version? My experience is that most hardware just works, except wireless stuff and some other less common devices. I suspect that due to ACPI not working, the AHCI controller is also not detected; thus it's not that your controller is not compatible; just the ACPI issue causing some ACPI-dependent stuff not to work.

    8) Well my experience is in the 20MB/s range; i guess it could depend on the chipset used in the external casing, which does the SATA/PATA to USB 2.0 conversion. Ideally you would want a USB 3.0 casing, as that would use newer chipsets and might work better. But the external casings i used (SATA/300 -> USB 2.0) worked without timeouts for me. But 20MB/s is not fast at all.
     
  6. ghost6303

    ghost6303 2[H]4U

    Messages:
    2,292
    Joined:
    Jul 24, 2004
    i bought one of the $7 newegg adapters and was only getting about 20-25MB/sec transfer rates with it. the card i used would do 80-90MB/sec if plugged into a card reader, so pretty sure that was the problem. it works well enough for loading 50 megs worth of linux kernels into memory at startup though. yours may have a better quality chip then mine i guess. if you are getting 100MB/sec then try it out.
     
  7. learnedbyerror

    learnedbyerror n00bie

    Messages:
    18
    Joined:
    Sep 20, 2010
    sub.mesa,

    1) These MB look interesting. I'll do a little reading and research on them this eveving. The SuperMicro appears to be limited to 4GB. I do like the quick read on the Zotac.

    3) Sorry, idle was the incorrect word, standby is what I should have said. Here are the specs from the WD web site

    I would like to configure the power management to spin down the drives during the large portions of the day when there is no activity.

    6&7) I have updated the BIOS and played with all of the settings whether I thought they would make a difference or not. In no case, did FreeBSD recognize anything other than ATA

    8) I would love to get 20MB/s on USB and have it run without crapping out on IO errors. To get around this, I currently have a shell script that re-runs my rsync command until it completes successfully. Very crude, but effective. Again, I think enough CPU and RAM will prevent the trigger. I'll see when I get my new MB ;)

    Ghost,

    The Ultra CF/IDE adapters aren't much more, $10 ea. on Amazon with free delivery if you have Prime. The only issue that I have run across with them is problem partition them with FreeBSD 8.0 or higher with Kingston CF cards. Apparently Kingston uses a utility that is based on an older version of FreeBSD. Older versions of FreeBSD set some bits improperly in the partition table that care 8.0 and higher to balk. a dd from /dev/zero for the first MB quickly takes care of the problem. Otherwise, they fly along.

    For a while now, I have used them only for boot media; however, in my current energy consciousness, I installed a second one with a photo grade card (good write charateristics) and am using that host the transactional data for processes like sabnzbd and sickbeard that would prevent me from being able to spin down my drives. Overall, I have been tickled to death with it. $35 for 4 GB (305x) CF and the CF/IDE adapter. Plus, it eliminates any disk contention if I have a large transfer running.

    lbe
     
  8. learnedbyerror

    learnedbyerror n00bie

    Messages:
    18
    Joined:
    Sep 20, 2010
    OK, got a few minutes to look into the motherboards in more detail. The SuperMicro is a non-starter for me since it is limited to 4GB. Technically, I like the Zotac with the I3; however, MB, CPU & 8 GB of RAM hit about $400. While this isn't necessarily a deal killer, it is more than I really want to put into this. Also, the power consumption on the Zotac can push 90W if the CPU is under full load. I'm guessing that a full load is highly unlikely and that there are some limitations that I can place on this via declocking. So power is a sligh concern, but not overriding.

    I need to mull this over a little and see if I can find something more attractive from a cost standpoint. the Zotac board has a lot on it that is not needed. I'm wondering if I can find an industrial PC mobo that is more stripped down. But I'm guessing if I do, then the cost savings will likely only be on the order of $30-40 :(

    The other way to get the cost down is to wait 6 mos until this is old technology and the prices will be down 30 - 40%. That is doable, though undesirable.
     
  9. odditory

    odditory 2[H]4U

    Messages:
    4,028
    Joined:
    Dec 23, 2007
  10. PigLover

    PigLover [H]ard|Gawd

    Messages:
    1,125
    Joined:
    Jul 11, 2009
    4GB is an Atom D510 limitation. Same limit for that CPU no matter which MB you use. I don't think you'll satisfy all 9 of the requirements you listed - some of them are pretty near mutually exclusive to each other. I don't think you'll get a closer match to what you are looking for than that SuperMicro Atom board. And realistically, even though ZFS can get pretty memory intensive at times, you won't suffer very much by living with 4GB.

    ps - think about the NAS application a bit more before you respond that you can't get maximum performance out of ZFS without that much memory...ultimately, you are limited to <100MB/s throughput to/from the NAS by the GigE link. And you'll only hit that with absolutely perfect conditions on the LAN and on the client at the other end. Realistic limit is 80-90MB/s and more practical day-to-day assumption is closer to 30. You just don't need 8GB for a small, low power NAS...
     
    Last edited: Oct 1, 2010
  11. learnedbyerror

    learnedbyerror n00bie

    Messages:
    18
    Joined:
    Sep 20, 2010
    Its amazing how much 24 hours can change ones perspective. I have traced the majority of my performance issues to a bad WD15EADS drive. While the drive isn't reporting any problems, it will not transfer at a speed greater than 10 MB/s. The drive is still under warranty, so I am going to file a warranty repair request with WD today. In the mean time, I have configured my 2 good WD drives in a zfs mirror that is large enough to hold everything that I need to have online for a while.

    Irrespective of the drive, I have finally been able to confirm that my current motherboards ICH7 implementation does not support AHCI so I am definitely limited to ATA100 performance. With the bad drive out and restoring with zfs send/receive, I did see all of my drives peak at 100 MB/s from time to time. I can now reliably get in the box transfer of in the 35 - 45 MB/s range. This is more inline with my expectations.

    PigLover, In general, I agree with the ps in your posting posting (assuming you mean MB/s instead of GB/s). The issue that I have with the RAM is as much a problem with FreeBSD as it is with zfs. If you are doing large transfers (i.e. > 10X RAM), FreeBSD 8.1 will not release its Inactive memory fast enough and the cache will starve off thereby at a minimum impacting performance. The worst part of this is that FreeBSD will panic and shutdown from time to time. And since the FreeBSD support for my current board's ACPI isn't solid, the box locks up and doesn't recover without physical intervention. So from a stability point, it seems to me that you need to have enough RAM to make it unlikely that you will starve off the cache and put the OS in a position to panic. I am hoping, that this will be addressed in 9.0 but I'm not enough of a FreeBSD kernel geek to know.

    So, given that I have reached a sufficient level of performance and the reliability is good, as long as I don't try to remotely execute big transfers, I am going to hang with my current motherboard. I like the I3's; however, the cost is a bit more than I want to go at this time. The current 4 GB limitation on the ATOM keeps me from wanting to make an investment in them at this time. I'll keep my eyes open and when chip makers present a sufficiently interesting motherboard, I'll look at upgrading.

    Thanks to all for the input

    lbe
     
  12. PigLover

    PigLover [H]ard|Gawd

    Messages:
    1,125
    Joined:
    Jul 11, 2009
    Yup. Simple typo on my part. Meant MB/s, not GB. I only wish we could see GB/s from a NAS! I've edited and fixed now.

    I do have trouble with the idea of throwing memory at FreeBSD to fix what is obviously a bug. Its impossible to ever have enough memory to completely avoid that problem - you just make it less likely.