Another quiet OpenZFS issue

Discussion in 'SSDs & Data Storage' started by ullbeking, May 25, 2019.

  1. ullbeking

    ullbeking n00b

    Messages:
    28
    Joined:
    Aug 17, 2017
    Hello,

    I am deploying ZFS-on-Linux as its own separate appliance on the following setup:
    • U-NAS 800 chassis with 8 bays
    • A1SRi-2758F board with 32 GB ECC RAM
    • Debian Buster running as the OS, with ZoL
    • A simple configuration where exports are shared over NFSv3 to my LAN
    • 4x WD Red 8 TB in RAIDZ2 as one vdev/pool
    • Currently configuring the other four bays with WD Re4/Gold HDD's, which will also be a RAIDZ2 pool
    My issue is that although everything has been going well, and the chassis is great, the constant and unpredictable whirring, clonking, tapping, and clicking of the moving-parts HDD's (WD Reds so far) is infuriating.

    I have started a plan whereby I reply this NAS by one the is implemented only with SSD's, but to get the same capacity cost is prohibitive. So I'm not sure if I can go through with this.

    Is it possible to create a ZFS array with HDD's, that is quiet and the disks don't make these annoying sounds?

    Is there a guide for tuning a ZoL for this?

    Any help would be vastly appreciated!!

    ullbeking
     
  2. Meeho

    Meeho [H]ardness Supreme

    Messages:
    4,371
    Joined:
    Aug 16, 2010
    A PC case? I don't hear my 5400/7200 NAS drives in Fractal Design Define cases.
     
  3. Spartacus09

    Spartacus09 Gawd

    Messages:
    1,008
    Joined:
    Apr 21, 2018
    Seconded on fractal design, I have had the FD R5 for my gaming rig for years now it cut the noise of my loud ass 970 by half, my current 1070 I can’t hear at all. I just snagged a scratch/dent define c for my new skylake plex box for 50$ on ebay (it turned out to be NIB condition by the way with only minor internal issues with the side panels that a hammer fixed). With good cpu coolers both are dead silent (both have only solid-state to be fair though). The dust filtration is top notch, sound dampening material is effective, and both the 120 and 140mm FD fan models are whisper quiet even at full rpm. Two of the 140mm in pull config cool my corsair 280mm aio on my 9900k sufficiently except for stress & benchmarking even.

    I have a caseless freenas build right now sitting on my desk I am playing around with. I’m using the fractal design cages I removed from my R5 and the vibration dampening trays work decently on the seagate LPs even in the cages open area the SL yate loons cooling the drives are the loudest thing near me. I do have two Dell R420s running in the opposite corner of the room in a half rack cage so probably can’t hear the clicking over that :) ~ I joke, I have done some ipmi commands to force lower rpm so they’re actually reasonably quiet lining it with egg crate foam.

    I’m looking at a freenas build myself and the FD XL R2 was my first choice, it has 8 internal bays and I was going to use this to get 3x 5.25 to 4x 3.5 with a fan slot to get enough bays (the last solo for a 4x 2.5” enclosure for a cache/ssd pool). However I’m quickly outgrowing my current 12 disk array and likely going to go with the 24 bay Norco with the 120mm fan wall and cougar fans.
     
    ullbeking and Meeho like this.
  4. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,930
    Joined:
    Jul 24, 2008
    Here is another tip of the hat to the Fractal. I am in the middle of replacing my primary workstation (Which was also in a Define XL) with an new XL. It has an i7-9700K, a 1080ti, 16x shucked 10tb WD Elements (8 in the included drive cages, 4 in an additional Fractal drive cage that replaces the bottom fan and 4 more in a Lian-Li 4 drive cage up top. Also has 2 1TB SATA SSD's and 2 1TB m.2 PCIe SSD's. When idle, and sitting 2 feet away from it I can hardly hear the fans, and can hear no drive noise whatsoever. When at load rendering or playing games, I can hear the fans from the Nvidia but not much more.
     
    ullbeking and Meeho like this.
  5. ullbeking

    ullbeking n00b

    Messages:
    28
    Joined:
    Aug 17, 2017
    Dear Meeho, Spartacus09, and mwroobel,

    Thank you so much for your replies. I initially started with a very complicated solution, where I had multiple NVMe disks in a RAID configuration, sync'ed to the main NAS in an upstream remote location. The possibilities for things going wrong are, well, I think it's pretty obvious that this idea is an expensive and fragile solution. It doesn't even really solve the whole problem unless I have an all-SSD NAS that is as large as the main NAS. In reality I'd have to rely on a tiering or caching strategy that can only hold a subset of the full data at any given time.

    I'm very interested and intrigued that the FD R5 and FD XL R2 seem to be top choices. If I understand correctly, the the R5 should be better for acoustical silencing? I've looked at the Fractal Design cases often, and I even have an XL R2 in waiting for a workstation that is in planning. Moreover, I understand that there are other models that are better for dampening acoustical vibration, such at the R5. So I will be looking into these more seriously.

    I'm also going to investigate Nanoxia Deep Silence, which I've heard good things about. Fans of Nanoxia have told me it is similar in build feel and construction to FD, but a little more solid and sturdy feeling. Does anybody have any opinions regarding this?

    I'm very happy that FD is now among my top solutions, and a very simple one at that.

    Finally, if we're looking at silent cases, does anybody have suggestions for silent rack chassis? Or is the market simply not there, i.e., rack chassis are designed for the data centre first and foremost.

    Kind regards,

    ullbeking
     
  6. Meeho

    Meeho [H]ardness Supreme

    Messages:
    4,371
    Joined:
    Aug 16, 2010
    What do you need an NVMe/SSD NAS for?

    I have 4x WD Reds 4 TB in R5 and 7x 7200rpm Toshibas in an XL R2 and they are both silent regarding HDD noise. I have no experience with Nanoxia but they look promising.
     
  7. ullbeking

    ullbeking n00b

    Messages:
    28
    Joined:
    Aug 17, 2017
    The point is that I don't need NVMe/SSD for... at first when I started on trying to solve this problem, the immediate answer seemed to be, build an SSD NAS so it's silent locally and then sync that remotely upstream to its master NAS. But this would either get very complicated if the SSD array layer implements only a subset NAS or a tier or cache layer. Plus there would be network latency to worry about. So I held back to see if there are simply and thus more resilient ways of solving the problem, such as what we are discussing here.

    I would use NVMe for local storage in a workstation, for example. But at this stage, not for a shared NAS, unless I had a TONNE of cash and could build one that was as large as its partner NAS, i.e, the backup NAS, i.e., the one it's syncing to.
     
  8. Meeho

    Meeho [H]ardness Supreme

    Messages:
    4,371
    Joined:
    Aug 16, 2010
    A quiet case with HDDs is a much better choice then.

    Keep in mind that R6 is the R5 successor and can house up to 12 drives I believe. Maybe even 12x 3.5 and 2x 2.5
     
  9. ullbeking

    ullbeking n00b

    Messages:
    28
    Joined:
    Aug 17, 2017
  10. Meeho

    Meeho [H]ardness Supreme

    Messages:
    4,371
    Joined:
    Aug 16, 2010
    R5 - 8 HDDs
    R6 - 11(12?) HDDs