[H]ard Forum Storage Showoff Thread

Discussion in 'SSDs & Data Storage' started by EnderW, Jan 1, 2015.

  1. wingfat

    wingfat [H]Lite

    Messages:
    115
    Joined:
    May 13, 2010





    I thought I would share my latest server build for folks looking for a relatively small server case that could accommodate
    at least 10 3.5" drives. ( Currently 11 x 4TB HGST) My requirements required hot swappable bays accessible from the front. (I like blinking lights :)
    After extensive searching and comparing cases, I decided on this iStar S-917 server case. I was a bit concerned about heat exhaust
    but all temperatures are well within specs. (Drive temps with the HSGT's @ 40 degrees C - ambient temp @ 83 degrees F)
    There is ample room within the case for all parts so it was easy to build. There are no welds holding the case together so it
    can actually be taken completely apart. Although not completely silent, the fan noise in negligible. I tried replacing the fans
    in the Supermicro racks and the system became silent. I opted however to stick with the stock fans as they pull twice as much
    air. Searching the net for information on this case is sparse which is one of the reasons why I posted this. The case actually comes with an aluminium front
    door which I removed. The hardware latches and hinges are bare metal underneath once removed. I painted these small areas with black touch up paint. The USB 2.0 connector was replaced with a USB 3.0 connector at the front top. The system is running FreeNAS with ZFS @ 37TB and one spare.



    1 x iStarUSA S-917 Black Aluminum / Steel Tower Compact Stylish mini-ITX Tower
    1 x iStarUSA T-7-SA 1 x 5.25" Bay Military Series SATA Mobile Rack - OEM
    11 x HGST Deskstar NAS H3IKNAS40003272SN (0S03664) 4TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" High-Performance Hard ...
    1 x Intel Xeon E3-1225 v5 SkyLake 3.3 GHz LGA 1151 80W BX80662E31225V5 Server Processor
    1 x ASRock E3C236D2I Mini ITX Server Motherboard LGA 1151 Intel C236
    2 X M391A2K43BB1-CPB 16GB DDR4-2133 ECC UDIMM 2RX8 1.2V -Samsung- (Memory.net)
    1 X ATX 550W Power supply
    1 x Noctua NH-D9L 92mm SSO2 Low-profile Premium CPU Cooler, NF-A9 PWM Fans
    1 X Phanteks 500-1500RPM, PWM Frame/Blades, 12025, 120mm Case Fan Retail Cooling PH-F120XP_BK Black/White
    1 X StarTech.com USB3SPNLAFHD 2 Port Panel Mount USB 3.0 Cable - USB A to Motherboard Header Cable F/F
    2 X Supermicro CSE-M35T-1B Mobile Rack (Black)
    1 X IBM SERVERAID M1015 ADAPTER PCI-E 2.0 X8 RAID CONTROLLER CARD 46M0861-LOW P
    1 X ADATA USA Premier SP600 2.5-Inch 64 GB SATA III Synchronous NAND SSD ASP600S3-64GM-C
     
    Last edited: Nov 15, 2016
    gigatexal, _anarion_ and Ranulfo like this.
  2. Ranulfo

    Ranulfo [H]ard|Gawd

    Messages:
    1,382
    Joined:
    Feb 9, 2006
    Nice build Wingfat. That just may be the case for me for one of my server builds.
     
  3. Trimlock

    Trimlock [H]ardForum Junkie

    Messages:
    15,054
    Joined:
    Sep 23, 2005
    Is there a cheaper place to get this case? I see it going for $200 in a lot of places.
     
  4. FLECOM

    FLECOM Modder(ator) & [H]ardest Folder Evar Staff Member

    Messages:
    15,569
    Joined:
    Jun 27, 2001
    added another 24TB

    [​IMG]

    brings me up to 72x 2TB drives
     
    Trimlock likes this.
  5. TeeJayHoward

    TeeJayHoward Limpness Supreme

    Messages:
    9,411
    Joined:
    Feb 8, 2005
    I'm curious - With HDD prices finally dropping and capacities increasing, why did you go with 2TB drives?
     
  6. 86 5.0L

    86 5.0L [H]ardness Supreme

    Messages:
    6,773
    Joined:
    Nov 13, 2006
    yeah, thats a bit overboard, i think. A few 8TB drives would pay off in the power savings alone haha
     
  7. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,637
    Joined:
    Jun 9, 2004
    Storage has grown rapidly in the two years since my post on the first page. Added a bunch to my colo (only counting personal use below). All said and done, ~400TB currently I think?

    Primary colo server (bottom of rack):
    Supermicro 216E16
    Supermicro X10SDV-4C-TLN2F and 8gb ECC RAM
    Transcend 32gb SSD
    Areca 1882i
    24 x 4TB 2.5" Seagate

    Secondary colo server (middle of rack):
    Supermicro 847E1
    Supermicro X8DT3-F with Xeon L5630 and 8gb ECC RAM
    Intel 320 40gb SSD
    Areca 1880i
    26 x 2TB 3.5" Hitachi
    10 x 1TB 3.5" Hitachi

    Home server #1:
    Supermicro 846E16
    No board at the moment as it's just being used as a SAS expander
    Areca 1882x
    24 x 2TB 3.5" Hitachi

    Home server #2:
    Synology DS2415+
    12 x 6TB 3.5" Hitachi helium

    Home server #3:
    Infortrend EonNAS Pro 850
    8 x 6TB 3.5" WD

    Consolidating a bunch at the moment, so I'll eventually go down to a single file server at the datacenter and home (have everything mirrored between the two sites). Have another ~60 drives in boxes at home that aren't in use too.

    [​IMG]
     
    WhoBeDaPlaya likes this.
  8. Welsper

    Welsper Gawd

    Messages:
    639
    Joined:
    Jun 30, 2005
    ...400TB...good lord.
     
  9. gigatexal

    gigatexal [H]ardness Supreme

    Messages:
    7,120
    Joined:
    Jun 22, 2004
    That's a lot of uh pr0n right? ;)
     
  10. Welsper

    Welsper Gawd

    Messages:
    639
    Joined:
    Jun 30, 2005
    That...uh...might literally be all the pr0n.
     
  11. FLECOM

    FLECOM Modder(ator) & [H]ardest Folder Evar Staff Member

    Messages:
    15,569
    Joined:
    Jun 27, 2001
    because that's the largest drive my array supports

    would take about 5.5 years for the power savings to surpass the difference in drive cost... so.... no
     
    jlbenedict and Master_shake_ like this.
  12. Irockzz

    Irockzz n00b

    Messages:
    39
    Joined:
    May 27, 2016
    a little update
    2x Dell 2100's 2x Xeon x5650 - 112Gb ram - 850 evo boot - 12x 3TB Red's Raid 6
    3x Dell r420 2x Xeon x5570 - 32GB ram 4x 1,5Tb Raid 0

    [​IMG]
     
    Cool Runnings likes this.
  13. Machupo

    Machupo Gravity Tester

    Messages:
    4,797
    Joined:
    Nov 14, 2004
    Took the SFF NAS to a smaller case. 36.3TiB raw in a 7L enclosure :)

    C2550d4i
    32GB ECC
    10x 4TB Segate 2.5" drives
    pico PSU 160xt
    Lian Li PC-Q02 case

    uSFF_NAS.jpg
     
    Last edited: Apr 1, 2017
  14. WhoBeDaPlaya

    WhoBeDaPlaya 2[H]4U

    Messages:
    2,485
    Joined:
    Dec 16, 2002
    ^ Very nice! How have those fat 4TB 2.5" Seagates been holding out so far?
    Did you shuck them from externals (eg. Backup Plus)?
     
    FLECOM and 404 - File not found like this.
  15. Machupo

    Machupo Gravity Tester

    Messages:
    4,797
    Joined:
    Nov 14, 2004
    No issues so far (I switched to these a couple months ago) -- they were, in fact, shucked from externals (well, I don't know about four of them -- got the bare drives off of ebay and haven't bothered to check the warranty status). Speeds are fine (no issues w/ multiple plex HD streams), and I only resilvered when I was testing the system initially.
     
    FLECOM and 404 - File not found like this.
  16. m0po

    m0po n00b

    Messages:
    16
    Joined:
    May 14, 2014
    Well, time for an upgrade!

    Raw storage - 60TB (10 x 6TB raidz2)
    Actual storage - 43.6TB

    Case: Lian-Li PC-Q26B
    Motherboard: Supermicro X10SDV-4C-7TP4F
    CPU: Xeon-D 1518
    RAM: Kingston KVR21E15D8/8 x 2 (16GB)
    PSU: Silverstone SX600-G
    OS Disk: Intel 600p 128GB
    Storage Disks: 10 x Western Digital Red 6TB WD60EFRX
    OS: FreeBSD

    PSU had to be SFX as the motherboard is a bit too big for the case. Loving the case though!

    [​IMG]

    [​IMG]
     
  17. WhoBeDaPlaya

    WhoBeDaPlaya 2[H]4U

    Messages:
    2,485
    Joined:
    Dec 16, 2002
    ^ Very clean build. I need to up my game soon - my current dual 128TB builds are kinda holdovers from the better part of a decade ago :S
     
  18. m0po

    m0po n00b

    Messages:
    16
    Joined:
    May 14, 2014
    thanks! i wish i could've routed the 24pin and 8pin but theres just not enough room. even tried with extenders, doesnt work :(
     
  19. FLECOM

    FLECOM Modder(ator) & [H]ardest Folder Evar Staff Member

    Messages:
    15,569
    Joined:
    Jun 27, 2001
    adding some drives

    [​IMG]

    [​IMG]
     
  20. wra18th

    wra18th [H]ardness Supreme

    Messages:
    7,760
    Joined:
    Nov 11, 2009
  21. mstrzyze

    mstrzyze n00b

    Messages:
    3
    Joined:
    Dec 13, 2015
    Those 4tb WD you are running in netapp ds4243 shelf?(Looking at caddy)

    If yes where you connect that..just to PC?
     
  22. eptesicus

    eptesicus n00b

    Messages:
    62
    Joined:
    Jan 12, 2015
    Update to #202.

    [​IMG]


    7xSVR01
    Primary NAS and VMs (DC, Apps & Services, Downloads, WSUS, Monitoring, VPN, and Testing)

    Total Storage: 69.36 TB

    OS: Windows Server 2016
    Case: Supermicro SC846E16-R1200B
    PSU: Redundant 920W PWS-920P-1R 80PLUS Platinum (quieter over stock 1200W's)
    Mobo: ASRock E3C224-4L ATX
    CPU: Intel E3-1241v3 3.5GHz
    RAM: 32 GB Kingston 4x8 GB DDR3 ECC Unbuffered KVR16E11/8
    RAID: IBM M5015 w/ BBU
    SSD: 120 GB Samsung 850 Evo - OS
    SSD: 250 GB Samsung 850 Evo - VMs
    HDD: 14x 4 TB HGST NAS (Raid 5) - storage pool
    HDD: 4x 1 TB HGST Deskstar (Raid 5 w/ Hot Spare) - VM data pool
    HDD: 3x 3 TB Mixed Models (Raid 5) - temporary data pool
    Notes: I've added 2 more 4TB drives since the last posted on this thread and upgraded the host OS to Server 2016.


    7xSVR02
    Data Backup and VMs (TS, Nextcloud, Veeam/Allway Sync)

    Total Storage: 58.71 TB

    OS: Windows Server 2016
    Case: Supermicro SC846E16-R1200B
    PSU: Redundant 920W PWS-920P-1R 80PLUS Platinum (quieter over stock 1200W's)
    Mobo: ASRock E3C224 ATX
    CPU: Intel E3-1241v3 3.5GHz
    RAM: 32 GB Kingston 4x8 GB DDR3 ECC Unbuffered KVR16E11/8
    RAID: IBM M5015 w/ BBU
    SSD: 120 GB Samsung 840 Evo - OS
    HDD: 14x 4 TB HGST NAS (Raid 6) - backup pool

    HDD: 4x 320 GB Mixed Models (Raid 10) - VMs
    HDD: 3x 500 GB Mixed Models (Raid 5) - VM data pool
    Notes: I replaced the motherboard, CPU, and RAM with almost identical hardware with 7xSVR01. The OS was upgraded to 2016, and I added 12x 4TB HGST NAS drives for backing up 7xSVR01. I'm using Veeam to backup my VMs and Allway Sync to backup my data.

    Total storage between the 2? 128 TB


    I just moved to the country and somehow qualified for 1 GbE fiber internet. I'm running out of space and need to keep adding more HDDs to my servers... The downside to this is the reconstruction rate taking upwards of 10 days to expand the arrays. I wish I could just buy a large lot of drives to expand all at once, but right now, I'm just buying 2 HDDs a month.

    I have some Mellanox 10GbE cards coming in the mail from eBay that'll help in the transfer speed between the two servers. I'm anxious for 10GbE to become more affordable to consumers.


    What else is new? I built a new firewall to replace the Mikrotik.

    7xFW
    Firewall running OPNsense. No real storage here... But I love this little firewall.

    Model: Asus RS200-E9-PS2-F
    OS: FreeBSD 11.0 running OPNsense 17.1.8
    CPU: Intel i3-7100 3.90GHz
    RAM: 8GB Crucial 2x4GB
    SSD: 128GB ADATA SU800
     
    Last edited: Sep 15, 2017
  23. FLECOM

    FLECOM Modder(ator) & [H]ardest Folder Evar Staff Member

    Messages:
    15,569
    Joined:
    Jun 27, 2001
    yep, have 4x netapp ds4243 expanders attached to an IBM x3650 M3 (2x Adaptec 5085 gives me 4x 8088 external SAS ports)

    also adding some more drives :D

    [​IMG]
     
    Last edited: Aug 6, 2017
    spencers and WhoBeDaPlaya like this.
  24. Welsper

    Welsper Gawd

    Messages:
    639
    Joined:
    Jun 30, 2005
    Christ on a bike, FLECOM...
     
  25. Zepher

    Zepher [H]ipster Replacement

    Messages:
    16,650
    Joined:
    Sep 29, 2001
    I feel so tiny now, lol.

    HAL-9000 (Editing/Encoding Workstation)
    2TB WD Green 5400rpm
    2TB WD Green 5400rpm
    2TB Samsung F4 5400rpm
    4TB Seagate 5400rpm
    250GB SSD Crucial

    10.25TB
    __________________________________

    HAL-X100 (Gaming Rig)
    2TB WD My Book 5400rpm
    2TB WD Black 7200rpm
    4TB WD Black 7200rpm
    8TB WD Red 5400rpm
    512GB SSD Crucial

    16.5TB
    __________________________________

    Plex Server (Home Media Server)
    2TB Samsung F4 5400rpm
    2TB Samsung F4 5400rpm
    3TB Seagate 7200rpm
    4TB WD Green 5400rpm
    4TB WD Green 5400rpm
    4TB WD Red 5400rpm
    250GB SSD Crucial

    19.25TB
    ___________________________________

    Matrox (Burst Coin Mining Rig)
    2TB WD Green 5400rpm
    2TB WD Green 5400rpm
    4TB WD Black 7200rpm
    4TB Toshiba 7200rpm
    250GB SSD Crucial

    12.25TB
    ____________________________________

    58.25TB Total Drive Space
     
  26. WhoBeDaPlaya

    WhoBeDaPlaya 2[H]4U

    Messages:
    2,485
    Joined:
    Dec 16, 2002
    I think I found a new Ultrastar He8 bro - FLECOM :D
    <-- 32x He8 drives
     
    gigatexal likes this.
  27. Gavian

    Gavian Limp Gawd

    Messages:
    387
    Joined:
    Jan 13, 2016
    Wow that's some serious storage.
     
  28. Zarathustra[H]

    Zarathustra[H] Official Forum Curmudgeon

    Messages:
    27,595
    Joined:
    Oct 29, 2000
    I just finished my storage upgrade. Pretty happy with the results.

    upload_2017-12-29_16-25-23.png

    Doesn't come close to touching some of the extreme storage setups some of you have on here though!

    My 12x old 4TB WD Red drives have gone into a spare server and will be used for offsite backup.

    upload_2017-12-29_16-29-6.png
     
    Concentric and WhoBeDaPlaya like this.
  29. ashman

    ashman Gawd

    Messages:
    805
    Joined:
    Mar 28, 2011
    My backup server currently has this much in total storage, you can also see how much I've used and got left, it's running unRAID.


    [​IMG]
     

    Attached Files:

  30. jlbenedict

    jlbenedict [H]ard|Gawd

    Messages:
    1,305
    Joined:
    May 22, 2005
    unbelievable on some of the setups in here.. the only problem with this much storage: how do you back up that shit? i'm assuming prioritize on what you deem necessary to back up...
     
  31. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,637
    Joined:
    Jun 9, 2004
    I have a secondary ~100TB file server at a different location to back up my primary one (with a site-to-site VPN). The really important stuff has either one or two more copies on top of that (only a couple TB total). I have a SLC based USB flash drive for a safe deposit box as well which houses a copy of my legal documents and KeePass.

    Duplicate file servers and the likes might be viewed as overkill, but if you've ever experienced large data loss, you won't want to go through that ever again. A lot of my data is replaceable, just a real pain to do, so I'm paying a premium to save myself all the work. Safe deposit box costs very little too.
     
  32. Zarathustra[H]

    Zarathustra[H] Official Forum Curmudgeon

    Messages:
    27,595
    Joined:
    Oct 29, 2000
    As I hinted at in my post above, I just upgraded to 12x10TB drives in my ZFS Pool. I took the 12x4TB drives I already had and put them in an old server I had kicking around which will be going offsite for offsite backups.

    Now you'll note that my backup capacity is only roughly 40% of my total capacity and this is fine.

    You don't have to backup everything. I maintain different folders for different things. Irreplaceable files (my photo backups, tax records, etc. etc.)_ are all in folders that get backed up. My media library is all replaceable (though it would be a pain in the butt). Right now I have enough space to back it up, so I'm doing that, but if I run low on space I'll drop it.

    My 15TB DVR/PVR folder of recorded shows off of TV is disposable. I don't back it up at all.

    Etc. etc.
     
  33. ashman

    ashman Gawd

    Messages:
    805
    Joined:
    Mar 28, 2011
    I also have data duplicated on another server.
     
  34. Phelptwan

    Phelptwan [H]ardness Supreme

    Messages:
    6,466
    Joined:
    Jul 20, 2002
    My backup server is a DFS mirror of my primary server.
     
  35. Machupo

    Machupo Gravity Tester

    Messages:
    4,797
    Joined:
    Nov 14, 2004
    Another year and no issues with the drives. Just bought another 24 of them and am rebuilding into a custom short depth 2U case. Should triple my storage density (TB/L).
     
    WhoBeDaPlaya likes this.
  36. Orddie

    Orddie 2[H]4U

    Messages:
    2,401
    Joined:
    Dec 20, 2010
    what case are you using?
     
  37. Machupo

    Machupo Gravity Tester

    Messages:
    4,797
    Joined:
    Nov 14, 2004
    Found a relatively blank 15" 2U case, piece that together with the front end of an ebay supermicro sc-216 with some selective de-riveting/riveting. Toss in a platinum flex atx psu, a mish-mash of internals from the old NAS and the ebay special. Hopefully it works, lol.
     
  38. sirmonkey1985

    sirmonkey1985 [H]ard|DCer of the Month - July 2010

    Messages:
    21,079
    Joined:
    Sep 13, 2008
    lol didn't see that picture until now.. hope you bought them at the store like i did, pretty sure the lady thought i was nuts when i asked how many 8TB WD easystore drives they had in the back and said there was 14 of them. but the best reaction from her was when i said i'd take them all.. she kept asking me if i was sure i needed that many external drives. :p at the end of the day i didn't need them and actually gave away and sold a bunch of them to people. but hey it made my dream come true seeing it say 101TB free.. honestly never thought i'd see 100TB of storage be considered affordable this soon.
     
  39. Bigdady92

    Bigdady92 [H]ardness Supreme

    Messages:
    5,768
    Joined:
    Jun 20, 2001

    I've thrown those 4TB drives away, replaced them with 5TB drives. Realized that these 5TB are great in read speeds, awful in write speeds. Those went away and I have an 8x2.5" hole in my case with no drives to put in there.
     
  40. Zarathustra[H]

    Zarathustra[H] Official Forum Curmudgeon

    Messages:
    27,595
    Joined:
    Oct 29, 2000
    I find it interesting that 2.5" drives are moving into the mass storage market now.

    I've always been a little. It uncomfortable with them.

    How do you find the array performs with them compared to traditional 3.5" drives?

    When I recently upgraded my ZFS box, I went with 12x 10TB 7200rpm Seagate Enterprise drives. I wanted to make sure that the pool was responsive.

    If 10k rpm drives still existed in the top size tiers, I would have seriously considered them.