The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Discussion in 'SSDs & Data Storage' started by EnderW, Feb 16, 2009.

Thread Status:
Not open for further replies.
  1. dashpuppy

    dashpuppy [H]ardness Supreme

    Messages:
    6,165
    Joined:
    May 5, 2010
    smidly, looks so sexy, great work nice rack :)

    J'
     
  2. rsriprac

    rsriprac n00b

    Messages:
    8
    Joined:
    Mar 11, 2010
    56+2 disks in a 4U design, although I found out that you can't hook two HP SAS Expanders to a ARC-1680IX-24. Currently only 44 disk is supported and a second ARC-1680IX would be needed to support the full 56 disk. The +2 is for the system disk that will be hooked directly to the motherboard. It would be awesome to have 3TB disk in them but I don't have access to that many 3TB disks.

    I propose that we have a new ranking for density of disk per U. (;

    Amount of storage in the following system: 43TB+

    Case: Custom
    PSU: Multiple Zippy 650W Redundant
    Motherboard: Supermicro X8DTH-IF-O
    CPU: (2) E5520 QC LGA1366 2.26GHz 8MB
    RAM: (2) Crucial 4GB 240-Pin DIMM DDR3
    GPU: None
    Controller Cards: ARC-1680IX-24, HP SAS Expander
    Optical Drives: None
    Hard Drives: ST31000340NS, HD103UJ, HDS722020ALA330
    Battery Backup Units: ARC-1680IX BBU
    Operating System: Linux

    The case is a proof-of-concept to see its possible to build a case to hold >40 disk in a 4U and is cost-effective. Unfortunately the current setup only hold 44 disk since the ARC-1680IX-24 doesn't work with two HP SAS Expanders.

    Images:
    http://imgur.com/f8zcp.jpg
    http://imgur.com/lLKCH.jpg
    http://imgur.com/YnYtQ.jpg
    http://imgur.com/ZS5To.jpg
    http://imgur.com/oK3uk.jpg
     
    Last edited: Dec 7, 2010
  3. cybertron

    cybertron [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Dec 13, 2002
    Aww what a tease... lets see more of that red box.
     
  4. dashpuppy

    dashpuppy [H]ardness Supreme

    Messages:
    6,165
    Joined:
    May 5, 2010
    wish he would have followed the rules and not posted MILLION x MILLION resolution pictures.

    UGG!
     
  5. cybertron

    cybertron [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Dec 13, 2002
    yeah, I wasn't going to mention that, but I'm running 2560x1600 resolution on a 30" monitor and they still run off the screen. Sorry rsriprac, they are a bit big though.
     
  6. rsriprac

    rsriprac n00b

    Messages:
    8
    Joined:
    Mar 11, 2010
    I've fixed the images, when I was logged in, it didn't show me the images but rather the links so I didn't noticed it was all gimpy.
     
  7. justbenice

    justbenice Limp Gawd

    Messages:
    170
    Joined:
    Oct 8, 2010
    My tiny 26TB file server !!
    This file server for HDmovie and Blueray movie.

    Mainboard ZOTAC H55-ITX WiFi USB3
    CPU Core i3 530
    RAM Gskill PI 4GB 1600Mhz cas 8
    HDD Seagate LP 2TB x 13
    PSU Seasonic X series 650W
    SATA control Supermicro's SASLP-MV8


    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]





    [​IMG]

    [​IMG]



    [​IMG]


    [​IMG]




    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  8. cybertron

    cybertron [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Dec 13, 2002


    So I have to know, did you build this in the past 2 months? I saw your rats nets back in October, this is quite an amazing case you've created. This is the exact reason why I want to start working with Solidworks more.
     
  9. saedrin

    saedrin [H]ard|Gawd

    Messages:
    1,978
    Joined:
    Jan 5, 2009
    All that space and no RAID?!
     
  10. cybertron

    cybertron [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Dec 13, 2002
    I'm sure he's doing software raid like freeNAS or the like... ZFS maybe.
     
  11. justbenice

    justbenice Limp Gawd

    Messages:
    170
    Joined:
    Oct 8, 2010
    Yes, and you still remember my rats nets in October. :p . It take me almost 2 month to mod this case. Now i am quite please with this case .
     
  12. justbenice

    justbenice Limp Gawd

    Messages:
    170
    Joined:
    Oct 8, 2010
    Becase this "file server" for store movie only, there are no thing important in it so i just mount all of my 13 HDD to one folder and share it to my HDPLAYER via hanewinNFS :)
     
  13. cybertron

    cybertron [H]ard|Gawd

    Messages:
    1,072
    Joined:
    Dec 13, 2002
    Well, technically, I just looked up your posts, hehe, but I did remember seeing that when it was posted, though not identifying it as yours. :)

    With that much space, you should really consider something like ZFS. I know it might not be important data, but imagine if you ripped 100 DVD's/Bluray movies to that thing and a single drive failed... the world continues turning, but wow that'd be real annoying. Your setup is actually exactly what I'm wanting to do...

    So how did you get those parts made? I guess you could design it all on something like google sketch or solidworks, and then send the files off to one of those internet fabrication sites? sounds expensive.
     
  14. justbenice

    justbenice Limp Gawd

    Messages:
    170
    Joined:
    Oct 8, 2010
    Thanks cybertron, but with all of full bluray i'v download in past 2 month, i almost use all of 26TB, i know that if one driver fall, it will take like weeks to re-download what it contain in that driver. I will consider software raid when the 3TB harddisk price drop and i change all of my harddisk to 3TB :)
    About the case mod, i'v just imaging what this file server look like and ask my friend do the mod, he bought everything and do a cnc all the parts. It quite cheap though, i spend around 220usd in this case mod only. I am living in Viet Nam and everything cheap like hell here . :)
     
  15. justbenice

    justbenice Limp Gawd

    Messages:
    170
    Joined:
    Oct 8, 2010
    Some of pics that i did not upload here :

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  16. nilsga

    nilsga n00b

    Messages:
    25
    Joined:
    Nov 2, 2009
    Wow! I want that cabinet!
     
  17. Renegate1666

    Renegate1666 n00b

    Messages:
    12
    Joined:
    Oct 6, 2008
    My machine (upgrade)

    Power Supply: Corsair HX 1000W

    Network:
    *Intel Gigabit ET Server Adapter, 2x 1000Base-T

    RAID Controller:
    *Areca ARC-1680- ix- 24

    RAID6
    * 20 x Hitachi DeskStar 7K2000 2000GB (HDS722020ALA330)

    Single SSD:
    * 1 x Transcend SSD 2.5" 32GB (TS32GSSD25S-M)

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  18. dashpuppy

    dashpuppy [H]ardness Supreme

    Messages:
    6,165
    Joined:
    May 5, 2010
    ^^^ Renegate1666, thats a nice portable heater you built :) nice and clean!!
     
  19. Activate: AMD

    Activate: AMD [H]ard|Gawd

    Messages:
    1,995
    Joined:
    Nov 6, 2004
    Renegate, what case is that? thats a heck of a system and I love the having all the drive caddys in there. neat cable job too
     
  20. Dangman

    Dangman Ninja Editor SuperMod

    Messages:
    46,106
    Joined:
    Dec 15, 2005
    That's the old Coolermaster Stacker case. Too bad they discontinued that case a few years back
     
  21. Activate: AMD

    Activate: AMD [H]ard|Gawd

    Messages:
    1,995
    Joined:
    Nov 6, 2004
    yea I thought so, but with all the drives in there its hard to recognize. I was curious to see if it was a special box.
     
  22. sergi00

    sergi00 n00b

    Messages:
    17
    Joined:
    Dec 10, 2010
    Case : Supermicro 3U SC933 modded with a 2U PSU
    PSU : 2U Supermicro modded PWS-0047 with a 80mm PAPST fan
    Motherboard : Tyan 7002 dual Xeon
    CPU : dual Xeon 5504 cooled by 2 Thermalright HR01-3U
    RAM : 12 GB triple channel ECC
    Controller Cards : ARECA 1260
    Hard Drives : 15 Western Digital WD20EARS 2Tb Caviar Green 667GB/platter
    System is install on a SSD OCZ Vertex 2 60GB
    Operating System : Windows Server 2008

    This system is my new NAS. I managed to remove all fan and quiet everything that was possible. The drives were chosen because of the low noise and good performance. In order to cool them, I modded the case top put 3 120mm PAPST fan in front of them. The special bracket still need to be made, but it will be done quickly now.
    The main goal is silence and everything was made in order to attain this goal.
    At the present time the machine is still under test, I need to get the 2nd CPU cooler, the first one is just on top of the CPU... not bad, the CPU temp is 53C for each core. The temperature reported by the ARECA card are lower than 28C. I must precise that the 120mm fan are working at 5V via a special LIAN LI bracket. they are inaudible.
    I made some load test for the network transfert. I am around 70mB/s. I am fine with that.

    I plan to make 2 arrays, one with RAID 1, and the other one with RAID 6. I am still thinking using Solaris with ZFS... The system have a 2TB hard drive in order to host the HyperV system I plan to use.

    [​IMG]
    [​IMG]
    [​IMG]
     
  23. DarthBeavis

    DarthBeavis 2[H]4U

    Messages:
    3,147
    Joined:
    Feb 24, 2006
    50 terabytes thus far
    Turns out my cards are not raid. For CES I will do Flexraid as I need to get this done and shipped. When I get back I need to get a raid card and SAS expander. Anyway to use my existing SATA cables so I do not have to redo all my cable management?
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
  24. war9200

    war9200 Limp Gawd

    Messages:
    273
    Joined:
    Oct 9, 2010
    That is one awesome build...
    How long did it take you to make it?
     
  25. war9200

    war9200 Limp Gawd

    Messages:
    273
    Joined:
    Oct 9, 2010
    Nice to see it can be done (quiet supermicro)
     
  26. DarthBeavis

    DarthBeavis 2[H]4U

    Messages:
    3,147
    Joined:
    Feb 24, 2006
    been working on it a couple months. Thanks for the kind words
     
  27. shetu

    shetu [H]Lite

    Messages:
    118
    Joined:
    Jul 23, 2010
  28. brycejones

    brycejones [H]ardness Supreme

    Messages:
    4,381
    Joined:
    Jan 29, 2002
    amazing workmanship and consistency in pulling off the colors of the theme. Really cool rig.
     
  29. lizardking009

    lizardking009 Gawd

    Messages:
    663
    Joined:
    Nov 18, 2005
    I have 2 big boxes. One will be upgraded this week so I'll list its new specs. Both boxes serve NFS for vSphere and an 11.2tb iSCSI target for Windows file services. Both run Fedora 14 with minimal services and no X.

    nfs1:
    SuperMicro SuperChassis 745TQ-R800
    SuperMicro X8SIL-F
    12gb RAM
    Xeon X3430
    Adaptec 2405
    SuperMicro M14T rack for 2.5" drives
    4x2.5" 500gb Hitachi EA series (RAID 10)
    LSI SAS3081R in IT mode
    8x2tb Seagate drives (software RAID 6)

    nfs2:
    Lian-Li PC-7077 modded for an iStar redundant PSU
    SuperMicro X8SIL-F
    8gb RAM
    Xeon X3430
    LSI 80416E
    iStar mobile rack for 2.5" drives
    4x2.5" 500gb Hitachi EA series (RAID 10)
    3x AICIPC mobile racks for 4 3.5" drives
    12x WD20EADS 2tb drives, TLER enabled (RAID 6)
     
    Last edited: Dec 14, 2010
  30. makwy2

    makwy2 n00b

    Messages:
    41
    Joined:
    Dec 7, 2010
    Thank you for all those pictures and talking through the process.. amazing. when will we get more?
     
  31. Renegate1666

    Renegate1666 n00b

    Messages:
    12
    Joined:
    Oct 6, 2008
    My second machine (The Slim one) :D

    Power Supply: Corsair HX 650W

    RAID Controller:
    *Areca ARC-1680- ix- 12

    RAID6
    * 12 x Seagate 1TB

    Single Disks:
    * 1 x Seagate 80GB
    * 1 x Seagate 1TB
    * 2 x Seagate 500GB

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  32. picker

    picker n00b

    Messages:
    46
    Joined:
    Sep 14, 2010
    Renegate1666, Nice job on the sata power plugs! but I'm confused what the lower card is that attaches to the sata card... can you help me? raid battery?

    [​IMG]
     
  33. PigLover

    PigLover [H]ard|Gawd

    Messages:
    1,168
    Joined:
    Jul 11, 2009
    Looks like an Areca Battery Backup Unit (BBU).

    Agree on power distribution. Well done, very clean.
     
  34. pyrodex

    pyrodex [H]Lite

    Messages:
    105
    Joined:
    Mar 12, 2004
    Total Storage: 36TB + 2TB + 64GB + 64GB
    Usable Storage: 30TB (Raid 6 + Hot SPARE) + 1TB (Raid 1 Home) + 64GB (Raid 1 (Linux MD Raid) OS Drive)

    Missing ONE Drive, waiting for RMA from Seagate!

    Case: Li Lian PC-A77F
    Motherboard: Asus M4A89GTD PRO/USB3
    CPU: AMD Phenom II X6 1055T
    Memory: 2 x OCZ Technology 4GB DDR3-1600 Dual Channel Memory Kit
    Power Supply: Antec True Power 750w
    Raid Controller: Areca 1880ix-16 1GB
    Disk Trays (Raid1 OS Drives): 1 x Vantec MRK-525ST 2 in 1 (Provides 2 2.5" Drive bays in a single 5.25" bay)
    Disk Trays (Raid6 Drives): 3 x Icy Dock MB455SPF-B 5 in 3
    Disk Tray (Raid6 Drive Single): 1 x Icy Dock MB672SKGF-BB - Had to do this to get another Drive into the Case to utilize all the 16ports of the Areca
    External Disk Tray (HotSPARE, Raid6 Drive Single, Raid1 Pair of Drives): 4 bay SAS/SATA Hotswap Tower Enclosure (http://www.pc-pitstop.com/sas_cables_enclosures/sas4bay.asp)
    RAID Disks: 18 x Seagate Barracuda XT 2TB 6/GBs
    OS Drive: 2 x MicroCenter 64GB SSD (SandForce 1222 Controller)
    Optical Drive: Standard DVD Drive
    Home Disks: 2 x Western Digital Black 1TB 6/GBs
    Operating System: Ubuntu 10.10 X64
    Cables: SFF-8088 to SFF-8088 (Serves Raid1 Drives, Hot SPARE, and additional Raid6 Drive)
    Cables: SFF-8087 to SATA (Serves internal Raid6 Drives)
    Network Card: HP NC364T PCI Express Quad Port Gigabit Server Adapter (Work Freebie)

    Extras:

    Windows 7 32Bit Running under Virtual Box (Used for iTunes Media Server)
    NetaTalk serving AFP (TimeMachine for Home Macs)
    Samba serving CIFS (General usage, also serving PopCorn Hour A-110)
    SABNZBd
    Transmission

    Pictures:

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]

    Code:
    root@beast:~# pvs
      PV         VG             Fmt  Attr PSize   PFree  
      /dev/md1   vg_ssd         lvm2 a-    55.40g  41.96g
      /dev/sda   vg_areca_raid1 lvm2 a-   931.32g 861.32g
      /dev/sdb   vg_areca_raid6 lvm2 a-    25.47t  21.47t
    root@beast:~# vgs
      VG             #PV #LV #SN Attr   VSize   VFree  
      vg_areca_raid1   1   3   0 wz--n- 931.32g 861.32g
      vg_areca_raid6   1  10   0 wz--n-  25.47t  21.47t
      vg_ssd           1   6   0 wz--n-  55.40g  41.96g
    root@beast:~# lvs
      LV          VG             Attr   LSize   Origin Snap%  Move Log Copy%  Convert
      Home        vg_areca_raid1 -wi-ao  50.00g                                      
      Opt         vg_areca_raid1 -wi-ao  10.00g                                      
      SyslogNG    vg_areca_raid1 -wi-ao  10.00g                                      
      Documents   vg_areca_raid6 -wi-ao  10.00g                                      
      NMT         vg_areca_raid6 -wi-ao 500.00g                                      
      News        vg_areca_raid6 -wi-ao 500.00g                                      
      Scripts     vg_areca_raid6 -wi-ao   5.00g                                      
      Software    vg_areca_raid6 -wi-ao 120.00g                                      
      Source      vg_areca_raid6 -wi-ao   5.00g                                      
      TimeMachine vg_areca_raid6 -wi-ao   2.00t                                      
      Torrents    vg_areca_raid6 -wi-ao 200.00g                                      
      Videos      vg_areca_raid6 -wi-ao 500.00g                                      
      iTunes      vg_areca_raid6 -wi-ao 200.00g                                      
      rootvol     vg_ssd         -wi-ao 976.00m                                      
      swapvol01   vg_ssd         -wi-ao   1.91g                                      
      swapvol02   vg_ssd         -wi-ao   1.91g                                      
      tmpvol      vg_ssd         -wi-ao 976.00m                                      
      usrvol      vg_ssd         -wi-ao   4.81g                                      
      varvol      vg_ssd         -wi-ao   2.91g                                      
    root@beast:~# 
    
     
  35. Renegate1666

    Renegate1666 n00b

    Messages:
    12
    Joined:
    Oct 6, 2008
    Yes that's the Battery- Pack. It's not as good as a USV but it save your raid during a power breakdown.
    I'm happy that you like my work! :)
    I like to change the SAS to Sata adapters to clean the case like my big on.
     
    Last edited: Dec 18, 2010
  36. war9200

    war9200 Limp Gawd

    Messages:
    273
    Joined:
    Oct 9, 2010
    Have you had any disks fail or failing in that case?
     
  37. Renegate1666

    Renegate1666 n00b

    Messages:
    12
    Joined:
    Oct 6, 2008
    Yes, it's hard work to remove the disk and put an new one inside, but I like to work on my PC's :D
     
  38. datemike

    datemike n00b

    Messages:
    1
    Joined:
    Dec 17, 2010
    Who makes that bayconverter? Nice system renegate
     
  39. pherson

    pherson n00b

    Messages:
    6
    Joined:
    Sep 17, 2008
    Modified (bigger fan) Yeong-Yang W20x-cage for YY-W201 / R502 / Cube
     
  40. Renegate1666

    Renegate1666 n00b

    Messages:
    12
    Joined:
    Oct 6, 2008
    pherson you have them too! show us the original one! :D
     
Thread Status:
Not open for further replies.