The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Status
Not open for further replies.
56+2 disks in a 4U design, although I found out that you can't hook two HP SAS Expanders to a ARC-1680IX-24. Currently only 44 disk is supported and a second ARC-1680IX would be needed to support the full 56 disk. The +2 is for the system disk that will be hooked directly to the motherboard. It would be awesome to have 3TB disk in them but I don't have access to that many 3TB disks.

I propose that we have a new ranking for density of disk per U. (;

Amount of storage in the following system: 43TB+

Case: Custom
PSU: Multiple Zippy 650W Redundant
Motherboard: Supermicro X8DTH-IF-O
CPU: (2) E5520 QC LGA1366 2.26GHz 8MB
RAM: (2) Crucial 4GB 240-Pin DIMM DDR3
GPU: None
Controller Cards: ARC-1680IX-24, HP SAS Expander
Optical Drives: None
Hard Drives: ST31000340NS, HD103UJ, HDS722020ALA330
Battery Backup Units: ARC-1680IX BBU
Operating System: Linux

The case is a proof-of-concept to see its possible to build a case to hold >40 disk in a 4U and is cost-effective. Unfortunately the current setup only hold 44 disk since the ARC-1680IX-24 doesn't work with two HP SAS Expanders.

Images:
http://imgur.com/f8zcp.jpg
http://imgur.com/lLKCH.jpg
http://imgur.com/YnYtQ.jpg
http://imgur.com/ZS5To.jpg
http://imgur.com/oK3uk.jpg
 
Last edited:
rsriprac said:
56+2 disks in a 4U design, although I found out that you can't hook two HP SAS Expanders to a ARC-1680IX-24. Currently only 44 disk is supported and a second ARC-1680IX would be needed to support the full 56 disk. The +2 is for the system disk that will be hooked directly to the motherboard. It would be awesome to have 3TB disk in them but I don't have access to that many 3TB disks.

I propose that we have a new ranking for density of disk per U. (;

Amount of storage in the following system: 43TB+

Case: Custom
PSU: Multiple Zippy 650W Redundant
Motherboard: Supermicro X8DTH-IF-O
CPU: (2) E5520 QC LGA1366 2.26GHz 8MB
RAM: (2) Crucial 4GB 240-Pin DIMM DDR3
GPU: None
Controller Cards: ARC-1680IX-24, HP SAS Expander
Optical Drives: None
Hard Drives: ST31000340NS, HD103UJ, HDS722020ALA330
Battery Backup Units: ARC-1680IX BBU
Operating System: Linux

The case is a proof-of-concept to see its possible to build a case to hold >40 disk in a 4U and is cost-effective. Unfortunately the current setup only hold 44 disk since the ARC-1680IX-24 doesn't work with two HP SAS Expanders.

[IM]http://imgur.com/f8zcp.jpg[/IMG]
[IM]http://imgur.com/lLKCH.jpg[/IMG]
[IM]http://imgur.com/YnYtQ.jpg[/IMG]
[IM]http://imgur.com/ZS5To.jpg[/IMG]
[IM]http://imgur.com/oK3uk.jpg[/IMG]
Click to expand...

Aww what a tease... lets see more of that red box.
 
dashpuppy said:
wish he would have followed the rules and not posted MILLION x MILLION resolution pictures.

UGG!
Click to expand...

yeah, I wasn't going to mention that, but I'm running 2560x1600 resolution on a 30" monitor and they still run off the screen. Sorry rsriprac, they are a bit big though.
 
I've fixed the images, when I was logged in, it didn't show me the images but rather the links so I didn't noticed it was all gimpy.
 
My tiny 26TB file server !!
This file server for HDmovie and Blueray movie.

Mainboard ZOTAC H55-ITX WiFi USB3
CPU Core i3 530
RAM Gskill PI 4GB 1600Mhz cas 8
HDD Seagate LP 2TB x 13
PSU Seasonic X series 650W
SATA control Supermicro's SASLP-MV8


SIfel.jpg


KY94z.jpg


n9wQB.jpg


4Q7FU.jpg


Rk2f7.jpg


rPJnB.jpg


qwDIq.jpg


4Q7FU.jpg






kF5HU.jpg


4B3xs.jpg




YbLv0.jpg



AXvaE.jpg





xqJPy.jpg


i1JAm.jpg


css6D.jpg


vhyh5.jpg
 
justbenice said:
My tiny 26TB file server !!
This file server for HDmovie and Blueray movie.

Mainboard ZOTAC H55-ITX WiFi USB3
CPU Core i3 530
RAM Gskill PI 4GB 1600Mhz cas 8
HDD Seagate LP 2TB x 13
PSU Seasonic X series 650W
SATA control Supermicro's SASLP-MV8


[IM]http://i.imgur.com/SIfel.jpg[/IMG]

[IM]http://i.imgur.com/KY94z.jpg[/IMG]
[IM]http://i.imgur.com/n9wQB.jpg[/IMG]

[IM]http://i.imgur.com/4Q7FU.jpg[/IMG]

[IM]http://i.imgur.com/Rk2f7.jpg[/IMG]

[IM]http://i.imgur.com/rPJnB.jpg[/IMG]

[IM]http://i.imgur.com/qwDIq.jpg[/IMG]

[IM]http://i.imgur.com/4Q7FU.jpg[/IMG]





[IM]http://i.imgur.com/kF5HU.jpg[/IMG]

[IM]http://i.imgur.com/4B3xs.jpg[/IMG]



[IM]http://i.imgur.com/YbLv0.jpg[/IMG]


[IM]http://i.imgur.com/AXvaE.jpg[/IMG]




[IM]http://i.imgur.com/xqJPy.jpg[/IMG]

[IM]http://i.imgur.com/i1JAm.jpg[/IMG]

[IM]http://i.imgur.com/css6D.jpg[/IMG]

[IM]http://i.imgur.com/vhyh5.jpg[/IMG]
Click to expand...



So I have to know, did you build this in the past 2 months? I saw your rats nets back in October, this is quite an amazing case you've created. This is the exact reason why I want to start working with Solidworks more.
 
cybertron said:
So I have to know, did you build this in the past 2 months? I saw your rats nets back in October, this is quite an amazing case you've created. This is the exact reason why I want to start working with Solidworks more.
Click to expand...

Yes, and you still remember my rats nets in October. :p . It take me almost 2 month to mod this case. Now i am quite please with this case .
 
cybertron said:
I'm sure he's doing software raid like freeNAS or the like... ZFS maybe.
Click to expand...

Becase this "file server" for store movie only, there are no thing important in it so i just mount all of my 13 HDD to one folder and share it to my HDPLAYER via hanewinNFS :)
 
justbenice said:
Yes, and you still remember my rats nets in October. :p . It take me almost 2 month to mod this case. Now i am quite please with this case .
Click to expand...

Well, technically, I just looked up your posts, hehe, but I did remember seeing that when it was posted, though not identifying it as yours. :)

justbenice said:
Becase this "file server" for store movie only, there are no thing important in it so i just mount all of my 13 HDD to one folder and share it to my HDPLAYER via hanewinNFS :)
Click to expand...

With that much space, you should really consider something like ZFS. I know it might not be important data, but imagine if you ripped 100 DVD's/Bluray movies to that thing and a single drive failed... the world continues turning, but wow that'd be real annoying. Your setup is actually exactly what I'm wanting to do...

So how did you get those parts made? I guess you could design it all on something like google sketch or solidworks, and then send the files off to one of those internet fabrication sites? sounds expensive.
 
cybertron said:
Well, technically, I just looked up your posts, hehe, but I did remember seeing that when it was posted, though not identifying it as yours. :)



With that much space, you should really consider something like ZFS. I know it might not be important data, but imagine if you ripped 100 DVD's/Bluray movies to that thing and a single drive failed... the world continues turning, but wow that'd be real annoying. Your setup is actually exactly what I'm wanting to do...

So how did you get those parts made? I guess you could design it all on something like google sketch or solidworks, and then send the files off to one of those internet fabrication sites? sounds expensive.
Click to expand...

Thanks cybertron, but with all of full bluray i'v download in past 2 month, i almost use all of 26TB, i know that if one driver fall, it will take like weeks to re-download what it contain in that driver. I will consider software raid when the 3TB harddisk price drop and i change all of my harddisk to 3TB :)
About the case mod, i'v just imaging what this file server look like and ask my friend do the mod, he bought everything and do a cnc all the parts. It quite cheap though, i spend around 220usd in this case mod only. I am living in Viet Nam and everything cheap like hell here . :)
 
My machine (upgrade)

Power Supply: Corsair HX 1000W

Network:
*Intel Gigabit ET Server Adapter, 2x 1000Base-T

RAID Controller:
*Areca ARC-1680- ix- 24

RAID6
* 20 x Hitachi DeskStar 7K2000 2000GB (HDS722020ALA330)

Single SSD:
* 1 x Transcend SSD 2.5" 32GB (TS32GSSD25S-M)













 
Renegate, what case is that? thats a heck of a system and I love the having all the drive caddys in there. neat cable job too
 
Activate: AMD said:
Renegate, what case is that? thats a heck of a system and I love the having all the drive caddys in there. neat cable job too
Click to expand...

That's the old Coolermaster Stacker case. Too bad they discontinued that case a few years back
 
Case : Supermicro 3U SC933 modded with a 2U PSU
PSU : 2U Supermicro modded PWS-0047 with a 80mm PAPST fan
Motherboard : Tyan 7002 dual Xeon
CPU : dual Xeon 5504 cooled by 2 Thermalright HR01-3U
RAM : 12 GB triple channel ECC
Controller Cards : ARECA 1260
Hard Drives : 15 Western Digital WD20EARS 2Tb Caviar Green 667GB/platter
System is install on a SSD OCZ Vertex 2 60GB
Operating System : Windows Server 2008

This system is my new NAS. I managed to remove all fan and quiet everything that was possible. The drives were chosen because of the low noise and good performance. In order to cool them, I modded the case top put 3 120mm PAPST fan in front of them. The special bracket still need to be made, but it will be done quickly now.
The main goal is silence and everything was made in order to attain this goal.
At the present time the machine is still under test, I need to get the 2nd CPU cooler, the first one is just on top of the CPU... not bad, the CPU temp is 53C for each core. The temperature reported by the ARECA card are lower than 28C. I must precise that the 120mm fan are working at 5V via a special LIAN LI bracket. they are inaudible.
I made some load test for the network transfert. I am around 70mB/s. I am fine with that.

I plan to make 2 arrays, one with RAID 1, and the other one with RAID 6. I am still thinking using Solaris with ZFS... The system have a 2TB hard drive in order to host the HyperV system I plan to use.

IMG_1425_small.JPG

IMG_1426_small.JPG

bureau_small.JPG
 
50 terabytes thus far
Turns out my cards are not raid. For CES I will do Flexraid as I need to get this done and shipped. When I get back I need to get a raid card and SAS expander. Anyway to use my existing SATA cables so I do not have to redo all my cable management?
20100913191030.jpg

dscf0292b.jpg

dscf02900.jpg

dscf0289.jpg

dscf02880.jpg

dscf0285c.jpg

dscf0283tt.jpg

dscf02950.jpg

dscf0301v.jpg

dscf0300j.jpg

dscf0297e.jpg
 
amazing workmanship and consistency in pulling off the colors of the theme. Really cool rig.
 
I have 2 big boxes. One will be upgraded this week so I'll list its new specs. Both boxes serve NFS for vSphere and an 11.2tb iSCSI target for Windows file services. Both run Fedora 14 with minimal services and no X.

nfs1:
SuperMicro SuperChassis 745TQ-R800
SuperMicro X8SIL-F
12gb RAM
Xeon X3430
Adaptec 2405
SuperMicro M14T rack for 2.5" drives
4x2.5" 500gb Hitachi EA series (RAID 10)
LSI SAS3081R in IT mode
8x2tb Seagate drives (software RAID 6)

nfs2:
Lian-Li PC-7077 modded for an iStar redundant PSU
SuperMicro X8SIL-F
8gb RAM
Xeon X3430
LSI 80416E
iStar mobile rack for 2.5" drives
4x2.5" 500gb Hitachi EA series (RAID 10)
3x AICIPC mobile racks for 4 3.5" drives
12x WD20EADS 2tb drives, TLER enabled (RAID 6)
 
Last edited:
Thank you for all those pictures and talking through the process.. amazing. when will we get more?
 
My second machine (The Slim one) :D

Power Supply: Corsair HX 650W

RAID Controller:
*Areca ARC-1680- ix- 12

RAID6
* 12 x Seagate 1TB

Single Disks:
* 1 x Seagate 80GB
* 1 x Seagate 1TB
* 2 x Seagate 500GB







 
Renegate1666, Nice job on the sata power plugs! but I'm confused what the lower card is that attaches to the sata card... can you help me? raid battery?

d.jpg
 
picker said:
Renegate1666, Nice job on the sata power plugs! but I'm confused what the lower card is that attaches to the sata card... can you help me? raid battery?
Click to expand...
Looks like an Areca Battery Backup Unit (BBU).

Agree on power distribution. Well done, very clean.
 
Total Storage: 36TB + 2TB + 64GB + 64GB
Usable Storage: 30TB (Raid 6 + Hot SPARE) + 1TB (Raid 1 Home) + 64GB (Raid 1 (Linux MD Raid) OS Drive)

Missing ONE Drive, waiting for RMA from Seagate!

Case: Li Lian PC-A77F
Motherboard: Asus M4A89GTD PRO/USB3
CPU: AMD Phenom II X6 1055T
Memory: 2 x OCZ Technology 4GB DDR3-1600 Dual Channel Memory Kit
Power Supply: Antec True Power 750w
Raid Controller: Areca 1880ix-16 1GB
Disk Trays (Raid1 OS Drives): 1 x Vantec MRK-525ST 2 in 1 (Provides 2 2.5" Drive bays in a single 5.25" bay)
Disk Trays (Raid6 Drives): 3 x Icy Dock MB455SPF-B 5 in 3
Disk Tray (Raid6 Drive Single): 1 x Icy Dock MB672SKGF-BB - Had to do this to get another Drive into the Case to utilize all the 16ports of the Areca
External Disk Tray (HotSPARE, Raid6 Drive Single, Raid1 Pair of Drives): 4 bay SAS/SATA Hotswap Tower Enclosure (http://www.pc-pitstop.com/sas_cables_enclosures/sas4bay.asp)
RAID Disks: 18 x Seagate Barracuda XT 2TB 6/GBs
OS Drive: 2 x MicroCenter 64GB SSD (SandForce 1222 Controller)
Optical Drive: Standard DVD Drive
Home Disks: 2 x Western Digital Black 1TB 6/GBs
Operating System: Ubuntu 10.10 X64
Cables: SFF-8088 to SFF-8088 (Serves Raid1 Drives, Hot SPARE, and additional Raid6 Drive)
Cables: SFF-8087 to SATA (Serves internal Raid6 Drives)
Network Card: HP NC364T PCI Express Quad Port Gigabit Server Adapter (Work Freebie)

Extras:

Windows 7 32Bit Running under Virtual Box (Used for iTunes Media Server)
NetaTalk serving AFP (TimeMachine for Home Macs)
Samba serving CIFS (General usage, also serving PopCorn Hour A-110)
SABNZBd
Transmission

Pictures:

beast_1.jpg

beast_2.jpg

beast_3.jpg

beast_4.jpg


Code: 
root@beast:~# pvs
  PV         VG             Fmt  Attr PSize   PFree  
  /dev/md1   vg_ssd         lvm2 a-    55.40g  41.96g
  /dev/sda   vg_areca_raid1 lvm2 a-   931.32g 861.32g
  /dev/sdb   vg_areca_raid6 lvm2 a-    25.47t  21.47t
root@beast:~# vgs
  VG             #PV #LV #SN Attr   VSize   VFree  
  vg_areca_raid1   1   3   0 wz--n- 931.32g 861.32g
  vg_areca_raid6   1  10   0 wz--n-  25.47t  21.47t
  vg_ssd           1   6   0 wz--n-  55.40g  41.96g
root@beast:~# lvs
  LV          VG             Attr   LSize   Origin Snap%  Move Log Copy%  Convert
  Home        vg_areca_raid1 -wi-ao  50.00g                                      
  Opt         vg_areca_raid1 -wi-ao  10.00g                                      
  SyslogNG    vg_areca_raid1 -wi-ao  10.00g                                      
  Documents   vg_areca_raid6 -wi-ao  10.00g                                      
  NMT         vg_areca_raid6 -wi-ao 500.00g                                      
  News        vg_areca_raid6 -wi-ao 500.00g                                      
  Scripts     vg_areca_raid6 -wi-ao   5.00g                                      
  Software    vg_areca_raid6 -wi-ao 120.00g                                      
  Source      vg_areca_raid6 -wi-ao   5.00g                                      
  TimeMachine vg_areca_raid6 -wi-ao   2.00t                                      
  Torrents    vg_areca_raid6 -wi-ao 200.00g                                      
  Videos      vg_areca_raid6 -wi-ao 500.00g                                      
  iTunes      vg_areca_raid6 -wi-ao 200.00g                                      
  rootvol     vg_ssd         -wi-ao 976.00m                                      
  swapvol01   vg_ssd         -wi-ao   1.91g                                      
  swapvol02   vg_ssd         -wi-ao   1.91g                                      
  tmpvol      vg_ssd         -wi-ao 976.00m                                      
  usrvol      vg_ssd         -wi-ao   4.81g                                      
  varvol      vg_ssd         -wi-ao   2.91g                                      
root@beast:~#
 
Yes that's the Battery- Pack. It's not as good as a USV but it save your raid during a power breakdown.
I'm happy that you like my work! :)
I like to change the SAS to Sata adapters to clean the case like my big on.
 
Last edited:
Status
Not open for further replies.
Back
Top