The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Status
Not open for further replies.
Nothing fancy but meets the reqs;

Case: Norco RPC-3216A 3U
PSU: Seasonic X-560
MB: ASUS P8B-WS
CPU: Intel Xeon E3-1260L
Mem: 16GB ECC
JBOD: Areca ARC-1320ixl-16
UPS: APC Smart-UPS 750 with SNMP management-card

2x Western Digital 160 GB Black Scorpio System RAID MIRROR
8x Seagate Barracuda 3TB ST3000DM001 Storage RAID Z2 => 18TB (24TB)
1x OCZ Vertex4 128GB SSD (dedicated to a Virtualbox machine)
1x Samsung 320GB E-SATA (dedicated to a Virtualbox machine)

Running Solaris 11 11/11 and Virtualbox


Quite a bit of work to setup Solaris stable with this x86-hardware. Have for example noticed that scrubbing of the storage-pool hangs if the virtual-box machines are quite active (even when they are using own physical disks). The only remedy for this is to reboot the machine at the time and avoid starting the VM's. Could be something else that causes this situation, haven't investigated it that much (the scrub cron runs each 3 month so not that noticable).

Since there's no UPS-management software from APC for Solaris when running headless, so I've got a VB-script running on one of the VM's that logs in and checks status and if battery is low sends a Solaris shutdown. This script also does some other fancy things like checking the internet connections (have two), update a webpage with statistics and so on.

Didn't bother with the 4k, am quite ok with the performance from 512 emulated on the 3TB Seagate Barracuda's.

iphoto.jpg
 
The case in #1274 should be a Xigmatek Elysium, the one in #1275 might be a modded Sharkoon Rebel 12
 
rust0r said:
Thank you to EVERYONE who posts their system here, it allows people like me to see what options are out there, and what creative things people have done and incorporate or branch off of those ideas
Click to expand...


I agree,
by the way, it's kinda hard to see where the sata-power cable are going
and what it look like when it's closed

I also recommand SnapRaid over Disparity, http://snapraid.sourceforge.net/compare.html
but if disparity suit you...i've nothing to say :)
 
Asposium said:
and this one...
what case?

are there many other case with 12 5.25inch bays?

8casefinished.jpg
Click to expand...

It's a Sharkon Rebel 12, german case, it's hard to find. I found one on ebay for around $120+$20 shipping USD

asuranii said:
very good, I'm not as good for the cable :D
Click to expand...

Thank you :)

asuranii said:
I agree,
by the way, it's kinda hard to see where the sata-power cable are going
and what it look like when it's closed

I also recommand SnapRaid over Disparity, http://snapraid.sourceforge.net/compare.html
but if disparity suit you...i've nothing to say :)
Click to expand...

The sata-power cables and all other cabling (excess SAS cable, fan power cables and such) are all stuffed on the back side of the hard drive tower somewhat neatly. The back panel goes on without much fuss or force.

Hmm, thanks for the recommendation, I haven't heard of SnapRaid before but I will look into it, what benefit does it have over Disparity from your experience?
 
rust0r said:
Hmm, thanks for the recommendation, I haven't heard of SnapRaid before but I will look into it, what benefit does it have over Disparity from your experience?
Click to expand...
(sorry for my english )


well I haven't tested disparity
but from what I've read SnapRaid work by block size, not by file

and it's Opensource, so maybe it's better for getting more updates and bug patch
from the link I've provided you can see the developpment of Disparity has not been updated since 2009, and snapraid last update is more recent (2011)

and what I needed for my snapshot raid/parity was RAID6 feature (2 disk failure)
 
this is a virtualized (ESXi 5U1) unRaid server

hal (unRaid DAS)
Case: Norco RPC-4220
PSU: Corsair AX-850
SAS Expander: Chenbro CK23601 (paired to an IBM M1015-IT mode)
Hard Disks: 20x 2TB (different brands) + 1x parity disk


GHETTO RACK


DAS SERVER (hal)
i changed the hx850 to an ax850 later


unRaid SimpleFeature web interface


ESXi client
 
asuranii said:
(sorry for my english )


well I haven't tested disparity
but from what I've read SnapRaid work by block size, not by file

and it's Opensource, so maybe it's better for getting more updates and bug patch
from the link I've provided you can see the developpment of Disparity has not been updated since 2009, and snapraid last update is more recent (2011)

and what I needed for my snapshot raid/parity was RAID6 feature (2 disk failure)
Click to expand...

Ah, yeah they seem very simliar, the main difference as you mentioned was block size vs file, but disparity detects file size changes (ie: document growing or replacing media with higher def) and adjusts accordingly. I do like that SnapRaid is supported on Linux as well as a few other OS', something Disparity is not. I think both are a good bit better than unRAID due to the fact they can be run within an operating system, allowing you to task your system for other things.

Disparity creator has just posted on the site recently that he was working to add some features, I'm waiting to see what he is doing :D
 
Amazing seeing all this, I have about 20TB of drives but not all hooked up at once (mainly backups). Ive read back a good few pages but was wondering what most people use 10 / 20+ TB of online storage for mainly?
 
saedrin said:
Is that an old Dell XPS T500?!
Click to expand...

certainly is !

i purchased it in late 1999 as my first machine and i love how trusty it has been during all these years.

it's currently my pfsense firewall and although i could buy a netgate or a hacom, i won't .. ill wait til its time comes :)
 
Rikki said:
Amazing seeing all this, I have about 20TB of drives but not all hooked up at once (mainly backups). Ive read back a good few pages but was wondering what most people use 10 / 20+ TB of online storage for mainly?
Click to expand...

porn, lots and lot of porn
:D
 
Rikki said:
Amazing seeing all this, I have about 20TB of drives but not all hooked up at once (mainly backups). Ive read back a good few pages but was wondering what most people use 10 / 20+ TB of online storage for mainly?
Click to expand...


6TB 1080p Movie
4TB 720p TV Series
4TB Anime
4TB personal docs
4TB work (adobe etc)

and lot of parity/mirror/raid (4 to 8TB)

and I need access to them any time I want.
 
Rikki said:
I was the same, till I found streaming haha
Click to expand...

But what if it's something you really want to keep? Some people are into some very boutique stuff, you know?
;)

Anyway, I use my rig to be a digital hoarder--if I've downloaded it, I'm keeping it, because you never know what you might want/need.
Pics of my system coming soon, need to clean up my wires first so I don't embarrass myself...
 
16.2 TB

Cooler Master Cosmos II
Enermax MaxRevo 1350W power supply
MSI Big Bang-XPower II Intel LGA 2011 X79 mainboard
Intel Core i7-3960X Extreme Edition Sandy Bridge-E 3.3GHz LGA 2011 CPU
Corsair Dominator GT 32GB (4 x 8GB) DDR3 1866Mhz (PC3 15000) 1.5v CAS 9-10-9-27 SDRAM
Two XFX Double Dissipation Black Edition Radeon HD 7970 in CrossFireX
Two Pioneer BDR-207DBKS Blu-ray optical drives
1TB RAID0 boot set: 2 x Samsung 830 Series 512GB solid state drives
4TB RAID0 data set: 2 x Western Digital RE4 2TB SATA 3.0Gb/s 3.5" 7200 RPM disk drives
4TB RAID0 data set: 2 x Western Digital RE4 2TB SATA 3.0Gb/s 3.5" 7200 RPM disk drives
1.2TB 2 x Western Digital VelociRaptor 600GB SATA 3.0Gb/s 3.5" 7200 RPM hot-swap data drives
6TB RAID5 archive set: 4 x Western Digital Green 2TB SATA 3.0Gb/s 3.5" 5400 RPM disk drives
CyberPower PP2200 1500W UPS
Windows 7 Professional 64-bit (Native)
CentOS 6.3 64-bit (VM)
Ubuntu 12 64-bit (VM)
Windows XP SP1 32-bit (VM)
Apple MacOS 64-bit (VM)


This workstation is used for post-production editing, software engineering and some occasional gaming.
16.2TB of total formatted storage capacity, 10.2TB internal and 6TB of external RAID5 archival storage (Drobo).
The RAID0 SSD set boots Windows 7 in 13 seconds, loads applications, providing scratch, swap and temporary rendering space.
The RAID0 data sets are primary data storage areas.
The RAID5 6TB external Drobo array stores infrequently used information and longer-term near-line storage.
Eventually, very old projects are burned onto BluRay discs for permanent archival.


Storage1.png


Storage3.png


Storage5.png


Power2.png


Storage7.png


Storage6.png

​
 
Last edited:
SonataSys said:
16.2 TB

This workstation is used for post-production editing, software engineering and some occasional gaming.
The RAID0 data sets are primary data storage areas.
Click to expand...
Can you explain why you use RAID0 for storing important post prod data? Would not RAID 1/6/10 be a better choice? Are you not concerned about losing that data?
 
Bigdady92 said:
Can you explain why you use RAID0 for storing important post prod data? Would not RAID 1/6/10 be a better choice? Are you not concerned about losing that data?
Click to expand...

Regarding data loss, there is no ignoring the risk involved with RAID0, but the speed and capacity advantages it provides cannot be denied. Given our storage requirements and budget, the cost of doubling capacity to account for mirrored RAID configurations is a luxury we cannot justify. Therefore, the following back-up methodology has worked well for us over the years to insure active, inactive and dead projects remain safe:

  • Active projects living on the RAID0 sets are scheduled in Ghost for a nightly back-up directly to one or two external SATA hot-swap drives, depending on project size (HD or standard definition). This disk-to-disk copy is very fast using a WD VelociRaptor for smaller projects, or spare 2TB WD RE4 or Caviar Green disks for larger HD projects.
  • These hot-swap back-ups are frequently used to share projects with another workstation, so work can progress in parallel on two machines, thus providing a third point of redudnancy since the same active project usually exists on two workstations and the transfer hot-swap disk.
  • Critical milestones during projects are Ghosted in the background overnight to a more redundant external RAID5 set for near-online storage, offering a fourth redundancy point.
  • Inactive projects are moved to the external RAID5 set for near-online storage up to 9 months.
  • Eventually, dead projects are burned to BluRay discs and removed whenever space runs short on the RAID5 set.

This approach insures we can never loose more than a day's work while making full use of the speed and capacity advantages provided by RAID0. Moreover, we have used RAID0 for over 8 years without a single array failure. The key to RAID0 is purchasing high-quality drives and verifying their surfaces are free of defects early on. In general, a bad disk usually fails early or not at all, meaning disks fail within the first month of use or perform well for many years until service times surpass MTBF. We have experienced good results with WD RE4 disks, both in terms of performance and long-term reliability in RAID configurations.

The 2TB capacity is also another crucial factor when editing multiple projects at the same time. When striped together in a RAID0 array, each 4TB data set supports six large standard-definition projects (around 3 hours of content) or two equivalent HD (1080i) projects. Mirroring cuts capacity in half, cutting the number of possible projects available on the set in half.

This heavy RAID0 configuration is all about speed and high capacity. The Samsung SSD array will load applications and provide plenty of scratch space for each application, including Adobe Bridge, Premiere Pro, After Effects, Audition, SpeedGrade, Photoshop and Encore. We find ourselves using most of these applications together in various combinations, depending on the project. When each application's recommended memory and maximum scratch space requirements are summed, the 32GB of SDRAM and Samsung 830 1TB RAID0 SSD array are sized to handle this load with 30% headroom for future growth. We seriously thought about doubling down on the SDRAM, but that would would break the budget or some other important hardware would be sacrificed.

Having two independent RAID0 data arrays is critical for our editing work, since we have determined through experience that each separate RAID0 set allows clean editing of either two standard-definition (SD) timelines or one HD timeline in an absolutely defect-free fashion, meaning no I/O congestion with resulting stuttering or lost frames when quickly navigating around timelines while moving large chunks of content. By moving all scratch activity to a dedicated RAID0 SSD set and working with two additional RAID0 data sets, we can confidently and quickly edit projects with up to four SD or two HD timelines without compromising productivity or quality. In a nutshell, editing several video/audio timelines using a single disk or RAID set just doesn't work very well.
 
thx for the infos

SonataSys said:
The 2TB capacity is also another crucial factor when editing multiple projects at the same time. When striped together in a RAID0 array, each 4TB data set supports six large standard-definition projects (around 3 hours of content) or two equivalent HD (1080i) projects. Mirroring cuts capacity in half, cutting the number of possible projects available on the set in half.
Click to expand...

in HD video project (premiere/after effects etc...) I use RAID0 too, can't work without speed :D
 
Update... More manly storage added :D

Amount of total storage (if posting multiple systems): 33,6TB

Gaming Box:
Amount of storage in the following system: 12,2TB

Case: Chieftec LCX-01
PSU: Enermax Liberty II 400W
Motherboard: Asus P7P55D
CPU: i7-860
RAM: 4x 4GB TeamGroup DDR3-1333
GPU: MSI GTX460 Cyclone
Controller Cards: Intel 1000PT Dual NIC
Optical Drives: Random DVD-RW
Hard Drives: Intel Postville G2 160GB
3x Samsung HD204UI
3x Hitachi 5K3000 2TB
Operating System: Win 7 Ultimate

Not so surprising the computer I use every day, no RAID as it is not needed so far - and because the platform sucks, as more PCIe slots are needed.
Next update are 6 Seagate Savvio 15K.2 and a SAS-HBA, so I need another GbE NIC for PCIe x1 as the onboard chip does not work. Somehow every NIC not made by Intel sucks...

Backup box - The Bucket (tm)
Amount of storage in the following system: 19,1TB
Usable: 15TB

Case: Inter Tech 4324L (aka Norco 4224)
PSU: Enermax Pro 82+ II 625W
Motherboard: Asus P6T WS Pro
CPU: Xeon W3503
RAM: 3x 4GB TeamGroup DDR3-1333
GPU: Matrox G550 PCIe
Controller Cards: LSI 9260-8i + HP Expander
Adaptec 19160
Intel 1000MT Dual NIC
Hard Drives: Seagate Momentus 7200.4 320GB for OS
2x WD RE.3 320GB
1x WD10EADS
5x Seagate Barracuda LP 2TB @ RAID5
3x Seagate Barracuda 7200.11 1,5TB @ RAID5
4x Seagate Cheetah 15K.5 73GB, 4x Fujitsu MAX3073RC 73GB @ RAID0
2x Maxtor Atlas 10K V 300GB @ RAID1
10x Seagate Savvio 10K.2 146GB @ RAID50
Misc: Tandberg 8 Slot LTO2 Autoloader
Operating System: Win Server 2008 R2

The name was given by a friend not really convinced those cases would be ok for home users...
SAS drives and the autoloader came as I got them really cheap - and as I learned at a LAN party we had at school some years ago the pr0n-storage has to be really fast :D
Files on this are a 1:1 copy of my gaming pc and additional backups of really important files.

Backup box #2
Amount of storage in the following system: 2268GB

Case: Chieftec CA-01
PSU: Old Enermax 350W
Motherboard: Asus P5WDG2 WS Pro
CPU: E6600
RAM: 2x 2GB DDR2-667
GPU: Matrox
Controller Cards: LSI 320-2E
Adaptec 29160
Hard Drives: Samsung HD502HJ
3x Seagate Cheetah 15K.5 146GB
2x Seagate Cheetah 10K.7 300GB
4x Seagate Cheetah 10K.7 146GB
2x Seagate Cheetah 10K.7 73GB
Operating System: Win Server 2008 R2

A classic Because-I-can-project - some older hardware and a boring day. Not a good combo :D
This holds an additional set of backups, mostly my photos.

Shitload-of-SCSI-box - not for the stats as this will be given to a friend and is not finished yet. This is just built from stuff thrown out at work.
We just want to see his housemate cry about the sheer number of drives. This guy tried to impress us with his sweet 5TB-machine, "Hey, look, I got 2 1TB-drives! Whoa, 6 drives now, a lot, hu?" So it's our turn. :D

Amount of storage in the following system: 2476GB :eek:
Case: Huge old tower
PSU: Old Enermax 550W
Motherboard: Intel SDS2
CPU: Dual P3-S 1400MHz
RAM: 4x 512MB Reg-ECC SDR
Controller Cards: 2x U320 RAID (ICP Vortex and maybe LSI 320-2X)
Optical Drives: random DVD-R
Hard Drives: 4x 146GB/15K, 4x 146GB/10K, 4x 73GB/15K at onboard HBA
5x 73GB/15K, 7x 73GB/10K, 12x 36GB/15K at RAID in HP drive cages
Operating System: ?

Pics will be added later...
 
JensS said:
Shitload-of-SCSI-box - not for the stats as this will be given to a friend and is not finished yet. This is just built from stuff thrown out at work.
We just want to see his housemate cry about the sheer number of drives. This guy tried to impress us with his sweet 5TB-machine, "Hey, look, I got 2 1TB-drives! Whoa, 6 drives now, a lot, hu?" So it's our turn. :D

Amount of storage in the following system: 2476GB :eek:
Case: Huge old tower
PSU: Old Enermax 550W
Motherboard: Intel SDS2
CPU: Dual P3-S 1400MHz
RAM: 4x 512MB Reg-ECC SDR
Controller Cards: 2x U320 RAID (ICP Vortex and maybe LSI 320-2X)
Optical Drives: random DVD-R
Hard Drives: 4x 146GB/15K, 4x 146GB/10K, 4x 73GB/15K at onboard HBA
5x 73GB/15K, 7x 73GB/10K, 12x 36GB/15K at RAID in HP drive cages
Operating System: ?

Pics will be added later...
Click to expand...

This one sounds like it would make a nice (but not quiet) space heater :D
 
Oh yes... :D My filer is around 400W idle, so I hope the 550W PSU will be enough for that monster... :D

btw: The drives are, for SCSI stuff, quite modern. Seagate 15K.4 and 15K.5, they are not that evil in terms of noise. Even if silent is something else, yes...
 
My home network is running for soms time now.
Pictures are on the site of the data center and data center@home

http://foto.steambay.eu/DataCenter/

Isp connection
Cable 120/10 down & up
VSDL 40/3 down & up

Switch
Dell 5324 24 port gigabit
Tp-link 24 port gigabit

Nas01
Case Norco RPC-2208
PSU Zippy/emacs P2h-6550p
Motherboard Asus P5G41T-M LE
CPU Intel® Core™2 Quad 9550
RAM 4 GB DDR3 Kingston Valueram
Controller Cards Areca 1222
Hard Drives 8x WD20EADS 2TB Raid5 2x ST9320325AS 320GB
Battery Backup Units APC Smart-UPS 750VA
Operating System Openmediavault 0.3

Nas02
Case DS211j
PSU 100V
CPU 1.2GHz
RAM DDR2 128MB
Hard Drives 2x 2TB Samsung HD204UI raid 0
Operating System DSM 4.1

Nas03
Case Chenbro ES34069
PSU 120 watt
Motherboard Itx Mobo
CPU AMD Athlon 64 X2 5000+
RAM 4 gb ram Kingston
Hard Drives 4x 2TB Samsung HD204UI raid5
Battery Backup Units APC Smart-UPS 750VA
Operating System Openmediavault 0.3

ESXI01
Case Norco RPC-2208
PSU Recom 850 Watt
Motherboard Asus P5G41T-M LE
CPU Intel® Core™2 Quad 9550
RAM 8 GB DDR3 Kingston Valueram
Hard Drives 4x Samsung HD204UI
Battery Backup Units APC Smart-UPS 750VA
Operating System ESXI5

ESXI02

Case 4U Oem
PSU Enlight 350watt
Motherboard Gigabyte p41
CPU Intel® Core™2 Duo E6300
RAM 8 GB DDR2 OCZ
Hard Drives 4x 750gb Samsung f1
Battery Backup Units APC Smart-UPS 750VA
Operating System ESXI5

Test01

Case Hp360 g4p
PSU 450watt
Motherboard Hp oem
CPU 2x dual xeon 3.4GHZ
RAM 8 GB DDR2 ecc
Hard Drives 2x 72gb 15k scsi
Operating System Debian 6

Web-02 colo backup @ home

Case 1U Super mirco
PSU 450 watt
Motherboard msi 939
CPU Amd Athlon 3700+
RAM 4 GB DDR1 kingston
Hard Drives 2x 160gb raid1
Battery Backup Units APC Smart-UPS 750VA
Operating System Cent Os 5.8

Web-01 colo @ Serverius in Dronten

Case 1U Super mirco
PSU 450 watt
Motherboard Supermirco
CPU Atom dual core 1.6ghz
RAM 4 GB DDR2 kingston
Hard Drives 2x 250gb raid1
Operating System Cent Os 5.8
 
Thought i may as well put my server up.

Lian Li PC-343
i7 980 @ 3.33ghz
Gigabyte X58A-UD7 Rev 1
24gb Kingston 1333C9 Non ECC
Corsair TX-750
Seagate 500gb 2.5 inch drive (Boot)
Gigabyte ATI 4550
Intel X520 SFP 10gb NIC
Areca 1880ix-24
Rocketraid 2320
No Name 5in3's
Seagate ST3000DM001 x20 (12 drive Raid 6, 8 drive Raid 6)
WD20FEAX x8 (Raid 5)
Seagate 7200.12 1.5tb x2 (Individual)
Samsung 103SJx1 (Individual)
Seagate 7200.11 1tb (Individual)
WD10EAC x1 (Individual)
WD20EARSx4 (Individual)
Windows 7 Ultimate

Advertised Space: 90tb
Useable Space: 68.95tb

[URL="http://i.imgur.com/n7Sdy.png?1"]http://i.imgur.com/n7Sdy.png?1[/URL]
Pics of the actual box in the morning.
 
Last edited:
Copie247 said:
Thought i may as well put my server up.

Lian Li PC-343
i7 980 @ 3.33ghz
Gigabyte X58A-UD7 Rev 1
24gb Kingston 1333C9 Non ECC
Corsair TX-750
Seagate 500gb 2.5 inch drive (Boot)
Gigabyte ATI 4550
Intel X520 SFP 10gb NIC
Areca 1880ix-24
Rocketraid 2320
No Name 5in3's
Seagate ST3000DM001 x20 (12 drive Raid 6, 8 drive Raid 6)
WD20FEAX x8 (Raid 5)
Seagate 7200.12 1.5tb x2 (Individual)
Samsung 103SJx1 (Individual)
Seagate 7200.11 1tb (Individual)
WD10EAC x1 (Individual)
WD20EARSx4 (Individual)
Windows 7 Ultimate

Advertised Space: 90tb
Useable Space: 68.95tb

[URL="http://i.imgur.com/n7Sdy.png?1"]http://i.imgur.com/n7Sdy.png?1[/URL]
Pics of the actual box in the morning.
Click to expand...

how the heck did you get the areca working in the gigabyte mobo?? :(
I tried everything to get my LSI to work in my GB mobo, but never could get it to work :(
 
gmutale said:
how the heck did you get the areca working in the gigabyte mobo?? :(
I tried everything to get my LSI to work in my GB mobo, but never could get it to work :(
Click to expand...

Worked perfectly the moment i plugged it in. Might be an IRQ error perhaps?
 
gmutale said:
how the heck did you get the areca working in the gigabyte mobo?? :(
I tried everything to get my LSI to work in my GB mobo, but never could get it to work :(
Click to expand...

Areca has the best mobo compatibility from what I have seen. Especially with desktop (non server/workstation boards).
 
houkouonchi said:
Areca has the best mobo compatibility from what I have seen. Especially with desktop (non server/workstation boards).
Click to expand...

oh ok, cool! :)
I have an old Areca 1210 - you reckon that'll be just as easy going as well?
I have a s1366 GB UD7 mobo
 
Status
Not open for further replies.
Back
Top