The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Status
Not open for further replies.
Total Advertised: 16TB
Total Available: 12TB

Case: Lian Li PC-TU200B
CPU: Intel Core i5-3330 Ivy Bridge 3.0GHz
Motherboard: Asus P8Z77-I Deluxe
16GB Memory: 2 x Kingston 8GB DDR3 SDRAM 1600
Power Supply: Corsair CX430M 430W
SAS HBA: IBM ServeRAID M1015
SSD: Crucial M4 128GB
Wireless: Atheros AR5BHB92-H AR9280 802.11n (AP/master mode)
OS: Gentoo Linux running software raid -> bcache -> dm-crypt -> LVM -> ext4
16TB RAID 5: 4 x Seagate STBD4000400 4TB

My goal for this build is to allow for a portable media server that could be carried easily back and forth between a house, yacht, private jet, etc (not that I own a yacht or private jet ;)).
The idea is that I could power this thing anywhere within wireless range of a media client to watch movies, tv and listen to music in places that may not have internet access.

I've currently chosen to use Plex Media Server and I am testing playback with a Roku 3, so far with very good results :D

Album for more photos!

ztvdxznh.jpg
 
Inside pics coming... They're hideous tho my cabling is terrible. 1u sinks work great with the fans blowing on them and they're quiet.
 
I should have pics of my D8000 build soon, waiting on some parts for the finishing touches. :D
 
I should have pics of my D8000 build soon, waiting on some parts for the finishing touches. :D

I just got mine done around two weeks ago.

Motherboard: ASROCK EP2C602-4L/D16
Processor: Two Intel Xeon E5 2620 6 core processors
Heatsinks: Cooler Master Hyper 212 EVO x 2
Memory: 56gb of DDR3 @ 1600mhz
Raid Controller: Dell Perc 6i
Hard Drives: 8 Samsung F4 2tb Drives in a raid 10 (8 x 2tb)
Case: Lian Li d8000
Network Card: Broadcom 5715 Dual Port 10/100/1000 NIC
Sata Controllers: AOC-SAS2LP-MV8 x 2
Power Supply: SeaSonic M12II 850 SS-850AM

Operating System: vSphere 5.1.0

I am running an unRaid array with a large mix of drives. 8 drives are of course for the Raid 10 array, then a mix of 15 disk for storage using UnRaid, the total of the unRaid array is 34tb

unraid_sm.png


esx1.png


mlb1_sm.jpg


server6_sm.jpg


server1_sm.jpg


server2_sm.jpg


server3_sm.jpg


server4_sm.jpg


server5_sm.jpg
 
Last edited:
Snoop it was your post on the unraid forums that pushed me off the fence on buying the D8000. :D I haven't tallied up the total project cost yet, but it's high lol. The hot swap adapters are redic expensive.

What CPU heatsinks are you using, and are they touching the mid-plane?
 
Last edited:
Snoop it was your post on the unraid forums that pushed me off the fence on buying the D8000. :D I haven't tallied up the total project cost yet, but it's high lol. The hot swap adapters are redic expensive.

What CPU heatsinks are you using, and are they touching the mid-plane?

Thank you for the kind words, mine are the Cooler Master Hyper 212 EVO. Yes, they absolutely do touch the tops of the heatsinks. Right now that piece is out but I will be putting it back in there the next time I have a chance to bring it all down. The server does so much it could be a month when I have everyone log out of the servers, etc.
 
Thank you for the kind words, mine are the Cooler Master Hyper 212 EVO. Yes, they absolutely do touch the tops of the heatsinks. Right now that piece is out but I will be putting it back in there the next time I have a chance to bring it all down. The server does so much it could be a month when I have everyone log out of the servers, etc.

Can you fully screw the mid-plane in with them installed? My D14 is 2-5mm too tall and I need to find a replacement,preferably cheap, but an OC'd FX-8350 isn't as easy to cool as I would like..
 
Decided to post my setup.

Total Advertised: ~24 TB
Total Available: ~21 TB

Name: thumper (yes, not very imaginative)
Case: SUN X4500
PSU: Redundant 1300W supplies came with unit.
Mobo: X4500 Stock board
CPU: Sun 2.6GHz Dual Core CPU, AMD Opteron 285, E6 Stepping, 95W
RAM: 8 x 2 GB (16 GB total) DDR SDRAM - ECC
GPU: Integrated ATI RAGE XL
Disk Controller: Marvell 88SX6081

Drives:
Product (Count)
Hitachi HDS72202 (7)
Hitachi HDS72101 (2)
ST3000DM001-9YN1 (1)
WDC WD6401AALS-0 (2)
ST31000520AS (3)
Total (15)

They are:
2 WD black 640's mirrored (OS)
7 WD 2TB drives (raidz)
5 1TB drives (raidz)
1 Seagate 3TB drive.


OS: Solaris 11.1
Networking: 4 x Gigabit Ethernet Intel 82546GB
Power Consumption: Unknown, I don't have an amp meter, and my kill-a-watt wont do 240V ;)

So, as you can see I have 15 drives so this leaves me with 33 empty slots.
I just use it as a storage server. A lot of people have newer/faster units, but I have no sas expanders and no cables to deal with and can read 3TB drives. Its pretty clean inside as the drives screw into a small metal caddy and then slide into the unit.

Hope to throw 7 more 2TB drives in shortly giving me another 12TB usable space.

I can include pics, but since its just a stock X4500 there are a lot on the web.
 
Can you fully screw the mid-plane in with them installed? My D14 is 2-5mm too tall and I need to find a replacement,preferably cheap, but an OC'd FX-8350 isn't as easy to cool as I would like..

Yes, but it is tight :) There is definitely some pressure on the MLB but not enough for me to worry about, it is just snug.
 
Last edited:
My weekend project...

HTPCofTheAbyss

Amount of total storage: 18TB
Amount of usable storage: 15TB

Case: Fractal Design Node 304 (pulled HDD activity and power lights)
PSU: Some shitty 250watt one included in the last case, gonna upgrade this next.
Motherboard/CPU/GPU: ASRock E-350M1 Zacate APU
RAM: 4GB of some cheap ass shit that I can't remember
Controller Cards: IO Crest SATA III 4 Port PCI-e Card SI-PEX40064
Hard Drives:
1x Samsung 840 120GB SSD
1x Western Digital Red WD30EFRX 3TB
1x Western Digital Green WD30EZRX 3TB
1x Hitachi GST Deskstar HD32000 3TB
3x Seagate Barracuda ST3000DM001 3TB
Operating System: Windows 8 Pro

Just got my HTPC setup with FlexRAID. XBMC front end. Thinking about using CrashPlan for backups.

Family Photo of the hard drives I had laying around when swapping out the 2TB with 3TB's before installing everything in the HTPC (the dead one will be send out under warranty soon):
hdd01_zps60d5a7c0.png


6x 3TB (5 for data 1 for parity) in the FD Node... yeah I need to work on the rat's nest. I'll do that when I replace the PSU:
hdd02_zps6cf537cc.png


More rat's nest... clever mounting of the SSD:
hdd03_zpsf9c0cfcb.png


Success! The FlexRAID pool is finished building (it took 16 hours):
flexraid_zps5f9fd236.png


In the cabinet. Killzone 3 sniper protecting it and Sackboy happily blocking the lights on my router.
htpc_9-22-2013_zps5713d399.png
 
Great weekend project!

Compact case too! I have been eyeing one up :)

Thanks!

My wife and kids weren't amused they couldn't watch any of the stuff on it for the last day or so though.

The case is great. Glad I got it for $55. It keeps the hard drives cool even in my cabinet.
 
My wife and kids weren't amused they couldn't watch any of the stuff on it for the last day or so though.

Well, at least you're able to get it done in a weekend.

I often tell my girlfriend that "the NAS will be down for the rest of the day", but then a week later, it is still a work in progress..
 
Well, at least you're able to get it done in a weekend.

I often tell my girlfriend that "the NAS will be down for the rest of the day", but then a week later, it is still a work in progress..

+1 (my was down for like 2 weeks this past 'upgrade')
 
I guess this is why you need a ZFS rig. And a backup ZFS rig. And a backup of the backup, offsite. Just incase the Mrs wants to beat you because the NAS is down :p
 
I guess this is why you need a ZFS rig. And a backup ZFS rig. And a backup of the backup, offsite. Just incase the Mrs wants to beat you because the NAS is down :p
You just described my setup exactly. Except I don't keep everything on my offsite, just the high-priority stuff.
 
Well, at least you're able to get it done in a weekend.

I often tell my girlfriend that "the NAS will be down for the rest of the day", but then a week later, it is still a work in progress..

True. We don't have cable so it can't stay down for too long.
 
My home network is running for soms time now.
Pictures are on the site of the data center and data center@home

Isp connection
Cable 150/15 down & up

Switch
Dell 5324 24 port gigabit
Tp-link 24 port gigabit

Esxi

System specifications
Component Value
Case Norco RPC-231
CPU Intel Q9550
RAM 8 GB DDR3
Motherboard Asus
Operating system Esxi 5.1
Controller HP Smart Array P212 Controller
Hard drive 2x seagate 1,5 tb
Hard drive 4x Samsung 400Gb
RAID array configuration Raid 1
Network 4x Onboard 1000MBit's

Nas01

Synology ds411j
4x WD 2tb Raid5

Nas02

System specifications
Component Value
Case Norco RPC-2208
CPU Intel Q9550
RAM 4 GB DDR3
Motherboard Asus P5G41T-M LE
Operating system Openmediavault 0.5
Controller Areca ARC-1222
Hard drive 8x WD Black 3 tb
Hard drive 1x 160
RAID array configuration Raid 5
Netto storage capacity 21.00 TB
Network 4x Intel E1000 1000MBit's
UPS apc 750

Nas03

System specifications
Component Value
Case RPC-3216
CPU Intel I3 3220
RAM 32 GB DDR3
Motherboard Asus P8B75-v
Operating system Openmediavault 0.5
Controller Adaptec 5445 Intel RAID Expander Card RES2SV240
Hard drive 16x Seagate Desktop HDD.15 , 4TB
Hard drive 3x Seagate Desktop HDD.15 , 4TB
Hard drive 1x 250Gb
RAID array configuration Raid 5
Netto storage capacity 66.58 TB
Network 3x Intel E1000 1000MBit's
UPS apc 750

Nas04

System specifications
Component Value
Case RPC-3216
CPU Intel I3 3220
RAM 4 GB DDR3
Motherboard Asus P8B75-M
Operating system Openmediavault 0.5
Controller Areca ARC-1231ML
Hard drive 16x Seagate Desktop HDD.15 , 4TB
Hard drive 1x 160
RAID array configuration Raid 5
Netto storage capacity 54.58 TB
Network Onboard 1000MBit's
UPS apc 750
Extra 10X samsung 2tb spare

Nas05

synology ds211j
Wd 2 tb Raid 1

HomeLab01.jpg


HomeLab02.jpg


HomeLab03.jpg
 
Thought id update mine.

The Norco 4224 consists of:
i7 980,
24gb 1333c9
HD5450 PCI
Silverstone 1200G
Areca 1880ix-24
24x3tb ST3000DM001/Toshiba ACA 3tb Raid 60
Intel Quad NIC,
Intel x520 SFP+ 10gb
Intel x540 10g base-t
120mm fan wall w/ Delta 252cfms

The SAS box:
Rackmountable S3016
16x 2tb samsung 204UI Raid 6 attached via SFF-8088 to Areca 1880ix-24

HP Server
8gb ECC ram
60gb samsung 830 ssd
5x WD20EARX 2tb's
Intel X520 SFP+ 10gb
Some no name usb 3.0 pcie x1 card
Attached via USB is the 4 bay Proway box which has 4x 3tb Hitachi 7K3000 drives

Networking gear is a Dell 8132F, so 24x 10gb SFP+ ports, also got both a 4x 10g base T module on its way along with a QSFP module. Its connected to the Dell 6224 via 2x SFP+ (the 6224 has an SPF+ uplink module) still in the process of setting it all up and waiting for the final lot of parts to show up (mainly SFP+ transcievers and some actual fibre cable)

zBigtvC.jpg

4vKgSYc.jpg

NNB7zk4.jpg

P2V80ZQ.jpg
 
Also, a look at my PWM fan controller that makes it all silent :cool:

That fan control is brilliant. Do you monitor the disk temperatures and if so what are the results? I'd love to know how the hell to make these, I have 3 SC847E1-R1400LPB's and when I turn them all on its like a Vulcan bomber taking off.
 
10195882625_13d5f8cee0.jpg


I ended up upgrading from my old supermicro tower server with 8 bays to a 24 bay supermicro chassis that I actually bought from ebay that I heard about through a good deals thread here.

Here's how she started:

10206914414_6fc684e782.jpg


From the front:

10207029416_5a8f877e46.jpg


Upgraded to dual Quad Opterons ($14 shipped for the pair), also upgraded from 8GB memory to 32GB.

10207042216_3923979159.jpg


10207049205_d5b1250426.jpg


Then 13 4tb hdds (12 for the array + 1 cold spare):

10207061075_08b5018e2a.jpg


10207077746_42fd918816.jpg


Side by side with old nas to new:

10207083815_9934a9180b.jpg


Trying to transfer data out of the old failing array:

10207094785_970b0052be.jpg


10207006954_ef0aed3705.jpg



Supermicro 24 Bay
Dual Opteron quad processors
32GB memory
500GB boot drive
128GB SSD l2Arc cache
32GB SSD Zil cache

12 Western Digital 4K 2TB HDDs
12 4TB Seagate 4K HDDs

Advertised space:

72TB

Usable Space:

46.5 TB

OS: Freenas

I'm using 3x supermicro SATA II controller cards in pass-through mode, then going ZFS + RaidZ2 (one RaidZ2 array for the 4TB drives and one for the 2TB drives in one zpool ) for my parity. In this configuration I loose 12TB to parity, but for anyone who has ever only had one disk parity and lost a second disk during a rebuild, you'll know it's worth it. Along with the SSD's for caching so I can do de-duplication and compression on the fly.
 
(wish it was mine) :) Just wanted to share....apologies if 'not personally owned' is not allowed.
VM server at work running 2012 Datacenter
2x E5 2690's
8x16GB DDR3 1600 ECC
8x Cheetah 600GB (4 each in RAID 1+0 x 2, one set is OS, other set is "fast storage")
8x 4TB WD Blacks (don't know how the boss wants to set these up yet...don't know if the card came in yet)
2x 3TB WD Red
Supermicro MBD-X9DR3-F-O board & CSE-846A-R1200B 4U chassis


Cheetahs

Blacks


Never thought I'd get to hold 32TB in my hands :eek:
 
I currently have a 12Tb file server which is currently housed in a Fractal Design Array R2 that is running: Intel D510MO / D510 Intel MN10, 4Gb DDR2 800Mhz.

But I am looking to upgrade into a 24 bay hot swap case and the X-Case looks lovely :D
 
Status
Not open for further replies.
Back
Top