The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Status
Not open for further replies.
I need to update my post. A bunch of people are catching up to my old value now. My actual storage is just under 400TB now though I just haven't updated the post in forever.
 
1586224daaamn.jpg
 
how the hell do you guys afford all of these expensive drives? lol

A lot of them live in parts of the country where you can actually buy a home for under $100k (unlike the ~$450k minimum for something you don't want to live in, and $650k for something you'd tolerate around here)

Incomes are on average slightly lower in these areas, but not enough by far to offset the ~$3,000 - $4,000 a month mortgage for a modest home anywhere I'd want to live.

That leaves a lot of money to spend on things like cars, or computers, if that's your thing (which, let's face it, it is for a lot of us in here :p ) or to buy a mansion.

I'd sooner shoot myself than have to live anywhere more than 10 miles away from a major coastal city in New York, New England, California, Oregon or Washington, but some people either prefer living in the boonies, flyover country or the south, or are willing to make those sacrifices in order to save money.

I'm not one of them, so I scrape by here in Boston, on a salary that would allow me to live like a king in many other places in the country.
 
how the hell do you guys afford all of these expensive drives? lol
Assuming you make the nationwide median household income of $55K/year, with a 30% tax rate, you should have about $3200/month. Spend half of that on bills, a quarter of it on retirement, and you've got about $800/month for "fun money". That's a new drive every week until you run out of slots to stick 'em in.
 
Assuming you make the nationwide median household income of $55K/year, with a 30% tax rate, you should have about $3200/month. Spend half of that on bills, a quarter of it on retirement, and you've got about $800/month for "fun money". That's a new drive every week until you run out of slots to stick 'em in.

Yeah, unless you live where $3000/month pays for a mortgage on a basic house.

My ex and I owned half of a two family condo. 1250 sq ft, ugly as hell, cost $400,000

Half of one of these:

14186761984_a301902f16_z.jpg


Thats about the cheapest you'd find anything for around here. There are some condos that go for ~$300,000, but that is rare. A decent single family home at about 2,500sqft is impossible to find under $600k

Unless you are making well over $100K here, you are scraping by.

...and I wouldn't live anywhere else.

Anyway, this is getting way off topic.
 
Zarathustra[H];1041160732 said:
Yeah, unless you live where $3000/month pays for a mortgage on a basic house.

My ex and I owned half of a two family condo. 1250 sq ft, ugly as hell, cost $400,000

Thats about the cheapest you'd find anything for around here. There are some condos that go for ~$300,000, but that is rare. A decent single family home at about 2,500sqft is impossible to find under $600k

Unless you are making well over $100K here, you are scraping by.

...and I wouldn't live anywhere else.

Anyway, this is getting way off topic.

Man, that's ridiculous. Sounds like you need to find a new place to live.

Around here you can get 2 acres of land and a brand new 2500sq ft. house for $190,000.

$650/mo mortgage, Gasoline is $2.94/gallon and natural gas/electric in the winter will cost you about $250/mo. Trash is $75 every 3 months and water/sewer is $100 every 3 months. Property taxes are $2000/yr. Heck, for a family of 6 we spend $800/mo on food & household supplies.

$100k/yr jobs are easy to come by here, and you'll live like a king.
 
Last edited:
um, where is here?

Michigan

Kalamazoo/Portage/Scotts/Pavilion Township/Texas Township area

Crappy winters though!

Brand new houses with land cost between $190k-$275k depending on finishing materials and appliances.

If you've got 5-10yrs experience and a good resume there are some good engineering, programming, networking & management jobs out here. $85k is usually where those jobs start at, moving to $100k in 3-5yrs.

Average commute is 45 miles 60 mins.
 
um, where is here?

My here is a ~10mile radius around Boston, but the same could be said about New York, San Francisco, San Diego, Seattle, or pretty much anywhere else I'd have any interest in living.

(Think major coastal city in highly educated area with flourishing arts, dining and excellent public transit)

Not sure where BDS1904's. "here" is. :p
 
hmm michigan and 100k...that sounds doable since things are probably really cheap like you said, i'm in Portland Oregon making 60k in a house that cost me 280k...i could rent it and move...thinking about grad school though so i can earn more later
 
Zarathustra[H];1041160270 said:
A lot of them live in parts of the country where you can actually buy a home for under $100k (unlike the ~$450k minimum for something you don't want to live in, and $650k for something you'd tolerate around here)

Lol making some assumptions there. I wish I could buy a home for $100k, thats how much my car cost. I live in Southern California.
 
Zarathustra[H];1041160732 said:
Yeah, unless you live where $3000/month pays for a mortgage on a basic house.
If you're only making $3.2K/month, you can't afford a $3K/month house. Make more, spend less, or move. :D
 
I hold myself responsible for derailing this thread. Wish I could take it back.

Let's see some awesome storage setups!
 
Don't let those "Michiganders" trick you into moving to that economic disaster of a cess pool. There's nothing going on in that state. Hell, they're so desperate they hired a marketing company to try and repackage that bag of shit as "Pure Michigan" but a bag of shit is still a bag of shit no matter what's printed on the wrapper.
 
Zarathustra[H];1041161638 said:
I hold myself responsible for derailing this thread. Wish I could take it back.

Let's see some awesome storage setups!

LOL, me too! Let's get this bitch back on the rails :cool:
 
Zarathustra[H];1041160732 said:
My ex and I owned half of a two family condo. 1250 sq ft, ugly as hell, cost $400,000

Unless you are making well over $100K here, you are scraping by.

...and I wouldn't live anywhere else.

The median family income in New York CIty is about $50K. If half the families in New Your CIty can live there on that little, whereever you live anything over $50K in income can go toward a nice storage system.

Set your priorities.
 
Another Storage showoff thread coming ?

It would be nice to move to the same model they use in the photography, and show off your rig threads.


Have an edition for every year.

That way things stay fresh, and we don't wind up with these ridiculous huge threads that are difficult to muddle through.
 
Michigan

Kalamazoo/Portage/Scotts/Pavilion Township/Texas Township area

Crappy winters though!

Brand new houses with land cost between $190k-$275k depending on finishing materials and appliances.

If you've got 5-10yrs experience and a good resume there are some good engineering, programming, networking & management jobs out here. $85k is usually where those jobs start at, moving to $100k in 3-5yrs.

Average commute is 45 miles 60 mins.

How did I know you were going to say Michigan...
I was and continue to get random job offers over there (my resume must still be floating around on monster or something) but OMG the winters would drive me insane, like clinically...
 
How did I know you were going to say Michigan...
I was and continue to get random job offers over there (my resume must still be floating around on monster or something) but OMG the winters would drive me insane, like clinically...

The intent by the over-weekend freezing of the thread was to get it back on track to discussing storage.
 
Zarathustra[H];1041179933 said:
It would be nice to move to the same model they use in the photography, and show off your rig threads.


Have an edition for every year.

That way things stay fresh, and we don't wind up with these ridiculous huge threads that are difficult to muddle through.
I'd be okay with this. Moving forward, it might even become a great idea, as storage moves away from the traditional drive and gets implemented more as cards.
 
I'd be okay with this. Moving forward, it might even become a great idea, as storage moves away from the traditional drive and gets implemented more as cards.

I messaged EnderW about creating a new 100TB thread with a sticky. 100TB is the next iteration of the massive storage pods. 10TB is nothing. 50TB is within reason. 100TB is HUGE and impressive.
 
I messaged EnderW about creating a new 100TB thread with a sticky. 100TB is the next iteration of the massive storage pods. 10TB is nothing. 50TB is within reason. 100TB is HUGE and impressive.
100TB in one chassis... Damn. That's a minimum of 5TB drives for a 24-bay chassis. It'd be a very short thread!
 
100TB in one chassis... Damn. That's a minimum of 5TB drives for a 24-bay chassis. It'd be a very short thread!

I'm not sure it'd be just one chassis. We have people in here that have way more storage than 100TB spread over 2-3 systems. 100TB is a stretch but not impossible and since we want to show the [H]ardest of the [H]ard we must put that reachable target out there.


The only rule is no Enterprise/Business systems.
 
I messaged EnderW about creating a new 100TB thread with a sticky. 100TB is the next iteration of the massive storage pods. 10TB is nothing. 50TB is within reason. 100TB is HUGE and impressive.

I don't think we need to be THAT restrictive. The list of people on this forum with 100TB of storage in total, let alone in one chassis is going to be vanishingly small.

How about instead we just limit it to any home built RAID application (other than striping or mirroring). I say home built, as looking at a bunch of Synology/QNAP/Drobo's isn't very interesting.

This way it will let both those working on their first 6 drive RAID6 on consumer hardware and those with more imopressive setups play.

Would be nice if we restrict it final setups too, as looking at a bunch of pictures of boxes full of drives is kind of a drag.

We should also discourage anyone from using RAID5 or lower, as it is functionally obsolete these days.
 
Zarathustra[H];1041185001 said:
I don't think we need to be THAT restrictive. The list of people on this forum with 100TB of storage in total, let alone in one chassis is going to be vanishingly small.

How about instead we just limit it to any home built RAID application (other than striping or mirroring). I say home built, as looking at a bunch of Synology/QNAP/Drobo's isn't very interesting.

I have yet to see any of those types posting. 100TB is massive.

This way it will let both those working on their first 6 drive RAID6 on consumer hardware and those with more imopressive setups play.

Would be nice if we restrict it final setups too, as looking at a bunch of pictures of boxes full of drives is kind of a drag.

We should also discourage anyone from using RAID5 or lower, as it is functionally obsolete these days.

Personal Preference.

People say using RAID at all is obsolete technology and ZFS/BTFS/ResierFS is TEH FUTRE. Tech agnosticism 100%.

We shouldn't care HOW you did it, but the fact that you hit 100TB IS impressive.

We set the bar high, minimum requirements, and we see who gets there and how they did it. HOCP is a hive of creative genius' that will find a a way to GITRDUN!
 
People say using RAID at all is obsolete technology and ZFS/BTFS/ResierFS is TEH FUTRE. Tech agnosticism 100%.

I'm using RAID in the general sense, not in the technique specific sense.

I - too - prefer ZFS, but I definitely consider ZFS a form of RAID. Not familliar with BTFS, and only very marginally familiar with Reiser though.

When I say RAID, I mean RAID in it's dictionary sense. An array of redundant drives.

When I mentioned RAID5 by name, I probably should have been more general, instead saying "any system with only one drive parity/redundancy" be that RAID5, RAIDz1 or whatever else the other systems call it.

Any system today in order to have safe data will need to still have redundancy during a resliver/resilver/rebuild process, as if it doesn't drive read/write errors will propagate while the array is being rebuilt from its non redundant state.

This was less of a problem in the past when drive sizes were much smaller, but with today's HUGE single drives (many many more bits) and the error rates (expressed in bits) staying the same, copying non-redundant data from even a single drive is going to result in more or less guaranteed corruption of at least one bit somewhere in the transfer.

As long as you have at least one parity drive left in your degraded state, then flipped bits can be automatically corrected from parity during the read process which the full array is being rebuilt.

So, for best practice RAID6/RAIDz2/whatever the others call it should be considered a bare minimum if dealing with data you care about.
 
Back on topic.

Here is my latest creation:

Storage.jpg


2 HP DL160 G6
4 HP MSA60

2 - Intel 730 240GB
48 - 4TB WD RED

Setup as:

HP DL160 G6 with ESXi installed onto 16GB USB Thumbdrive attached to 2 MSA60 via Dell SAS external flashed to LSI firmware. Dell SAS external passed into FreeNAS VM. Intel 730 used as datastore for ESXi and ZIL for FreeNAS VM in first drive bay of DL160. Storage setup as 4 stripe of RAIDz2 with 6 x 4TB drives for 64TB per server or 128TB total storage.
 
Back on topic.

Here is my latest creation:

http://s28.postimg.org/uyq18knfx/Storage.jpg

2 HP DL160 G6
4 HP MSA60

2 - Intel 730 240GB
48 - 4TB WD RED

Setup as:

HP DL160 G6 with ESXi installed onto 16GB USB Thumbdrive attached to 2 MSA60 via Dell SAS external flashed to LSI firmware. Dell SAS external passed into FreeNAS VM. Intel 730 used as datastore for ESXi and ZIL for FreeNAS VM in first drive bay of DL160. Storage setup as 4 stripe of RAIDz2 with 6 x 4TB drives for 64TB per server or 128TB total storage.

Ahh, knew I recognized those caddies :p

I have an unused HP DL180G6 in the basement. Was too loud for me, only lasted a month before I built my own into a 4U Norco case.
 
Back on topic.

Here is my latest creation:

http://s28.postimg.org/uyq18knfx/Storage.jpg

2 HP DL160 G6
4 HP MSA60

2 - Intel 730 240GB
48 - 4TB WD RED

Setup as:

HP DL160 G6 with ESXi installed onto 16GB USB Thumbdrive attached to 2 MSA60 via Dell SAS external flashed to LSI firmware. Dell SAS external passed into FreeNAS VM. Intel 730 used as datastore for ESXi and ZIL for FreeNAS VM in first drive bay of DL160. Storage setup as 4 stripe of RAIDz2 with 6 x 4TB drives for 64TB per server or 128TB total storage.

And I thought my power useage was excessive, lol.

Got any pics of the wiring?
 
I need to update my post. A bunch of people are catching up to my old value now. My actual storage is just under 400TB now though I just haven't updated the post in forever.

can I do my offsite backups to your house? :D

(I'll pay - seriously) :p
 
Last edited:
Woah nice. How much of a dent does that do on the hydro bill? :D

I have like 20TB after my last upgrade which is not much compared to some of you but I'm only using like 1/4 of it. I need to download more things. :p
 
And on a side note, ZFS just saved my butt for the second time in 2014.

One of my remaining old as dirt 3TB WD Green drives just failed.

(previously this year an absolutely ancient even older 3GB/s Sata 2TB WD Greens Died)

Loving the fact that I decided to go redundant when I did.

I realize full well that these drives are probably failing prematurely because of the Intellipark issue that I failed to disable

(and I shouldn't have used WD Greens for this in the first place, but it's what I had around when I built my first RAID)

I'm just happy for the redundant protection.

Getting 2 more 4TB Reds from Amazon!
 
Woah nice. How much of a dent does that do on the hydro bill? :D

I have like 20TB after my last upgrade which is not much compared to some of you but I'm only using like 1/4 of it. I need to download more things. :p

not as much as you would think, according to the 2 UPS' the whole rack is drawing <2500W

from top to bottom

Sun X4170 2x X5570 24Gb DDR3 2x 146Gb 10k RAID1 2x 120Gb EVO RAID0
Sun X4170 2x X5570 24Gb DDR3 6x 146Gb 10k RAID6
Sun X4170 2x X5570 24Gb DDR3 6x 146Gb 10k RAID6
Sun X4600 M2 8x Opteron 8389 64Gb DDR2 2x 120Gb Intel 330 RAID1 2x 80Gb 10k RAID1

EonStor A16F-G1A2 16x 1Tb Seagate 7200.11/7200.12 drives RAID5 + hot spare
EonStor A24F-R222416x 2Tb WD Green WD20EADS (old school, TLER disabled, these drives just won't die, 8x empty slots in this chassis)
EonStor A24F-R2224 24x 2Tb Hitachi UltraStar 2x 12 disk RAID6
EonStor A24F-R2224 24x 2Tb Hitachi UltraStar 2x 12 disk RAID6
 
Last edited:
Status
Not open for further replies.
Back
Top