The [H]ard Forum Storage Showoff Thread - Post your 10TB+ systems

Status
Not open for further replies.

Lost-Benji

Limp Gawd
Joined
Jun 23, 2013
Messages
321
\
IMG_2640_zpsf7f6a77f.jpg

As much as I hate quoting with pics, this FAIL get the prize.

APC 1U UPS (way too small for the setup) that runs hot normally and hotter when on battery sitting on carpet floor with 50-70 KG's stacked on top.
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
Well I would hope that a rackmount UPS is designed for front to back air flow.

Then again I've seen some that don't even have any fans at all. Kinda bad design.

Personally I'd put that stuff in a rack in a separate room though but that's just me. But that gets a pass, because seeing a box full of hard drives is just damn sexy. :D
 

TeeJayHoward

Limpness Supreme
Joined
Feb 8, 2005
Messages
11,147
As much as I hate quoting with pics, this FAIL get the prize.

APC 1U UPS (way too small for the setup) that runs hot normally and hotter when on battery sitting on carpet floor with 50-70 KG's stacked on top.
Look under the desk in the back. There's a standalone 1500vA UPS which is powering the two 4Us. The two 2Us are running off the 1U UPS. And even if it wasn't, the 1U is a 1000vA (or 1500, I can't recall) UPS. More than enough to run those two boxes at full tilt (not that they ever are) for the 2-5 minutes it would take them to shut down. Combined, they draw something like 400W from the wall.

The airflow issue is the reason that they're sitting so far out in front of the desk instead of being flush with it. Temps aren't bad, but it is a bit of a space heater. Warmest room in the house, year round. (And yeah, I'd love to have another room to put them in... But this IS my "other room". My house only has two bedrooms!)
 

cantalup

Gawd
Joined
Feb 8, 2012
Messages
758
I am gonna cry by looking you pic. poor supermicro server :(.

get decent UPS with remote monitoring :D. used one is not expensive on 3bay and make sure remote/daugher adapter is included.
I pick 2 APC and 2 remote adapters. 1500VA each.
well batteries are extra, each UPS need 4 bats :(.


other thing, get half size rack for your poor server :p.
 

UltraHD

n00b
Joined
Nov 19, 2014
Messages
3
TorrentStation

* Game Tiger Mage Mi-3
* Seasonic G-500 (SSR-550RM)
* ASRock Z97 Extreme6
* Crucial Ballistix Tactical DDR3 (BLT8G3D1608DT1TX0) x 4
* Intel® Core™ i5-4690
* Thermalright True Spirit 120M BW
* Thermalright X-Silent 140 x 3
* Syba SI-PEX40064 x 2
* APC Back-UPS ES 700

* HDD Seagate 3TB (ST3000DM001) x 5
* HDD WD Green 3TB (WD30EZRX) x 5
* HDD Toshiba 2TB (DT01ACA200)
* SSD Samsung 840 EVO 500GB (MZ-7TE500LW)
* SSD M.2 Samsung XP-941 128GB with Windows 7 64bit

* Pioneer BDR-206M (outside the case)



= CLICKable =



Handmade adapter for HDDs like Procase 5T3-3T5.





















100 Mbps
I7MCxdq.gif
24/7
QTupnGE.gif
160 W


Total: 5x3 TB + 5x3 TB + 2 TB + 500 GB = 32,5 TB in One Chassis ;)


-------------
UPD №1.
-------------

* HDD Seagate 5TB (STBV5000200 = ST5000DM000)

>>



Total: 5x3 TB + 5x3 TB + 2 TB + 500 GB + 5 TB= 37,5 TB in One Chassis ;)
 
Last edited:

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
33,073

Impressive amount of storage for a consumer case.

How are you handling redundancy?
 

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
33,073
I don't get it. What part is a fire hazard?

The desktop versions of those APC UPS;es get very hot when they are under battery discharge.

The 1U rackmount revisions get even hotter (less space for the battery to air out)

That 1U UPS is sandwiched between many pounds of servers and a rug.

I'm guessing that is what he is thinking.
 

PigLover

[H]ard|Gawd
Joined
Jul 11, 2009
Messages
1,181
Having the chassis of the APC rackmount unit in direct contact with combustible materials (carpet/flooring). Major no-no. If it burns your insurance company is almost guaranteed to refuse to cover.

The stack of stuff on top if it isn't really a problem, regardless of "weight". There is no expectation of ventilation from the top or bottom of rack-mount equipment. They are expect to almost touch with no airflow between them.
 

Quartz-1

Supreme [H]ardness
Joined
May 20, 2011
Messages
4,257
I was referring to the whole lot. Not only the UPS on the carpet but the servers above it. They may have no expectation of ventilation, but they'll still get hot, and that's a wood table. Wood burns quite easily. Add in household fluff, perhaps pet hair, to act as kindling, and you have a disaster waiting to happen.
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
Is there anything in the APC instructions that say to leave a U of space between the UPS? If not then technically he's not doing anything wrong. I would install fans in the UPS if there arn't any though. They just put a new one at work in one of the racks that's not on inverter power and they get high temp alarms all the time even if it's not hot in the room.
 

PigLover

[H]ard|Gawd
Joined
Jul 11, 2009
Messages
1,181
Is there anything in the APC instructions that say to leave a U of space between the UPS? If not then technically he's not doing anything wrong. I would install fans in the UPS if there arn't any though. They just put a new one at work in one of the racks that's not on inverter power and they get high temp alarms all the time even if it's not hot in the room.
Not surprising that you'd take this position given the fire hazard you've posted pictures of in your basement...

No. Nothing in the APC instructions say to leave any space. But it does clearly say not to leave in contact with combustibles due to battery runaway risk. They expect that rackmount chassis will be mounted in a rack (doh!) and that the other things it touches above/below will be metal (double doh!)
 

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
33,073
Is there anything in the APC instructions that say to leave a U of space between the UPS? If not then technically he's not doing anything wrong. I would install fans in the UPS if there arn't any though. They just put a new one at work in one of the racks that's not on inverter power and they get high temp alarms all the time even if it's not hot in the room.

You don't necessarily need to leave space. Metal cases conduct heat. A rug does not.
 
Last edited:

bds1904

Gawd
Joined
Aug 10, 2011
Messages
1,007
Not surprising that you'd take this position given the fire hazard you've posted pictures of in your basement...

No. Nothing in the APC instructions say to leave any space. But it does clearly say not to leave in contact with combustibles due to battery runaway risk. They expect that rackmount chassis will be mounted in a rack (doh!) and that the other things it touches above/below will be metal (double doh!)

I would be more worried about battery acid from bulged/broken batteries than anything. The batteries in that apc will be under far more pressure with the weight of those servers, add the extra heat and you have the perfect combination for broken batteries.

At least in a rack you wouldn't have the additional smashing effect.

Battery acid on the feet at 3 am sounds great. The dog/cat/kid injesting it sounds even better.:rolleyes:
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211

skypine27

Gawd
Joined
Apr 18, 2008
Messages
733
17TB storage (see sig). All internal. The 2 x SSDs are mounted via double sided tape behind the mobo tray. The other 5 x mechanical HDDs are mounted in the front drive rack behind 2 x 120 mm fans. Case handles it well, but with no side windows (as well as sound dampening insulation on the side, top, and bottom panels, I totally give up on cable management. You will be disgusted :)

Edit to include Pic when I get home (on vacation till early Dec)
 

Master_shake_

Fully [H]
Joined
Apr 9, 2012
Messages
17,795
Current inventory:
49x2TB Seagate SAS
19x3TB Seagate SATA
3x320GB Western Digital SATA
2x160GB Western Digital SATA
2x256GB Samsung 840 Pro SATA SSD
1x128GB Samsung 840 Pro SATA SSD
1x300GB Western Digital VelociRaptor SATA

Failed inventory:
7x HP 300GB 10K SAS 2.5" (In various states of failure)
3x3TB Seagate SATA (In various states of failure)
12x2TB Seagate SAS (In various states of failure)
2x40GB Maxtor IDE (Untested in years)

I've also thrown away another two or three 3TB Seagate SATA drives over the years. Dunno why I didn't just keep 'em for the pile. I'd love to pick up another 24-bay chassis and make use of the 3TB drives, but finances don't allow for it at the moment. Maybe in the future.

IMG_2640_zpsf7f6a77f.jpg


Both chassis are identical, snapshot'd and rsync'd nightly:
Code:
[root@nas ~]# zpool status pool
  pool: pool
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support
        feature flags.
  scan: scrub repaired 0 in 5h7m with 0 errors on Sat Sep 27 15:07:01 2014
config:

        NAME                        STATE     READ WRITE CKSUM
        pool                        ONLINE       0     0     0
          raidz2-0                  ONLINE       0     0     0
            scsi-35000c50034f36cff  ONLINE       0     0     0
            scsi-35000c50034eb58bb  ONLINE       0     0     0
            scsi-35000c50034f44577  ONLINE       0     0     0
            scsi-35000c50034e85e4b  ONLINE       0     0     0
            scsi-35000c50034f422b7  ONLINE       0     0     0
            scsi-35000c50034e85c3f  ONLINE       0     0     0
            scsi-35000c50040cf0c4f  ONLINE       0     0     0
            scsi-35000c500409ae567  ONLINE       0     0     0
            scsi-35000c500409946ff  ONLINE       0     0     0
            scsi-35000c5003c95a907  ONLINE       0     0     0
            scsi-35000c50034fbe17b  ONLINE       0     0     0
            scsi-35000c50034f3dfc7  ONLINE       0     0     0
          raidz2-1                  ONLINE       0     0     0
            scsi-35000c50034f3cc5f  ONLINE       0     0     0
            scsi-35000c50034f3e81f  ONLINE       0     0     0
            scsi-35000c50034ea0857  ONLINE       0     0     0
            scsi-35000c50034ff6167  ONLINE       0     0     0
            scsi-35000c50034f3decf  ONLINE       0     0     0
            scsi-35000c50034f421c7  ONLINE       0     0     0
            scsi-35000c50034f3daeb  ONLINE       0     0     0
            scsi-35000c50034ff1b8b  ONLINE       0     0     0
            scsi-35000c50034f42db7  ONLINE       0     0     0
            scsi-35000c50034f3d3ab  ONLINE       0     0     0
            scsi-35000c50034e011d3  ONLINE       0     0     0
            scsi-35000c5003c95abdf  ONLINE       0     0     0

errors: No known data errors
[root@nas ~]# df -h|grep -v tmpfs|grep -v oot
Filesystem               Size  Used Avail Use% Mounted on
pool                      36T   14T   22T  39% /pool

too bad you're not local.

got a 31u rack for sale

:D
 

Blue Fox

[H]F Junkie
Joined
Jun 9, 2004
Messages
11,813
Right, so, back on topic I think. My setup as of late. Currently about half has been moved to a datacenter 15 minutes away to act as offsite backups.

Primary server consists of the following:
Supermicro 846E1
Supermicro X9SCM-F with Pentium G630T and 8gb ECC RAM
Intel 320 40gb SSD
Areca 1882i
24 x 2TB Hitachi disks

Secondary server consists of the following:
Supermicro 847E1
Supermicro X8DT3-F with Xeon L5630 and 8gb ECC RAM
Imation 32gb SSD
Areca 1880i
26 x 2TB Hitachi disks
10 x 1TB Hitachi disks

File servers #3 and #4 only have a couple disks each, so won't bother mentioning them. Third (and fourth, not pictured) 4U chassis aren't in use at the moment, but should be soon seeing as I'm completely out of space. Have only ~3TB total free.

P1140017.jpg
 

Blue Fox

[H]F Junkie
Joined
Jun 9, 2004
Messages
11,813
It's not perfect, but I tried my best with velcro wraps. There's an HP 1800-24G and 1U PDU at the back that aren't visible as well.
 

Lost-Benji

Limp Gawd
Joined
Jun 23, 2013
Messages
321
Look under the desk in the back. There's a standalone 1500vA UPS which is powering the two 4Us. The two 2Us are running off the 1U UPS. And even if it wasn't, the 1U is a 1000vA (or 1500, I can't recall) UPS. More than enough to run those two boxes at full tilt (not that they ever are) for the 2-5 minutes it would take them to shut down. Combined, they draw something like 400W from the wall.

The airflow issue is the reason that they're sitting so far out in front of the desk instead of being flush with it. Temps aren't bad, but it is a bit of a space heater. Warmest room in the house, year round. (And yeah, I'd love to have another room to put them in... But this IS my "other room". My house only has two bedrooms!)
I rebuild the APC UPS's regularly (they are strong and easy to source parts for) and I even have one of the 1U UPS's myself. They are only 1KVA (big claim) but realy only handle about 300W before being rather nasty on the output. I have pushed them to 500W but only with brand new batteries, once aged a little, the UPS is not suited for heavy systems. I also know these get very warm in operation hence the fail, same for your comment "Temps aren't bad", you cant tell when it is sandwiched like that!
Lift the hole lot off floor (if even only an inch) and get the weight of the servers off the UPS. it was never designed to have a weight on top (active components just under lid and fan is plastic and easy to crush/seize).

Is there anything in the APC instructions that say to leave a U of space between the UPS? If not then technically he's not doing anything wrong. I would install fans in the UPS if there arn't any though. They just put a new one at work in one of the racks that's not on inverter power and they get high temp alarms all the time even if it's not hot in the room.
The UPS's come with a single rail kit and if you have seen one, you would know that they don't need space but also are not to have any weight on them.


Zarathustra[H];1041253897 said:
Impressive amount of storage for a consumer case.

How are you handling redundancy?

English dude!

So all that for a JBOD box with a screen-shot of drives too full....
 
Last edited:

xtream1101

n00b
Joined
Dec 17, 2014
Messages
9
Raw: 86T
Usable: 59.073T

Supermicro SC846E16 & 2 x SE3016 SAS expanders
2 x SuperMicro 900 Watt
Supermicro X8DTE-F
2 x Xeon 5570 @ 2.93GHz
4 x G.SKILL Ripjaws 4Gb sticks
Adaptec 5805 HW Raid card
8 x Samsung Spinpoint F4EG 2T (HD204UI)
10 x Seagate Barracuda 3T (ST3000DM001)
10 x WD Red 4T (WD40EFRX)
Tripp Lite SMART1500LCDT 1500VA 900W UPS
Windows Server 2012

This system runs my Plex server and my downloads (usenet).
The hdd's are set up in 3 main storage arrays each in a Raid6 array:
Array #1: 8 x 2T Samsung (HD204UI)
Array #2: 8 x 3T Seagate (ST3000DM001)
Array #3: 10 x 4T WD Red (WD40EFRX)

I also have a Raid1 setup using 2 x 3T Segate hdd's to download to.


Inside the Supermicro case
7HRsJICl.jpg


Array #1 is stored in the supermicro case (SC846E16)
Array #2&3 are in the middle case (SE3016)
Raid1 for downloads is in the bottom case (SE3016)
GyRcLuMl.jpg


This is all in my computer room in a 26U rack:
SEuCLlol.jpg
 

artbird309

Weaksauce
Joined
Jul 26, 2012
Messages
100
@xtream1101

What are you using to dampen the sound in the rack and how well does it work? And how do you deal with the heat dissipation then?
 

xtream1101

n00b
Joined
Dec 17, 2014
Messages
9
@xtream1101

What are you using to dampen the sound in the rack and how well does it work? And how do you deal with the heat dissipation then?

I got the rack off CL, not sure of the model.
It is a sound proofing foam that outlines the interior of the rack. It works very well though I do still want to get quieter fans for the supermicro case I just added.

There are 3 high flow fans in the top rear of the rack as seen in the picture of the servers. They pull air from the side vents at the bottom of the rack and through the gaps around the front door. All my hdd's stay under 32C So I am very happy with the cooling.

Edit:
Here is a video I just took so you can hear the noise difference https://vid.me/eWol
 
Last edited:

cantalup

Gawd
Joined
Feb 8, 2012
Messages
758
^^^
That is telecom rack, not server rack
Pretty old telecom rack with sound breaker to reduce noise from noisy equipment.

You can replace with dc fans with controller.
I have 19u telecom rack that picked from near trash dumpster 10 years ago.
Solid and heavy as metal dumpster.
 

xtream1101

n00b
Joined
Dec 17, 2014
Messages
9
@cantalup

I always thought that telco racks were not very deep. So what makes a telco rack different from a server rack?

I will be replacing the rear fans eventually, they are not very loud.
Either way I love this rack!
 

bds1904

Gawd
Joined
Aug 10, 2011
Messages
1,007
@cantalup

I always thought that telco racks were not very deep. So what makes a telco rack different from a server rack?

I will be replacing the rear fans eventually, they are not very loud.
Either way I love this rack!

Most telco racks used to be 23", not 19" like modern rackmount equipment. 99% of new equipment telco's use now is 19". "Telco Racks" typically have a lower weight capacity. Most telco equipment is fairly lightweight in comparison to servers, with the exception of chassis routers that are also used in regular network's. Example, the Juniper chassis that I use is heavy as all heck and 4 post but the alcatel-lucent equipment is typically lighter, and mounted in 2-post configuration.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,995
^^^
That is telecom rack, not server rack
Pretty old telecom rack with sound breaker to reduce noise from noisy equipment.

You can replace with dc fans with controller.
I have 19u telecom rack that picked from near trash dumpster 10 years ago.
Solid and heavy as metal dumpster.

WAT?


It's not a telco rack. It's also not that old. Square hole racks are usually fairly recent and also not widely used by telco companies (in fact, ATT techs will look at you crooked if you ask them to install on a square hole rack, they also don't even carry the captive nuts either).

Old telco racks usually aren't that deep and are 23" wide instead of the 19" rail spacing.
 

cantalup

Gawd
Joined
Feb 8, 2012
Messages
758
WAT?


It's not a telco rack. It's also not that old. Square hole racks are usually fairly recent and also not widely used by telco companies (in fact, ATT techs will look at you crooked if you ask them to install on a square hole rack, they also don't even carry the captive nuts either).

Old telco racks usually aren't that deep and are 23" wide instead of the 19" rail spacing.

I would say, not old, but just telco rack.

I have telco rack with square hold rack and has captive nuts.
it is very deep.
It was silly at me when misplaced all parts in apt closet. all square rail and captive nuts were none to be found. got it in 2005( this was may assumption on old).

can be mounted front and back.
 

cantalup

Gawd
Joined
Feb 8, 2012
Messages
758
Most telco racks used to be 23", not 19" like modern rackmount equipment. 99% of new equipment telco's use now is 19". "Telco Racks" typically have a lower weight capacity. Most telco equipment is fairly lightweight in comparison to servers, with the exception of chassis routers that are also used in regular network's. Example, the Juniper chassis that I use is heavy as all heck and 4 post but the alcatel-lucent equipment is typically lighter, and mounted in 2-post configuration.

thanks for the info,

It was my bad, that I did not save neatly and realized some parts were gone during moving in to my house from apt at that time. The rack was missing side panels when I got it.

I know, it was telco rack, when saw telco cables (ports) in it, and trashed out the guts since not needed.
 

cantalup

Gawd
Joined
Feb 8, 2012
Messages
758
@cantalup

I always thought that telco racks were not very deep. So what makes a telco rack different from a server rack?

I will be replacing the rear fans eventually, they are not very loud.
Either way I love this rack!

you will love your rack,

replacing the fan with good power consumption by assuming running 24/7
most DC fans are easy to control and very efficient too.
if you know microcontroller or something..
Arduino is the easy to learn, you can create DC fan controller with PWM monitoring and add ethernet capability to control.
let say...max fan speed when temperatur raising to XX Celcius... and do notification. including funky your imagination


for me, telco rack is like a box/drawer with front air intake hidden, server rack is "open" box ( grilled/mashed door on the back and front)
once again, that my understanding.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,995
you will love your rack,

replacing the fan with good power consumption by assuming running 24/7
most DC fans are easy to control and very efficient too.
if you know microcontroller or something..
Arduino is the easy to learn, you can create DC fan controller with PWM monitoring and add ethernet capability to control.
let say...max fan speed when temperatur raising to XX Celcius... and do notification. including funky your imagination


for me, telco rack is like a box/drawer with front air intake hidden, server rack is "open" box ( grilled/mashed door on the back and front)
once again, that my understanding.

You made your own definition right there.

Many modern server racks have enclosed front ends, either for privacy, security, dust control, or sound isolation. Great lakes rack for example still produces their racks with the option of a front Plexiglas window and even offer solid front doors for a NEMA 12 rating.


Your definition of a telco rack is not the standard definition of a telco rack. This is a server rack. Telco racks are generally 23" wide instead of the 19" width.

You also assume since you see telecommunication equipment in a rack that it's a telco rack. We use APC AR3100 rack cabinets for all of our telecommunication equipment and carrier equipment. Why? Because the 19" rack size is a better approach at space management and stays align with the cabinet design.
 
Last edited:

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,995
Ockie, I remember your build :p Have you done some upgrades? :)

welcome back Ockie, I hope you will post in the new 2015 thread :)

I've been lurking the past few years. Moved several times in the process. Picked up a pile of new hobbies that makes hard drives look cheap :p

In any case, I'll be back. I'm actually sitting on the sidelines until those 10TB's come out to make it a nice even number.
 
Status
Not open for further replies.
Top