[H]ard Forum Storage Showoff Thread

[Machupo]

I have a direct 10G BaseT copper link between my desktop and my NAS. It's been great.

Before it I used a Brocade SFP+ fiber direct link and it was garbage. Has turned me off from fiber for life.

Yeah -- I was looking at adding the Unifi 16XG to my stack, SFP+ down to my POE switch, then going 10GbE to the NAS. Keeping central management of the Unifi kit (APs, etc) would be nice. I'm sure there's better stuff out there, though.

All that SSD love over 1Gbe LAGs? Like running on a blown hamstring...

Nice touch on the mounting though. I do hope you have direct airflow through those things. They will get plenty hot packed in that dense.

I'm moving this summer, so keeping it on the avoton mobo for now, but along with the 10G switch mentioned above, i'm looking at a x10 supermicro board w/ dual 10GbE. There are a couple fans in the top pic immediately behind the SSD stack


[edit]
Went back to two stacks of SSDs (instead of inverting a middle set) as it allowed cleaner cable management (and better airflow):

[/edit]

 

[IdiotInCharge]

Unifi 16XG

They're reportedly fairly solid if using SFP+- their 10Gbase-T ports can be iffy, and if you're throwing in their new 10Gbase-T SFP+ RJ45 transceivers or MikroTik's versions, you might be in for a bit of a ride.

Honestly I'd recommend looking at MikroTik switches. They have a 4-port SFP+ one with 1 x 1Gbase-T port that would likely be the most that homelabbers would need. That would likely take their RJ45 transceivers pretty well as needed (as you can get motherboards with that for consumer desktops).

 

[Zarathustra[H]]

They're reportedly fairly solid if using SFP+- their 10Gbase-T ports can be iffy, and if you're throwing in their new 10Gbase-T SFP+ RJ45 transceivers or MikroTik's versions, you might be in for a bit of a ride.

Honestly I'd recommend looking at MikroTik switches. They have a 4-port SFP+ one with 1 x 1Gbase-T port that would likely be the most that homelabbers would need. That would likely take their RJ45 transceivers pretty well as needed (as you can get motherboards with that for consumer desktops).

I would love an affordable managed 24 port layer 2 switch with a small number (2-4) 10GBaseT "uplink" ports.

My two Intel 82598EB 10GBaseT adapters I got on eBay used a few years back just work, and just work beautifully.

I've actually hit 1.2GB/s file transfers over them using NFS.

By contrast my two Brocade BR1020's with transducers and OM3 cable from The Fiber Store were a miserable experience that never worked right.

I'm done with fiber. I had such terrible experiences with it, I just don't want to do it ever again.

 

[vxspiritxv]

Dell PowerEdge R740xd2

26 x 12TB HGST HUH721212AL5200 7200RPM SAS drives
(2 in RAID 1) + (12 in RAID 5) + (12 in RAID 5)
312TB Raw, as the image shows below 267.65TB usable.

2 x Xeon Silver 4110
96 Gigs of Hynix HMA81GR7AFR8N-VK memory
Controller: H730P
2 x 10GbE Fiber , 2 x 1GbE Copper (failover)
Work machine, used for Veeam, Backups and backup testing.

 

[Zarathustra[H]]

Dell PowerEdge R740xd2

26 x 12TB HGST HUH721212AL5200 7200RPM SAS drives
(2 in RAID 1) + (12 in RAID 5) + (12 in RAID 5)
312TB Raw, as the image shows below 267.65TB usable.

View attachment 152363

2 x Xeon Silver 4110
96 Gigs of Hynix HMA81GR7AFR8N-VK memory
Controller: H730P
2 x 10GbE Fiber , 2 x 1GbE Copper (failover)
Work machine, used for Veeam, Backups and backup testing.

View attachment 152364


View attachment 152365

How loud is it? I had a HP 2U storage server for a while, but I wound up not using it because it sounded like a Jumbojet taxiing.

(Literally)

(Not my video)

 

[vxspiritxv]

How loud is it? I had a HP 2U storage server for a while, but I wound up not using it because it sounded like a Jumbojet taxiing.

(Literally)

(Not my video)

http://www.quickesthosting.com/~spirit/SERVER.MOV

Very short clip, yes it's loud. It's co-located.

 

[Deadjasper]

http://www.quickesthosting.com/~spirit/SERVER.MOV

Very short clip, yes it's loud. It's co-located.

Very short indeed.

 

[IdiotInCharge]

By contrast my two Brocade BR1020's with transducers and OM3 cable from The Fiber Store were a miserable experience that never worked right.

I'm done with fiber. I had such terrible experiences with it, I just don't want to do it ever again.

I have a pair of DACs from them- one end to an Aruba S2500 off eBay, other to HPE managed workgroup switch. We use fiber at work for everything, but 'everything' is Cisco or Intel. That's reliable, not sure if I'd want to try doing that at home though.

 

[Factum]

840'ish TB of all flash test-storage...not in production, just for test, certification ect ^^
I am rather fond of those 15.3 TB SSD's

 

[ltcharles]

My set up after it got an upgrade from Athena power single bay hot swap bays to 4 hdd backplanes. Can hold 24 drives currently only holding 17 drives roughly 68 TB as of now plan to be at 100TB by end of year.


As well as having a UNAS back with 20TB for family backs up split in Raid 1

 

[blade52x]

Just setup my first NAS ever - its primary purpose will be to hold 8K 3D 360 video so that family can relive memories from within the capture. They're already coming out looking amazing on a Pimax 5k+ headset and should only get better with good 8k headsets (I went for the 5k+ over the 8k since reviews seemed to mention it is better).

But I think getting this setup was a sleep deprived decision (my newborn will be 4 weeks old soon) as looking back I could have done a DS1618+ with 6x12TB for not much more, and had the option of upgrading it to 10gbe down the line. But right now this works fine - the transfer speed is not a bottleneck. This NAS will probably last me about a year, and definitely not more than 2 with how much space these files need. By then I should have a significantly upgraded workstation and LAN arrangement.

 

[Zarathustra[H]]

I have a pair of DACs from them- one end to an Aruba S2500 off eBay, other to HPE managed workgroup switch. We use fiber at work for everything, but 'everything' is Cisco or Intel. That's reliable, not sure if I'd want to try doing that at home though.

I should retract part of my statement you quoted.

I have indeed gone back to fiber in the home.

it wasn't fiber that was my problem. It was brocade.

Now I have Intel 10G fiber adapters and instead of Fiberstore 3rd party transducers I am using "Made for Intel" Finisar ones, and the experience is great.

I'll fully admit, I was wrong to blame fiber for my woes. Fiber done right is actually pretty awesome.

 

[Zarathustra[H]]

In process of moving from my old Norco RPC-4216 to a 24 bay Supermicro SC846.

I expected the supermicro case to be better in every way. I was wrong.

To be clear it is better in MOST ways, but there are a few things I liked better about the Norco.

The Norco unit has much better drive caddies. The Supermicro ones are flimsy by comparison.

Also, the fact that you can buy an optional 120mm fan wall for the Norco is HUGE for home users. I was able to ghetto mod in 3 120mm fans in the Supermicro though. Between this and the optional dual "quiet" 900 watt power supplies it is much more "home compatible" now.

Side note, the 80mm screamers the Supermicro case ships with are not just loud and obnoxious, they are also dangerous. Never stick your finger in one that is powered on. It took the skin on my finger tip clean off!

 

[Welsper]

Never stick your finger in one that is powered on. It too the skin on my finger tip clean off!

Blood sacrifice! That system will now run perfectly for as long as you care to own it.

 

[Zarathustra[H]]

Blood sacrifice! That system will now run perfectly for as long as you care to own it.

Let's hope something good comes out of the blood letting.

I left a trail of blood up from the basement to where we keep the bandages in the kitchen.

I had forgotten how much fingers bleed

 

[IdiotInCharge]

Side note, the 80mm screamers the Supermicro case ships with are not just loud and obnoxious, they are also dangerous. Never stick your finger in one that is powered on. It took the skin on my finger tip clean off!

I remember putting Delta fans on CPU coolers, back when 80mm fans were considered large- so I can absolutely sympathize!

And no rig is built properly without bloodletting.

 

[EnderW]

Ha! You think that’s ghetto, checkout the front 140mm fan mod for my 846.

 

[Zarathustra[H]]

I remember putting Delta fans on CPU coolers, back when 80mm fans were considered large- so I can absolutely sympathize!

Yeah, I did that as well.

I also remember mounting the screaming Delta to some crazy heavy machined copper block heatsink, which knowing what I know now probably wasn't a very good heatsink due to the rather limited surface area it gave me, but it was all the rage back then.

It had a thick copper base and individual pins screwed into it. I rememeber it being pretty damned heavy.

Some googling suggests it may have been a Swiftech MCX series cooler.

Something like this:

That Delta fan pissed off my college roommate something awful

As loud and fast as those Delta's were, I don't recall them coming anywhere close to what the OEM's have been putting in server chassis in the last 10 years or so. You practically need hearing protection to be in the same room with these things.

And no rig is built properly without bloodletting.

Fair, but usually that's due to sharp edges in the case. I have those scrapes too

 

[Zarathustra[H]]

Ha! You think that’s ghetto, checkout the front 140mm fan mod for my 846.

It's not dumb if it works.

 

[SamirD]

That Delta fan pissed off my college roommate something awful

I brought all my computers to our dorm room, so I had 7 in there with a kvm switch--my roommate got a transfer, lol.

Looking back, I'm glad I left college--it was a tremendous waste of money compared to what I learned outside of school. I might go back someday for the piece of paper and just to really expand my knowledge, but there's only so much you can do in this life before you're gone and I would have no regret not having a piece of paper that cost $120k.

 

[Zarathustra[H]]

I brought all my computers to our dorm room, so I had 7 in there with a kvm switch--my roommate got a transfer, lol.

Looking back, I'm glad I left college--it was a tremendous waste of money compared to what I learned outside of school. I might go back someday for the piece of paper and just to really expand my knowledge, but there's only so much you can do in this life before you're gone and I would have no regret not having a piece of paper that cost $120k.

Much cheaper to go to a good state school. (At least it used to be 20 years ago. These days things seem to keep getting more and more expensive.)

College worked out very well for me, but I'd agree that most majors aren't worth the money from a strict return on investment perspective.

I'd argue that college has other benefits, broadening knowledge and life experiences to the point where people tend to become more interesting and compassionate people, but those benefits become a difficult sell at $120k...

 

[Deadjasper]

I'll take a marketable singing voice over a college degree any day.

 

[SamirD]

Much cheaper to go to a good state school. (At least it used to be 20 years ago. These days things seem to keep getting more and more expensive.)

College worked out very well for me, but I'd agree that most majors aren't worth the money from a strict return on investment perspective.

I'd argue that college has other benefits, broadening knowledge and life experiences to the point where people tend to become more interesting and compassionate people, but those benefits become a difficult sell at $120k...

I had just sold $15k of Intel stock to pay for one quarter and my next sale was going to be Microsoft. I started looking at the return I had been getting on MSFT and looked at the return that a degree would get me and they were VASTLY different. I kick myself that I sold Intel now as that was back in 1995--today that $15k would have been worth nearly $100k. And MSFT has been even better to me. Best decision I ever made was to keep it and leave school!

I think the lack of looking at college as an investment vs an 'experience' is what keeps people going and also being in debt for the rest of their lives. If you look at college as an roi, there's much cheaper ways to learn what you need. If you look at it as an experience, there's much better value out there for your money if you just want to party, lol. Almost either way, it doesn't make the best financial sense--at least to me anyways.

It's interesting that you mentioned broadening knowledge and life experiences--I found that the ones inside the academic bubble were just that--experiences inside a bubble. It was only until I started using my talents for what I wanted to do in this world did I start the real journey of becoming 'me'.

 

[SamirD]

I'll take a marketable singing voice over a college degree any day.

There's a lot of singers and talented people out there that never make it. I have a friend who's talented as an actress, and even after some small breaks she's actually considering giving it up. It's not easy at all.

 

[TheSlySyl]

I've got nothing on NAS or RAID arrays, but for a machine that's both gaming and a Plex Media Server, I'm happy with how it works. The hodgepodge of HDDs are from years of acquiring, mostly shucked.

3x NVME drives, one's hiding between the cooler and GPU.

Temps are good with all the drives and the case is absolutely silent in every circumstance that doesn't put the GPU fans at full load. (Even then its still quiet.)

 

[Zarathustra[H]]

I had just sold $15k of Intel stock to pay for one quarter and my next sale was going to be Microsoft. I started looking at the return I had been getting on MSFT and looked at the return that a degree would get me and they were VASTLY different. I kick myself that I sold Intel now as that was back in 1995--today that $15k would have been worth nearly $100k. And MSFT has been even better to me. Best decision I ever made was to keep it and leave school!

I think the lack of looking at college as an investment vs an 'experience' is what keeps people going and also being in debt for the rest of their lives. If you look at college as an roi, there's much cheaper ways to learn what you need. If you look at it as an experience, there's much better value out there for your money if you just want to party, lol. Almost either way, it doesn't make the best financial sense--at least to me anyways.

It's interesting that you mentioned broadening knowledge and life experiences--I found that the ones inside the academic bubble were just that--experiences inside a bubble. It was only until I started using my talents for what I wanted to do in this world did I start the real journey of becoming 'me'.

I think this is an interesting topic, but I am going to go ahead and self moderate here. We are so far off the topic of this thread it's pretty silly

I may reply to this post via PM if I don't forget. Feel free to reply

 

[SamirD]

View attachment 180433Temps are good with all the drives and the case is absolutely silent in every circumstance that doesn't put the GPU fans at full load. (Even then its still quiet.)

Nice. What case is that?

 

[TheSlySyl]

Fractal Design Define R5

 

[SamirD]

Fractal Design Define R5

Sweet.

 

[WesM63]

In process of moving from my old Norco RPC-4216 to a 24 bay Supermicro SC846.

I expected the supermicro case to be better in every way. I was wrong.

To be clear it is better in MOST ways, but there are a few things I liked better about the Norco.

The Norco unit has much better drive caddies. The Supermicro ones are flimsy by comparison.

Also, the fact that you can buy an optional 120mm fan wall for the Norco is HUGE for home users. I was able to ghetto mod in 3 120mm fans in the Supermicro though. Between this and the optional dual "quiet" 900 watt power supplies it is much more "home compatible" now.

Side note, the 80mm screamers the Supermicro case ships with are not just loud and obnoxious, they are also dangerous. Never stick your finger in one that is powered on. It took the skin on my finger tip clean off!

I have that case and as an FYI, you can replace those 80mm fans with quieter ones. You can't retain the power mechanism that supermicro uses but you can use the beige plastic housing and easily feed the 3pin down through the gap where the original power adapter was. I replaced all the 80's with Silenx fans, that plus the 900w psu made it bearable.

 

[EnderW]

Here’s a picture of my R5 with 16 drives before I moved to the 846

 

[TheSlySyl]

You definitely used every bit of space in the case you could for that setup.

My motherboard only has 6 SATA ports, I guess the logicial next step would be to get a PCI-E adapter, but I think I'm just gonna retire my nearly 10 year old 3TB instead.

It has over 7 years of active use.

 

[EnderW]

I thought about getting a low profile CPU HSF and hanging another 4 drive cage from one of the top fan mounts but decided against it.

 

[Zarathustra[H]]

I have that case and as an FYI, you can replace those 80mm fans with quieter ones. You can't retain the power mechanism that supermicro uses but you can use the beige plastic housing and easily feed the 3pin down through the gap where the original power adapter was. I replaced all the 80's with Silenx fans, that plus the 900w psu made it bearable.

Yeah, I decided to just rip out the fan wall. I was concerned that replacing the screamers with slower 80mm fans just wouldn't get enough airflow across the drives.

Instead - as luck would have it - the fan wall is exactly 360mm wide, so it fits 3x 120mm fans in its place. I ordered 3 120mm Noctual iPPC 3000rpm pwm fans, zip-tied them together through the screw holes and zip tied the bottom to the cable passthrough. There is still about a 3/4" space open on top that would allow back-flow of air, so I made a custom shaped wood bar to be installed there, with carved out channels for zip ties to tie it to the top of the fans. I also drilled a little hole through the shroud that goes around where the wires come out of the power supplies, to fasten the top of the rightmost 120mm fan there, and drove a wood screw through one of the holes in the side of the case on th eleft side, to hold the wood in place.

I also painted the wood black for aesthetic purposes, just for shits and giggles.

In addition to make sure enough air was being pulled over the drives, I also blocked any vent holes on the side of the case that allow air to enter after the hard drives. As I wanted to force all air to enter the case over the drives.

You can sort of see my handiwork in that pic above in the right case, but I didn't take any closeups. If anyone is interested I can do that when I get home from work.

 

[Zarathustra[H]]

I have that case and as an FYI, you can replace those 80mm fans with quieter ones. You can't retain the power mechanism that supermicro uses but you can use the beige plastic housing and easily feed the 3pin down through the gap where the original power adapter was. I replaced all the 80's with Silenx fans, that plus the 900w psu made it bearable.

Yeah, I decided to just rip out the fan wall. I was concerned that replacing the screamers with slower 80mm fans just wouldn't get enough airflow across the drives.

Instead - as luck would have it - the fan wall is exactly 360mm wide, so it fits 3x 120mm fans in its place. I ordered 3 120mm Noctual iPPC 3000rpm pwm fans, zip-tied them together through the screw holes and zip tied the bottom to the cable passthrough. There is still about a 3/4" space open on top that would allow back-flow of air, so I made a custom shaped wood bar to be installed there, with carved out channels for zip ties to tie it to the top of the fans. I also drilled a little hole through the shroud that goes around where the wires come out of the power supplies, to fasten the top of the rightmost 120mm fan there, and drove a wood screw through one of the holes in the side of the case on th eleft side, to hold the wood in place.

I also painted the wood black for aesthetic purposes, just for shits and giggles.

In addition to make sure enough air was being pulled over the drives, I also blocked any vent holes on the side of the case that allow air to enter after the hard drives. As I wanted to force all air to enter the case over the drives.

You can sort of see my handiwork in that pic above in the right case, but I didn't take any closeups. If anyone is interested I can do that when I get home from work.

Some pics if anyone is interested.

It works very well. Fans are nice and quiet enough for home use.

(though I'd probably do some more work to slow down the fans if it were in the same room as I am, because I am a silence freak)

It did require ditching the little 2U heatsinks and plastic fan duct it came with though, so I picked up a pair of Supermicro 4U active heatsinks, which were surprisingly cheap new on Amazon with prime.

Those are the 3000rpm versions of Noctuas 120mm iPPC PWM fans.

My unairconditioned basement is probably about 85 degrees right now, and with those fans spinning at 1450rpm there is enough airflow to keep my 7200rpm Seagate Enterprise drives at 33C at low to moderate load, They spin up if it gets hotter/busier.

 

[SamirD]

My unairconditioned basement is probably about 85 degrees right now, and with those fans spinning at 1450rpm there is enough airflow to keep my 7200rpm Seagate Enterprise drives at 33C at low to moderate load, They spin up if it gets hotter/busier.

I have a question--do you not have an issue with thermal runaway as the heat from the server heats up the garage? Generally that's what I've run across and the next thing you know, the fans are floored and the room temp is 90F+.

 

[Zarathustra[H]]

I have a question--do you not have an issue with thermal runaway as the heat from the server heats up the garage? Generally that's what I've run across and the next thing you know, the fans are floored and the room temp is 90F+.

I've never had that issue, but this is is basement the full floor plan of the house, the basement is probably a good 800+ sqft. That and and basements tend to be cooler due to being below ground.

 

[SamirD]

I've never had that issue, but this is is basement the full floor plan of the house, the basement is probably a good 800+ sqft. That and and basements tend to be cooler due to being below ground.

Ugh, I said garage. That's a good size sqft, and the cooling effect of being underground probably helps keep it under wraps then. Surprised it's 85F in there though--that's pretty hot!

 

[TeleFragger]

View attachment 180433
I've got nothing on NAS or RAID arrays, but for a machine that's both gaming and a Plex Media Server, I'm happy with how it works. The hodgepodge of HDDs are from years of acquiring, mostly shucked.

3x NVME drives, one's hiding between the cooler and GPU.

Temps are good with all the drives and the case is absolutely silent in every circumstance that doesn't put the GPU fans at full load. (Even then its still quiet.)

working on friends computers, I get to keep all the old hardware as they don't want it... with that I have a ton of drives that are different sizes...

I was googling and landed on a site where a guy used this same case, but he used paper to label his drives, and he said he should of used a labeler... was this you? did you label your drives with a labeler later???

my machine is just a file server for family pics/movies, family data, plex movies and dvr tv shows on a windows server 2019 box...

anywho... this case seemed to intrigue me and I almost went his way... but then was looking at the Deep Silence 5 to which I figured I would get instead; however my OCD went even farther and I landed my next case...
hope ebay links are allowed
https://www.ebay.com/itm/264403896522

Corsair Obsidian Series 750D CC-9011078-WW Airflow Edition Computer Case Black

specs say..
Case Drive Bays 3.5" 6
Case Drive Bays 2.5" 10

but I'm not sure how you get 10x 2.5 in it... but doesn't matter.. (re reading I think 6 in 3.5 bays and 4 hidden in side panel)

trying to find the optional 3 drive bay cage (3 of them as I can see 2 will fit but I'm hoping 3 will too...) for this case... I think the 450 series all the way up to 750d are the same...
but with that I can get ( if I can get the 3x cages added )...

15x 3.5
18x 2.5" (2x icy dock 6x2.5" owned - need to buy 1 more)
_____________
33 drives

BUT another killer thing about this case.. there are also 4 spots for ssd behind the side compartment where all the cables run so...
33 + 4 = 37 drives.....

I am running server 2019 now with Stablebit Drive Pool and love it. I get it that it isn't real raid, isn't gonna be the best performance but the SSD pool alone is not bad. I have 8x 500gb ssd now and my plans are as follows:
3x 4tb - 3.5"
12x 2tb - 3.5"
12x 500gb - 2.5" ssd
_________________
27 drives and room to grow....

ive got 1tb drives and other stuff too.. so who knows.
since drivepool allows you to add any size drives to the pool, it works well...

case should be here by next friday, but I'm so busy with my kid in travel soccer, and both of us in MMA, doesn't leave much time after work... and were going on vacation so will be some time till I can get to it...

 

[SamirD]

Can't wait to see that--37 drives is amazing. I thought we did it big back in the late 1990s with a cases similar to this one by California PC Products:

Ours has drive bays all the way to the floor total of 14 HH 5.25". This was a Cyrix P166 set up back in the day with a Mylex DAC960SUI SCSI-SCSI Raid running Raid 5 and Seagate Cheetah drives. Would do 10MB/sec raw in DOS. Still have the system and need to get the Supermicro motherboard repaired as it still booted until the battery must have leaked and damaged the board.