Advice needed: server for 48 SSDs (professional project)

S

shadowwyvern

Gawd
Joined
Sep 22, 2005
Messages
604
A friend recently asked me for advice on a problem he was facing at work.

The company he works for has systems that record data to large numbers of SSDs, and then the SSDs get physically transferred to a server-type box to offload the data onto some other storage medium (eg, a large disk array). He currently works with 48 drives at a time.

Their current solution has 12 hot-swap bays, and they load the drives in groups of 12 and cycle through until all the drives are offloaded. This is, obviously, very inefficient, and they are looking to upgrade to a better system.

The requirements are basically:
1) 48 hotswap drive bays

2) Support for 48 SSDs. This is really tricky, because SSD's draw most/all of their power from the 5V rail, so for 48 of them, that could be 240+W

3) Ideally, support for 20+TB of non-SSD storage to store the offloaded data on. My one immediate suggestion was that a 2-system solution, connected by 10Gb ethernet or similar, should be considered.

I found this chassis: http://www.supermicro.com/products/chassis/4U/418/SC418E16-R1K62B2.cfm, which looks great, except it's power supply doesn't have enough 5V amps.

Since this is a professional project, my friend's company is probably going to go with a proper commercial solution, but does anyone have any insight as to what that specifically might entail?

Thanks!
 
shadowwyvern said:
I found this chassis: http://www.supermicro.com/products/chassis/4U/418/SC418E16-R1K62B2.cfm, which looks great, except it's power supply doesn't have enough 5V amps.
Click to expand...
What makes you think that the power supply is lacking in amperage? That's a 1620W PSU from one of the most solid PSU manufacturers in the world. Their 500W PSUs pump out 42A@5V. Something three times larger won't be enough? Which SSDs are you using?

If you'd like additional non-SSD storage, you can use one of SuperMicro's JBOD systems attached to a controller in the above chassis. Works like a charm.

Edit: You aren't confusing the 5V standby numbers for the 5V max, are you? The 4A listed is the power available to the motherboard when the system is off. It allows IMPI features to work, Wake On LAN, etc. (Simple, stupid, and technically "wrong" way to put it - It's actually used for a ton of stuff. But it's not the max amps for the 5V line.)
 
Last edited:
Yes, I definitely confused those. I was looking for a quick answer to my concern - in retrospect 4A is very tiny :) Sorry.

I suppose you have to email them for the specs on the the power supply (or rather the distribution board, since that seems to be what influences 5V availability)?
 
shadowwyvern said:
Yes, I definitely confused those. I was looking for a quick answer to my concern - in retrospect 4A is very tiny :) Sorry.

I suppose you have to email them for the specs on the the power supply (or rather the distribution board, since that seems to be what influences 5V availability)?
Click to expand...
Email or phone call. Supermicro has some downright EXCELLENT tech support. You can get ahold of a proper engineer very quickly, and they're incredibly knowledgeable about their products. It puts every other company I've ever had to deal with to shame.
 
time to move to a storage array of sometype with the ability to clone / snap luns.
 
TeeJayHoward said:
Email or phone call. Supermicro has some downright EXCELLENT tech support. You can get ahold of a proper engineer very quickly, and they're incredibly knowledgeable about their products. It puts every other company I've ever had to deal with to shame.
Click to expand...

I can vouch for that. Called Supermicro yesterday with a motherboard issue and had someone on the phone in less than 10 minutes. He didn't even ask me for the board's serial number, just started troubleshooting with me right away.
 
This sounds like an awkward workflow.

Do they screw in the drives to the hot-swap trays and then load them, copy, remove them, and repeat? I'm guessing the device they're getting their initial data from isn't using the same hot swap bays as any server solution like the SuperMicro.

SATA connectors are not made to be plugged and unplugged repeatedly - most of them have rated lifespans in the 10's or 100's of plugins at best. This is exacerbated if they're not properly held in place with a hot-swap cage or similar.
 
obrith said:
This sounds like an awkward workflow.

Do they screw in the drives to the hot-swap trays and then load them, copy, remove them, and repeat? I'm guessing the device they're getting their initial data from isn't using the same hot swap bays as any server solution like the SuperMicro.

SATA connectors are not made to be plugged and unplugged repeatedly - most of them have rated lifespans in the 10's or 100's of plugins at best. This is exacerbated if they're not properly held in place with a hot-swap cage or similar.
Click to expand...

As a point of curiosity I looked up how many pluggings they're rated to - turns out it's 50.

(from the wiki - esata section http://en.wikipedia.org/wiki/Serial_ATA)

Sounds like the OP's described situation might involve rather a lot more than 50 over the lifespan of the drives.
 
Thanks for the replies everyone! I'm sorry I neglected this thread for so long.

I confirmed with Supermicro that the power supply in the 418E16-R1K62B2 has 50A on the 5V rail, which should be sufficient for the drives that will be populated. I also confirmed that despite different part numbers, the drive trays they use in that chassis and the ones they use in their mobile rack (M28SAB) are compatible.

Obrith is very much correct about the workflow. According to my friend, this is exactly how it is done - sometimes as many as 15 cycles per day are put on the drives. Neither I nor my friend had really considered this. According to him, the drives in use today have > 300 mating cycles on them. In light of the rated life, its impressive they have lasted this long.

The "other" device is a completely custom-built system, and apparently effectively irreplaceable.

Could anyone that works with Supermicro servers on a regular basis share how much a full server tends to weigh? Their website lists a "net" and a "gross" weight - but I am not entirely sure that that means. The gross weight of the is something like 96lbs! Which seems pretty heavy for just a (albeit nicely built) metal box and power supply. This server would be shipped around a lot, and there is a "magic" line of around 150lbs per box where that becomes more difficult.
 
shadowwyvern said:
Could anyone that works with Supermicro servers on a regular basis share how much a full server tends to weigh? Their website lists a "net" and a "gross" weight - but I am not entirely sure that that means. The gross weight of the is something like 96lbs! Which seems pretty heavy for just a (albeit nicely built) metal box and power supply. This server would be shipped around a lot, and there is a "magic" line of around 150lbs per box where that becomes more difficult.
Click to expand...
I just recently received a pair of SC846 chassis. On the box, they're listed at 76lbs, if I recall correctly, and it felt about that when moving it through the house. SSDs tend to weigh about as much as the blank drive carriers they replace, so I'd guess you'd be under 100lbs after adding disks, cpu, ram, mobo, controller, etc. I can't realistically forsee a situation in which the chassis would weigh 150lbs unless you're using spinners.

As for net vs. gross weight, I'd guess that gross includes the box and foam, and net is just the chassis.
 
Indeed, Supermicro confirms they call "gross weight" the shipping weight - basically the weight of the box going out the door.

The challenge is, this company wants to put the server in a shock-mounted shipping case (the ones with isolated 19" racks inside), and most of those cases weigh 50+lbs!

We might go with their "72 2.5 inch" chassis, because it's 16lbs lighter and several inches shorter.
Checking with fedex, a 160lb box costs roughly 150% of what a 150lb box costs, due to crossing the boundary to "freight" :(
 
obrith said:
Do they screw in the drives to the hot-swap trays and then load them, copy, remove them, and repeat? I'm guessing the device they're getting their initial data from isn't using the same hot swap bays as any server solution like the SuperMicro.
Click to expand...

This is the biggest appeal of the Supermicro solution. They seem to be one of the only companies that offer both a server chasis, and a 5.25 "mobile rack" that use compatible drive trays.
 
shadowwyvern said:
This is the biggest appeal of the Supermicro solution. They seem to be one of the only companies that offer both a server chasis, and a 5.25 "mobile rack" that use compatible drive trays.
Click to expand...

So whatever is feeding the initial data to the drives is apparently SAS connected? That's a lot better than what I was envisioning... there are products that you plug bare SSDs into (and sometimes use outdoors) that record lots of data to them and then you're expected to take that bare SSD and put it into a computer to get the data off - many consumers don't seem to get that there is a reason (quality conscious) companies enclose disks that are not permanently mounted in a device.

I'm still a bit skeptical that there isn't a better workflow. For example, it might make more sense to have 2x external SAS arrays, take one off the "production" unit that feeds in data and move it to the archival server, plug the second array into the production unit, then swap them as needed. Or there are a variety of networking options - some very fast and cheap if you're going from one server to another.
 
You must log in or register to reply here.
Back
Top