Best Storage Chassis for Multipath W/O SAS Expander

B

Beta4Me

Weaksauce
Joined
Nov 3, 2011
Messages
103
Hi Guys,

I'm planning a storage build with dual heads and a few storage shelves. Everything will be SAS multipathed (one connection to each head).

There is to be 3 shelves with 16 x 1TB 3.5" HDDs each with a backplane that has a dual-path expander.

The last (fourth) shelf will contain 10 x 2.5" SSDs. Therefore, I don't want to have an expander as this will severely limit bandwidth to the drives BUT I still want to multipath it to the dual heads for path redundancy.

What are the best chassis that suit this purpose, i.e. chassis that has a multipath-capable backplane but does not use a SAS expander?

Forgive me if any of that sounds stupid, I'm still learning and haven't any hands-on experience with this sort of stuff...yet ;)

Thanks! :D
 
AFAIK you can not have multi path without a SAS expander (a SAS expander built into the chasis backplane is a requirement of multi-path I/O).

While I am not 100% sure on this I am pretty sure this is the case.
 
houkouonchi said:
AFAIK you can not have multi path without a SAS expander (a SAS expander built into the chasis backplane is a requirement of multi-path I/O).
Click to expand...
That's the impression I was getting but was hoping that it wasn't the case :(

What might an alternative be so as not to compromise my SSD performance (i.e. saturate the SAS expander's link) but still maintain multipath? Hmm...
 
There are dual-port solid state and magnetic SAS disks that support multipath with no expander.

You could use 6x external SFF-8088/SFF-8088 multilane cables, 6x 4-port SFF-8088/SFF8087 brackets, and 10x dual-port SSDs (but you might as well go with 12x as you'll have 24 channels to the array) to get redundant full bandwidth paths to the SSD array.

You would need to be pretty picky about your controller and platform, though, for all of that cabling effort to not go to waste at a bus throughput or controller IO/s bottleneck.

Alternately, you could just use multiple expanders in the SSD enclosure.
 
Last edited:
Tiberizzle said:
There are dual-port solid state and magnetic SAS disks that support multipath with no expander.

You could use 6x external SFF-8088/SFF-8088 multilane cables, 6x 4-port SFF-8088/SFF8087 brackets, and 10x dual-port SSDs (but you might as well go with 12x as you'll have 24 channels to the array) to get redundant full bandwidth paths to the SSD array.

You would need to be pretty picky about your controller and platform, though, for all of that cabling effort to not go to waste at a bus throughput or controller IO/s bottleneck.

Alternately, you could just use multiple expanders in the SSD enclosure.
Click to expand...
Thanks for that, it's exactly what I plan to do.

The thing is, what chassis has a dual-port backplane with SFF-80887 ports (i.e. 2 per 4 drives) without using an expander. That's what I'm really struggling to find :(
 
Do you even have SAS drives? If not, don't bother. If so, why are you not wanting any expanders? You're going to be HBA limited in the end and cost shouldn't be an issue if you can afford SAS SSDs.
 
Blue Fox said:
Do you even have SAS drives? If not, don't bother. If so, why are you not wanting any expanders? You're going to be HBA limited in the end and cost shouldn't be an issue if you can afford SAS SSDs.
Click to expand...

Yes, SAS SSDs are planned :D

Expanders will limit the SSD throughput. It's not an issue for the SLC ones for ZIL/SLOG, but the MLC ones will almost saturate a 600MB/s (6Gbps) link on their own so I definitely don't want to reduce my bandwidth with an expander ;)

Therefore, I'm looking for a chassis with a dual-port backplane with SFF-8087 ports (dual ports per 4 drives). Therefore, I can maintain full bandwidth using 6 x SFF-8087 which will be converted to SFF-8088s for piping through to the dual heads (3 each) :) Finding such as chassis is proving to be difficult... :confused:
 
Last edited:
No they won't. You do realize that with a single mini-SAS port, you have four lanes, right? That's 2.4gb/s...and you could use two ports and have double that. There isn't a single HBA out there capable of handling that much throughput, so it won't be a limiting factor.
 
Yes, I know I have four lanes. So 12 drive chassis = 12 lanes = 3 x mini-SAS. They will be converted to external mini-SAS ports and piped through to the dual heads.

No, I never said that I would be running all 3 mini-SAS ports to an HBA. I have 7 PCIe slots on my mobos, so 1 will be for 10Gbe, and I'll have 6 free for SAS HBAs and what not. I'm going to use 4 x 9205-8e HBAs...2 for the HDD shelves and 2 for the SSD shelf. There'll be 1 mini-SAS free on each pair with the HDDs one being used for the internal drives and the SSD one for expansion for up to 4 more drives, if I needed it.

Am I missing something?
 
Last edited:
I just don't see how 4.8gb/s is going to be a limiting factor. Also makes the second HBA in each chassis (for the SSDs) unnecessary.
 
It's not 4.8GB/s it's 2.4GB/s per head (for redundancy not speed).

Split 2.4GB/s between 12 drives and you get 200MB/s per drive. Hopeless for SSDs.
 
No, it's 4.8gb/s. 8 lanes x 600mb/s as that's what each HBA has. Effectively you have 9.6gb/s throughput due to the fact that you are using both ports on each disk (with one HBA per chassis). I have no idea how you're getting the 2.4gb/s figure.
 
It's 2.4GB/s with an expander or 7.2GB/s without. 4.8GB/s is just not applicable in either scenario. It's irrelevant what the throughput of the HBA is for this purpose as either a 4-port one is used with an expander or two HBAs are used without, which is what I want to do.

But, all this is beside the point...I'm searching for a chassis with a dual-port backplane.
 
Blue Fox said:
Effectively you have 9.6gb/s throughput due to the fact that you are using both ports on each disk (with one HBA per chassis).
Click to expand...
Also, that's just not correct. It's an active-passive dual head setup for redundancy only.
 
No it isn't. You need to realize that you can use 8 lanes between devices. My math is not off. Utilizing a dual SAS expander backplane, you have 9.6gb/s effective throughput when using two HBAs, and for this very reason, manufacturers don't produce a product without SAS expanders because it's just unnecessary. Multipath IO is not only redundant, but increases throughput as well.
 
Blue Fox said:
No it isn't. You need to realize that you can use 8 lanes between devices. My math is not off. Utilizing a dual SAS expander backplane, you have 9.6gb/s effective throughput when using two HBAs, and for this very reason, manufacturers don't produce a product without SAS expanders because it's just unnecessary. Multipath IO is not only redundant, but increases throughput as well.
Click to expand...

Let's take a SuperMicro 826E26 (SAS2 Dual Expander Backplane) as an example.
This provides up/down ports per expander. In my scenario, I'd only use uplink, so I'd have two mini-SAS uplinks of 2.4GB/s each (one per expander).
So, I'd have one mini-SAS link to each head with 2.4GB/s to each head. That's it. The number of HBA's and their bandwidth is completely irrelevant.

Is this not correct?
 
It's incorrect; there are no such things as up or down ports. They are all the same. You can use multiple ports for increased throughput (two per HBA/expander in this case). Supermicro just labels them as such to demonstrate a possible configuration.
 
Blue Fox said:
It's incorrect; there are no such things as up or down ports. They are all the same. You can use multiple ports for increased throughput (two per HBA/expander in this case). Supermicro just labels them as such to demonstrate a possible configuration.
Click to expand...

So there are 2 mini-SAS per expander and I can use both to HBA(s) and get 4.8GB/s per expander? :D
 
Yes, been saying that all along. Plenty of us here doing that (myself included).
 
Incidentally, I saw a 5-in-3 SAS/SATA backplane module that broke out the dual ports into a pair of SATA connectors for each slot the other day. Huge footprint for a 2.5" SSD array so I won't bother finding the link again unless you actually want it, but it is confirmation that things like this exist at least :p
 
Tiberizzle said:
Incidentally, I saw a 5-in-3 SAS/SATA backplane module that broke out the dual ports into a pair of SATA connectors for each slot the other day. Huge footprint for a 2.5" SSD array so I won't bother finding the link again unless you actually want it, but it is confirmation that things like this exist at least :p
Click to expand...
Thanks for that info :) Probably doesn't really suit, especially as it breaks out to SATA rather than SAS. Good to know though!
 
The SAS/SATA 5-in-3 modules use SATA-style connectors because they support 5 drives, while there are only 4 channels on a SFF-8087 connector. You would use a pair of SFF-8087 to SATA fanout cables to attach it to the controller, with 3 channels from one fan-out left over for internal drives or another 5-in-3 module. Neither a SAS drive nor a SAS controller would know or care that it was a SATA-style connector on the host side of the backplane :p
 
Last edited:
Tiberizzle said:
The SAS/SATA 5-in-3 modules use SATA-style connectors because they support 5 drives, while there are only 4 channels on a SFF-8087 connector. You would use a pair of SFF-8087 to SATA fanout cables to attach it to the controller, with 3 channels from one fan-out left over for internal drives or another 5-in-3 module. Neither a SAS drive nor a SAS controller would know or care that it was a SATA-style connector on the host side of the backplane :p
Click to expand...
But a dual-port SAS drive would not work though, no? :p
 
You must log in or register to reply here.
Back
Top