High availability storage using SAS expanders

A SAS expander for example can have 2 SFF ports (8 SAS ports) connected to the host. What would happen if you connected for example a 36 port SAS expander to a bunch of hard drives and then one SFF port to one HBA in one server and another SFF port to another HBA of a different server.

I see even if this would work I guess it would need to be active/passive. The expander is a single point of failure, but if it could work this way it should be possible to have multiple expanders to allow full access to the data even if one fails. Even with one SAS expander it seems less likely to lose the one item than any CPU, RAM stick, NIC, HBA, etc that could take down a single server.

I can't seem to find out any information on if this would work, but I have seen information for budget clusters using DAS boxes.

So does anyone know if this could work? If my explanation is not clear enough I can make a diagram.

My guess is that both SAS HBA's would detect that there is more than one controler on the SAS network and shutdown to prevent you mangling the data.

It is not intended to work that way.

HA in storage systems like that is not possible that way as SAS topology doesn't permit it. For HA, one would have the first HBA going to a SAS expander connected to the first port on each disk and the second to another connected to the second port on each disk. This is possible with SATA disks as well so long as you have a mux interposer, but that's not something you can generally build on your own.

Lots of fud here.

It should shutdown? not permit it?

It was designed to do this, and it works perfectly fine.
If you use a non-clustered filesystem, yes, you will hose your filesystem in active/active mode.

You also must make sure your disks support multible hba's talking to them, but it's been a long time since I have seen ones that don't support 32 hba's controlling them.

It does nothing to protect against single points of failure. This is why sas disks are dual ported, and you create two paths to your systems, leaving only the disks themselfs as a spof, that you would use raid or something ontop, so remove that spof.

It is fully supported, to attach a single disk to an expander, and attach 10 esxi systems to that expander, and use vmfs on that disk. Though, this is normally not desired, due to lack of any kind of protection/raid/...

only one way to find out

http://www.ebay.com/itm/New-6Gbps-16-Port-SAS-Expander-PCI-E-Card-for-9240-8i-9261-8i-M5015-more-/111545362219

2 in 4 out.

Not true, that is a SAS2 card, there are no in/out ports, they are all the same.

For SAS1 it did have them, things where alitle more different with sas1 though.

Also, this is the whole point of the lsi sas switches, the hp blade sas switches, ...

patrickdk said:

You also must make sure your disks support multible hba's talking to them, but it's been a long time since I have seen ones that don't support 32 hba's controlling them.

Click to expand...

But this only works with SAS disks, right? Do SATA disks also work with multiple HBAs?

Depends on the sata disks firmware, on how many initiators it supports. This I don't know, as I never cared to look into doing something like this using sata. But also sata doesn't directly even support sas, and uses stun, and that might also have side effects also.

I would not even attempt to do something like this with sata personally. If you want to have HA, sata is not the way to go, unless you mirror two totally and completely seperate systems.

Poor mans experimental HA Cluster:



With 2x8 disk RAID 6. The most you could lose from an expander or HBA failing is 2 disks. You could do RAID 6+0 and as long as you did not have a drive failed the system could take any other failure or as long as you don't have a failed expander the system could take a failed disk and any other failure. At least to my understanding. Raid 6+1 would provide more fault tolerance, but you may as well have two 8 drive servers then.

This would work even better with RAID-Z3 and a few hot spares, but I'm not sure if ZFS is cluster aware or not.

You could also do just 2 HBA's per a node but then you would have to fail over if an HBA were to fail. I have not really heard of HBA's failing often anyway, So it may not be a poor choice.

This setup would be mainly for learning how to create a cluster and testing the survive-ability of different failures than actual production.