share sas target between two initiator

T

tuxtof

n00b
Joined
Aug 25, 2011
Messages
6
Hello everyboy

how to share JBOD between two HBA ?

my goal is to access some hard disk (vmfs formatted) between two ESX server

logically it's one goal of an sas extender

did someone have success with this multiple host access ? with witch extender ?

thanks for your help

regards
 
Very interesting question.....

I'd love to know what software is required to do this or hardware required.
 
@OP

I'm might be wrong, but I'm not sure you can do this in the way you describe, otherwise you risk filesystem corruption.

Yes, LUNs can be shared among HBAs; SANs effectively do this to enable SAS zoning, as far as I am aware. However, once a LUN is assigned, no other HBA can meddle with it.
 
Not sure this has been attempted with sas yet.
This has been done with scsi with gfs.

But the firmware on the drive would have to be adjusted to handle locking, and to lock on a per hba level.
 
lock is not done on hba level but fs level
we done this everyday with SAN, iscsi or anything else who allow to share block device

concurrent access is manage/done/allow by filesystem, VMFS (VMware) in my case

my only question is to share sas target between two initiator with sas expander
when i read expander's data sheet it seems possible

but i don't find any information on this kind of usage


regards
 
I think you might be thinking of redundancy

ie one server has 2 x HBA's connected to dual ported SAS drives

That is if a controller fails, the server can still get a path to the SAS drives thru the second working HBA

Not for 2 seperate servers tho

In which case you would setup the storage as NFS or iSCSI and connect the 2 servers to the same storage pool that way.


Example...

HDD's have port A and Port B on them (each has a seperate WWN)

Controller one connects to all of the HDD's ports A's
Controller two connects to all of the HDD's ports B's
Either directly or via the expander

.
 
hi

my goal is not redundancy but really Storage Networking

i want two server , each with 1 HBA , who share some disk device

for example if you looking the following expander datasheet

http://files.astekcorp.com/Products/A33606-PCI/A33606-PCI_PB_20100322.pdf

you see in the features

"Supports connection to multiple
initiators allowing Storage-Area- Networking and system redundancy"

the first part is my goal "allowing Storage-Area- Networking"

are somebody already try this kind of architecture

regards
 
Last edited:
For Storage "Networking" as such you are into SAS Switching territory.

ie think of network setup.... where each server connects to others thru a Switch. and VLans etc

Same can be done in a way with SAS Switch...

That is, each server has a HBA which connects to a SAS switch.

the Sas switch is connected to one or many JBOD Disk Units. and you (similar to Vlans on networks) configur the switch to decide which server can see which Disk.

in this case tho, you STILL Cannot have multiple Servers using the same Disk.

.
 
@stanza33

ok you think it's not possible to do this only with an expander, i read everywhere than sas expander and sas switch was the same think

and so in this case what means "Supports connection to multiple
initiators allowing Storage-Area- Networking and system redundancy" in the astek datasheet
 
with version 1 of SAS there were two types of expanders
Edge and Fan Out

Snip from Wikipedia for you
SAS 1 defined two different types of expander; however, the SAS-2.0 standard has dropped the distinction between the two, as it created unnecessary topological limitations with no realized benefit:

An edge expander allows for communication with up to 255 SAS addresses, allowing the SAS initiator to communicate with these additional devices. Edge expanders can do direct table routing and subtractive routing. (For a brief discussion of these routing mechanisms, see below). Without a fanout expander, you can use at most two edge expanders in your delivery subsystem (because you will connect the subtractive routing port of those edge expanders together, and you can't connect any more expanders). To solve this bottleneck, you would use fanout expanders.
A fanout expander can connect up to 255 sets of edge expanders, known as an edge expander device set, allowing for even more SAS devices to be addressed. The subtractive routing port of each edge expanders will be connected to the phys of fanout expander. A fanout expander cannot do subtractive routing, it can only forward subtractive routing requests to the connected edge expanders.
Direct routing allows a device to identify devices directly connected to it. Table routing identifies devices connected to the expanders connected to a device's own PHY. Subtractive routing is used when you are not able to find the devices in the sub-branch you belong to. This will pass the request to a different branch altogether.

Expanders exist to allow more complex interconnect topologies. Expanders assist in link-switching (as opposed to packet-switching) end-devices (initiators or targets). They may locate an end-device either directly (when the end-device is connected to it), via a routing table (a mapping of end-device IDs and the expander the link should be switched to downstream to route towards that ID), or when those methods fail, via subtractive routing: the link is routed to a single expander connected to a subtractive routing port. If there is no expander connected to a subtractive port, the end-device cannot be reached.

Expanders with no PHYs configured as subtractive act as fanout expanders and can connect to any number of other expanders. Expanders with subtractive PHYs may only connect to two other expanders at a maximum, and in that case they must connect to one expander via a subtractive port and the other via a non-subtractive port.

SAS-1.1 topologies built with expanders will generally contain one root node in a SAS domain with the one exception case being topologies that contain two expanders connected via a subtractive-to-subtractive port. If it exists, the root node is the expander which is not connected to another expander via a subtractive port. Therefore, if a fanout expander exists in the configuration, it must be the domain's root node. The root node contains routes for all end devices connected to the domain. Note that with the advent in SAS-2.0 of table-to-table routing and new rules for end-to-end zoning, more complex topologies built upon SAS-2.0 rules will not contain a single root node.
End Snip

SAS switch is a different beast for an example
see here http://www.lsi.com/channel/products/storagecomponents/Pages/LSISAS6160Switch.aspx

here for the video
http://www.youtube.com/watch?feature=player_embedded&v=3OUcEmce96Q&list=PL407B5D616ABDAF01

It's a blurry term SAS networking I believe, as the cabling between SAS HBA's and some SAS expanders is what's known as an Infiniband Cable (also used for 10Gb networking)

Also SAS terminology is similar to networking (ala Fibre Channel) as you have Targets and Hosts throughout both setups.

"Supports connection to multiple
initiators allowing Storage-Area- Networking and system redundancy"
that description from Astek I would read as
Supports connection to multiple HBA's to provide system redundancy.

Skip the Storage Area Networking part of the sales blurb.

To get SAS Networking (As such) you would have to setup say Fibre over SAS connections

That is, just like you can setup IP over Fibre that is networking using fibre channel transport)..... Pretending you have a network link but transporting it over fibre channel

Which you can also do backwards....

That is Fibre over Ethernet FCoE .... Pretending you have a Fibre Channel SAN but transporting it over an IP Network

Plenty of different ways to transport different things ie encapsulate one protocol inside another (which is actually what SAS does to talk to a SATA Device.

SCSI over IP (commonly known as iSCSI
ATA Over ethernet is another example

You can even have IP or Fibre Channel over a SAS connection

But your best bet is either using Fibre Channel or iSCSI ESX knows how to communicate over both and is smart enough to lock drives so both systems can access the same device at once.

Solaris box with FC cards shared to ESX is a nice cheap and fast solution in your case.
 
i just receive the following response from Astek

Christophe,

It is possible to share a single SAS-1 or SAS-2 device between two Host
Bus Adapters using the Astek A33606 series of expander boards. The caveat
is that the HBAs and expanders do not provide any distributed computing,
SAN or shared volume support internally, you must have a suitable
software/OS stack to manage the shared use of a disk. An example is GFS on
Linux, or Xsan on Apple OS X.

Ted
Click to expand...

i asked for more information and i waiting next response

i keep you informed
 
Yes you CAN share, but it's not going to work as each machine will overwrite the others info on the shared Disk and without a suitable OS Filesystem corrupt the data.

ESX is aware and can LOCK / Share the filesystem for concurrent access between multiple ESX boxes with Fibre Channel / NFS and iSCSI.... I doubt it can share as you want it to with direct SAS access.

but hell give it a go and see what breaks...

.
 
Ok i will explain that further for you... (had coffee now ;) )

Even tho it will physically work...

If you connect ESX up this way

Each ESX Server is going to think the SAS drives are direct connected internally and their own.

So you won't have any High Availablility etc available (on the fly transfering of one running virtual machine from one server to another) like iSCSI or NFS etc would give you

I am guessing you will also have some very weird things happening.

That is
Server one thinks there is say 3 virtual machines stored on the HDD, and thinks there is XYZ amount of space free.

Server two might think there is say 6 virtual machines on the same HDD, and think there is a different amount of free space.

Until each Server refreshes the their storage pool.... and finds differing amounts of Free space etc.

Are you planning on also using a SHARED SAS Drive as an ESX backup destination as well?:eek:
 
parityboy said:
Fixed. :) What you were talking about is FCIP, or iFCP
Click to expand...

Double wink back at you

http://www.google.com.au/search?q=fcoe&surl=1

http://en.wikipedia.org/wiki/Fibre_Channel_over_Ethernet

oh and the difference is

The standardized FCIP protocol works by transmitting FC frames over TCP/IP, which can route FCIP FC frames locally or remotely over existing Ethernet switches and routers. The emerging FCoE standard also uses existing Ethernet switches and routers, but runs without TCP/IP at the Ethernet layer so it's not routable. That means it requires administrators to create point-to-point configurations, or zones, in Ethernet switches using the MAC addresses found on servers and storage network cards. This is similar to how admins create zones in FC SAN switches using the world wide name addresses presented by FC host bus adapters (HBAs).

.
 
I think we're saying the same thing...your original post made FCoE sound as if it was designed to be transported over IP networks, hence my correction. :)
 
I had a similar question about this. I was looking into the LSI SAS 6160 switch and creating a scale-out file server (Server 8 tech), but I'd need to create a cluster shared volume for the failover cluster. Would pairing the SAS 6160 switch to a Supermicro JBOD chassis and some MegaRAID SAS 9285-8e cards in a couple servers allow me to achieve this functionality? Gets very blurry right there.
 
Are you referring to a setup like the following?
http://www.dataonstorage.com/images/products/DNS/9470/CiB-9470_70-70_Configuration_HBA.pdf

I spoke to them yesterday and this 70-70 setup was explained as a single domain setup.

They also have another setup where with 12 disks shared between 2 heads setup in active-active manner shown in the following. In this case it was described as a double domain setup where data access is available via either route.
http://www.dataonstorage.com/images...Private_Cloud_Storage_CiB-9220_FeatureSet.pdf

I guess unless the LSI Synchro cards are used, to manage which node has the access to the disk at any point in time, we need to manage this via software. Can someone please comment on this?
 
You must log in or register to reply here.
Back
Top