SAS Expander Card Experience

B

bkelly

n00b
Joined
Sep 18, 2009
Messages
3
Hi Everyone,

When I was searching for information about SAS expander cards, this forum had some helpful information but also a lot of my same questions. Now that I'm a little further down the road, I thought I'd share my experience.

My goal is to consolidate two 24-drive pieced together NAS systems into one system's direct attached storage. The plan was to use an Adaptec RAID 8085 to run a Tripp-Lite 8088 cable to a SAS expander card in each storage node. The expander card would then use six Tripp-Lite 8087 cables to connect to a 4-drive backplane created by Guanghsing.

In April of 2009, I put in an order for Chenbro CK13601 SAS Expander Cards from CDW-G. I received them in May, installed it in one of the storage nodes, booted the main system into the Adaptec BIOS and immediately had trouble. First the SAS expander was not seen by the raid card and then later not all the drives would be seen by the expander and occasionally the Adaptec software would either hang or crash. I spent weeks troubleshooting, swapping drives, backplanes, cables and Chenbro cards. It was very frustrating as a cable would find four drives when plugged into CD1 and only two drives when moved to CD5. My questions to Chenbro support went unanswered except to forward me the card's manual and ask if the card was properly jumpered. I finally decided the ports on the CK13601 were low quality and returned the cards in August.

As a replacement, I ordered a single HP SAS Expander Card from by.com as a trial and an 8087 to 8088 adapter from pc-pitstop.com. To power the card, I stole the motherboard from a spare eMachines computer around the office and wired it up with no boot devices. When I booted back up into the Adaptec BIOS, things went much smoother. A couple of drives were still missing but it was now possible to create an array with the visible disks and see which drives were not being accessed while the RAID rebuilt. (With the Chenbro card, some visible drives would error out when initialized.) I replaced a couple of cables and soon had all 24 drives visible and functioning.

I have since ordered another HP SAS Expander Card. To power them, I've gotten a JetWay JI31GM4-LF motherboard, an Intel Celeron 430 and a stick of 512 MB Geil DDR2 memory from newegg.com. The total cost for the power solution is $91 but now the power switch on the front of the storage nodes works.

Today, this one node holds about 4.6 TB of data and I am almost ready to plug the second one into the Adaptec RAID 8085. I've been pleased with the performance so far. I've also learned that I can plug the 8088 cable directly into the back of the HP SAS Expander Card even though that port is labeled "external connection for tape", saving the need for an adapter. My only complaint with the HP so far is that it has no way to be powered directly from the power supply.

Hope this information is helpful to others.

Brian
University Corporation for Atmospheric Research
 
There is a jumper on the chenbro expander to use either "original" firmware or "updated" firmware. Updated is the default, and I had problems with not seeing all of my drives with it. The chenbro works fine for me with the original firmware setting. Don't know if that would solve any of your problems or not

http://www.hardforum.com/showthread.php?t=1441062
 
I used to also have an Adaptec 5 series card (51645) and it did not want to play nicely with my Chenbro SAS expander at all, original firmware or not. I ended up replacing it with a Highpoint 4320 (using an internal to external miniSAS PCI bracket) to connect my JBOD, and it works like a charm (with the Chenbro in "original" firmware mode), but I've run into some compatibility issues with various hard drives:

1. Any refurbished Seagate seems to cause detection and dropout issues - regular Seagates seem to work fine. I have 7 1TBs in a raid5.

2. Creating a raid5 with more than 3 drives using the Samsung 1.5TB F2EG hangs during initialization and fails - just 3 drives works, and the 1.0TB spinpoint F1 drives seem to work fine in a 4 drive raid5.

Right now I'm using a Norco 4020 as the JBOD chassis, and it's only turned on when I need it (I use it mostly as an offline mirror). The Highpoint also supports online array roaming, which means you can safely add/remove entire raid arrays while the system is still on, unlike the Adaptec.

I didn't know HP also had a standalone SAS expander card, looks like they work better w/ the Adaptec cards.
 
PS poor man's way of powering on a PSU is just to short the PWR and GND wires and use the rocker switch on the PSU itself to turn on the unit. There's also dedicated powerboards you can get if you want a more elegant solution, without having to actually power a motherboard for no reason other than to act as a PSU switch.
 
I'm using a Chenbro CK13601 with an Adaptec 5805. The only problem I had was caused by a loose SATA cable.
 
kharan5876, I tried multiple versions of the Chembro firmware but my problems persisted. You have had no loose cable problems at all?

alamone, thanks for the description. The motherboard/CPU/DIMM solution isn't to make the power switch work but to power the HP SAS Expander Card. Since the card doesn't have a molex connector, I could think of no easier or cheaper way to get the card power.

shabazkilla, that is good information. Perhaps the Chenbros work well with certain cables. What brand do you use?

Brian
University Corporation for Atmospheric Research
 
I have an adaptec 4505 that when connected has the same problem as stated above.. I read you can get a firmware update from them by sending your card back but they never even respond to any of my messages.. I've sent over 5 in the past 3 months trying to get a response.. Any idea how to get someone from chenbro to respond?
 
Chenbro's support unfortunately really does suck. While I own a couple of their expanders, I don't own any Adaptec cards, so I can't really help you there (not to mention I haven't had any issues with my Areca cards).
 
HP SAS EXPANDER CARD

hmmm....

Linux support? ability to compile driver?


All this card does is hook up (via external port) to a RAID card that must support SAS expanders, correct?
 
Last edited:
I was going to buy an areca but the 4505 was cheap with 1 sas port but with dual core on a chip.. Wish I would of known about this site before buying.. I actually have a promise card but it doesn't seem to like the sas expander either and that promise card is hooked to my 16x 1tb drives. (just found out today I can turn that setting off for raid. Thanks to this forum my raid is even safer so thanks =) ) I originally was going to move everything over to adaptec but guess that got halted.. Thanks for the quick response.
 
It would appear that I solved my own problem.. I'm researching more into it but apparently my bios was out of date (even though I just recently bought the card) I also found out how bad adaptec sucks when it comes to updating the stuff since I had to figure out how to make a dos usb drive. Mainly because I would to dig out a floppy drive and 10 floppies if i didn't.. It seems to be much more compatible though now..
 
shabazkilla said:
I'm using a Chenbro CK13601 with an Adaptec 5805. The only problem I had was caused by a loose SATA cable.
Click to expand...

So you use two SAS cables for 8 native drives from the 5805 to the Chenbro which gives you total 36 drive capacity? Or did I miss how this thing works.

Also how is speed with the expander?

I'm looking to do this vs buying a super expensive Areca or using multiple little cards.
 
I found its what your really looking for.. I wanted a raid 6 for my data's protection. An Areca i've read and heard is one of the best to do the job but yes.. SAS can be expanded multiple times over. My 5405 for instance supports up to 256 drives hooked to a single sas connector on the hardware raid card. I can't seem to find a program to do a decent speed test.. If you can tell me a good one (not HD tune) I'll do a speed test on it and tell you how well it works out for me.. I currently have 5 drives through a chenbro expander in a raid 0 attempting to test the speed..
 
thebeephaha said:
So you use two SAS cables for 8 native drives from the 5805 to the Chenbro which gives you total 36 drive capacity? Or did I miss how this thing works.

Also how is speed with the expander?

I'm looking to do this vs buying a super expensive Areca or using multiple little cards.
Click to expand...
That expander has 36 ports, but of those 36 ports, only 24 are for internal drives. As for speed, it should be the same as if there drives were connected directly to the card (there is a 1.2gb/s connection between the expander and card). If you look at the design of RAID cards incorporating the IOP348 (like my Areca 1680ix-24), it all makes sense. The chip itself only supports 8 drives, but 4 of the lanes are for the external port and the other 4 go to an internal 28 port expander (so the internal drives are already connected to an expander anyway).
 
thebeephaha said:
So you use two SAS cables for 8 native drives from the 5805 to the Chenbro which gives you total 36 drive capacity? Or did I miss how this thing works.

Also how is speed with the expander?

I'm looking to do this vs buying a super expensive Areca or using multiple little cards.
Click to expand...

One mini-sas->mini-sas cable (SFF-8087 to SFF-8087) going from the expander to the Adaptec 5805. Then mini-sas->sata fanout cables (SFF-8087 to SFF-8448) from the expander to the drives. All cables are Adaptec.

The expander has 6 internal mini-sas ports for capacity up to 24 drives internally, plus and external mini-sas connection. There is also an additional mini-sas port internally for connection to the controller, and an extra external mini-sas port for connection to an other expander or controller.
 
Not to derail this thread from the OPs questions / issues I'd love to see a pic of a setup using one of these expanders. A PM is fine to avoid crapping up this thread.
 
thebeephaha said:
Not to derail this thread from the OPs questions / issues I'd love to see a pic of a setup using one of these expanders. A PM is fine to avoid crapping up this thread.
Click to expand...
I have an Areca 1680ix-24 and 2 of the CK13601 Chenbro SAS expanders. What specifically do you want to see?
 
Has anyone tried using the PMC Sierra based HP SAS expander card on a non-HP machine with a non-HP SAS card? If this works, is it possible to flash the firmware? The latest version supports SATA 6gbps, also supported by my SAS card, the LSI 9260-8i.
 
Anybody any experience with the Areca 1300 series cards and expanders? A setup like that would be a nice cheap way of building a large WHS box for example...
 
Hm se areca got one to. For 12/16 SATA/SAS drives 450/550$ on ebay

ARC-8020.jpg

http://www.areca.com.tw/products/sascableexpander.htm
 
Hey all,

Sorry for resurrecting this older thread but it kinda outlines a direction I'm thinking about.

I have been mulling over the idea of building a NAS to better expand my system storage (mainly media). I currently have 4x1.5TB Samsung F3EG drives that I want to utilise in an array but I am stumped as to what my best option is with regard to cost and performance.

The type of NAS is was looking was either a purpose built appliance (such as something from QNAP, which although start reasonably cheaply at £300 for the 310, you don't seem to get the features I would like including WOL until you get to the 439 at around £600), or I could build my own based on something like the Intel DH57JG M/B with Intel i3 Clarkdale CPU and a Chenbro ES34169 case running something like FreeNAS or unRAID (which would probably come to around £300 or so). Both of these would kind of give me what I want for now - a low power file storage system - that I can set up reasonably quickly and cheaply, however, future expansion would be minimal as I would essentially need to build another unit and I'm not sure of the performance (especially running the Intel M/B with only one Gb Ethernet port as I have read that this is generally the limiting factor in this type of NAS performance).

On the other hand, I currently run a Windows 2003 (SBS) server and have an Adaptec 5405 RAID card (which is running 4x250GB WD RE drives in a RAID10 configuration), currently housed in an IcyDock 4 drive housing. I am going to visualise this system using VMWare ESXi with a virtual copy of the Win 2003 server on top, as I would also like to run MythTV (or some such) on the same box and gives me the option to "play" with other OS as well. I have seen that although this card only has 1 SAS port (controlling 4 SATA drives) that it can be expanded using an SAS Expander card.

So, I have been wondering if I might as well buy a better server case (I would like to go rack mount at some point in the future) that maybe has 9x5.25" external drive bays so that I could utilise multiple IcyDock (or similar) hot swap drive bays running from this single 5405 card (or just go for something a bit more purpose build like this The cost of purchasing a new rack mount case, an SAS expander card, and potentially more HDD cages would be more that the other two, but it would give me room to add in the future. I could then run a single box with the WD drives as the main Virtual OS store, and all other drives as storage. I'm guessing that performance would be better too, but I don't quite see how the expander works - I would think that although my card is supposed to be able to support 256 devices, there would be a performance hit a some point if they are essentially all running off a single port. I'm not sure if this won't be too much for the single port on my Adaptec card?

Anyhow, any thoughts welcome.
 
Last edited:
Correction: My first post said I was using an Adaptec RAID 8085. I believe I was wrong about that model number and it is an Adaptec RAID 5085.

pgjensen, no driver is needed for any SAS expander card because, as you said, it hooks up to your RAID card that must support expanders. Think of it like a USB hub for the SAS protocol. For me, the expander connects to my Adaptec RAID 5085 through an SFF-8088 cable which manages the RAID volume and presents it to the Fedora 11 system through a OS provided driver.

swinster, you're right about the performance hit using SAS expanders (1.2Gb/s for a SFF-8088 cable) so I would not recommend using them unless you are connecting drives that reside in another case. In your case, I think you are on the right path looking at at a 16/20 drive chassis with internal SFF-8087 drive connectors that you can then attach to something like an Adaptec RAID 52445.

Brian
University Corporation for Atmospheric Research
 
You must log in or register to reply here.
Back
Top