HP SAS Expander Owner's Thread

BestGear said:
Hi

Not sure if there is much interest in this now, but in my Norco I used the Supermicro JOBOD power board (old one) and a PCI-e slot intended for mining.

Works a treat and at £4 for the power board much less hassle than an old motherboard or indeed a SBC riser/daughterboard.

The mining board looks to have a 3.3v regulator on it as its clearly 5 and 12v from the molex.




Happy days.

David
Click to expand...

Hi, where could I purchase the power board to use the SAS expander without a motherboard ?

Regards,
Giuseppe Chillemi
 
Hey guys, new member and let me just say that this thread has been INCREDIBLY informative. Thank you all!

One question that I can't seem to find an answer to is if the max storage limitation is real or not.

I currently have 48TB (8x 6TB in a raid 6) and I just purchased a Norco RPC-4224 and 8 additional drives to double my capacity to 96TB

I was planning on using a HP 12Gb SAS Expander (769635-B21) along with my LSI 9260-8i but in the spec sheet is says that it's limited to:
"Up to 90TB of total storage with 15 x 6TB 3.5" SATA MDL HDD"

Is this true?

I've read about a lot of people purchasing this product to use with the Norco RPC-4224 and I haven't seen many of them using SAS drives.

Is there really a limit to 90TB and 15 SATA drives?
 
Forgive me, and I'm sure it's already been posted somewhere on this thread, but after 56 pages of it, I got lost.........

Soooo.... Here's my plan, I want to expand my storage capacity, internally and externally. So I was either going for another RAID card, or this HP SAS Expander, the 24-port will be enough even for future expansion.

Here's my server config:

Intel E3-1230V3
Supermicro X10SL7-F
16GB RAM ECC
2x RocketRAID 2720 SGL, 1 on x8, 1 on x4

I'm taking out the second RR2720 as I will be putting that on my offsite backup server I'm expanding, and in place I'm installing a 4-port NIC on the PCIe x4 slot.

Since, this motherboard has an LSI 2308 onboard, I'm planning on getting 1 (or 2 if needed) SATA-SAS breakout cable that will go to the HP SAS Expander, and from there you can imagine the rest right ?

So, my questions are...
1. The on-board LSI 2308 can expand to the HP SAS Expander as it's known to be compatible, right ?
2. If 2 is true, what else do I need to buy except for a SATA to SAS reverse breakout cable ? Or, what kind of cable do I really need ?
3. On the HP SAS Expander...if I understand this correctly I can configure which of the port can be inbound signal from the 2308, and setup the rest to be the outbound signal going to the HDDs. True ?
4. The PCIe x8 requirement of the Expander doesn't need any sort of connection to the motherboard, it's only meant to power up the Expander, right ?
5. If 4 is true, then theoretically I could use one of those PCIe extension power adapter thingy commonly used bit bitcoin miners, like this: MENGS® PCI-E 1x Express16x Adapter Riser Karte Erweiterung Powered USB 3.0-Kabel für Bitcoin Mining: Amazon.de: Auto ..right ?
6. If 5 is true, I technically don't need to plug the USB to PCIe x1 to the motherboard, right ?

I'm not looking into getting hi speed data transfers, I just need data archival, and backup purposes.

Anyway, I'm re-reading this whole thread but I'm open to quick answers ;)

I'm getting a good price for the Expander soon, and if the Expander won't be compatible with LSI 2308 then I can get an HP Smart Array P410. And yes, I don't really need to have the whole 6Gbps speed. Heck, not even SATA300 can't be saturated by one HDD anyway.

Much much much appreciated sirs and ladies !
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
1. Yes, generally any of the LSI controllers work without issue
2. You'll want forward breakout cables, not reverse
3. You don't need to configure which port is the uplink. SAS topology functions like ethernet essentially and you can just plug it in anywhere
4. Correct, power only
5. Yes, people have done that with similar adapters
6. Shouldn't need to

If the HP expander doesn't wind up doing what you want, you could get the Intel one instead of getting the HP controller
 
Blue Fox said:
1. Yes, generally any of the LSI controllers work without issue
2. You'll want forward breakout cables, not reverse
3. You don't need to configure which port is the uplink. SAS topology functions like ethernet essentially and you can just plug it in anywhere
4. Correct, power only
5. Yes, people have done that with similar adapters
6. Shouldn't need to

If the HP expander doesn't wind up doing what you want, you could get the Intel one instead of getting the HP controller
Click to expand...

Uhmmm..I thought I would need a reverse cable instead ? Since it's 8x SATA ports from motherboard (LSI 2308) which will be the host, then goes to the expander, to which I would need a reverse breakout SATA to SAS cable ? Right ? Or I'm totally misreading somewhere on this thread........I mean, on my RR2720, the SFF8087 to SATA cable I'm using is a forward cable though right ?

Cheers !
 
I didn't realize you had SATA ports on the board. From the LSI controller to the SAS expander, you will need a reverse cable indeed. From the SAS expander to drives, it will be a forward one.
 
Darn, I already have 2 sets of the regular/forward spare cables laying around, I felt hope on saving some cash there :p ......But thanks though mate, I appreciate the confirmation !
 
Just one final thing, I want to get clarified, specially this was mentioned a few times on forums and on this thread. But it's now 2016....

...do I need to switch the onboard LSI 2308 on my X10SL7 to IT-mode if I want to use the HP SAS Expander ? ...sorry, looking for a more 2016 era kind of answer for this question :)

And what are the pros and cons between the IR and IT mode when used with the HP SAS Expander ? (beside the no RAID thing, I won't RAID anyway)
 
RAID/non-RAID mode won't affect the attached expander, so you'll want the IT firmware seeing as you're not wanting the RAID capabilities.
 
I've just purchased a 487738-001 from eBay, I'm in the Houston area. I was not able to verify its firmware level prior to purchase, so if it comes in below v1.52, is there anyone in the Houston area with an appropriate HP RAID adapter who could assist me in flashing the expander?
 
I'm still making my way through this thread but this is what I am looking to setup

DL360 G7 with a P812 HP Card
Dumb PC case with an old motherobard, PSU and a load of disks connected with Mini SAS to SATA breakout cables and the following SAS Expander
https://www.amazon.co.uk/HP-468406-B21-Expander-Smart-Array/dp/B0025ZQ16K
SFF cable to connect the dumb PC case to the DL360
Configure the drives in the RAID's BIOS

Ta-da?

Does that follow right?
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
J89eu said:
I'm still making my way through this thread but this is what I am looking to setup

DL360 G7 with a P812 HP Card
Dumb PC case with an old motherobard, PSU and a load of disks connected with Mini SAS to SATA breakout cables and the following SAS Expander
https://www.amazon.co.uk/HP-468406-B21-Expander-Smart-Array/dp/B0025ZQ16K
SFF cable to connect the dumb PC case to the DL360
Configure the drives in the RAID's BIOS

Ta-da?

Does that follow right?
Click to expand...

That looks good to me. I will have that exact HP SAS expander available for sale in a few days. I ordered one from eBay for a project that has now been scrapped. Where are you located?
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
NobleX13 said:
That looks good to me. I will have that exact HP SAS expander available for sale in a few days. I ordered one from eBay for a project that has now been scrapped. Where are you located?
Click to expand...

Thanks for confirming. I actually have the hardware ready to go. Just need to buy a PSU and wait for the delivery of some breakout cables :)
 
I bought the newer 12gb expander and would like to update the firmware. Is it possible to buy an older p410 card to do the online flash, or will I need to buy a card designed for gen 9 proliant servers?
 
Just wanted to confirm, that after installing latest firmware 2.10 on HP SAS Expander, my disks are now connected through 3.0 Gbps link! Previously, on firmware 2.08, reported link speed was only 1.5Gbps.
To upgrade firmware I had to put Expander card into HP Proliant DL380 G6 server. Important - you have to connect cable from port 1 from built in P410 controller to port 9 on Expander card.

Firmware and upgrade guide is available here: Drivers & Software - HPE Support Center.
 
My Chenbro CK23601 just bit the dust after 3 years. The fan broke and it seems to have overheated, since it puts my Areca 1882i card in an endless "Waiting for F/W to become ready" loop.

I've had no luck finding an HP SAS Expander that is confirmed to have Firmware v1.52 or later on it.

If someone has the ability to flash a card for me, what would you charge me, assuming shipping is taken care of both ways?
 
Last edited:
As an eBay Associate, HardForum may earn from qualifying purchases.
I just realised......

...how do you know if you're running in dual uplink ? I know, plugging 2 cables gives you that answer, in a physical sense, but what if 1 cable is not working, right ? ;) ...anyway how do you know ?

Thanks !
 
Your HBA should be able to tell you about the topology and show you what links are functional. My Areca does that and I presume LSI supports it as well.
 
Just signed up to add the info that there is no need for a HP controller to update the firmware of this expander. Virtually any compatible LSI based HBA it IT mode and a Linux live cd will do:

1. Boot from x86 Linux Live CD, ie. Ubuntu
2. Open a console with Ctrl+Alt+T
3. sudo apt install lsscsi
4. lsscsi -g should show something like that: enclosu HP HP SAS EXP Card 2.06 – /dev/sg1
5. Download and install (via doubleclick) sg3utils and libsgutils: http://sg.danny.cz/sg/p/p/sg3-utils_1.39-0.1_i386.deb and http://sg.danny.cz/sg/p/p/libsgutils2-2_1.39-0.1_i386.deb
6. Download firmware 2.10: http://h20564.www2.hpe.com/hpsc/swd...=MTX_264388b1a7724246bee1748a95&swEnvOid=4168
7. Copy the firmware .scexe to /home, execute this: mkdir tmp && chmod +x CP022989.scexe && ./CP022989.scexe –unpack=tmp && mv tmp/PUF21000.bin . && rm -r tmp/
This should put a file named PUF21000.bin in /home.
9. sudo sg_write_buffer –mode=dmc_offs_defer –bpw=4096 –in=PUF21000.bin /dev/sg1
10. sudo sg_write_buffer –mode=activate_mc /dev/sg1
11. Reboot via Power off/on

Hope that helps. ;-)
 
Blue Fox said:
Your HBA should be able to tell you about the topology and show you what links are functional. My Areca does that and I presume LSI supports it as well.
Click to expand...


Hmmmm I can't see it in MSM..or maybe I guess I don't know what is the indicator ? ..Anyone knows how and where to look for this using MSM ?

Cheers !
 
MSM doesn't see the expander as a separate device.
HPSSA does see it, but it doesn't tell you if dual link is active, either. Source: my expander is connected to a HP P812 with two cables. The only difference I "see" when I pull one of the links is that the controller initializes significantly faster with just one link.
 
MSM only shows it as an "Enclosure", but I just can't find an indicator that could tell me that I'm running in dual link. Unless, there's another way to find out outside of MSM.
 
Well, there is one way: connect enough drives to the expander to saturate more than one uplink (2,4 GB/s), RAID0 them and benchmark. ;-)
In other words: if your drive setup can't provide more than 2,4 GB/s, it doesn't really matter if both links are used - that's assuming that you aren't going for path redundancy with the dual link.
 
raylangivens said:
Just signed up to add the info that there is no need for a HP controller to update the firmware of this expander. Virtually any compatible LSI based HBA it IT mode and a Linux live cd will do:

<snip>

Hope that helps. ;-)
Click to expand...

And I just signed up specifically to thank you raylangivens !
Thanks to your instructions I was able to install FW2.10 on the two expanders I just purchased, from 2.08.

I used a Dell PERC H200 in it mode, and sg3 utils 1.42, command syntax was a bit different:
- sudo sg_write_buffer -m 14 -b 4096 -I PUF21000.bin /dev/sg0
- sudo sg_write_buffer -m 15 /dev/sg0

Did I say thank you ?
;-)
 
Thanks for this forum!

Just upgraded my HP DL580 G5 to LSI 9260-8i and HP SAS 3Gbs Expander which came with 2.06 firmware.
After following all the instructions for upgrading the LSI 9260-8i to 12.15.0-0248 and HP SAS Expander to 2.10,
I was able to dual link the 9260-8i to the HP SAS Expander which shows correctly in MSM 16.11.00.03.
By "shows correctly" I mean that it now displays port 0-3 & port 4-7 both being used by SAS Expander can someone
please update the summary at the beginning to indicate that 9260-8i and HP SAS Expander will also dual link.

Also the server has 16 SAS drives some are SAS Dual Port which linked at 6Gbps to the HP SAS Expander.
The single port drives linked at 3Gbps which is the max each drive allowed.
This server has just jumped from ~400MB/s per HP400 Raid card (2) using RAID0 to
1000MB/s writes / 1200MB/s reads on a single RAID5 array.
NOTE: each drive only benchmarked at around 78MB/s to start with so these numbers are exactly what
I expected.
If anyone has a secret to making the LEDs work again on the drive bays please send word. I do miss that feature.

Thanks again for helping add new life to this old server!
 
getting close to running out of space on my multimedia rig again

so thinking about reusing the drives from the last upgrade and throwing them into a 24 bay enclosure as an JBOD DAS with snapraid

whats the cheapest way to setup a HBA SAS these days?

I still have a spare pci-e slot to break out an external SAS cable or four , but not wanting to setup another PC just to run an HBA card inside the new enclosure
disappointed in how high all the prices have rose over the last few years
but would rather use up my old drives sitting here in storage than buy larger ones

as long as it can stream a Blu-ray rip it will have all the speed I need
 
https://www.servethehome.com/sas-expanders-diy-cheap-low-cost-jbod-enclosures-raid/

You can do what they did at STH ?

Actually, don't bother with the extra PSU and motherboard either, especially if your current PSU can power another 24 drives. Just get 2 of those external molex plug on the PCI bracket thingy, and you can power the other 24 drives from the server's PSU for the DAS enclosure, and also power the expander at the same time.

...I hope I'm making sense on my explanation !
 
Hi,

I've got an LSI 9260 Controller connected to the HP SAS Expander, and i keep getting an odd error in the log. it doesnt seem to be causing an issue. everything works. so i'm just curious as to what it is and how to stop it as its happening evey few seconds

the error message is :

ID = 167
SEQUENCE NUMBER = 2793683
TIME = 15-04-2017 14:10:30
LOCALIZED MESSAGE = Controller ID: 0 Communication restored on enclosure: 1

ID = 167
SEQUENCE NUMBER = 2793683
TIME = 15-04-2017 14:10:30
LOCALIZED MESSAGE = Controller ID: 0 Communication restored on enclosure: 1

ID = 187
SEQUENCE NUMBER = 2793682
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Hardware error on enclosure: 1

ID = 187
SEQUENCE NUMBER = 2793682
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Hardware error on enclosure: 1

ID = 113
SEQUENCE NUMBER = 2793681
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Unexpected sense: PD = 20No addtional sense information, CDB = 0x1d 0x10 0x00 0x01 0x08 0x00 , Sense = 0x70 0x00 0x04 0x00 0x00 0x00 0x00 0x0a 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

ID = 113
SEQUENCE NUMBER = 2793681
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Unexpected sense: PD = 20No addtional sense information, CDB = 0x1d 0x10 0x00 0x01 0x08 0x00 , Sense = 0x70 0x00 0x04 0x00 0x00 0x00 0x00 0x0a 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
 
Hi everyone

I recently bought this lovely HP expander as a way of freeing up at least one PCI-E slot in my server, so I can put in a PCI-E SSD. Thanks to everyone who has contributed to all the great knowledge here!

My expander works well, but I have a performance issue I hope someone can advise on.

I am currently only able to use the HP expander with 12 drives if I want to get full performance, and that requires 2 connections from my HBA. So I'm only able to achieve 2 ports -> 3 ports, a net gain of one, which is rather disappointing.

My total array is 27 x 2TB 7.2k SATA3 HDDs: 11 x Seagate ST2000DM001 (2014 - 2015) and 16 x Toshiba DT01ACA200 - (2012 - 2016). Configured in ZFS as 3 x 9-drive RAIDZ2 VDEVs, running in Solaris 11.3.

The controller I am connecting to my HP expander is an LSI 9207-8e (2308 chipset) running the latest v20 IT firmware. I have 2 x 8088 cables going from this LSI to the HP, one to the HP's external port #1 and one to #8 via an 8088-8087 converter. (I've also tried both cables going to the HP's internal ports.)

In an iozone write test, the maximum speed I expect to get from my drives in a sequential write test is around 150-160MB/s per drive, giving a total array speed of around 3GB/s. I achieve that performance when I have all drives connected direct to LSI cards, which requires four LSIs connected to a total of 7 x 8087 cables (each going to a 4-port Chenbro SAS/SATA backplane.) I can't use 4-port LSI cards as I have a 2U low profile case.

I can achieve that same performance with the HP expander only if it's connected to 11 or 12 drives via 3 x 8087 cables. If I connect another 8087 to put a total of 15 drives on the HP (one JBOD drive slot is empty), overall performance drops to approx. 115MB/s per drive.

I can't quite follow the logic of these numbers, given that my 2 x connections to the LSI HBA should provide at least 2400MB/s bandwidth. That's certainly what the HP specifications say. In fact given lsiutil reports I have 8 x 6G lanes between expander and LSI, maybe I should get 4800MB/s. Although I guess that given SATA is only 3G, this also limits the connection to the controller to 3G even though it's negotiated at 6G? I really can't be getting 2400MB/s per cable, as performance with just one cable from controller to HP is awful.

But even 2400MB/s should be enough - divided into 15 drives that's 160MB/s per drive, which is exactly what I need to reach maximum performance. Unless there's a lot of overhead? But 115MB/s * 15 drives is only 1725MB/s total, rather a huge drop from the expected 2400.

I am running the latest HP 2.10c firmware.

Can anyone confirm if the performance numbers I am seeing are expected, or whether there might be some issue somewhere? I would love to be able to run 16 drives from the HP (well, actually only 15 right now) as it would free up another PCI-E slot and enable me to sell another LSI HBA (and help pay for the dual-port 10GBe card and 1.2TB PCI-E SSD I want to free the slots up for.. :) )

Thanks in advance
 
Last edited:
I also wanted to add that I tonight upgraded my HP firmware from 2.02 to 2.10 using the Linux sg_write_buffer method described in this ServerFault post and repeated earlier in this thread.

It's really awesome that this works and that an HP-branded HBA is not required. And my upgrade resulted in the version number being correctly updated to 2.10; I didn't have the problem mentioned earlier in which the FW upgrade worked but the version number remained unchanged.

This is how lsiutil reports my newly-upgraded expander:
Code: 
     B___T  Type       Vendor   Product          Rev      SASAddress     PhyNum
 1.  0  25  EnclServ   HP       HP SAS EXP Card  2.10  5001438007a0cea5    36

I did the upgrade from an Ubuntu Live CD written to a USB stick and the whole process - including downloading and writing Ubuntu - took no more than about 20 minutes. Highly recommended for anyone wanting to do this upgrade.
 
Last edited:
Malakym said:
I went back through this thread and found other confirmed use of this type.
Probably worth adding to the sticky under the power section.
Click to expand...

That's exactly what I'm using. I use the Molex port - I very briefly tried the USB 3 port and it didn't seem to power it. Might have done it wrong, though - I just plugged it into a standard USB power adapter, like I use for powering my phone.

 
Thank you odditory. (Just finished going through the whole thread)

If you have some time, please update original post re flash updating the card. It would save some angst... :happy:
With a LSI based HBA in IT mode (Dell PERC H200), I successfully updated two expander cards following raylangivens advice at post #2222:
https://hardforum.com/threads/hp-sas-expander-owners-thread.1484614/page-56#post-1042688412

And there was another method shown earlier.

Cheers.
 
Last edited:
I'm looking for advice in bulding a 30 disk storage. The case will be a Lian Li PC-343 with 6x Icy Box IB-555SSK bays. I am quite experienced with building "standard" desktop PCs, but when it comes to SAS and RAID adapters it's a pretty new territory.

RAID is actually not needed, I need the OS to see them as individual drives.

I am thinking of a IBM Serveraid M5015 controller (supports up to 32 disks) https://www.proshop.dk/Controller/IBM-ServeRAID-M5015-SASSATA/2195243 with the HP SAS Expander http://www.ebay.co.uk/itm/HP-24-Bay-PCI-E-SAS-Expander-Card-468406-B21-487738-001-/161906886211

Would this configuration work together, both the raid card and expander on the same mobo, connected with an SFF-8087 cable? (or two for for dual linking?).

I'd be running Windows 10.

Thanks!
 
I bought new HP 12GB expander. It was impossible to find out which chipset was inside, so once I got it I opened it and realized it's same manufacturer as on the older expander, just newer chipset from MicroSemi PM8044.
 

Attachments

  • IMAG1367.jpg
    IMAG1367.jpg
    138.6 KB · Views: 101
Hi I have same problem with HPE 769635-B21 12Gb SAS /fw:2.10/ and IBM M5210 /fw: 24.16.0-0104/ (both latest firmwares).

ID=187 Critical 2, Hardware error on enclosure:1
ID=113 Information 0, Unexpected sense: PD=21 No additional information CDB=0x1d 0x10 0x00 0x01 0x30 0x00 , Sense = 0x70 0x00 0x04 0x00 0x00 0x00
ID=167 Information 0, Communication restored on enclosure:1

It keeps repeating in the loop forever. Has anyone found the reason why these two cards don't like each other.

Anyone knows why this happens ???
TY !

PS : sense_key_name HARDWARE_ERROR is misleading.

Disk is just fine, it works in the other computer. So it's compatibility problem...

Dicara said:
Hi,

ID = 113
SEQUENCE NUMBER = 2793681
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Unexpected sense: PD = 20No addtional sense information, CDB = 0x1d 0x10 0x00 0x01 0x08 0x00 , Sense = 0x70 0x00 0x04 0x00 0x00 0x00 0x00 0x0a 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

ID = 113
SEQUENCE NUMBER = 2793681
TIME = 15-04-2017 14:10:21
LOCALIZED MESSAGE = Controller ID: 0 Unexpected sense: PD = 20No addtional sense information, CDB = 0x1d 0x10 0x00 0x01 0x08 0x00 , Sense = 0x70 0x00 0x04 0x00 0x00 0x00 0x00 0x0a 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
Click to expand...
 
Last edited:
You must log in or register to reply here.
Back
Top