HP SAS Expander Owner's Thread

As an Amazon Associate, HardForum may earn from qualifying purchases.
Unfortunately, I can't find any other online sources for the HP expander where I can read the language and will ship to Norway. What you can try is http://h20465.www2.hp.com/gpl/search.aspx?country_code=NO&language_code=en and choose Servers and then Proliant Accessories and put in your postal code. It may cost you a more but you could purchase from someone authorized locally who would also be able to flash it for you to the latest version and have someone you can go back to if there is a problem.

Edit - Damn, I found an online authorized source in Oslo http://www.dropzone.no/scripts/wgate/zz20b2b/!?OSTORE=B2B-DIR&~okcode=START&zshop=XP&zmenu=HP and put in 468406-B21 - 2929 kr I think that is about $400something in US$ (Amazon sells it for $350 in the US usually) which isn't bad for a retail product from an authorized dealer in a foreign country (which often cost more)
 
Last edited:
What a fantastic language! I only understand "HP Elitebook" and the information given for the size of their screens! :)
And some words seem to be very odd:
Halvliterstort kraftverk.
Click to expand...
Kraftverk probably means power station, but what's the other word?
Forretningsmesterstykke.
Click to expand...
And this isn't really an actual word, is it? :D

Anyway :D
It is probably a good idea to buy from a local, authorized HP store because they can indeed provide proper support.
It is really expensive though. According to the latest exchange rates 2929 NOK is about 389 EUR or 484 USD which is a big pile of money for that thing.
 
I broke out google translate. It seems that Forretningsmesterstykke. makes more sense as Forretnings Mesterstykke which is "Business Masterpiece". The first one translated as "Pint of Large Power Stations" ????
As to the cost, yeah it is more expensive but several times in various places around the world I have had to pick up parts which cost me 5x what I could get it elsewhere because a> They had it now b>I needed it now or c> No one else in the area even stocked it at any price.
 
Oh my, you guys really did some research :D "Halvliterstort kraftverk" does not make any sense! Maybe their site was google translated :p
As for the language translations, google translate does some wierd stuff :D
Btw, Norwegian is supposed to be the language that is easiest for a native english speaker to learn :)

I am really trying to get the card at a price below $350. The closest I get is $415.
Oh, well - guess I'll just have to pay up.
 
eg1l said:
Oh my, you guys really did some research :D "Halvliterstort kraftverk" does not make any sense! Maybe their site was google translated :p
As for the language translations, google translate does some wierd stuff :D
Btw, Norwegian is supposed to be the language that is easiest for a native english speaker to learn :)

I am really trying to get the card at a price below $350. The closest I get is $415.
Oh, well - guess I'll just have to pay up.
Click to expand...

Actually, you are right. words like ProductInformasjon, OperativSystem, Prosessor, Grafikk, Batteri, Dimensjoner, Kontact, antireflects etc made things relatively easy to pick up the general gist, even some of the paragraphs I could follow to a point by picking out similar words and interpolating what I could. Easier than Russian! :)
 
What I did to get my two HP SAS Expanders to Norway, was to buy them from a reputable seller on ebay (SynergyDustin here on Hardforum, budgetservers on eBay) and had them shipped to Norway with Jetcarrier.com.

From a consumer point of view it may be safer to buy a product in Norway, since we got laws that are very consumer friendly, if something happends to the product you've purchased. (Manufacturer may have only 1 year warranty, but consumerlaws gives you 3 or 5 years right to have a problem fixed, product replaced or money back). The downside is usually a higher saleprize and specialityitems may not be so easy to get hold of.

SynergyDustin did flash all the cards he sold to the latest firmware, and from the high volume of cards he sold, and great feedback both on email and on this forum, I didnt see any risk in getting a non-functional card. Both my cards have now worked flawlessly for almost 2 and 2.5 years respectfully.

I dont wanna buy a HP SAS Expander from a company in Norway, if they dont have first hand knowledge of which type of revision or firmware thats on the board they try to sell.


Offtopic: Infographic on which languages is the most easy to hard to learn, for a native english speaking person:
http://voxy.com/blog/index.php/wide-infographics/what-are-the-hardest-languages-to-learn-infographic/?post=3322
 
I've sent probably a dozen of these things to people overseas through this forum. Send me a PM and I can help you out. Should still be cheaper than getting one locally even after shipping.
 
I heard that some people here sell these cards? In the other topic I made, I finally got a good understanding of the setup of the cards and I'm about to purchase one of these in the coming weeks. I'd like to bring my business to the forum :)
 
Crest said:
I heard that some people here sell these cards? In the other topic I made, I finally got a good understanding of the setup of the cards and I'm about to purchase one of these in the coming weeks. I'd like to bring my business to the forum :)
Click to expand...

Some do, you can peruse the expander thread if you want to which lists some people, and you can post a WTB (Want To Buy) thread in the forsale forum here on hardforums http://hardforum.com/forumdisplay.php?f=17
 
JensS said:
What a fantastic language! I only understand "HP Elitebook" and the information given for the size of their screens! :)
And some words seem to be very odd:

"Halvliterstort kraftverk. "

Kraftverk probably means power station, but what's the other word?
Click to expand...
Break up the words, and then google:
Halv - liter - stort - kraft - verk
Half - litre - big - power - station

A power station big as a half litre.
 
Is anyone willing to upgrade the firmware on my expander for me? I'm stuck at 1.01, and can't find a cheap way to get access to an HP SAS controller long enough to upgrade it. I'll pay shipping both ways (or, if you're within an hour's drive of Baltimore, drive it out in person) and give you a couple bucks for the trouble.

PM me if you can flash expander firmware. You'd be my hero.
 
unhappy_mage said:
Is anyone willing to upgrade the firmware on my expander for me? I'm stuck at 1.01, and can't find a cheap way to get access to an HP SAS controller long enough to upgrade it. I'll pay shipping both ways (or, if you're within an hour's drive of Baltimore, drive it out in person) and give you a couple bucks for the trouble.

PM me if you can flash expander firmware. You'd be my hero.
Click to expand...

What color is the PCB on your card? Green (Hopefully its what you have, it can be upgraded), Blue (50/50 shot on upgrades) Yellow (Can't be upgraded to latest firmware). One other thing to keep in mind (though it really doesn't affect anyone not using all ssd's for the most part) is that as of 1.52 there was an update:

Enhancements/New Features:
Support for 6Gb SAS / 3Gb SATA HDD.

Added support for HP ProLiant DL 580 G5 and HP ProLiant DL 785 G5 Servers. Support is limited to 3Gb SAS / 1.5Gb SATA HDD.

This means that sometimes the SAS connections will maintain as much as a 6Gb/s connection and that SATA HDD's will sometimes maintain just a 1.5GB/s connection (per link). There seems to be no rhyme or reason to this, and even though the card just obstensively uses the PCIe bus for power, it seems to have a bearing on the PCIe architecture of the board (whether the lanes come off the CPU, PCH or 3rd party PHY chip). I have had seemingly identical systems next to each other, one links the SATA drives at SATA300 and one links at SATA150. It also has no bearing on the speeds with spinning disks, only the latest drives are exceeding this performance and that is just for STR, average random usage never comes close to even SATA150 throughput (per drive as part of an array).
 
Last edited:
mwroobel said:
What color is the PCB on your card? Green (Hopefully its what you have, it can be upgraded), Blue (50/50 shot on upgrades) Yellow (Can't be upgraded to latest firmware).
Click to expand...
My PCB is green.
mwroobel said:
One other thing to keep in mind (though it really doesn't affect anyone not using all ssd's for the most part) is that as of 1.52 there was an update:

Enhancements/New Features:
Support for 6Gb SAS / 3Gb SATA HDD.

Added support for HP ProLiant DL 580 G5 and HP ProLiant DL 785 G5 Servers. Support is limited to 3Gb SAS / 1.5Gb SATA HDD.

This means that sometimes the SAS connections will maintain as much as a 6Gb/s connection and that SATA HDD's will sometimes maintain just a 1.5GB/s connection (per link). There seems to be no rhyme or reason to this, and even though the card just obstensively uses the PCIe bus for power, it seems to have a bearing on the PCIe architecture of the board (whether the lanes come off the CPU, PCH or 3rd party PHY chip). I have had seemingly identical systems next to each other, one links the SATA drives at SATA300 and one links at SATA150. It also has no bearing on the speeds with spinning disks, only the latest drives are exceeding this performance and that is just for STR, average random usage never comes close to even SATA150 throughput (per drive as part of an array).
Click to expand...
Yeah, I don't mind so much if the sata links go slowly. My main issue is that the overall throughput is apparently limited to about 300 MB/s with 18 disks. That's pretty lame.

If I pick 6 disks and dd off them, then I get about 45 MB/s each (from disks that should do 100 MB/s each); if I bump it up to 12, it drops to about 23 MB/s each; if I go for all 18, it's more like 17 MB/s each. lsiutil confirms that there are 4 links at 3.0gbps to the expander, but somehow it's not using them to the fullest.

Interestingly, it does manage more than 300 MB/s even, which I might expect if it were only using one lane or something. With 18 disks, it gets about 337 MB/s; with 12, 307 MB/s, and with 6 disks 263 MB/s. Beats me why, but I want to take "old broken firmware" out of the picture.
 
Last edited:
unhappy_mage said:
My PCB is green.

Yeah, I don't mind so much if the sata links go slowly. My main issue is that the overall throughput is apparently limited to about 300 MB/s with 18 disks. That's pretty lame.

If I pick 6 disks and dd off them, then I get about 45 MB/s each (from disks that should do 100 MB/s each); if I bump it up to 12, it drops to about 23 MB/s each; if I go for all 18, it's more like 17 MB/s each. lsiutil confirms that there are 4 links at 3.0gbps to the expander, but somehow it's not using them to the fullest.

Interestingly, it does manage more than 300 MB/s even, which I might expect if it were only using one lane or something. With 18 disks, it gets about 337 MB/s; with 12, 307 MB/s, and with 6 disks 263 MB/s. Beats me why, but I want to take "old broken firmware" out of the picture.
Click to expand...


What SAS HBA are you using and what *nix OS are you running? How do you have the array configured? mdadm, zfs etc? What CPU and how much RAM? Another thing to try... Do a test with say 4 drives right off your HBA, and then test the same 4 drives hooked to the expander. Is there any material difference?
 
Last edited:
mwroobel said:
What SAS HBA are you using and what *nix OS are you running? How do you have the array configured? mdadm, zfs etc? What CPU and how much RAM? Another thing to try... Do a test with say 4 drives right off your HBA, and then test the same 4 drives hooked to the expander. Is there any material difference?
Click to expand...

It's an LSI2008 controller, specifically the AOC-USAS2-L8i. I have three six-drive vdevs, each currently in its own pool. Two are raidz2, one is raidz3. Note, though, that my tests above involve only reading directly from the drives, not through zfs.

I have one Opteron 4170 and 16GB of memory allocated to the virtual machine that's running the OpenIndiana install, out of 32GB physical. I haven't tried with the same drives directly connected, but I do have two SSDs connected directly to the HBA. If I read from one of them while reading the same six drives, I see the SSD dip in performance there as well, so maybe my issue is more related to the HBA than the SAS expander. I'll also have to try bare-metal at some point. I wonder now if maybe it's linking at PCI Express 2.0 x1? That'd be a 500 MB/s limit, and I haven't seen anything exceed that yet. I dunno, further testing is required.

Thanks for suggesting things that made me re-check my assumptions :)
 
unhappy_mage said:
It's an LSI2008 controller, specifically the AOC-USAS2-L8i. I have three six-drive vdevs, each currently in its own pool. Two are raidz2, one is raidz3. Note, though, that my tests above involve only reading directly from the drives, not through zfs.

I have one Opteron 4170 and 16GB of memory allocated to the virtual machine that's running the OpenIndiana install, out of 32GB physical. I haven't tried with the same drives directly connected, but I do have two SSDs connected directly to the HBA. If I read from one of them while reading the same six drives, I see the SSD dip in performance there as well, so maybe my issue is more related to the HBA than the SAS expander. I'll also have to try bare-metal at some point. I wonder now if maybe it's linking at PCI Express 2.0 x1? That'd be a 500 MB/s limit, and I haven't seen anything exceed that yet. I dunno, further testing is required.

Thanks for suggesting things that made me re-check my assumptions :)
Click to expand...

Well, I always try and remove any particular software from the equation first in weird performance issues. If you could either setup a quick boot drive with the OS of your choice, or even boot a USB LiveCD and then try again, that would be a good start. Whatever FS you choose or mdadm, create a 4 drive RAID0, so you can pull top throughput from the drives. Connect the 4 drives first to the HBA by itself and run your performance tests. Then connect the HBA to the expander and the drives to the expander and run the tests again. First, did the initial bare-metal tests significantly increase over your VM tests, and then did the expander tests pretty much equial your bare metal HBA tests? If you see improvement, then the failure is somewhere in either the VM itself or the hypervisor. If these results are just as poor then it is likely a hardware problem. What motherboard is the HBA in, and what slot #? Is it a x16 physical slot (and if so, what is it electrically?). What other PCIe cards are in the system, and do you have them passed through to the VM? The AOC you have does PCIe1 and PCIe2 autoswitch, but let me know the motherboard specs and slots anyway.
 
mwroobel said:
Well, I always try and remove any particular software from the equation first in weird performance issues. If you could either setup a quick boot drive with the OS of your choice, or even boot a USB LiveCD and then try again, that would be a good start. Whatever FS you choose or mdadm, create a 4 drive RAID0, so you can pull top throughput from the drives. Connect the 4 drives first to the HBA by itself and run your performance tests. Then connect the HBA to the expander and the drives to the expander and run the tests again. First, did the initial bare-metal tests significantly increase over your VM tests, and then did the expander tests pretty much equial your bare metal HBA tests? If you see improvement, then the failure is somewhere in either the VM itself or the hypervisor. If these results are just as poor then it is likely a hardware problem. What motherboard is the HBA in, and what slot #? Is it a x16 physical slot (and if so, what is it electrically?). What other PCIe cards are in the system, and do you have them passed through to the VM? The AOC you have does PCIe1 and PCIe2 autoswitch, but let me know the motherboard specs and slots anyway.
Click to expand...
I have an H8DCL-iF. Everything's either x4 or x8 electrically. I think the HBA is in the x16/x8 slot, but I'll re-check that tonight. I have a bunch of add-on cards that I'm not using right now installed in it---a Ceton InfiniTV, an Intel wireless card, and a USB 3.0 card---so I think I'll take those out while I have the chance to remove some complexity.
 
First I'm wondering how to check the expanders version. And secondly is it anyone in Norway that can help me upgrading my card?
 
Rudde93 said:
First I'm wondering how to check the expanders version. And secondly is it anyone in Norway that can help me upgrading my card?
Click to expand...

What HBA do you have attached to the expander(s)? What OS? The cards management software in Windows can usually display this information. In *nix, there are different choices. hpacucli or areca cli for manufacturer specific uses or smartctl in general.

Unfortunately, I am on the other side of the pond so I can't help. You will need to find someone with an compatible HP adapter with the required amount of RAM and the right driver level to flash your card.
 
The card is connected to IBM M1015 flashed to LSI9211 IT mode

I use ESXI passthrough, to solaris 11 express, can be redirected to windows if its easier to find out there.
 
Rudde93 said:
The card is connected to IBM M1015 flashed to LSI9211 IT mode

I use ESXI passthrough, to solaris 11 express, can be redirected to windows if its easier to find out there.
Click to expand...

You can use the OS of your choice. Get a copy of MegaCLI form LSI for the OS of your choice, and try these commands:

MegaCli -AdpAllinfo -aALL
MegaCli -PDList -aALL
MegaCli -EncInfo -aALL
MegaCli -CfgDsply -aALL
 
Edit: Ugh the P411 seems to have gone bad, will try to resume loaning it out to people if I get a replacement back from HP. Sorry guys.
 
Last edited:
Yep you're right -- edited. Been sitting in a box for a year so I forgot HP's naming conventions. Terrible, terrible RAID card with the one redeeming quality of being able to flash an HP expander.
 
Last edited:
Very true, the P41x controllers aren't very good. Like most HP controllers acutally. One aspect is that they are very old now.
I didn't manage to get my hands on a P420 (or P421 or P822) so far though. That's the brand new one and maybe it's better. I can't tell much about it at the moment apart from that it seems to be a very recent design supporting PCIe 3.0 and using DDR3 memory as its cache.
When talking about this controller HP doesn't waste a word regarding the expander though. That applies to the new DL380p Gen8 servers as well. That comes with expander backplanes for the drive cages if you choose a model with more that eight HDD bays.

I guess the expander is or will be EOL very soon.
 
I have a problem that I'm hoping someone can help me with.

So my setup is an LSI 9620-4i and I'm connected to my HP SAS Expander using a SFF-8088 to SFF-8087 cable. I basically have the cable going from the RAID card outside of the case through a rubber grommet and in to the back of the HP SAS Expander.

For an operating system I'm using Windows Server 2008 R2.

Now my problem is, when my server first starts up and I log in to Windows the HP SAS Expander is not being detected correctly by the LSI RAID Card. In the LSI log I have lots of errors like this:

"Controller ID: Communication lost on enclosure: 1" and "Link restored on SAS Wide Port: 0 PHY = 0/1/2/3"

So basically when I first boot up I cannot access my RAID arrays. But here is the thing, after 20-30 minutes the HP SAS Card is then correctly detected by the LSI RAID card, the errors stop completely, now obviously by this time the LSI card thinks my arrays are gone. But luckily I can quickly recover them by selecting "load forign configuration" where by it reads the RAID Array information from the disks.

But this does have me worrying about the longevity of this experiment I'm conducting with the HP SAS Card if every time I boot up there is a chance it's not gonna work properly. My HP SAS Card has firmware 2.08 and I'm running almost the latest LSI Firmware (I'll be upgrading it in a little while).

Any thoughts or advice about this gents?
 
Had to discontinue P411 loaning out temporarily since it's developed a problem.
 
Last edited:
Are Hitachis still a good choice for hardware RAID now that they've been taken over by WD? I'm just wondering if the TLER nonsense has permiated down to the current set of Hitachi drives? I'm half looking at some of the new 4TB ones.


G
 
Yes Hitachi's are still a good choice - assuming you can find them - still the perfect choice in my book for hardware RAID relative to alternatives. WDC's buyout of Hitachi wasn't with the intention of taking over and continuing to manufacture Hitachi drives - it was to take the Hitachi desktop segment products off the market altogether with the assumption people will just buy WDC's preexisting products instead ("You don't want steak, why do you want steak, here try our spam in a can,.. mmm mmm good, see? Try some..").

However WDC did want hitachi's enterprise stuff - a segment where Hitachi was far stronger and more successful than WDC, so I'm sure that will influence WDC's future enterprise offerings. On the bright side WDC was court ordered to give Toshiba some of the Hitachi IP, design patents and tech as part of the terms of the buyout and so some of us are waiting to see what Toshiba delivers before EOY in the hopes it's a closer spiritual successor to the former Hitachi lineup than what Seagate and WDC have on offer (which ain't much -- I can point of a list of flaws for every single one of each of their sub-enterprise offerings including the new Reds).
 
Last edited:
The sas2ircu utility can be used to check the firmware revision of your SAS expander when used with LSI cards. Here's the process in verbose detail:


Download the sas2ircu utility. Unpack and change file mode on the file that matches your distribution and architecture. Here I'm using Solaris on x86.

Code: 
$ wget 'http://www.lsi.com/downloads/Public/Host%20Bus%20Adapters/Host%20Bus%20Adapters%20Common%20Files/SAS_SATA_6G_P14/SAS2IRCU_P14.zip'
$ unzip SAS2IRCU_P14.zip
$ cd SAS2IRCU_P14/sas2ircu_solaris_x86_rel/
$ chmod +x sas2ircu

List all installed and recognized adapters.

Code: 
# ./sas2ircu list

Run sas2ircu with the adapter index as the first argument. Here I'm displaying information about the first adapter; adapter at index 0.

Code: 
# ./sas2ircu 0 display

The firmware revision of my SAS Expander is 1.52.

Code: 
------------------------------------------------------------------------
Physical device information
------------------------------------------------------------------------
Initiator at ID #0

Device is a Enclosure services device
  Enclosure #                             : 2
  Slot #                                  : 0
  State                                   : Standby (SBY)
  Manufacturer                            : HP      
  Model Number                            : HP SAS EXP Card 
  Firmware Revision                       : 1.52
  Serial No                               : PMCSIERA
  Protocol                                : SAS
  Drive Type                              : SAS_HDD

Anybody have a loaner HP SmartArray controller in the US?
 
odditory said:
Q: How many harddisks can you attach to the HP SAS expander?
A: Up to 32. I've stress tested various configurations including a RAID6 array with 32 x 1Tb drives and had no issues.
Click to expand...

So one problem on this topic.

I am currently using a LSI 9260-8i and the expander card (Firmware 2.08) in my filer. When connecting 24 disks to the expander everything is fine, as soon as I plug in more disks the controller reports 'Invalid SAS-topology detected' when initializing. Even if it works fine, if I add the drives when the server is running.
So what causes this problem? The Controller, the expander, my stupidity?

Edit: This also happens when I connect 24 disks to the expander and the other 4 to the controller.
 
Last edited:
JensS said:
So one problem on this topic.

I am currently using a LSI 9260-8i and the expander card (Firmware 2.08) in my filer. When connecting 24 disks to the expander everything is fine, as soon as I plug in more disks the controller reports 'Invalid SAS-topology detected' when initializing. Even if it works fine, if I add the drives when the server is running.
So what causes this problem? The Controller, the expander, my stupidity?

Edit: This also happens when I connect 24 disks to the expander and the other 4 to the controller.
Click to expand...

99% of the time this error will appear with LSI controllers when you have a cable problem. Either you have incorrectly connected them or if you have a bad cable. How do you have your rig cabled? The easiest way would be dual linking the card to the expander, and then run all the drives off the expander. Also, do you have sufficient cooling for the card itself in your box?
 
With all drives connected to the expander I had port 9 as uplink and 2-8 connected to the backplanes.
Then I took the backplane on port 4 over to port 2 of the controller.

Meanwhile I worked on this with a friend. Closing the 'gap' at port 4 (port 2-7 for disks, 8 as uplink, 9 unconnected) worked. Now the box boots up fine.

The cards are cooled by a 120mm fan fixed above. This should be ok as the heatsink is a bit warm but not hot.
 
It is good you have a cooling fan trained on your cards. This expander was originally designed to be placed in an HP Proliant which has extensive cooling, and if the expander gets too warm can start acting flakey (dropping drives, etc). A lot of people just stuff it in a regular case and don't think about it.
 
I'm using the HP SAS Expander with an LSI MegaRAID SAS 8344ELP RAID Controller. You can add that to the list.
 
You must log in or register to reply here.
Back
Top