HP SAS Expander Owner's Thread

reimann3 said:
What I haven't figure out is whether I need a second case.


Can I just the SAS expander or do I need another card to use it?
Click to expand...
You don't need a second case for the expander, but obviously you might if your primary case doesn't have space for all your drives.

A SAS expander as analogous to a SATA port multiplier. It takes a single SAS port and expands it to more ports. So it needs a compatible SAS port to plug it into, just like you can't use a SATA port multiplier without first having a SATA port. Unless you happen to have a motherboard with compatible SAS ports, then yes, you will need a SAS HBA to provide a SAS port to be expanded into more ports. See the first post of this thread for some examples of known compatible SAS RAID/HBA cards, as well as some SAS port equipped motherboards.
 
dscline said:
A SAS expander as analogous to a SATA port multiplier.
Click to expand...

In general? No!
It might be but it won't necessarily be like that.
If you have a port multiplier that does command based switching then you are wrong. That would be like saying an ethernet switch is the same as an ethernet hub.
If it does frame information structure based switching then you are right.

Some advantages of the SAS expander:
1. It is defined in the SAS spec as a standard whereas port multiplier aren't defined as a SATA standard. In theory that creates less incompatibilities than the port multipliers do.

2. Because SAS usually uses multilane ports you have more bandwidth available between the raid card or HBA and the expander which might be useful in bigger environments.

Port multipliers are cheaper though.
 
JensS said:
dscline said:
A SAS expander as analogous to a SATA port multiplier.
Click to expand...
In general? No!
Click to expand...


Yes. In general.


anal·o·gous :
1. Having similar function but a different structure and origin
2. Similar or alike in such a way as to permit the drawing of an analogy.​


A SAS expander is comparable to a SATA port multiplier in that it is used to provide more connections from the one (or few) that you have. Analogous does not mean they are the same. The comparison was used to help someone who isn't familiar with SAS components understand what a SAS Expander does. A SAS expander provides the same function in the SAS world as a port multiplier provides in the SATA world. That doesn't mean that they have to achieve that means the same way, or that one can't be a superior solution to the other. In fact, "analogous" implies that they ARE different, yet perform a similar function.
 
Last edited:
Granted, thats true.
I wouldn't have used this example though. For someone who isn't familiar with the topic a expander is probably as unknown as a port multiplier. Even though it is some kind of SATA component it isn't used often in home environments.
I think if someone doesn't know (or can't imagine) what a SAS expander does or how it works then he doesn't know what a port multiplier is either.
 
I'm looking at a norco rackmount case. I want to know if I need the HP410 or can I just install the sas expander with the appropriate cables? I will probably be using Windows 2k8 R2 as an OS
 
reimann3 said:
I'm looking at a norco rackmount case. I want to know if I need the HP410 or can I just install the sas expander with the appropriate cables? I will probably be using Windows 2k8 R2 as an OS
Click to expand...

Have you bothered to look at the OP? aka Post #1
 
odditory said:
Have you bothered to look at the OP? aka Post #1
Click to expand...

Yes and it wasn't clear to me. I don't want to go and spend money when I don't have to. my build will all be in a single case possibly FreeBSD/ZFS but more than likely windows server 2008R2
 
Do you have problems understanding the first post because of missing technical knowledge or is it a problem because of the language?

You don't need a P410 but you will need ANY compatible SAS card. Can be a P410, can be something else.
In general it might be a raid controller or a simple HBA. It even might be a integrated raid controller on your board.
What sort of card you should use depends on what you want to build.
A list of compatible cards is in the first post in this thread. This list isn't complete so the actually compatible cards aren't only the ones listed there. The 3ware 9690SA series is still listed as compatible although from what I have experienced so far it isn't.
 
I want to stick with HP as much as possible. My employer contracts with HP so I get a discount. This also isn't the first place I've heard the p410 mentioned. It seems to be a really piece of hardware and it's well supported among several different OS including havinf native support in the latest release of FreeBSD.
 
You usually don't want to use a P410 for several reasons:

1. It is quite old now. It is already replaced with the P420 in HPs Gen8 servers and hopefully this will be a better piece of hardware. I don't know much about the P420 yet though.

2. It is quite bad in terms of interoperability with any other mainboards than those used in HP ProLiant servers. And this isn't a surprise really because that is the only job they are made for.

3. Write back (using the cache for write access) is only possible when a battery backup unit is present. This involves a NiMh battery which has to be 100% charged to enable the write cache and this is only the case in 24/7 operation. If you don't run the machine 24/7 you will always have to wait for the battery to be charged before you get decent performance.
This also means you have to pay for something you don't really need (since no one needs a battery on a raid controller) and the battery has to be replaced eventually when it is worn out.
There is a flash backed option available which doesn't use a battery but this is even more expensive.

4. There is NO write caching for SATA drives under ANY circumstances. This means bad write performance when using SATA drives and you can't do anything about it apart from using SAS drives which is really, really expensive.

The only advantage of the P410 is that you can use it to flash the FW of the expander which you can't do with the controllers from other manufacturers!
 
JensS is 100% correct and those are only SOME of the downsides. UNLESS you are running an HP server with HP SAS drives then a P410/P411 is a chariot to hell. I know, I've had a P411 a long time for testing purposes and nothing has changed even in the latest FW. The choices are simple:

Want Hardware RAID, get an Areca 1880/1882 or LSI 9260/9265 card.
Want Software RAID, on Windows2008R2 then get an IBM M1015 card from ebay, then flash it to IT mode firmware (instructions in the forum). I'd also suggest FlexRAID software for the drive pooling and RAID duties.
Want Software RAID, on a linux/solaris variant then get an IBM M1015 card from ebay, then flash it to IT mode firmware (instructions in the forum). FreeBSD, or Nexenta+Nappit GUI or OpenIndiana+NappItGUI are popular
 
Last edited:
odditory said:
JensS is 100% correct and those are only SOME of the downsides. UNLESS you are running an HP server with HP SAS drives then a P410/P411 is a chariot to hell. I know, I've had a P411 a long time for testing purposes and nothing has changed even in the latest FW.
Click to expand...

Even worse, even if used in a Proliant, don't ever have the audacity to try and use it with non-HP drives (SAS or SATA). Lost a lot of hours way back when before I learned that one! Sometimes even with HP drives mysterious incompatibilities pop up which HP just replaces the controller always, never the drive.
 
odditory said:
......
Want Software RAID, on a linux/solaris variant then get an IBM M1015 card from ebay, then flash it to IT mode firmware (instructions in the forum). FreeBSD, or Nexenta+Nappit GUI or OpenIndiana+NappItGUI are popular
Click to expand...

you do Not need to flash to IT on linux, just flash to the newest firmware from LSI directly...
all drive will treat as jbod via passthrough, you can query via smartctl too.
the kernel module this LSI HBA is stable in linux

When we are speaking on "open" solaris... it is a different story .
 
JensS said:
You usually don't want to use a P410 for several reasons:

1. It is quite old now. It is already replaced with the P420 in HPs Gen8 servers and hopefully this will be a better piece of hardware. I don't know much about the P420 yet though.

2. It is quite bad in terms of interoperability with any other mainboards than those used in HP ProLiant servers. And this isn't a surprise really because that is the only job they are made for.

3. Write back (using the cache for write access) is only possible when a battery backup unit is present. This involves a NiMh battery which has to be 100% charged to enable the write cache and this is only the case in 24/7 operation. If you don't run the machine 24/7 you will always have to wait for the battery to be charged before you get decent performance.
This also means you have to pay for something you don't really need (since no one needs a battery on a raid controller) and the battery has to be replaced eventually when it is worn out.
There is a flash backed option available which doesn't use a battery but this is even more expensive.

4. There is NO write caching for SATA drives under ANY circumstances. This means bad write performance when using SATA drives and you can't do anything about it apart from using SAS drives which is really, really expensive.

The only advantage of the P410 is that you can use it to flash the FW of the expander which you can't do with the controllers from other manufacturers!
Click to expand...

on P212 (firmware version 5.14) with expander :

I slapped 6 500G Sata (seagate 7200rpm ) on P212 with gigabyte 990FXA-UD5.
the option rom can not load due on no I/O address assignment. I need to use HP configuration tools. has been ran since in the beginning of this year.
the machine is running centos 6.2 with update currently.

you can force to enable write cache ( without the battery or the battery not full), "Enable Write Cache When Battery/Capacitor Not Present or Not Completely Charged" <-- the option is enable or disable

you can enable/disable hd write cache ( in linux term is barrier 1 or 0) -> "Physical Drive Write Cache State" <-- the option is enable or disable

I guess P410 should be identical with P212 or could not..
 
Last edited:
I'm going to probably want some form of ZFS. I have a really good friend who's a unix sysadmin for a large ISP so I'll have the help I need to get started. I work for an HP contractor albeit on the consumer side but that still gets me a nice discount. That's why I'm looking at HP hardware initially.
 
@cantalup:
Not sure if this is new in a recent version of the ACU or if it is specific for the P212 controller. Haven't used that one so far.
It is true that you can enable the local cache on the drive but that is not what you want. Not sure if that is possible for SATA drives though. If I remeber correctly it isn't.

@reimann3:
Even if you get them for free it wouldn't matter. Basically you don't get what you want. In the end it is up to you to make the decision. But don't come back crying if something doesn't work.
If you are going for ZFS you don't need a proper raid card anyway. A M1015 would be the better choice.
 
JensS said:
@cantalup:
Not sure if this is new in a recent version of the ACU or if it is specific for the P212 controller. Haven't used that one so far.
It is true that you can enable the local cache on the drive but that is not what you want. Not sure if that is possible for SATA drives though. If I remeber correctly it isn't.

.....
Click to expand...

as my understanding and could be wrong :p:
it is not the local cache, "Physical Drive Write Cache State" is the cache on eachdrive when "Physical Drive Write Cache State" is enabled or disabled. once you disabled, the rate is degraded a bit..not much where depends on you SATA or SAS drive :p, SATA drives sometimes lie to boost transfer rate ( even you disable write cache by sending command, the Sata drive lies by saying" the write cache is turn off now", but it is not! in reality)

the detail is in HP CLI tools, I just use ACU when I am in lazy mode :D.

as my habit, I never trust HD physical cache enabled :) .
on mdadm software raid with ext3, I added extra param -> barrier=1, to flush drive cache right away.
But!! :( some HDs lie whether their cache already flush to the disk/plate physically.
I had some SATA WD drives that lied.. took weeks to figure out and got confirmation from one of liunx filesystem author:| via private email ( company email :))
you can read on http://lwn.net/Articles/479142/


Have fun!!
 
Local cache in this case meant the cache on each drive. It's the same thing.
The fact that drives with enabled local cache "lie" is no surprise. That is how caching works. That they lie about the state of their local cache though is new to me.
Raid controllers work in exactly the same way. It tells the OS that the given I/O operation is done although it isn't written through onto the drives. This is much faster than a immediate write through obviously and it's the reason why caching speeds up your storage.

I don't have a working setup with a P410 controller at home at the moment but maybe I will have a look at the latest HP drivers and software at work on monday.
This was the major subject we were talking about and if HP now allows the write cache on the controller to be activated without a fully charged BBU or flash cache then my point #3 above isn't valid anymore. I will also check what it does with SATA drives with recent drivers and software.
 
HD write cache
SAS drive never lies ( as I know until know)
Some SATA driver lies to make a good look an benchmarks:D, How an easy way to know? it is pretty simple, connect to direct SATA port, enable and disable write cache via command, and do write dd (tested on linux). writing rate would drop significantly on a big writing (for example 2G size of a file).

my rule of thumb while using Raid Hardware, I always disable HD write cache. let the card handle caching.

on linux, I used barrier=1 (ext3/ext4)to enable flushing that filesystem tell to write all data to the disk at the end of the current cycle, well.... this depends on the SATA drive to execute the command or not. doh!.
I believe linux XFS is set barrier=1 by default

as I learnt, UPS is the key . when I am not sure whether all data has been flush to disks, I just execute sync command ( not really useful but would help)
the other thing is, shutdown the machine gracefully ...for example halt or other commands
 
Last edited:
Doesn't the P410 make use of write cache on SATA drives at all? Even with a fully charged battery module?

I have 5 drives in my Microserver with 5x1TB Samsung F1 in RAID5.
On a virtual XP Pro machine I get:
write_raid5_5x1tb_esxi-11-07-03.png

Not flying numbers but it's not a flying machine either :p
 
As an eBay Associate, HardForum may earn from qualifying purchases.
mwroobel said:
I haven't seen that particular part number on the HP SAS cards, either retail or service parts. Ask him how old the card is and what color the board is (Yellow, Blue or Green). Also, ask him if he knows what firmware is on the card.
Click to expand...

Already did, he does not know what firmware is on the card or the production date. It looks like a green card, and the price is quite good :) However, I came across the Intel sas expander. How does the Intel card compare to the Hp one?
 
eg1l said:
Already did, he does not know what firmware is on the card or the production date. It looks like a green card, and the price is quite good :) However, I came across the Intel sas expander. How does the Intel card compare to the Hp one?
Click to expand...

The intel isn't a bad expander. It is a 24 port which will yield you 20 ports if single linked to the SAS controller. It also can be powered by molex connectors instead of taking up a slot if you don't have one available. That said, I personally have a number of them in systems and they have been rock solid. The HP SAS expander also offers 36 ports (32 internal). What SAS controller are you planning on connecting this to?
Btw I am basing this upon you meaning the Intel RES2SV240, correct?
 
mwroobel said:
The intel isn't a bad expander. It is a 24 port which will yield you 20 ports if single linked to the SAS controller. It also can be powered by molex connectors instead of taking up a slot if you don't have one available. That said, I personally have a number of them in systems and they have been rock solid. The HP SAS expander also offers 36 ports (32 internal). What SAS controller are you planning on connecting this to?
Btw I am basing this upon you meaning the Intel RES2SV240, correct?
Click to expand...

I was thinking RES2CV240, as the one you mentioned is being discontinued later this year. I have a Norco 4220, but if I want another case just for drives, would it be smarter to go with the HP one, cause I would loose one connector on the Intel one, and not be able to connect all the drives in the 4220 case? However, I am using a Intel SASUC8I controller which has two connectors, would I be able to connect one of them to the backplane, one to the expander, and still have one connector available for linking another case? Maybe not the best solution?
 
You don't want to cascade expanders, it has very flakey results (I have had issues with both the intel and HP when trying this). You are better off single linking one port of your HBA to each of the two expanders and then connecting the drives/backplanes that way. The 4220 having 20 slots you could use either expander internally, and still have one port on the HBA for a second expander in another case.
 
Right, so I could use the Intel expander and connect all the drives to that one in my current setup, and use the second connector on my HBA to connect to a PCI bracket, and then link externally to another case with a second expander in that one?
 
eg1l said:
Right, so I could use the Intel expander and connect all the drives to that one in my current setup, and use the second connector on my HBA to connect to a PCI bracket, and then link externally to another case with a second expander in that one?
Click to expand...

That would be fine.
 
mwroobel said:
You don't want to cascade expanders, it has very flakey results (I have had issues with both the intel and HP when trying this). You are better off single linking one port of your HBA to each of the two expanders and then connecting the drives/backplanes that way. The 4220 having 20 slots you could use either expander internally, and still have one port on the HBA for a second expander in another case.
Click to expand...

I want to point the same results. I've gotten flacky results from chaining 2 expanders as well. I've tried to chain pairs of HP expanders without luck. Also tried to chain 2 new chenbro ones without much success either. This currently leaves me in a bind as I have 3 norco chassis but only 2 ports on my Areca. I'm not sure how I'll go about attaching that 3rd (and possibly future 4th) case to the same machine. I may end up just making it an entirely separate machine for that reason alone.

Mixing and matching the chained expanders didn't get any better (actually worse) results.
 
Another thing to keep in mind. These expander boards are not made for desktop machines. They are designed to live in server-class machines (such as Proliant) which are most often specced with extremely fast, loud high CFM cooling. It is very important to keep them very well cooled, as they will get rather warm otherwise and will flake out if they get too hot.
 
HP uses several different part number for different reasons for the same component.
468405-002 seems not to be a valid part number for any HP component. Their PartSurfer database doesn't know this number. 487738-001 refers to the SAS expander though.
You can see the bigger heatsink on the picture which has been fitted to the later revisions so this should be a blue or green card and you should be fine with that one. Firmware might be old but should be upgradeable.

And a quick word on the temperature topic:
It is very true that actually most enterprise/server grade cards are designed to work in environments with a high airflow for proper cooling. And you can easily see why when you put up a system on a test bench or something so that you can touch the heatsinks of the cards after powering on the system. After a very short time even without heavy workload these heatsinks may be so hot that you can't touch them for longer than a second or so.
That's why I always have an extra fan blowing between the PCIe cards to stop the cards overheating which otherwise will happen at some point. Doesn't have to be a 120 mm high power server fan with 5000 rpm, a normal 120 mm desktop class case fan with around 1000 rpm will do nicely.
120 mm works fine for me because there are several cards in my systems. You might go for a smaller one if you have only one or two cards.
 
JensS said:
HP uses several different part number for different reasons for the same component.
468405-002 seems not to be a valid part number for any HP component. Their PartSurfer database doesn't know this number. 487738-001 refers to the SAS expander though.
You can see the bigger heatsink on the picture which has been fitted to the later revisions so this should be a blue or green card and you should be fine with that one. Firmware might be old but should be upgradeable.
Click to expand...

On this picture: http://img.inkfrog.com/click_enlarge1.php?image=IMG_0414.JPG&username=telnetliquid&aid=256441522 I can see the S/N: 487738-001, so I would be safe buying this?
 
eg1l said:
On this picture: http://img.inkfrog.com/click_enlarge1.php?image=IMG_0414.JPG&username=telnetliquid&aid=256441522 I can see the S/N: 487738-001, so I would be safe buying this?
Click to expand...

487738-001 and 486406-B21 are the standard and service part model numbers. The additional sticker is the one that says 468405-002, that could be some third party designation or it could also be some internal HP number for a demo system that someone is parting out (I have seen that before). In any case the sticker below has the right numbers. This is no 100% guarantee but it looks right.
 
SP usually stands for Spare Part, so this is the number used to identify what spare part HP has to send you in case of a defective part. That's why 487738-001 only refers to the expander card itself, they usually will not replace the accessories.
If a seller offers you a 487738-001 part then you'll probably get a naked expander card.
468406-B21 is a option partnumber which refers to a option you can specify when buying a new server or buy seperately to upgrade an existing server. These option kits can be easily identified by the -B21 ending and they usually supply everything you need to immediately start using the component. In this case there would be four SFF-8087 to SFF-8087 cables (two short and two long ones) in the box. Short ones to connect the controller to the expander and long ones to connect the expander to the backplane. These are specifically tailored to be used in a ProLiant Server though.

Basically you could take this like the difference between a bulk and a retail component.
You should be fine with that one but you might get the naked card.
Is $299 significantly cheaper than the price for the retail -B21 version?
 
Last edited:
JensS said:
SP usually stands for Spare Part, so this is the number used to identify what spare part HP has to send you in case of a defective part. That's why 487738-001 only refers to the expander card itself, they usually will not replace the accessories.
If a seller offers you a 487738-001 part then you'll probably get a naked expander card.
468406-B21 is a option partnumber which refers to a option you can specify when buying a new server or buy seperately to upgrade an existing server. These option kits can be easily identified by the -B21 ending and they usually supply everything you need to immediately start using the component. In this case there would be four SFF-8087 to SFF-8087 cables (two short and two long ones) in the box. Short ones to connect the controller to the expander and long ones to connect the expander to the backplane. These are specifically tailored to be used in a ProLiant Server though.

Basically you could take this like the difference between a bulk and a retail component.
You should be fine with that one but you might get the naked card.
Is $299 significantly cheaper than the price for the retail -B21 version?
Click to expand...

That makes sense, the seller states that this is a bulk card, and no other accessories are included with the sale. It's not very cheap as a bulk card, but with the cheapest shipping option to Norway, it ain't that bad.

I am only concerned about the firmware, as I do not have the possibility to flash the card.
 
You could always be unlucky and buy from a shop with a rather old inventory. These expanders aren't sold in big numbers, at least not by most shops. If they order something like 10 expanders it might probably last for a year or more.
So buying the retail -B21 version makes no difference in terms of firmware, it might still be old and needs to be flashed.
You should look out for someone who is able to flash the firmware for you if you decide to buy one. Just in case.
I would do it but I don't live in Norway.
 
eg1l said:
That makes sense, the seller states that this is a bulk card, and no other accessories are included with the sale. It's not very cheap as a bulk card, but with the cheapest shipping option to Norway, it ain't that bad.

I am only concerned about the firmware, as I do not have the possibility to flash the card.
Click to expand...

I don't know if this helps, but amazons EU focused page offers this http://www.amazon.co.uk/SAS-Expande...Q16K/ref=sr_1_1?ie=UTF8&qid=1339433024&sr=8-1 which is the B21 retail part for about L184 (with at least UK delivery included), I don't know if exchange rates to norway make that more or less favorable than the ebay deal. Amazon takes returns/exchanges where this particular ebay guy doesn't.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
eg1l said:
That makes sense, the seller states that this is a bulk card, and no other accessories are included with the sale. It's not very cheap as a bulk card, but with the cheapest shipping option to Norway, it ain't that bad.

I am only concerned about the firmware, as I do not have the possibility to flash the card.
Click to expand...

I could flash the expander for you but I live in Sweden...
 
This one isn't sold by Amazon though which might have an effect on how easy it is to return the card. It is a marketplace offer.

It is a bit cheaper though, according to Google 180 GBP is about 280 USD at the moment.
Shipping to Norway from Germany, where I live, is 15 EUR or about 20 USD so I would assume it costs about the same to ship the card from the UK to Norway.
This shop, LambdaTek, also offers the expander via the marketplace on amazon.de and they charge 5.50 EUR shipping costs from the UK to Germany which is actually pretty cheap.
So you might end up with ~300 USD including shipping if they ship to Norway at all.

These expanders are about 230 EUR in Germany which is about 288 USD. So you might also get a deal in Germany if you can find a shop that ships to Norway.
 
You must log in or register to reply here.
Back
Top