HP SAS Expander Owner's Thread

[Dgephri]

+1 on the Hitachi 5K3000. IMO, best drive right now for data archival, especially where striped arrays are concerned (hardware RAID, ZFS, etc).

As far as sourcing them, Fry's > Amazon > Newegg. I say Fry's because they sell retail package - meaning drive suspended in packaging thats superior to Hitachi's official OEM packaging that Amazon sells. Check Fry's on thursdays for new week's sale pricing - these drives go down to $69 or less frequently. Otherwise, anywhere else you can get the retail SKU is preferred. Amazon sells Hitachi OEM which is packaged less well than retail box as mentioned, and Newegg sells individual drives removed from OEM 20-packs which are then "repacked" by teenagers in the Newegg warehouse like the aspergers superstars they are.

/derail

Any reason for a RAID5 to spend the extra on the 7k3000? Newegg has the 7k for $100 each or Newegg vs. Amazon 5k3000 at $80 each...

 

[Bookmage]

Is there any news on that? i intend to buy one of these but how do you switch in the Backplane with that? or is there any other hardware to switch on the powersupply and the harddrives?

Just to update, the IEI HPE-3S2-R10 is back ordered due to manufacturer part problems?

I was going to order the PE-2SD1-R10
but then I discovered this: HPE-4S2-R40 which looks like the same thing with an extra PCI slot.
it should be in the first week of April and i'll post back whether it works with the HP SAS expander. As of 03/24/2011 it's less than $5 more expensive than the 3 slot version, but still under $30 + Shipping.

Might be nice to link these parts in the OP as compatible power sources for those who want to use it in an enclosure with no motherboard.

Is there any news on that? i intend to buy one of these but how do you switch in the Backplane with that? or is there any other hardware to switch on the powersupply and the harddrives?

Yes I ordered the R40 and it finally shipped this month. I just received the part last week and was able to set it up over the weekend.
The HPE-4S2-R40 is the same with 3 PCI slots. THE HP SAS Expander works with it. It takes a regular ATX 20 pin to power it . It requires a permanent on/off switch, not a momentary on/off switch that's found in most computers nowadays.. I was able to pull an old power switch from an older computer and have it temporarily set up now, but I'm debating just putting a jumper on it and using the power supply or finding a better switch to screw to the case.

The mounting holes don't fit any of the standard ATX mount holes on the case. I was able to use the mounting standoffs to the Norco 4020 to provide some support and I have it secured with the HP SAS Expander and an old PCI Modem... still good for something...

But for $40 shipped, it works.

 

[bigdogchris]

We're comparing servers. How is the performance of the P410i with 512MB BBWC compared to something dedicated? Will be a Raid 10 setup with SAS drives.

 

[jmk396]

What's a cheap RAID/HBA card that works with the HP SAS expander?

Here's my story...

I have a PERC 6i connected to eight SATA drives. It's working great but I've just ordered a Norco 4220 and want to use more than eight drives.

The PERC 6i apparantly doesn't work with SAS expanders so I'm stuck buying a new card. I'm leaning towards using software RAID (ZFS?) so what is a cheap HBA card that will work with the HP expander?

(By the way, this is used as a home fileserver... [complete overkill, but that's half of the fun])

 

[rkand]

Has anyone had any experience with the adaptec 6000 series and the HP SAS Expanders? I'm thinking of getting either the 6405 or the 6445. They actually seem pretty cheep for Canadians compared to alternatives with the same feature set.

Am I any better off to connect externally to the HP SAS expander or is internal fine?

Just a note for anyone else interested, the Adaptec 6405 raid controller is not compatible as of FW 5.2-0 18301 and my HP SAS fw 2.02

 

[jmk396]

Do all of the HP SAS expanders support 32 drives?

I'm looking at this document and it says only 24 drives.

It also says maximum is 14 TB.

Are these facts true?

Also, does this expander support blinking the LEDs of the Norco 4220 case? (to find a defective drive, etc)

 

[nitrobass24]

Well the norco backplane doesnt support Failed Drive LED because it does not have any logic (SGPIO).

The HP is just like a USB hub, it will pass through whatever it gets.

There is not a 14TB max

If you read the second page of that article it intends for the user to connect a P400 or similar controller to 2 of the Internal Connectors for Dual Link, which would only leave 6 MiniSAS ports x4 = 24 Drives.

Technically that expander *by itself* only supports 32 drives due to port limitations. You could daisy chain another expander for more.

 

[anthony_chief]

Hi guys, I have a adaptec 5405 and have herd reports that the HP Requires firmware 1.00 to work with the adaptec 54xx series.
Can anyone confirm this or am i just geting nevious over nothing ?
Cheers

 

[odditory]

Do all of the HP SAS expanders support 32 drives?

I'm looking at this document and it says only 24 drives.

It also says maximum is 14 TB.

Are these facts true?

If the HP documentation had any merit there would not have been any reason to start this thread. Read the first post.

 

[madburg]

Need some help/direction.

SuperMicro X8SI6-F rev1.02 (onBoard SAS2008) flashed to lastest FW 9.00.00.00-IT (2/18/2011, config utility v7.17.00.00) Brand new
HP SAS expander firmware 2.08 (Latest) received from Dustin (Synergy)
HITACHI Deskstar 5K3000 HDS5C3020ALA632 2TB 32MB Cache SATA 6.0Gb/s 3.5" Brand new
Norco 4224 Chassis Brand new

First I connected both ports from the onboard SAS2008 to two Norco Backplane with the HITACHI's HD's one per backplane, in the LSI Bios config utility it shows Negotiated Link Speed(Gbps) = 6.0; for each drive.

Then I connected one port from the onboard SAS2008 to Port #8 on the HP SAS Expander (single-link), then Port #2 from the expander to one Norco backplane; same (1) Hitachi drive. Upon boot and LSI Bios post I have receive errors, exact same problem as "Gigas-VII" on post #1539 (page 77) screenshot there. In the LSI Bios config utility it shows Negotiated Link Speed(Gbps) = 1.5

Proceeded to connect the second port from the onboard SAS2008 to Port #9 on the HP SAS Expander (dual-link), then Ports #2 & #3 from the expander to two Norco backplanes; same Hitachi drives. Upon boot and LSI Bios post I STILL receive errors. Negotiated Link Speed(Gbps) = 1.5 per drive.

I also tried all this with a brand new LSI 9211-8i LSI SATA/SAS 6Gb/s HBA/RAID Controller card (flashed to lastest FW 9.00.00.00-IT (2/18/2011, config utility v7.17.00.00), same POST Errors with the HP SAS Extender and always Negotiated Link Speed(Gbps) = 1.5 per drive

I then tested same without the Norco backplanes (to see if the card does not like them) from ports #2 & #3 on the HP SAS Extender directly to drives (forward brakeout cables) exactly the same. Then tested with nothing but LSI controllers to ports #8 & #9 to the HP SAS Expander no cables/connection to drives, same post errors.

I know it should be Negotiated Link Speed(Gbps) = 3.0 per the forums for SATA Drives throught the HP SAS Expander right?

I think I was sent the same card as "Gigas-VII" from Dustin from Synergy. Same status LEDs, bottom three are lit, then two are unlit, and the next one up is blinking steadily.

If I am not mistaken "odditory" and maybe a few others tested my exact board and config and all worked. Did anyone ever check to see that it stated "Negotiated Link Speed(Gbps) = 3.0" or 6.0 in the LSI Config utility and you definitly did not receive those error as show in "Gigas-VII" post #1539 on page 77?

 

[BigXor]

If I build a 2 array raid 6 with 8 5k3000 drives each using a 1880i, would there be any point to dual linking the expander? Would the array rebuild time be shorter?

Also what stripe size should I use for video storage?

Also should I align the array on a stripe boundary?

 

[SynergyDustin]

Oh, wow. I didn't realize these HP SAS expanders are out of production already. Is it so?

That's not the case, we simply receive large lots of them directly from HP demo equipment. Test, upgrade firmware, then put into stock.

 

[OldSchool]

If I build a 2 array raid 6 with 8 5k3000 drives each using a 1880i, would there be any point to dual linking the expander? Would the array rebuild time be shorter?

At 16 drives you would be at the point that you would benefit from dual linking. Four 3gbps channels gets you ~1500MB/s before overhead, 16 5K3000s can push a little more than that at their combined peak.

 

[BigXor]

At 16 drives you would be at the point that you would benefit from dual linking. Four 3gbps channels gets you ~1500MB/s before overhead, 16 5K3000s can push a little more than that at their combined peak.

Thanks, that info has helped determine my purchasing decision as to controller. I need 20 ports as I have 1-SSD, 2-15k SAS Savvio's and a CF Adapter in my ODD slot as well as the 16 data drives (I moved my ODD to external).

 

[Blue Fox]

So, I've come across a newer revision of the card with a blue PCB. Anyone know the differences (if any) between it and the other versions? The layout seems to be similar to the one with the green PCB (same heatsink), but has a few differences (namely the IO pins above the external connector like the original yellow PCB version).

 

[vv111y]

forgive the noob , but isn't a x4 pci-e fast enough for a current SAS2 HBA? I see the recommends are x8 cards. I was hoping to conserve some slots to keep expansion options open.

 

[OldSchool]

forgive the noob , but isn't a x4 pci-e fast enough for a current SAS2 HBA? I see the recommends are x8 cards. I was hoping to conserve some slots to keep expansion options open.

PCI-E x4 can sustain 1000MB/s in each direction, if that is enough to suit your needs then rock on

 

[tormentum]

Need some help/direction.

SuperMicro X8SI6-F rev1.02 (onBoard SAS2008) flashed to lastest FW 9.00.00.00-IT (2/18/2011, config utility v7.17.00.00) Brand new
HP SAS expander firmware 2.08 (Latest) received from Dustin (Synergy)
HITACHI Deskstar 5K3000 HDS5C3020ALA632 2TB 32MB Cache SATA 6.0Gb/s 3.5" Brand new
Norco 4224 Chassis Brand new

First I connected both ports from the onboard SAS2008 to two Norco Backplane with the HITACHI's HD's one per backplane, in the LSI Bios config utility it shows Negotiated Link Speed(Gbps) = 6.0; for each drive.

Then I connected one port from the onboard SAS2008 to Port #8 on the HP SAS Expander (single-link), then Port #2 from the expander to one Norco backplane; same (1) Hitachi drive. Upon boot and LSI Bios post I have receive errors, exact same problem as "Gigas-VII" on post #1539 (page 77) screenshot there. In the LSI Bios config utility it shows Negotiated Link Speed(Gbps) = 1.5

Proceeded to connect the second port from the onboard SAS2008 to Port #9 on the HP SAS Expander (dual-link), then Ports #2 & #3 from the expander to two Norco backplanes; same Hitachi drives. Upon boot and LSI Bios post I STILL receive errors. Negotiated Link Speed(Gbps) = 1.5 per drive.

I also tried all this with a brand new LSI 9211-8i LSI SATA/SAS 6Gb/s HBA/RAID Controller card (flashed to lastest FW 9.00.00.00-IT (2/18/2011, config utility v7.17.00.00), same POST Errors with the HP SAS Extender and always Negotiated Link Speed(Gbps) = 1.5 per drive

I then tested same without the Norco backplanes (to see if the card does not like them) from ports #2 & #3 on the HP SAS Extender directly to drives (forward brakeout cables) exactly the same. Then tested with nothing but LSI controllers to ports #8 & #9 to the HP SAS Expander no cables/connection to drives, same post errors.

I know it should be Negotiated Link Speed(Gbps) = 3.0 per the forums for SATA Drives throught the HP SAS Expander right?

I think I was sent the same card as "Gigas-VII" from Dustin from Synergy. Same status LEDs, bottom three are lit, then two are unlit, and the next one up is blinking steadily.

If I am not mistaken "odditory" and maybe a few others tested my exact board and config and all worked. Did anyone ever check to see that it stated "Negotiated Link Speed(Gbps) = 3.0" or 6.0 in the LSI Config utility and you definitly did not receive those error as show in "Gigas-VII" post #1539 on page 77?

I recently blew up my LSI 9211-8i and had to RMA it. In the mean time I purchased another 9211-8i card off Newegg with express shipping to get the box back up. Being the good little IT guy that I am, I flashed to latest firmware (9.00.00.00) and upon boot I started getting the EXACT same errors.

I ended up going sequentially back through the firmware archive (9.00.00.00 -> 8.00.00.00 -> 7.00.00.00) until I found a firmware that seemed to work again (v7.00.00.00). I'd be curious to see if this works for you too.

I had to use the DOS based floppy disk sas2flsh utility (!! Must be DOS version!) and ran the following commands:

sas2flsh -o -e 6
sas2flsh -o -f 2118it.bin -b mptsas2.rom

The first command wipes the card firmware/bios. The second command flashes the card with the older firmware/bios combo.

WARNING: Do not power off machine after erasing the card, you will toast it and need to RMA. I accept no responsibility if you screw it up! This worked fine on a 9211-8i card, but verify with your LSI card's documentation for correct downgrade procedure.

My HP SAS Expander is at FW 2.06, but I don't think updating to 2.08 would resolve the issue. It's the LSI card you need to back-grade.

 

[HitechRedneck]

Well just got my HP expander card along with an areca 1880 and 10x 2tb hitachi deskstar hard disks. I must report I did have a few hiccups during the build but nothing major. I even got a 9x 2tb wdc ears raid to work with it. (10th drive had issues and caused most hiccups it is being replaced). I also created a 12x 1tb raid with great results. I must admit these 2 cards together are a great pair

 

[DJ_Datte]

Actually, shutting down the computer after erasing the card (SAS2008) won't damage or inconvenience it. I had to do it due to some unfortunate circumstances of not having the proper software available and not being able to introduce it in the environment without a shutdown. But nothing went wrong, the card was there, just in a reset state, waiting for firmware to be introduced to it.

EDIT: When you erase the card like you did with "-o -e 6 or 7" you delete the sasadress. you may want to put a new sasaddress in with "-o -sasaddr (16digit number)" or you may encounter issues down the road by not having a unique sas adress for your HBA.

//D

 

[tormentum]

Actually, shutting down the computer after erasing the card (SAS2008) won't damage or inconvenience it. I had to do it due to some unfortunate circumstances of not having the proper software available and not being able to introduce it in the environment without a shutdown. But nothing went wrong, the card was there, just in a reset state, waiting for firmware to be introduced to it.

EDIT: When you erase the card like you did with "-o -e 6 or 7" you delete the sasadress. you may want to put a new sasaddress in with "-o -sasaddr (16digit number)" or you may encounter issues down the road by not having a unique sas adress for your HBA.

//D

Interesting. I might try flashing my faulted 9211-8i before sending it back for RMA, even though I have the RMA number already. Strange thing was that in the system I had the card installed in, the system would not POST and would keep rebooting in an infinite loop.

The reason I mention the issue with powering down the computer is the documentation is quite adamant that you should not do it. YMMV.

I'm curious to hear if anyone has gotten the HP SAS Expander to work with the latest 9211-8i firmware (9.00.00.00)?

 

[treadstone]

If the firmware of the controller was written properly, it should have two different firmware images at all times. All systems I design have a boot loader or I call it a 'base image' which is either a simplified or scaled down version of the normal firmware (a base OS) to get the controller in a defined state at which point it waits for a new firmware upload. The second image is the normal firmware or what I call the 'run-time image'. In case the run-time image fails or got corrupted, the controller would simply fall back to the base image to allow you to fix the issues the run-time image has. Some controllers don't have that level of recovery or operate in this way and for those you would need specialized tools (software and/or hardware) to reprogram the controllers firmware (e.g. a JTAG based programmer). Based on the fact that most of these LSI controllers have a serial port on the card, I would assume that they do have a base image and run-time image. The serial port can be used for diagnostic and debug purposes as well as firmware uploads.

 

[Mpkuno]

Hi to all of you people!
I have a couple of questions. If I create an array using only the controller and then I plug in the HP sas exp is there any problem recognising the array?
And from the posts a few pages back you talk about the dell H800 controller. But I din't understood if it is compatible or not...

 

[DJ_Datte]

As long the controller itself is the same, it does not matter if the disks are connected to the controller or to the sas expander that's connected to the controller.

//D

 

[odditory]

I would like to know if anyone has managed to get staggered spin up working?

Here odditory says it is working
http://75.126.99.220/showpost.php?p=1036260668&postcount=996

and here treadstone says the opposite http://hardforum.com/showpost.php?p=1037214881&postcount=10

To clarify, I was talking about staggered spinup from sleep, not from coldboot. It's correct that the HP Expander doesn't default SATA drives into standby (wait for spinup signal) when power is first applied, and that might even be a good thing given I'm not sure every SAS controller necessarily sends a spinup signal when its scanning the bus at bootup - a scenario that might hold potentially far more complaints and confusion in people thinking their drives are dead when they're not recognized or unresponsive.

There may be a way to set a parameter on the controller firmware of certain drives for this behavior to occur, but consider also that cheaping out on a power supply to save $25 has to be very important to you, important enough to ignore the fact its the hinge pin driving the thousands you've invested in drives, controllers, motherboard/cpu/ram, etc. Personally I'd never size a power supply smaller than being able to comfortably handle all components in a system running at full power draw at once, especially so for a server. Anything less is russian roulette, considering the PSU is the one component that if overdriven may behave erratically to the point its stressing or even frying other more expensive components. Been there, done that.

That said, would it be nice if staggered spinup from coldboot worked from an academic perspective, sure, but I don't think its worth losing sleep over given the bigger picture.

 

[Jeroen1000]

Since my server will be idle 99% of the time, a smaller sized PSU may (will?) be more cost effective in the long run.
But I do agree it is better to be safe than sorry. So I did by a good 600W single rail Silverstone PSU.

Have you per chance tested what happens when drives are directly connected to the controller (so without the expander sitting between the controller and the drives)?

edit: also, I do not believe the HP expander has any say whatsoever in whether a drive defaults to stand-by at start up (cold boot, just to be clear). If a drive supports staggered spin up via pin 11 on the power connector, it should by all means prevent the motor from spinning up until it has successfully completed the PHY-initialization (if you want I'll post the source for this information). You see, staggered spin up is not invoked by sending an ATA-command. Putting a drive in stand-by (like Windows can achieve for instance) IS invoked by using an ATA-command. We must be aware that staggered spin up is a one time deal during a cold boot. Putting a drive to sleep (and waking it) after a cold boot is something else.

edit2: I've reread all my research and sources on this. If the drives spins up on cold boot (= as soon as power is applied to it), pin 11 on the power connector is most likely grounded OR the hard drive simply ignores pin 11. If pin 11 is not grounded (floating high) the HBA "wakes" (or should wake) every drive sequetially by doing a PHY-initialisation.

 

[treadstone]

@Jeroen1000: I had done all those test on my original installation in a quest to get staggered spin-up working on my 100TB server. The chassis I use is a Chenbro RM-91250 with backplanes that do support the staggered spin-up of the attached HDDs. IIRC, when the drives where connected to the Areca ARC-1680i controller directly and the controllers BIOS was configured accordingly, the staggered spin-up worked as intended. As soon as I added the HP SAS expander in-line, the drives always start up no matter if pin 11 on the HDD was floating or pulled high. This is most likely due to the fact that the SAS expanders processor initiates the PHY initialization between the HDD and the expander even before the controller gets a chance to talk to the expanders processor.

Currently, my server is configured for staggered spin-up, however all drives spin-up during a cold boot at the exact same time (hence the power draw of over 1100W at cold boot)!

Also, PUIS doesn't seem to be supported by the Areca controller either, at least I could never get this to work. If I strapped a drive for PUIS, the controller would not recognize the drive, no matter if it was connected via the SAS expander or directly to the controller...

I also agree with Odditory, it doesn't make any sense to skimp on the power supply, especially if this is your server holding all you 'important' stuff!
Workout what your typical power consumption would be as well as what the expected worst case load might be and size your power supply accordingly. Remember that even with your expected worst case load, this should be about 80% of your power supplies capacity.

 

[Jeroen1000]

So the expander does seem to have a say in this. I thought it behaved like a "dumb" ethernet switch and kept silent until it gets to pass "things" that come from the Areca controller.

PIUS is indeed not supported and Areca have confirmed this to me. There does seem to be one brand (Highpoint) that might support PIUS. There are also tools that can activate/deactivate PIUS. However, if the correct ATA-command is not sent to the drive when it is in stand-by, it simply will not show up (from the controllers point of view) nor will the spindle motor spin up. But drives can be queried (without causing the spindle motor to spin up) in PIUS mode. I don't know what kinds info you can query but I do know it is possible to ask the drive whether it is in stand-by or not.

So it shoud be possible to wake your PIUS drives with the correct software. Areca just decided not to support this method. After all, controller firmware is just software.
I think everyone know this already, but PIUS != the pin 11 method.

If staggered spin up would work reliably (with either method), I feel you are not skimping on the PSU. I know the fact that using the PSU efficiently might be low(er) on the priority list for a server that holds important data. But if it were to work nicely, I see no reason to overspec the PSU just for that brief power spike during a cold boot.

 

[treadstone]

@Jeroen1000: First, a minor correction to your post, it's PUIS (Power Up In Standby) not PIUS!

The problem with trying to query drives via a program once the OS is loaded is that during the boot phase, the controller (and the OS) try to query your drives for presence and status. If they do not respond (because of the PUIS state they are in and the controller not waking them up due to the lack of PUIS support), they will not be available to the OS or even show up. I heard (but have no first hand knowledge of this myself), that there are a couple of motherboards that do support PUIS with the build-in SATA ports. this however is quite useless to most of these server builds as the majority of HDDs are not connected to or controller through these on-board SATA ports.

Of course the idea of using staggered spin-up is to not have to over spec the PSU. On the other hand, those brief moments you were referring to during the time all drives spin up simultaneously and hence put the most stress on the PSU, is exactly the moment that can break your PSU if not sized properly. Depending on what drives you use and how many you have in your system, this period can last anywhere from below 10 to well over 30 seconds. For an overload condition, that's a LONG time. Most drive manufacturers list the maximum current draw during spin-up in their drive specs. Most new drives limit the current draw and some models you can even program what the maximum draw should be to help to keep the PSU small.

Personally I prefer the Gold series certified PSUs as they have a slightly higher efficiency at around 50% of their total capacity. I try to spec the PSU to 50% for normal operating mode of the server which should give me plenty of headroom when the system puts the most stress on the PSU (which I try to keep at 80% of the PSUs capability). This still leaves me with an additional 20% in case I add more equipment later or if something goes wrong (which I hope it doesn't).

In regards to the HP SAS expander, in essence the functionality is similar to a 'dumb' Ethernet switch, however there is a lot more going on behind the scenes than you would expect. For one, the expnader has a build in microprocessor (even a build-in 10/100Mbps Ethernet interface which is not used on this controller). This micro-controller initializes the interfaces and PHYs and negotiates the maximm capabilities of each interface with the corresponding link partner. All this happens (and has to happen) before the controller even gets a chance to talk to the expander. The firmware of the expander could be changed to query the link partners capabilities without spinning up the drives and wait for the controller to start the spin-up sequence. But at the same time, if the controller doesn't support staggered spin-up, but expects the drives to be ready when it queries them to see if they are available, the controller would not get the reply it would expect and simply indicate that there are no drives present. It just a matter of getting all these individual components to work properly with each other, but unfortunately, some of the higher level functions are not supported by all vendors as they implement the bare minimum to be able to work with each other. Also, keep in mind that HP doesn't really care if this works with any other brand of controller as this expander was designed to work with their hardware in mind and nothing else!

 

[Jeroen1000]

@Jeroen1000: First, a minor correction to your post, it's PUIS (Power Up In Standby) not PIUS!

Yes, sorry lol, kind of funny now I read it back

The problem with trying to query drives via a program once the OS is loaded is that during the boot phase, the controller (and the OS) try to query your drives for presence and status. If they do not respond (because of the PUIS state they are in and the controller not waking them up due to the lack of PUIS support), they will not be available to the OS or even show up. I heard (but have no first hand knowledge of this myself), that there are a couple of motherboards that do support PUIS with the build-in SATA ports. this however is quite useless to most of these server builds as the majority of HDDs are not connected to or controller through these on-board SATA ports.

You are probably right on this one. I thought that maybe it would be possible to wake up the drives AFTER the OS has booted (the OS drive obviously should not be in a PUIS state as we need at least one working drive with an OS).

Of course the idea of using staggered spin-up is to not have to over spec the PSU. On the other hand, those brief moments you were referring to during the time all drives spin up simultaneously and hence put the most stress on the PSU, is exactly the moment that can break your PSU if not sized properly. Depending on what drives you use and how many you have in your system, this period can last anywhere from below 10 to well over 30 seconds. For an overload condition, that's a LONG time. Most drive manufacturers list the maximum current draw during spin-up in their drive specs. Most new drives limit the current draw and some models you can even program what the maximum draw should be to help to keep the PSU small.

A overload spike lasting that long? That is contrary to what I've read. Startup overloads should be in the order of milliseconds AFAIK? Remember it is only the spike to get the motor spinning that could pose a problem.

I agree we simply have no control over what the HP expander does. Too bad, it woud have been nice to have. I wonder whether that Intel expander fares better in this department. I'm going to check the HP website. There problably won't be any mention regarding staggered spin up, because if there is...there might be some hope to get it working.

 

[kirodar]

In regards to the HP SAS expander, in essence the functionality is similar to a 'dumb' Ethernet switch, however there is a lot more going on behind the scenes than you would expect. For one, the expnader has a build in microprocessor (even a build-in 10/100Mbps Ethernet interface which is not used on this controller). This micro-controller initializes the interfaces and PHYs and negotiates the maximm capabilities of each interface with the corresponding link partner. All this happens (and has to happen) before the controller even gets a chance to talk to the expander. The firmware of the expander could be changed to query the link partners capabilities without spinning up the drives and wait for the controller to start the spin-up sequence. But at the same time, if the controller doesn't support staggered spin-up, but expects the drives to be ready when it queries them to see if they are available, the controller would not get the reply it would expect and simply indicate that there are no drives present. It just a matter of getting all these individual components to work properly with each other, but unfortunately, some of the higher level functions are not supported by all vendors as they implement the bare minimum to be able to work with each other. Also, keep in mind that HP doesn't really care if this works with any other brand of controller as this expander was designed to work with their hardware in mind and nothing else!

Do you assume that the HP expander does not support staggered spin-up, based on your argumentation, or do you actually know it?

In my opinion it's still not 100% clear if the HP Expander prevents staggered spinup. In this thread we have at least one report where HP Expander + staggered spinup does seem to work. "mredz" is invited to give further proof

Not true. I have staggard spinup working with my Highpoint 4320 and WD RE4 drives no problem.

Besides staggered spinup from coldboot, some people seem to use staggered spinup from sleep. Aren't you afraid of the higher wearout due to the starting and stopping? Which inactive timeout do you use? And how do you prevent a spinup each time your file system is only accessed for listing a directory which results in the wake up of all disks of your RAID (which could be prevented if only the listing was cached in RAM or on the system disk for instance).

 

[treadstone]

A overload spike lasting that long? That is contrary to what I've read. Startup overloads should be in the order of milliseconds AFAIK? Remember it is only the spike to get the motor spinning that could pose a problem.

If you have a lot of drives (e.g. as it is the case in my server) the drives do not start up EXACTLY all at the same time. Also, an overload condition can last a few seconds. The current draw will rise sharply during the first few milliseconds to charge up capacitors and when the spindle motors are started and then will drop off as the spindles come up to speed. It does take a few seconds not just a few milliseconds for a possible overload condition (I am talking about a server with LOTS of HDD not just one or two!).

I agree we simply have no control over what the HP expander does. Too bad, it woud have been nice to have. I wonder whether that Intel expander fares better in this department. I'm going to check the HP website. There problably won't be any mention regarding staggered spin up, because if there is...there might be some hope to get it working.

I doubt you'll find any information in regards to staggered spin-up on the HP SAS expander. It's hard enough to find ANY kind of useful information on this card on their website!

Do you assume that the HP expander does not support staggered spin-up, based on your argumentation, or do you actually know it?

If you read my previous posts and/or my build log, you'll see that I went through a LOT of different tests to get this going. I tried the HP SAS expander with an Areca ARC-1680i as well as with a Supermicro AOC-SASLP-MV8 as the controller. The staggered spin-up would worg with drives connected directly to the controllers but once the HP SAS expander was in-line, the drives always spin up as soon as they receive power from the PSU!

In my opinion it's still not 100% clear if the HP Expander prevents staggered spinup. In this thread we have at least one report where HP Expander + staggered spinup does seem to work. "mredz" is invited to give further proof

Well as I said, I did a LOT of different tests and no matter what I tried, the HP SAS expander simply refuses to cooperate when it comes to staggered spin-up from cold boot!

Besides staggered spinup from coldboot, some people seem to use staggered spinup from sleep. Aren't you afraid of the higher wearout due to the starting and stopping? Which inactive timeout do you use? And how do you prevent a spinup each time your file system is only accessed for listing a directory which results in the wake up of all disks of your RAID (which could be prevented if only the listing was cached in RAM or on the system disk for instance).

I don't use a hardware based RAID setup anymore. I use FlexRAID and it works just like I wanted my system to operate! FlexRAID actually helps my server to reduce the wear and tear on the drives, so no, I am not afraid that my drives will wear out due to the sleep mode. I would have to take a look again as to what I had actually programmed into the controller to see what the settings are for the sleep timers.

 

[Jeroen1000]

Kirodar, WD drives use a different implementation AFAIK. They do not use pin 11 on the power connector but change a setting in the drive's firmware. That fully explains why it can work on those drives.

 

[odditory]

Also, keep in mind that HP doesn't really care if this works with any other brand of controller as this expander was designed to work with their hardware in mind and nothing else!

thank you treadstone, that really is the #1 thing people should keep in mind about this device when pondering why it isn't configurable or able to behave in certain ways to fit everyone's unique requirements, I really need to put that line in the OP in big bold letters since it gets old having to repeat it.

 

[Jeroen1000]

Knowing that should not stop us from trying though.

 

[Rudde93]

Hello I just got my Expander card installed in the cumputer and I don't get up the hard drives (LSI 1068E controller) on the SUPERMICRO X8ST3-F motherboard. the card get pretty hot and I have 3 green ligths on it solig (of the 6) and one flashing, the one on the top is flashing and the three on the bottom are solid.

WHAT UPS? The hard drive show up if I connect directly to the controller.

I see they write open and closed on jumper settings in the manual, I was wondering is Closed means its going to be connected or not connected?

 

[thideras]

Odditory, you can add the LSI 8708EM2 and the Intel SRCSASBB8I to the list of cards that work. I'm not sure if it uses both 8087 connectors between the RAID card and the SAS expander, but it doesn't cause it to fail. I own the Intel SRCSASBB8I and tested the stock LSI (8708EM2) firmware and both worked fine.

I did have some issues with three SAS drives when I wasn't using the "To Controller" ports on the SAS expander. Once I swapped to those, I could successfully build an array with the SAS drives. What is odd is the seven Hitachi 7k1000 in RAID 6 were working fine while the other array kept failing.

http://thideras.com/images/projects/project_rackmount_overkill/lsi_8708em2/physical.png
http://thideras.com/images/projects/project_rackmount_overkill/lsi_8708em2/logical.png
http://thideras.com/images/projects/project_rackmount_overkill/lsi_8708em2/lsi_8708em2_moved.JPG

 

[kennypowers]

Didnt feel like resurrecting the chenbro thread, so excuse me this is the wrong thread.

Anyone have experience with the Chenbro CK12804 28 port expander? And in particular in combination with the 1880i. The sites compatibly .pdf says it works with 1680ix/LP, but it hasnt been updated since 2009.

Any information is appreciated. =)

 

[JensS]

I finally got my SAS Expander and a LSI 3ware 9690SA-4I raid controller which should work according to the compatibility list in this thread.
At the moment I use it for testing purposes on a Supermicro H8DCE, which is an older dual socket 940 board, and I can't get it to work.
The controller works fine, I've tried that individually. But as soon as I connect the expander the system hangs during boot when the controller tries to initialize. I get "Waiting for 3ware controller to initialize" on the screen forever (or at least for 30 minutes, I then ran out of patience ).
Since the 9690 already had its most recent firmware I wanted to flash the expander to see if it makes any difference and so I got my SmartArray P410 out. That was flashed to FW 3.66 quickly which HP says is required for updating the expander to FW 2.08.
The P410 as well takes ages to initialize during boot with the expander connected but at least it will get through at some point, taking somewhere between 5 and 10 minutes I think.
Unfortunately the HP Update Tool doesn't work properly, it boots via USB and then stops with a black screen doing absolutely nothing. I will try this in a ProLiant server later, but since that lies on my workbench in bits I will rely on the working hardware for the moment.
The expander shows up in ACU when conncted to the P410 and it told me that it has FW 2.06, so that is pretty recent as well.
So at this point I can tell that both controllers have the recent firmware and the expander has FW 2.06.

Long story short:

1. Is it normal that controllers take ages to initialize with the expander connected?
2. Any ideas why the LSI controller doesn't initialize at all with the expander connected?
3. Are there any other ways to update the FW on the expander?

Thx in advance, any help is much appreciated

 

[RESTfulADI]

can I use this expander in any way to connect several old Supermicro servers with LSI controllers and 16 disks each to make one SAN?