HP DL180 G6 and SAS HBA Hell. (Help?)

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
38,866
Hey all,

So this is a duplication of a thread I have going in another forum, but I figured I'd try here as well because - you know - more eyeballs. When you are dealing with relatively rare problems, anything might help.

Anyway, let's go.


Background:


About 3.5 of years ago I bought a used HP DL180 G6 server on eBay, because it looked like a great deal. Dual Xeon 1366 sockets and a built in backplane with SAS expander sounded great to me. I didn't opt to buy one of the official HP RAID cards because I preferred an HBA, as I wanted to run ZFS with a JBOD configuration.

Instead I bought an IBM M1015, flashed it to an LSI IT mode and it worked great... EXCEPT

Apparently HP servers look to make sure that you use officially supported HP PCIe cards, and if you don't go into a full fan speed freakout mode because they can't tell the temp of the PCIe card and err on the side of safety and blast the shit out of it with full fan speed. When you have 8x little 18krpm fans blowing through a 2U chassis this is LOUD to the point of being practically unusable for home use.

At over 100db and literally sounding like a jumbojet taxiing down the runway preparing for takeoff, even when it was hidden in the basement, I could hear it two floors up in my bedroom with all doors shut in between.

A noise reference:



At the time I searched for a noise solution. HP at some point released a firmware update for the 4U DL380 G6 that eliminated this fan problem, but the 2U DL180 G6 never got one.

I eventually gave up, harvested the CPU's and RAM from the server, and got a Supermicro board and did a custom server build in a Norco case and have been happy with my silence ever since...


...until a couple of weeks ago.

Slightly inebriated, I was browsing eBay and came across a matched pair of L5630 Xeon's for ~$5, and was like, why the hell not. I have 12x 8GB of registered DDR3 just hanging around, why not play with the old Dreamliner again?

I took to the internet, and someone suggested that HP had a compatible HBA called the H220 that wouldn't trigger fan freakout mode, and this was enough to get me thrilled.

Reason being, I am going through a storage update on my main server (related threads here, here and here) and since my current backup solution, Crashplan is exiting the consumer backup market, I was thinking, I could use this old server to do some backup of my own, with my old drives.



The Problem:


So I went ahead and ordered one of those HP H220 HBA's, and it looks like the DL180G6 recognizes it as a friendly device, because thus far we haven't gone into fan freakout mode, which is a good thing!

I do have some problems though.

I installed the HBA, and hooked it up to the backplane in the DL180G6, and powered up my dreamliner.

The BMC rewrote the firmware (like it always does when you insert a new PCIe card for some reason) and everything looked good, until the H220 option ROM loaded, at which point it froze eternally on the following:

Code:
Initializing.../

With the slash spinning in circles for all eternity. (I let it sit overnight, just to make sure it wasn't just doing something, and it still stayed in that state.)

Alright. Some troubleshooting. Disconnected the backplane, and connected some random drives I had kicking around using one of those SAS to SATA breakout cables, and this worked just fine, with the devices detected properly.

37646348806_2820c77305_c.jpg


So, it seems like the H220 has some trouble initializing when connected to the backplane, but otherwise works fine. I know that the backplane works, as I have had it working with 12 disks connected to a IBM M1015 SAS controller with everything working. The server was just deafening, because it ddidn't recognize that SAS controller as a known card, so I decided not to use it that way.

So, it looks like I am dealing with some sort of incompatibility between the HP H220 controller and the backplane in the HP DL180G6.

Here is what I have tried thus far:



1.) Disable Boot ROM:

Gone into the SAS configuration page and disabled the boot ROM. This allows me to boot with the backplane connected. I was hoping it would just be a bootrom init problem, and then it would work in OS, but that is not the case. Ubuntu 16.04LTS freezes during boot at the following point:

Code:
mpt2sas_cm0: host_add: handle(0x0001), sas_addr(0x500605b00503c950), phys(8)
mpt2sas_cm0: expander_add: handle(0x0009), parent(0x0001), sas_addr(0x50001c1071540000), phys(19)

So, it looks like it is having trouble with the SAS expander in the backplane.



2.) Unplug Backplane I2C Connector


Doing some more googling about I found someone suggesting that they got a troublesome H220 to boot with their HP blackplane by pulling out the i2c cable that goes between the backplane and the motherboard in their server.

I tried this, and I was able to get the server to boot!

Only problem is, now it doesn't see any of the drives connected tot he backplane, so I guess that didn't really solve anything.



Things I can think of trying next (but I could use some help)


- I have read that the backplane has updateable firmware, but I don't quite know how to do this, and I don't know if I can do so without a working connection to the HBA... Any suggestions?

- Also, I want to try to update the H220 firmware, but I don't know where HP hides their firmware and firmware update tools for these cards. I've googled a bit without much success. I found the official HP support page, but when I filter on H220 it just seems to contain tons of hard drive firmware?? Any suggestions?

- I could try to flash the controller with standard LSO 2308 firmware, but I'm guessing this will remove whatever the BMC in the server uses to recognize it as a properly supported PCIe card, and send it back into fan freakout mode, which was the problem I was trying to avoid in the first place.

I have done some other googling, and there are a few instances of people with similar problems, but none of them have any solutions. If anyone knows anything I might try, I'd appreciate it.


Of course, now that I read HP's docs, they list support for Gen 8 servers only, so maybe this is a lost cause, and I should just buy a cheap case, another used Supermicro server board, and just recycle this damned HP server and never let it cause me any aggravation ever again...


I'd appreciate any suggestions!
 
Last edited:
I have zero experience with this server and controller, but I used https://duckduckgo.com/?q=DL180+G6+firmware and found the following.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=3884342

On the left in Drivers & Software, under Operating environment, expand the Software type category and look inside the following: BIOS (Entitlement Required) (5), Firmware (Entitlement Required) (3) and Firmware (98).

I tried search terms backplane, firmware, CD, DVD and supplemental on this HP page "Search this product for..."

Smart Update Firmware DVD Version 10.10 Jun 04 2012, within Firmware (Entitlement Required) category (a collection of firmware)

Firmware DVD Supplemental Update - HP ProLiant DL180 G6 Version 2.2 Feb 28 2013, within Firmware category (a backplane update)

The second result from the above duckduckgo search says to download the HP Service Pack for Proliant DVD.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html

https://duckduckgo.com/?q=HP+H220+firmware found the link below.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=5263567 (same one you found BUT try search term H220 on this page "Search this product for...")

Online ROM Flash Component for Linux - HPE Host Bus Adapters H220, H221, H222, H210i and H220i Version 15.10.10.00 Apr 21 2017

I am guessing the HP Service Pack for Proliant DVD is what you want and to use the last SPP that contained drivers and firmware for G6 servers, Version: 2016.10.0 (24 Oct 2016).

871790_001_spp-2016.10.0-SPP2016100.2016_1015.191.iso

MD5: 3e93873632c5758666c5b1c305d557ea
SHA-1: 829eb4a5d7276aceead5254aeaabe488a11a55bd
SHA-256: 3e7943ae12f6071efa087e327086b81aa06b13d4ea0c2d60290cd442aca3c289

I tried to download the HP Service Pack for Proliant DVD but it says you need an active warranty or support agreement.

The hpe support center links may have older (or newer) software so compare version numbers and dates for any particular BIOS, Firmware or Driver etc.
 
Last edited:
I have zero experience with this server and controller, but I used https://duckduckgo.com/?q=DL180+G6+firmware and found the following.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=3884342

On the left in Drivers & Software, under Operating environment, expand the Software type category and look inside the following: BIOS (Entitlement Required) (5), Firmware (Entitlement Required) (3) and Firmware (98).

I tried search terms backplane, firmware, CD, DVD and supplemental on this HP page "Search this product for..."

Smart Update Firmware DVD Version 10.10 Jun 04 2012, within Firmware (Entitlement Required) category (a collection of firmware)

Firmware DVD Supplemental Update - HP ProLiant DL180 G6 Version 2.2 Feb 28 2013, within Firmware category (a backplane update)

The second result from the above duckduckgo search says to download the HP Service Pack for Proliant DVD.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html

https://duckduckgo.com/?q=HP+H220+firmware found the link below.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=5263567 (same one you found BUT try search term H220 on this page "Search this product for...")

Online ROM Flash Component for Linux - HPE Host Bus Adapters H220, H221, H222, H210i and H220i Version 15.10.10.00 Apr 21 2017

I am guessing the HP Service Pack for Proliant DVD is what you want and to use the last SPP that contained drivers and firmware for G6 servers, Version: 2016.10.0 (24 Oct 2016).

871790_001_spp-2016.10.0-SPP2016100.2016_1015.191.iso

MD5: 3e93873632c5758666c5b1c305d557ea
SHA-1: 829eb4a5d7276aceead5254aeaabe488a11a55bd
SHA-256: 3e7943ae12f6071efa087e327086b81aa06b13d4ea0c2d60290cd442aca3c289

I tried to download the HP Service Pack for Proliant DVD but it says you need an active warranty or support agreement.

The hpe support center links may have older (or newer) software so compare version numbers and dates for any particular BIOS, Firmware or Driver etc.


Thanks, I'll definitely look into that!

I was about to just try to over-flash the controller with the LSI firmware and see if it works without tripping the server into "all the fans" jumbojet failsafe mode, but maybe this is worth trying first.
 
I have zero experience with this server and controller, but I used https://duckduckgo.com/?q=DL180+G6+firmware and found the following.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=3884342

On the left in Drivers & Software, under Operating environment, expand the Software type category and look inside the following: BIOS (Entitlement Required) (5), Firmware (Entitlement Required) (3) and Firmware (98).

I tried search terms backplane, firmware, CD, DVD and supplemental on this HP page "Search this product for..."

Smart Update Firmware DVD Version 10.10 Jun 04 2012, within Firmware (Entitlement Required) category (a collection of firmware)

Firmware DVD Supplemental Update - HP ProLiant DL180 G6 Version 2.2 Feb 28 2013, within Firmware category (a backplane update)

The second result from the above duckduckgo search says to download the HP Service Pack for Proliant DVD.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html

https://duckduckgo.com/?q=HP+H220+firmware found the link below.

https://support.hpe.com/hpesc/public/home/driverHome?sp4ts.oid=5263567 (same one you found BUT try search term H220 on this page "Search this product for...")

Online ROM Flash Component for Linux - HPE Host Bus Adapters H220, H221, H222, H210i and H220i Version 15.10.10.00 Apr 21 2017

I am guessing the HP Service Pack for Proliant DVD is what you want and to use the last SPP that contained drivers and firmware for G6 servers, Version: 2016.10.0 (24 Oct 2016).

871790_001_spp-2016.10.0-SPP2016100.2016_1015.191.iso

MD5: 3e93873632c5758666c5b1c305d557ea
SHA-1: 829eb4a5d7276aceead5254aeaabe488a11a55bd
SHA-256: 3e7943ae12f6071efa087e327086b81aa06b13d4ea0c2d60290cd442aca3c289

I tried to download the HP Service Pack for Proliant DVD but it says you need an active warranty or support agreement.

The hpe support center links may have older (or newer) software so compare version numbers and dates for any particular BIOS, Firmware or Driver etc.

Do you know I should expect from this DVD? Is it something I can boot from and use to flash BIOS, or something I'll need to read from an OS?
 
Do you know I should expect from this DVD? Is it something I can boot from and use to flash BIOS, or something I'll need to read from an OS?
There are different modes, so yes and yes. Read the installation instructions here.

Some documentation is here, but use the Search current category for 2016.10.0 to narrow the list. The PDF at the bottom (SPP 2016.10.0 Release Notes Revised Jan 2017) may have more detailed instructions.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html
 
Last edited:
There are different modes, so yes and yes. Read the installation instructions here.

Some documentation is here, but use the Search current category for 2016.10.0 to narrow the list. The PDF at the bottom (SPP 2016.10.0 Release Notes Revised Jan 2017) may have more detailed instructions.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html


I have to say I am very impressed. Literally none of this shit showed up in my searches, and I thought I was pretty good with the search terms...

Thanks again!
 
There are different modes, so yes and yes. Read the installation instructions here.

Some documentation is here, but use the Search current category for 2016.10.0 to narrow the list. The PDF at the bottom (SPP 2016.10.0 Release Notes Revised Jan 2017) may have more detailed instructions.

https://www.hpe.com/us/en/product-catalog/detail/pip.5104018.html


Hmm....

Maybe I need an earlier disk. The DL180 G6 is not listed in the massive matrix in the server support guide pdf.

How did you figure that this was the correct disc? Was there something on their site that said it was?

Thanks again.
 
Hmm....

Maybe I need an earlier disk. The DL180 G6 is not listed in the massive matrix in the server support guide pdf.

How did you figure that this was the correct disc? Was there something on their site that said it was?

Thanks again.
Revision History says 2016.10.0 is the last SPP release to support G6 servers. You are right, I can't find G6 or Gen 6 references to the DL180 in those PDFs.

Okay, wow, I cannot find DL180 G6 in any of the SPP documentation.

Now I'm confused as this guy was able to use SPP from September 2014 on his DL180 G6. None of the 2014.09.0 documentation mentions DL180 G6 either so....

This explains a lot, but it is for the DL320 so it may not apply.

I would try 2016.10.0 from a burned DVD first and work back if you need to (but I don't think you will).
 
Last edited:
Got it solved.

In the end what did it was this.

Oddly enough, even though the HP SAS expander / backplane combo in the G6 is older than the H220 SAS controller, older firmwares refuse to work with this sas expander / backplane.

The solution is needing the latest firmware, but the latest HP firmware is too old.

So, I had to flash LSI's IT firmware, but this was not straight forward.


First problem:

The Sas2flsh application recognizes the card as an HP branded card, and refuses to flash LSI firmware to it.

The sneaky workaround. Download the older LSI P14 firmware update, and use the sas2flsh executable from that package. It predates the HP branded controller, and thus doesnt recognize it as such.

Now download the latest firmware update and use the P20 firmare bin file from that package. Use the old sas2flash executable to flash the new firmware.


Second Problem:


While the sticker on the card identifies it as a SAS 9205-8i, the 9205 firmware on LSI's site refuses to flash, stating the stepping of the card is a later revision than supported by the firmware.

I did some googling and read that others have had success with the 9207 firmware, so I downloaded the 9207-8i firmware and it worked like a charm.

The great news is the server didn't go into panic fan mode once flashed!


For posterity, here are the appropriate command lines, after creating a bootable freedos USB stick and booting to it:

Erase firmware first (optional)

I did this to get rid of the boot ROM, as I don't plan on booting from this controller and it just slows down the boot time. Warning If you do this, you can brick the controller if you don't flash a new firmware before rebooting.

sas2flsh -o -e 6

Now flash the new firmware

sas2flsh -o -f 9207-8.bin -c 0

Took a little while fir it to finish, but once done, worked like a charm.
 
Back
Top