• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

OI and hotswap

kidchunks

Weaksauce
Joined
Jul 26, 2011
Messages
110
I've been testing Openindiana (148) with my m1015 and when I disconnect a disk for simulation, the zpool the drive is in still shows as online. I have to issue a scrub command on the pool for it to show that the disk is no longer available. Isn't it suppose to detect it automatically?
 
Hmmm, mine is online now, so I can't easily check. If you boot OI and go into the HBA BIOS, is there any kind of hotplug setting?
 
Maybe not related here, but I found that to do hotswap right (mainly for auto-adding the drive back to the pool when plugged back in), in /etc/system, I had to set this:

set sata:sata_auto_online=1

or you need to use cfgadm which is a PITA.
 
Maybe not related here, but I found that to do hotswap right (mainly for auto-adding the drive back to the pool when plugged back in), in /etc/system, I had to set this:

set sata:sata_auto_online=1

or you need to use cfgadm which is a PITA.

Useful, I'll play with that once I get the zpool to recognize that the a disk is missing.

When I run dmesg I get warnings about an offline disk.
Aug 20 21:24:33 solaris scsi: [ID 107833 kern.warning] WARNING: /pci@0,0/pci1002,5a1f@b/pci1014,3b1@0/sd@1,1 (sd5):
Aug 20 21:24:33 solaris drive offline

But when I run cfgadm -al, it shows the disk is still connected.

:confused: Maybe a driver issue? Using the ones provided by lsi. Any settings in the card's bios for hot swapping? (9240-8i)
 
Last edited:
Hmmm, did you flash with IT firmware? I did. If you are still in raid mode, maybe that is it?
 
Hmmm, did you flash with IT firmware? I did. If you are still in raid mode, maybe that is it?

Had the card with the IT firmware but reverted back to stock. Which drivers are you using with it flash with the IT firmware?
 
Why did you revert? OI really wants discrete disks, so IR has no upside. With IT mode, it should be the default driver. Don't remember what the name is, but it works out of the box.
 
Why did you revert? OI really wants discrete disks, so IR has no upside. With IT mode, it should be the default driver. Don't remember what the name is, but it works out of the box.

I was not receiving temperatures on the drives using the IT firmware.
Also, I had issues with the disk path when cfgadm was called (it looked like a long string).
c4::w50024e9204db16f1,0 disk-path connected configured unknown


I followed this guide for installing the 9240-8i drivers which fixed the above issues.
 
Last edited:
The long string name is (I think) a WWW (world wide name.) You say "issues" - do you mean something was not working right, or it just looked weird. Here is my pool:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c8t50014EE2AEDF73CEd0 ONLINE 0 0 0
c6t50014EE2AF872299d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c7t50014EE2AEDF7498d0 ONLINE 0 0 0
c4t50014EE0AC01D8EDd0 ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
c10t50014EE204411A53d0 ONLINE 0 0 0
c9t50014EE0ABCEE0A9d0 ONLINE 0 0 0

Didn't notice anything about temperatures. Let me poke on mine...
 
What command line were you using with smartctl? I found to get anything in this mode, I had to use:

-d sat -T permissive
 
Let me rephrase and say that it may not be an issue with the long names but rather an annoyance for me.

Example of a mirror pool:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c3t50024E900359B6E8d0 ONLINE 0 0 0
c3t50024E9204D77474d0 ONLINE 0 0 0

You can see it's hard to tell what is the position of the drive (what sata cable it is connected with). I would have to issue a cfgadm -al which will show the following.

c4 scsi-sas connected configured unknown
c4::w50024e9204db16f1,0 disk-path connected configured unknown
c5 scsi-sas connected configured unknown
c5::w50024e9204d77474,0 disk-path connected configured unknown
c6 scsi-sas connected configured unknown
c6::w50024e900359b6e8,0 disk-path connected configured unknown
c7 scsi-sas connected configured unknown
c7::w50024e9204d9e1b0,0 disk-path connected configured unknown

You see my frustration? From here I can then tell that the two disk in that mirror pool are c5 and c6. This is with the card flashed with the IT firmware.
 
Yeah, I understand. WD drives at least have the WWW on the top of the drive, so you can tell, but this apparently is how IT mode works. For me it's a trade-off I am willing to go thru...
 
Yeah, I understand. WD drives at least have the WWW on the top of the drive, so you can tell, but this apparently is how IT mode works. For me it's a trade-off I am willing to go thru...

I'm using samsung 1tb drives so that also can contribute to the weird disk path. I'll give Nexenta a spin and see if I can reproduce the same.

Thanks for the help dan!
 
I don't think it's the drives, as I understand it, this is just how IT mode works. I have 600GB WD drives and get the funky names too. Oh well...
 
Hey danswartz,
I looked at an older thread of yours where you had a similar issue. I applied the "stmsboot -d" command and after a reboot I was able to see the drives with the c# attached to the WWN. So it wasn't the drives, you were right. At least now I know which drive is connected to which port. Before it was showing all drives with "c1t".

Thanks!
 
Last edited:
Hey dan,
I'm still having issues with the drives showing up as connected when I disconnect them (pool isn't showing degrade either).

If anyone is using OI/napp-it, can you please test pulling a drive out and seeing if the pool becomes degraded?

Did the test on eon storage and it works flawlessly, no idea why it's not on OI.

Thanks!
 
Not 100% sure of that. Might be necessary to recognize pulled drive too? Can't hurt to try?
 
You must have set as "AHCI" in bios. Check the SATA settings in bios, and see if it is "raid" or "ide legacy" or something else. I must be AHCI, otherwise hot swap will not work.

BEWARE this: if you have your boot disk as non AHCI, say "IDE legacy mode" and you change to AHCI - then most possibly your boot disk will stop work. I did this, changed from IDE to AHCI mode, and Solaris 11 Express did not boot anymore. This is a known old bug which has never been fixed. I know Windows needs some work after changing to AHCI, but it is doable. With Solaris changing to AHCI is quite messy, and you should avoid this. It is possible to repair Solaris installations, but quite messy and tricky. If you need to repair AHCI, it is written in the opensolaris.org forum somewhere. I saw the instructions there.

I changed from IDE to AHCI, and in the end, I had to reinstall everything. I could not repair it, it would not boot again. Later people posted instructions, but I never tried them.

Conclusion: if you need to change from IDE to AHCI in BIOS, be very prepared that you can never boot that install again. Very likely you need to reinstall everything and then run everything as AHCI.

I have now everything as AHCI, and it works fine.
 
I was talking about drives on a m1015 flashed with the IT firmware but thanks for the information brutalizer.
 
Rebooted the system with stmsboot -e (I guess stmsboot -d was breaking something) and now I see drives as unconfigured when disconnected. But the pool still shows as healthy.

Once I scrub the pool it becomes degraded.

Still can't get the the pool to auto degrade once a disk is disconnected though. :(
 
Last edited:
I notice the same issue. I currently have 13 drives on my system.

I only want 12 in the raidz2. Unplugged the single drive i didnt want in it. However still shows up in the drive configuration. How do i get to remove/auto degrade if i unplug a drive from the z2 config?

Current setup seems great, however still new to OS11 and learning.. cant figure out how to install drivers either but will keep working on it..

write 20.48 GB via dd, please wait...
time dd if=/dev/zero of=/Media/dd.tst bs=1024000 count=20000

20000+0 records in
20000+0 records out

real 18.4
user 0.0
sys 5.9

20.48 GB in 18.4s = 1113.04 MB/s Write

read 20.48 GB via dd, please wait...
time dd if=/Media/dd.tst of=/dev/null bs=1024000

20000+0 records in
20000+0 records out

real 16.9
user 0.0
sys 4.9

20.48 GB in 16.9s = 1211.83 MB/s Read
 
Figured it out! It was working this whole time. :eek:

I thought once the system realized a disk was offline it would automatically relay that to pool. This is not the case. When you create a pool and want to test drives (removing, ect). Be sure to write data to the pool after you remove a disk. The data would write to the pool and in doing so would check the disk(s) and relay any failure messages.

All this time I was testing a pool without disk activity so when I removed a drive from the pool, no errors would occur. :p
 
Last edited:
Huh, live and learn. I never saw this because my pool is backing an ESXi datastore, so there is always some activity :) Good to hear...
 
I notice the same issue. I currently have 13 drives on my system.

I only want 12 in the raidz2. Unplugged the single drive i didnt want in it. However still shows up in the drive configuration. How do i get to remove/auto degrade if i unplug a drive from the z2 config?


20.48 GB in 16.9s = 1211.83 MB/s Read

When you've created the pool with N drives, you can't shrink it - you would need to copy the data off and recreate the pool and copy back.
 
Back
Top