napp-it gui - drives not listed

dsumike · Jun 11, 2012

I have two SMC216 24-bay disk trays each connected to a separate LSI 9201-16e controller, but I am experiencing an interesting issue where the disks are not being presented in the GUI for use.

If I navigate to home --> disks I clearly see my two hard drives listed (2x 300gb, 10k SAS 6Gbps):

c11t5000C50033660355d0
c12t5000CCA00AA0EA99d0

However, if I navigate to home --> disks --> details there are no disks showing up (either in use nor available).

Knowing the disk ID's, I can manually create a pool using these drives from the console though:

Code:

# zpool create tank mirror c11t5000C50033660355d0 c12t5000CCA00AA0EA99d0

The command completes successfully and the pool is useable pool.

user@openindiana:/tank# df -h | grep tank
tank 274G 31K 274G 1% /tank

What's weird is that once I create the pool from the console the disks then show up in the "disks in use" list, however if I navigate to the home --> disks --> Smartinfo page, I get no readings from these disks.

If I manually type:

# /usr/sbin/smartctl -a -d scsi -T permissive /dev/rdsk/c11t5000C50033660355d0

I can get the smart info that should have parsed correctly to get Smart Info.

I did a little port swapping, and the drives are always recognized on the main home --> disks page, but it's like the napp-it gui won't display the drives if the controller sits on a card that has two digits (ie cXX vs cX as the first part of the label).

In my two disks, this is c11 and c12. When I get on to a port that is c1 through c9, everything seems to display just fine. I could just swap ports for now, but the long term goal is to use the 4 remaining ports. So I would like to figure out what would cause this.

Does anyone know if there is something in the GUI that would prevent cXX controllers from displaying?

_Gea · Jun 11, 2012

dsumike said:
Does anyone know if there is something in the GUI that would prevent cXX controllers from displaying?

In next napp-it, I have modified the library for WWN disk names detection.
You may download the new lib and replace the files in
/var/web-gui/data/napp-it/zfsos/_lib/

Please report if that helps with your problem.
http://www.napp-it.org/doc/lib.zip

dsumike · Jun 11, 2012

Thanks for the quick response. Unfortunately the updated files didn't seem to have any impact.

I have been trying to see how napp-it is reading the drive by playing in the get-disk-smart.pl file. If I add a print line right after the $id variable is set as it loops through the disks, I can see that it is finding each disk with an s0 at the end of the disk rather than a d0:

c11t5000C50033660355s0 ... c12t5000CCA00AA0EA99s0

When the smartctl command then runs, it's getting the following output:

Code:

smartctl -a -d scsi -T permissive /dev/rdsk/c11t5000C50033660355s0 smartctl 5.42 2011-10-20 r3458 [i386-pc-solaris2.11] (local build) Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net Smartctl open device: /dev/rdsk/c11t5000C50033660355s0 failed: No such device

This explains why it is not seeing any Smart Info. If I run the command from the console simply replacing the s0 with d0, I get results.

I'm just not seeing where and whyy napp-it is trying to use these as s0 rather than d0.

_Gea · Jun 11, 2012

dsumike said:
I'm just not seeing where and whyy napp-it is trying to use these as s0 rather than d0.

There is no consistent behaviour of smartmontools and "auto"-setting is not working
with Solaris. All my disks are ending with d0 and on my controllers, i need to add s0
to get proper smart values (after manually trying sat, scsi and ata settings)

Maybee a second run of the whole detection with d0 at the end is needed when there
was no success with s0

dsumike · Jun 11, 2012

_Gea said:
There is no consistent behaviour of smartmontools and "auto"-setting is not working
with Solaris. All my disks are ending with d0 and on my controllers, i need to add s0
to get proper smart values (after manually trying sat, scsi and ata settings)

Maybee a second run of the whole detection with d0 at the end is needed when there
was no success with s0

Yeah. The smart info is really a low priority, what I'm more concerned about is that the disks aren't showing up in the "unused" list, which means they do not show up in the Disk Details list are thus are also not able to be selected in the "Add vdev" menu.

This is a show-stopper for quickly adding disks to the storage pool using the GUI.

_Gea · Jun 11, 2012

Its hard to fix a problem that i do not have with my disks and controllers.
The detection of available disks is done in the two files in http://www.napp-it.org/doc/lib.zip

These libraries are always executed whenever a menu under disk is called.
The files check and collect disk infos in the hash %disk (see content when edit is enabled) prior of a menu action.

dsumike · Jun 11, 2012

I understand. It's hard to debug a problem blind. I have to imagine the problem is somewhere in how the system is seeing the disks. Using cfgadm, I see that the disks are presented to OpenIndiana without the d0.

Code:

# cfgadm -al
c11::dsk/c11t5000C50033660355  disk         connected    configured   unknown
c12::dsk/c12t5000CCA00AA0EA99  disk         connected    configured   unknown

This doesn't really help me much, but I'm sure this is part of the problem. It seems to reference the disk differently at different parts of the OS.

Of course, where I go from here I don't know yet...

Stanza33 · Jun 11, 2012

dsumike said:
I have two SMC216 24-bay disk trays each connected to a separate LSI 9201-16e controller, but I am experiencing an interesting issue where the disks are not being presented in the GUI for use.

How are the JBOD's connected to each controller?

Single Cable or Dual Cables?

If Dual, what happens when you use only one cable?

If single cable, have you disabled multipathing for the controller?

.

dsumike · Jun 12, 2012

Stanza33 said:
How are the JBOD's connected to each controller?

Single Cable or Dual Cables?

If Dual, what happens when you use only one cable?

If single cable, have you disabled multipathing for the controller?

.

The goal is to use multi-pathing, so I haven't tried disabling it. However, I have tried three different configurations:

Controller A, Port 1 --> Chassis A, Port 1
Controller B, Port 1 --> Chassis A, Port 2

Controller A, Port 1 --> Chassis A, Port 1
Controller A, Port 2 --> Chassis A, Port 2

Controller A, Port 1 --> Chassis A, Port 1

In all three configurations the drives appear to be presented the same way.

I have the the second configuration in a different system with a 2-port card instead of 4-port, using the same JBOD chassis, and everything works fine... so it's curious why it's causing so much problems all of a sudden.

dsumike · Jun 12, 2012

dsumike said:
I have tried three different configurations:

Controller A, Port 1 --> Chassis A, Port 1
Controller B, Port 1 --> Chassis A, Port 2

Controller A, Port 1 --> Chassis A, Port 1
Controller A, Port 2 --> Chassis A, Port 2

Controller A, Port 1 --> Chassis A, Port 1

In all three configurations the drives appear to be presented the same way.

I should clarify, I ran into a different problem with option 1 (it saw the drives twice), but options 2 and 3 had the same results.

I went ahead and disabled MPxIO in /kernel/drv/mpt_sas.conf and did a stmsboot -D mpt_sas -d to try and disable multipathing, but the even after disabling the drives are still presented the same way.

dsumike · Jun 12, 2012

Well I have continued to push forward with the implementation. I can't seem to figure out anything that says why the disk id's are being presented with and without the d0 at different spots. Using iostat -Ensr I see the drives with d0's,

However, under Disk Details if i click on the %disk option I see a larger version of the following:

Code:

c11t5000C50033660355_busy	-> 	'configured'
c11t5000C50033660355_online	-> 	'connected'
c11t5000C50033660355_phys	-> 	'unknown'
c11t5000C50033660355d0_cap	-> 	'300.00 GB'
c11t5000C50033660355d0_error	-> 	'Error: S:0 H:0 T:0'
c11t5000C50033660355d0_ill_request	-> 	'0'
c11t5000C50033660355d0_info	-> 	''
c11t5000C50033660355d0_product	-> 	'ST9300603SS'
c11t5000C50033660355d0_recoverable	-> 	'0'
c11t5000C50033660355d0_revision	-> 	'B53B'
c11t5000C50033660355d0_vendor	-> 	'IBM-ESXS'

What's interesting is that part of the disk is referenced without d0's and the rest is with d0's... ultimately this is probably the source of the problem... somewhere.

In the mean time, in case anyone else has ran into a similar issue, I was able to write a small fix for napp-it by updating the get-disk.pl file.

In the get-disk.pl on around line ~70 (search for "#unused") you'll need to change block of code within the if-statement to the following:

Code:

               #unused

		$key="$id\_pool";
                if ($disk{$key} eq "") {
                    $key="$id\_busy";
			# test to see for non-d0
                	$myid=$id;		# make a separate check
                	$myid=~s/d0//;		# if the disk is configured as non-d0
                	$mykey="$myid\_busy";	#
		        #print "$id <b>$disk{$key}</b>  $disk{$mykey}<br>";
			# add an or in the following if-check.
                    if ($disk{$key} eq "configured" | $disk{$mykey} eq "configured") {
                        if (!($disk{'unused'}=~/$id\b/)) {
                              $disk{'unused'}.="$id\t";
                        }
                        if (!($disk{'replaceable'}=~/$id\b/)) {
                              $disk{'replaceable'}.="$id\t";
                        }

                    }  else {
                       # opt remove from used
                       $disk{'used'}=~s/$id\t*//;

                    }
                }

I had to comment out use strict; at the top of the file, as I haven't found where I need to add the global variables to yet to create the $myid and $mykey variables.

Outside of that, this code update allows napp-it to properly display the disks as unused, and they become available to add as a vdev to a pool again. This has made the web-gui useable again, which is nice.

I may need to do some minor tweaking for smart info still, but at least I have a solution that works for both these drives in this chassis, as well as in a standard installation as well.

_Gea · Jun 12, 2012

dsumike said:
Well I have continued to push forward with the implementation. I can't seem to figure out anything that says why the disk id's are being presented with and without the d0 at different spots. Using iostat -Ensr I see the drives with d0's,

However, under Disk Details if i click on the %disk option I see a larger version of the following:

Code:

c11t5000C50033660355_busy -> 'configured' c11t5000C50033660355_online -> 'connected' c11t5000C50033660355_phys -> 'unknown' c11t5000C50033660355d0_cap -> '300.00 GB' c11t5000C50033660355d0_error -> 'Error: S:0 H:0 T:0' c11t5000C50033660355d0_ill_request -> '0' c11t5000C50033660355d0_info -> '' c11t5000C50033660355d0_product -> 'ST9300603SS' c11t5000C50033660355d0_recoverable -> '0' c11t5000C50033660355d0_revision -> 'B53B' c11t5000C50033660355d0_vendor -> 'IBM-ESXS'

What's interesting is that part of the disk is referenced without d0's and the rest is with d0's... ultimately this is probably the source of the problem... somewhere.

In the mean time, in case anyone else has ran into a similar issue, I was able to write a small fix for napp-it by updating the get-disk.pl file.

In the get-disk.pl on around line ~70 (search for "#unused") you'll need to change block of code within the if-statement to the following:

Code:

#unused $key="$id\_pool"; if ($disk{$key} eq "") { $key="$id\_busy"; # test to see for non-d0 $myid=$id; # make a separate check $myid=~s/d0//; # if the disk is configured as non-d0 $mykey="$myid\_busy"; # #print "$id <b>$disk{$key}</b> $disk{$mykey}<br>"; # add an or in the following if-check. if ($disk{$key} eq "configured" | $disk{$mykey} eq "configured") { if (!($disk{'unused'}=~/$id\b/)) { $disk{'unused'}.="$id\t"; } if (!($disk{'replaceable'}=~/$id\b/)) { $disk{'replaceable'}.="$id\t"; } } else { # opt remove from used $disk{'used'}=~s/$id\t*//; } }

I had to comment out use strict; at the top of the file, as I haven't found where I need to add the global variables to yet to create the $myid and $mykey variables.

Outside of that, this code update allows napp-it to properly display the disks as unused, and they become available to add as a vdev to a pool again. This has made the web-gui useable again, which is nice.

I may need to do some minor tweaking for smart info still, but at least I have a solution that works for both these drives in this chassis, as well as in a standard installation as well.

The more people are familiar with the code, the better.
You should keep the use strict, just predeclare your vars prior use like
my (var1,$var2);

I will check your modifications with my disks and add it to the source.
Thanks.

dsumike · Jun 12, 2012

_Gea said:
The more people are familiar with the code, the better.
You should keep the use strict, just predeclare your vars prior use like
my (var1,$var2);

I will check your modifications with my disks and add it to the source.
Thanks.

That seems easy enough. I'm working on updating the smart checks as well. It's a lot of checking-twice, once for drives that read correctly with d0's and if no results, then check again without. I'll try and post some code snippets back here once I get everything going.

smellyeagle · Aug 7, 2012

I have the same issue as dsumike. Could it be something as simple as too many characters between the starting 'c' and the following 't'?

for example, these are my 6 disks:
c10t5000CCA01A92E495d0
c5t5000CCA01A92E4C9d0
c6t5000CCA01A93E805d0
c7t5000CCA01A93E339d0
c8t5000CCA01A9428ADd0
c9t5000CCA01A9A2F69d0

The only one that doesn't work is the first one and has 2 digits between the 'c' and 't', while the rest have only 1 digit. The same pattern followed with dsumike.

This could be a coincidence, but I figured it was worth noting my observation. Glad to know I can do things manually by ID though.

mephistopoa · Aug 8, 2012

I had similar problems, drives now showing up and strange drive names. I gave up on napp-it and I'm now just using vanilla solaris 11

_Gea · Aug 8, 2012

some problems with WWN numbers are fixed in newest napp-it
the remaining problems:

output of iostat -Enr (used in menu disk) and
cfgadm -avl (used in menu disk details) are not identical (ex missing d0 at the end)

disk firmware problems (ex some Dell disks)

Serial from cfgadm and smartmontool are not exact identical (SAS2 monitoring problem)
Serial of several disk is the same (SAS2 monitoring problem)
WWN of several disks is identical (i have this with some of my SSD's, unfixable problem)

If I can handle problems with special configs I try to implement it.
But it is difficult to support all specials. (The reason why Nexenta or Oracle use a HCL)

or indeed, you can try to use CLI commands for these disks

napp-it gui - drives not listed

dsumike

Weaksauce

_Gea

Supreme [H]ardness

dsumike

Weaksauce

_Gea

Supreme [H]ardness

dsumike

Weaksauce

_Gea

Supreme [H]ardness

dsumike

Weaksauce

Stanza33

Gawd

dsumike

Weaksauce

dsumike

Weaksauce

dsumike

Weaksauce

_Gea

Supreme [H]ardness

dsumike

Weaksauce

smellyeagle

n00b

mephistopoa

Weaksauce

_Gea

Supreme [H]ardness