ZFS - Disk 'IDs'

[ldoodle]

Hiya,

I've had 2 failed disks in my array (no data loss), so I thought i'd take this oppertunity to backup all data, destroy the pools, remove Solaris 11 Express and re-install from scratch with full Solaris 11.

My IDs as reported by 'iostat -en' are, with all slots populated:

Controller 1: c5t7d0 - c5t14d0
Controller 2: c6d8d0 - c6t14d0

So if I do a dd if=/dev/dsk/disk_id of=/dev/dsk/null bs=8192k count=10 on each disk, the led flashes up long enough for me to see they're in order (meaning that slot 1 is the lowest ID and slot 15 is the highest ID). The failed drives are in slots 10 (c6t9d0) and 14 (c6t13d0) so I took these out and put working drives in their places, expecting them to use these same IDs thinking they're generated by port number on the controller, but they don't. The 2 failed disk IDs are no longer shown in iostat -en as I have physically removed them, but the drives that were in slots 13 and 15 and now where the failed drives were have maintained their original IDs. So if I do a new dd to flash up their led's they're out of order.

Is it because the first boot from USB initially generated them in the old slots and remembers them?

Does that makes sense, and is there a way to change this behaviour. Or better yet relabel them in sequential order - disk1, disk2, disk3 etc

 

[danswartz]

This is all kinda magic to me. I just assumed I can't count on the IDs ever making any kind of sense.

 

[_Gea]

Hiya,

I've had 2 failed disks in my array (no data loss), so I thought i'd take this oppertunity to backup all data, destroy the pools, remove Solaris 11 Express and re-install from scratch with full Solaris 11.

My IDs as reported by 'iostat -en' are, with all slots populated:

Controller 1: c5t7d0 - c5t14d0
Controller 2: c6d8d0 - c6t14d0

So if I do a dd if=/dev/dsk/disk_id of=/dev/dsk/null bs=8192k count=10 on each disk, the led flashes up long enough for me to see they're in order (meaning that slot 1 is the lowest ID and slot 15 is the highest ID). The failed drives are in slots 10 (c6t9d0) and 14 (c6t13d0) so I took these out and put working drives in their places, expecting them to use these same IDs thinking they're generated by port number on the controller, but they don't. The 2 failed disk IDs are no longer shown in iostat -en as I have physically removed them, but the drives that were in slots 13 and 15 and now where the failed drives were have maintained their original IDs. So if I do a new dd to flash up their led's they're out of order.

Is it because the first boot from USB initially generated them in the old slots and remembers them?

Does that makes sense, and is there a way to change this behaviour. Or better yet relabel them in sequential order - disk1, disk2, disk3 etc

short id's like c1t0do refer to a physical controller slot but also identifies members of a pool. If you just remove/replace a disk, this id is already 'used' and you have a problem.

If you need to replace a damaged disk, do a proper disk replace damaged -> new from another slot.
Otherwise you may need a reboot to get proper id's. With some controllers you even need a export-import to have it re-working properly.

Long id's like c3t600039300001EA56d0 are usually real disk-id's (from manufacturer) and are displayed with a modern SAS2 controller. If you move such a disk to a new slot, it keeps its number - but you need to write it down or you need a ses backplane with special software to identify the slot (beside the not alway working dd method)

 

[ldoodle]

Is it possible to reset or clear the memory that remembers them? They're all over the shop at the moment so identifying what slot contains what disk is stupid.

How does it even work. I loaded the Solaris 11 .usb image to a new memory stick and booted thinking they were saved on the first memory stick somewhere.