ZFS - State Unavail/offline disk RAIDZ problem

[xp0]

Hello,


Long story made short ... I had two drives in a 4 disk raidz that started having problems. The pool was working fine, all my data was still accessible. I decided I would replace one, scrub, let it replace the disk, etc than replace the other. Well, after offline-ing one drive and turning off the system so I could physically replace it, it kept rebooting with a ZFS Panic error. I went into single user mode, edited rc.local to stop ZFS from booting, and able to make a few changes. So anytime I tried to do anything ZFS related, it would Panic and reboot. I figured it was the other bad drive (/dev/ad18) so I removed that drive and now I can boot with ZFS enabled, check zpool status, etc.

So here is my dilemma ...

I have a pool which has one drive unavailable (/dev/ad18) and one drive that is offline (/dev/ad12). I can not online the drive because the state is UNAVAIL. I have a feeling if I can somehow online this drive, I can get things working.... so my question is, is there anyway I can possibly online this drive? reconnecting /dev/ad18 will not work because if I do, any command I run will just cause the system to panic and reboot.

Is there someway to edit the zdb to online the drive that is offline?
is there someway to "copy" /dev/ad18 to another drive and make ZFS think its the original drive, just without errors, and see if I can replace that drive to bring that back online?

Any help will be greatly appreciated!! If you need any additional information, commands run, etc... please let me know

 

[xp0]

Anyone have any ideas or thoughts? Or should I just scrap all the data, rebuild, and be more careful?


Thanks

 

[MentatBOB]

Anyone have any ideas or thoughts? Or should I just scrap all the data, rebuild, and be more careful?


Thanks

I'm still new to ZFS so I'm not sure about copying data but one thing you can try in a last ditch effort is to chill the drive that you think you might be able to recover data from. Wrap it up really well with a towel, stick it in a plastic bag and then throw it in the freezer for a while. The key is to make sure that it doesn't get any moisture on it, it needs to stay dry. Sometimes it works, sometimes it doesn't.

 

[xp0]

Thanks - I can give that a try, it has worked in the past for me ... I am just not sure this drive is to that stage yet. It doesnt click or make noise ... its just a few messages like:

ad12: FAILURE - READ_DMA48 status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=3907028224
ad12: FAILURE - READ_DMA48 status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=3907028736
ad12: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=1<ILLEGAL_LENGTH> LBA=256
ad12: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=768
ad12: FAILURE - READ_DMA48 status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=3907028224
ad12: FAILURE - READ_DMA48 status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=3907028736

 

[brutalizer]

I am really surprised that you have not lost all your data yet?

You are using raidz1 (similar to raid-5) which allows ONE disk to malfunction. raidz1 nor raid-5 can not handle problems on two disks. You have problems with two disks - the raidz1 should not be able to handle this scenario. But you can still access some data??

Your raidz1 setup is not able to handle two disk failures. So, I suggest you backup all data as fast as you can.