ZFS Pool Degraded

[BigPaulie]

Hey everyone,

I have been a lurker for a while... and have really enjoyed the community, but now I need your help.

I have setup a Solaris Express 11 file server with 5 x 2TB Seagate Drives.
Before I setup the server I did a long test using Seatools for DOS, all the drives were factory new and passed without error.

I have setup the server with raidz1 (raid5) and copied 265gb of data to the pool over samba using Solaris's built in cifs.

When I checked the status of the pool, because I plan on transferring another 3 TB of data soon, I saw that the pool was degraded because one of my drives was faulted.

Here is the output after a scrub:

root@FileServer:~# zpool status -v dtank0
  pool: dtank0
 state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
        Sufficient replicas exist for the pool to continue functioning in a
        degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
        repaired.
 scan: scrub repaired 0 in 0h10m with 0 errors on Sun Oct 30 13:25:44 2011
config:

        NAME        STATE     READ WRITE CKSUM
        dtank0      DEGRADED     0     0     0
          raidz1-0  DEGRADED     0     0     0
            c8t1d0  ONLINE       0     0     0
            c8t2d0  ONLINE       0     0     0
            c8t3d0  FAULTED      1   819     0  too many errors
            c8t4d0  ONLINE       0     0     0
            c8t5d0  ONLINE       0     0     0

errors: No known data errors

As you can see there seems to be no data errors. Basically I don't know how to proceed, and would really appreciate some advice, should I replace the drive, or clear the errors? I would prefer not to replace the drive but I can if necessary.

Thank you for your replies.

-Paulie

 

[hotzen]

Replace the drive, resilver the pool, check the degraded disk intensively and possibly use it for just another offline backup

 

[BigPaulie]

Okay, if I replace the drive, what do you recommended to intensely test the faulted drive?

-Paulie

 

[hotzen]

GSmartControl [1] or any other good SMART program , use the long scan which will check
every sector. Then check the smart attributes for pending reallocations and bad sectors, etc.

I dont know if you can trust this disk again and I think the write-failures are due to corrupted sectors and the disk gave up writing. But I am not sure, perhaps there are other minor causes for ZFS reacting like that. However I would not trust that disk too much again...

If it looks ok, I would TrueCrypt everything, put my most important stuff on it and hand it some friend for an offsite backup. Be prepared that the disk is failing on you again if you need it most, but better than nothing.

[1] http://gsmartcontrol.berlios.de/home/index.php/en/Home

 

[drescherjm]

I would do a 4 pass badblocks read write test then look at the SMART.

 

[young_einstein]

Have you had any crashes / power failures / other weird events that might have contributed to any read/write errors?

Since you're effectively running without any redundancy at all right now, I probably wouldn't take any chances with a potentially faulty drive.

Replace the disk ASAP,and double check your cabling etc while you're at it, then you'll have time to do some proper checking of the disk (without worrying about the potential of losing your entire pool).

If everything does check out OK, I'd consider adding the disk back into the pool as a hot spare.

 

[brutalizer]

If you get read/write error on another drive now, all your data is lost. Repair your raid as fast as possible. Dont do anything with it. Raidz1 is very dangerous, you might loose all your data now if you get read/write errors.