ZFS or Adaptec 51645?

D

DieTa

Weaksauce
Joined
Aug 17, 2009
Messages
114
Hey guys,

I'm currently in a dilemma. I'm not sure if I should set to ZFS or a "classic" Adaptec 51645.

I have bought a Yeong Yang Cube with place for 20 HDs.

I have here:
5x Samsung HD541UI 1.5TB
3x WD Green "EARS" (4k Cluster) 1.5TB
6x WD Green 1TB
2x Hitachi 1TB
3x WD Blue 640GB
+ a 32GB OCZ SSD for the OS

Yep, that means around about 22TB of storage ;)

The board is a ASUS M4A89GTD, the CPU an Athlon X4 605e and 4GB of ECC RAM (Kingston).

Also, I've for test here: Adaptec 51645 Controller w/ 4x SFF-8087 to SATA Breakout Cables

AND

I've here 2x Intel SASUC8i SAS Controllers w/ 2x Multilanes each. Means: In both cases, I would need to connect the 3x WD EARS drives to the mainboard itself. Because of 4k Clusters they are also incompatible w/ the Adaptec RAID controller.

I wanted to run:
- In case of HW RAID: Debian (which I know very well and like9
- In case of ZFS: FreeBSD or Solaris 11 Express

But I'm honestly not sure what makes MORE sense. The server should be a 99% fileserver only! No VMs or stuff like that. I also want to keep private stuff like our Aperture & iPhoto Libs on this server, so the ZFS arguments are very good regarding file security.

Just to bring in the financial part:
The Adaptec would cost me 670€, the 2x Intels only 230€.

What do you guys think?

Maybe I should highlight that this server has to serve decent speeds for video editing and of course, huge storage and also movies from out own DVDs & BRs. All clients are Macs, so SMB or NFS would be fine, AFP is too slow.

Thanks a lot in advance!

Ciao
Dennis
 
I will wait for someone else to chime in on ZFS performance.

I get 210Mbytes/sec with a 52445 on a pathetic Celeron M410 with 2 trunked GBE connections out of the server running Centos 5.4. My point is that that setup with Adaptec should have no problem saturating 2 GBE connections using SMB (Athlon X4 is way more powerful than my setup).
 
Hey Krobar,

many thanks for your feedback!

Meanwhile I've got a link from someone on another forum to Nexenta, which is what I want: It's Ubuntu (nearly Debian) with the OpenSolaris Kernel. So known environment with ZFS. Wow... need to test that :)
 
The LSI SAS3081E-R are functional equivalents of the Intel cards and are often on eBay for cheap. I bought one for $75, shipped.
 
I would easily choose ZFS, because hw raid such as adaptec card might corrupt your data. Here are lots of research papers about data integrity on hw raid: they all suck. Here is also research paper about ZFS which shows that ZFS DOES give you good protection. Research shows that XFS, ReiserFS, JFS, etc does not protect your data. Read this post if you are interested
http://hardforum.com/showpost.php?p=1036395000&postcount=49
 
Are you using 4x1GiB RAM or 2x2GiB RAM? If you go ZFS you might want to spend a little more on 8GiB DDR3 ECC instead, preferably the newer 1.35V memory.

Your SASUC8i would be suitable for ZFS; just make sure to flash it with SuperMicro or LSI IT firmware, so it behaves as normal HBA instead. Otherwise disk timeouts may still disconnect your disk.
 
Hi guys,

thanks for your feedback!

I've meanwhile installed the system which is now running on Nexenta Core (Ubuntu with OpenSolaris Kernel). I've installed the 2x Intel SASUC8i controllers, 12GB of ECC RAM and 2x Intel GBit Desktop/Pro 1000 CT cards.

Here are some photos of the setup: (of course, I'm currently copying files from the Onnto DataTale which is attached to my Mac mini to the server, so 8TB will built in once this is done ;) - currently installed: 8x1.5TB)

mediapool.jpg


hddin.jpg


view1.jpg


view2.jpg


backview.jpg


frontview.jpg


zahlen.jpg


@sub.mesa: Hmm interessting. I really have some issues, but only with my SSD where the OS is installed on. Maybe you can help me to understand WHAT went wrong on if it's a broken SSD or not.

Here's the log:
Code: 
  pool: syspool
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scan: scrub in progress since Sun Nov  7 19:27:21 2010
    250M scanned out of 4.50G at 83.2M/s, 0h0m to go
    0 repaired, 5.41% done
config:

        NAME        STATE     READ WRITE CKSUM
        syspool     DEGRADED     0     0     0
          c1d0s0    DEGRADED     0     0     0  too many errors

errors: No known data errors

Code: 
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: abort request, target=0 lun=0
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: abort device, target=0 lun=0
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: abort target, target=0 lun=0
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: reset target, target=0 lun=0
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: reset bus, target=0 lun=0
scsi: WARNING: /pci@0,0/pci-ide@11/ide@1 (ata1):
timeout: early timeout, target=0 lun=0
gda: WARNING: /pci@0,0/pci-ide@11/ide@1/cmdk@0,0 (Disk0):
Error for command 'read sector' Error Level: Informational

Sense Key: aborted command
Vendor: 'Gen-ATA ' error code: 0x3
gda: WARNING: /pci@0,0/pci-ide@11/ide@1/cmdk@0,0 (Disk0):
Error for command 'read sector' Error Level: Informational
Sense Key: aborted command
Vendor: 'Gen-ATA ' error code: 0x3

and:

Code: 
root@gigi:/backup# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c0t0d0 <ATA-SAMSUNG HD154UI-1118-1.36TB>
          /pci@0,0/pci1022,960b@3/pci1000,3140@0/sd@0,0
       1. c0t4d0 <ATA-SAMSUNG HD154UI-1118-1.36TB>
          /pci@0,0/pci1022,960b@3/pci1000,3140@0/sd@4,0
       2. c0t5d0 <ATA-SAMSUNG HD154UI-1118-1.36TB>
          /pci@0,0/pci1022,960b@3/pci1000,3140@0/sd@5,0
       3. c0t6d0 <ATA-SAMSUNG HD154UI-1118-1.36TB>
          /pci@0,0/pci1022,960b@3/pci1000,3140@0/sd@6,0
       4. c0t7d0 <ATA-SAMSUNG HD154UI-1118-1.36TB>
          /pci@0,0/pci1022,960b@3/pci1000,3140@0/sd@7,0
[B]       5. c1d0 <DEFAULT cyl 3889 alt 2 hd 255 sec 63>
          /pci@0,0/pci-ide@11/ide@1/cmdk@0,0[/B]
       6. c3d1 <WDC WD15-  WD-WMAVU274514-0001-1.36TB>
          /pci@0,0/pci-ide@11/ide@0/cmdk@1,0
       7. c4d0 <WDC WD15-  WD-WMAVU255804-0001-1.36TB>
          /pci@0,0/pci-ide@14,1/ide@0/cmdk@0,0
       8. c4d1 <WDC WD15-  WD-WMAVU218269-0001-1.36TB>
          /pci@0,0/pci-ide@14,1/ide@0/cmdk@1,0

Many thanks in advance!
 
Cool build. Makes me want to try zfs at home too.

Where did you get the YY case at? I have been looking for a new one for a couple of years now after I sold my last one :mad: BIG mistake.
 
Check the cable on your SSD. Also check SMART output for UDMA CRC Error Count (indicates cable errors).
 
schnell said:
Cool build. Makes me want to try zfs at home too.

Where did you get the YY case at? I have been looking for a new one for a couple of years now after I sold my last one :mad: BIG mistake.
Click to expand...

I got it from a local reseller here in Germany via eBay. Shipping took a few days because it was directly sent from Yeong Yang in Hamburg to my door step.

sub.mesa said:
Check the cable on your SSD. Also check SMART output for UDMA CRC Error Count (indicates cable errors).
Click to expand...

Ok - I'll try to replace the cable (it's a new one - that came with the mainboard, but of course this does not mean it's not damaged in any way).

I tried to get some SMART information:
Code: 
root@gigi:/backup# smartctl -a -d scsi /dev/rdsk/c1d0s0  
smartctl 5.39.1 2010-01-28 r3054 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

Standard Inquiry (36 bytes) failed [Inappropriate ioctl for device]
Retrying with a 64 byte Standard Inquiry
Standard Inquiry (64 bytes) failed [Inappropriate ioctl for device]
A mandatory SMART command failed: exiting. To continue, add one or more '-T permissive' options.

I'll reply when the next folder is copied and I will shut the server down, so I can replace the cable.

PS. I also noticed that this happens around about every 15-20min.
 
I've just replaced the cable and wait 30mins in idle, no issues, but once I run "zpool status" 5x, the fifth time, I get the same error message and the system lags.

I think the SSD is damaged :/
 
I tried, here's the result:

Code: 
root@gigi:/backup# smartctl -a -d scsi /dev/rdsk/c1d0s0 -T permissive
smartctl 5.39.1 2010-01-28 r3054 [i386-pc-solaris2.11] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net

Standard Inquiry (36 bytes) failed [Inappropriate ioctl for device]
Retrying with a 64 byte Standard Inquiry
Standard Inquiry (64 bytes) failed [Inappropriate ioctl for device]
Request Sense failed, [Inappropriate ioctl for device]

Error Counter logging not supported
Device does not support Self Test logging

This data is not very helpful. :(

I also recognized some issues with my 3x WD15EARS (4k Cluster HDs). They are now also marked a "error". But that's not dramatic, I will just put it back into one of my Onnto DataTales and will connect it to the server via eSATA. That's ok for backups (RAID5, I think I can't do JBOD in this case).
 
You must log in or register to reply here.
Back
Top