Project: No Name File Server. 56K beware

Discussion in 'Worklogs' started by axan, Aug 30, 2008.

  1. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Ok so it was time to upgrade my file server. My old one had 5TB in 2 raid 5 arrays(15 drives) and with 300 gb of space left it was time to expand. I couldn't come up with a good name for the new box(all the good ones are already taken ) so for now it will be "the no name" file server, if you have any suggestions for the name, post them.

    Part List

    Mobo Asus DSBF-DE
    CPU Intel Xeon e5405 2Ghz Quad Core
    Ram 2 x Kingston 2GB FBDIMM
    Optical Drive Linteon slim dvd drive
    OS Drive 80GB Seagate 7200.10 Sata Drive
    Raid Card Areca 24 port SAS Card with 2gb of cache and Battery Backup later replaced with Areca 1280ML due to incompatibility with the Seagate hdds
    Raid Storage Drives 20 x Seagate 7200.11 1TB drives in Raid 6
    Case Norco 4020 with 20 hot swap hdd bays
    PSU Corsair 1000HX

    Time for some pics:
    If you want to see full res version click HERE

    [​IMG]
    [​IMG]

    Originally I was going to go with dual cpus, but I realized that 2 quad cores in a file server is way too much of an overkill even for [H]

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
  2. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Some pics of the case:

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  3. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    More pics:
    Mobo:
    [​IMG]
    [​IMG]

    FBDIMM ECC RAM
    [​IMG]
    [​IMG]

    2gb ecc unbuffered ram for areca cache
    [​IMG]

    Starting to put things together
    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
  4. swatbat

    swatbat [H]ardForum Junkie

    Messages:
    12,805
    Joined:
    Apr 25, 2001
    What OS you going to run on it?

    Looking good so far. I'd personaly run a raid 1 for the boot os but other then that should serve you well.
     
  5. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Areca 24 port Raid controller

    [​IMG]

    [​IMG]

    [​IMG]

    External SAS port and Ethernet port (for management)
    [​IMG]

    [​IMG]

    Card with upgraded cache memory
    [​IMG]

    The card comes with sas to sas cables so in order to connect sata drives we need new cables (about ~$25 each)
    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  6. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Starting to put things together

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    At this point it was getting late and I was going to take a brake, as i was putting the top cover on the case I realized that the thermalright HR-01 X won't fit in this case :(
    Oh well thermalright ultima-90 on order and for now I'll run the case with the lid off.
     
  7. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    OK this was day 2. Time to hook up power and get this thing up and running.

    The 80mm fans that came with the case are actually pretty good, move decent air and are fairly quiet.
    Unfortunatelly 1 of them had broken power connector and the 2 exhaust fans had molex connectors instead of 3 pin. So they got the axe and got replaced with some scythe fans.

    [​IMG]

    Here's what will power this beast. Corsair really outdid themselves on this psu. It's awesome.
    [​IMG]

    [​IMG]


    Ok cable management on this thing is a bitch but I did the best I could
    [​IMG]

    [​IMG]

    The mini sas to sata cables
    [​IMG]

    [​IMG]

    Battery backup unit for the raid card
    [​IMG]

    OK the whole thing up and running, final shots:
    [​IMG]

    [​IMG]

    [​IMG]

    Right now I'm running stability test and building a 5 drive raid 5 array to play with.
     
  8. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Undecided for os yet, most likely gentoo with xfs as file system.
    As for raid 1 for boot there is really no room to mount an extra drive :D. And once I have the os setup and running all the services and stuff I want I will clone the os drive and keep it as backup in case this one dies.
     
  9. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Some thoughts on the Norco 4020 case. You get what you paid for, the case is not bad. It has plenty of space and lets you have 20 swappable hdds; however, the hdd trays are pretty poor quality, sometimes it's hard to put the tray back into the case, you kinda have to wiggle it around to make it align properly. Definitely no comparison to Supermicro quality. Given it's $300 price tag it's a very good case but if there was a version of Supermicro 846 without the loud ass redundant psu and fans for ~$600 I would upgrade in a heart beat.
     
  10. Dangman

    Dangman Ninja Editor SuperMod

    Messages:
    46,062
    Joined:
    Dec 15, 2005
    Nice hardware! Another new member to the Norco club :D

    Anyway some ideas for server names:
    Deep Thought
    Earth
    42
    The Library
    The Beast
    The Repository
    The Archive
    The Index
    The Depository
    Black Box
    Collection
    Kermit the Frog
    Memory Alpha
     
  11. gjvrieze

    gjvrieze [H]ard|Gawd

    Messages:
    1,423
    Joined:
    Sep 8, 2006
    Very nice build (almost patting myself on the back since you have almost a copy of my Plutonium 4!!!

    Congrats and enjoy the new space!!
     
  12. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    yes forgot to mention that, lots of credit goes to ockie, gjvrieze and others from whom I 'borrowed' some ideas :D
     
  13. ND40oz

    ND40oz [H]ardForum Junkie

    Messages:
    11,444
    Joined:
    Jul 31, 2005
    Did that board ship with the latest bios for the 45nm chips? I had forgotten that some of the 5000 series had been updated for them.

    Also, why'd you go with 533 mhz ram, you're running slow for the 1333 fsb. Throw them in the white slots so you're in dual channel and pick up 2 more to run quad channel.
     
  14. thecodemeist3r

    thecodemeist3r n00b

    Messages:
    58
    Joined:
    May 31, 2008
  15. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    yes it came preloaded with bios v1004 which is the newest bios asus has on their page. It works with my 45nm chip no problem. As for the ram, it doesn't matter, it's a file server even 4gb of ram is overkill for this thing, my old server had 2gb and I never saw more then 10% mem utilization.
     
  16. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,977
    Joined:
    Feb 20, 2001
    Very nice and welcome to the club!
     
  17. criccio

    criccio Fully Equipped

    Messages:
    12,381
    Joined:
    Mar 26, 2008
    Gotta love hardware porn... especially storage porn!
    I very soon plan to replicate this build, only with normal SATA PCIe cards and use WHS.
     
  18. Blue Fox

    Blue Fox [H]ardForum Junkie

    Messages:
    11,685
    Joined:
    Jun 9, 2004
    What is performance like with the 1680ix-24 with the SATA drives? I might sell my 1280ML later down the line since that thing supports expanders, so I'm curious...
     
  19. Met-AL

    Met-AL [H]ardness Supreme

    Messages:
    7,876
    Joined:
    Apr 9, 2002
    Is it good for raid cards to bend under pressure from the cables?
     
  20. swatbat

    swatbat [H]ardForum Junkie

    Messages:
    12,805
    Joined:
    Apr 25, 2001
    Anyway to mount the drive where the floppy drive could go? You could have always gone with 2 laptop drives in the single 3.5 inch bay but I would have tried for the second 3.5.

    Either way looking good.
     
  21. Kubrick

    Kubrick [H]Lite

    Messages:
    98
    Joined:
    Feb 18, 2008
    Name suggestions:
    Storage Atlas II
    Tera Server

    I really want this case!!
     
  22. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,977
    Joined:
    Feb 20, 2001
    Not really, I just looked back at the pics and saw that. Might want to move the cables to stop that, it does look pretty bad.
     
  23. Ockie

    Ockie *** Self Proclaimed Storage King ***

    Messages:
    17,991
    Joined:
    Mar 2, 2005
    Welcome to the club
     
  24. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    The picture is kinda deceiving there is virtually no pressure on the card. The cables are pretty stiff cuz technically that's 4 sata cables in fancy sleeving. You can sorta bend them to shape and they stay that way.
     
  25. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    hmm Tera Server I kinda like it, might use it, thanx
     
  26. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    I'm in middle of installing gentoo x64, my first time doing 64 bit system so it might take a while :D When that is done I'll post some benchmarks of various configurations.
     
  27. mjz_5

    mjz_5 2[H]4U

    Messages:
    3,612
    Joined:
    May 24, 2001
    if they made the case 4 inches longer, would allow for easier cable management
     
  28. gjvrieze

    gjvrieze [H]ard|Gawd

    Messages:
    1,423
    Joined:
    Sep 8, 2006
    Yes, with the Extended ATX mobo, it is tight....
     
  29. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Little update:

    Installing Gentoo 64 bit went very smoothly. I got it up and running with the following services:
    NFS
    Samba
    ProFTPD
    MediaTomb

    So software side everything is going great but I had a glitch with my array. Last night I set my array to expand from 5 disk raid 5 to 6 disk raid 6 (Just messing arround) Today morning I was woken by the beeping of the Areca controller. I looked at the event log and there was bunch of time outs and then it basically said migration failed and no matter what I did I couldn't get the array back online. Now I'm not sure what caused it, the box wasn't on ups and there might have been a power glitch. So right now I put it on apc 1500va ups and it's initializing new 5 disk raid 5, I plan to put some data on it and try to migrate it to raid 6 again and see how that goes.
     
  30. cereal_killer24

    cereal_killer24 Limp Gawd

    Messages:
    241
    Joined:
    Nov 26, 2005
    Name suggestion:
    Big box with lots of hds
     
  31. colinstu

    colinstu 2[H]4U

    Messages:
    3,563
    Joined:
    Oct 11, 2007
    That RAID card is going a little bendy on you... I'd move some of those cables around to prevent them from warping the card like that.
     
  32. EnderW

    EnderW [H]ardForum Junkie

    Messages:
    10,889
    Joined:
    Sep 25, 2003
    looks nice
    thanks for the worklog and lots of pics
     
  33. Greg-O

    Greg-O Limp Gawd

    Messages:
    252
    Joined:
    Nov 22, 2006
    looks nice, all I can say is I wish i had the money to be able to throw something like that together... :rolleyes:
     
  34. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    Update.

    Having issues with the raid array. Basically the drives keep timing out and destroying the array. Talked to Seagate but they were no help whatsoever. I also contacted Areca, they are aware of the issue and trying to come up with a solution. The problem is SAS processor is not compatible with the Seagate drives. Seagate released a firmware fix for the NS series drives but nothing for AS. As of right now they have no ETA for a fix.
    I need my array operation now so I'll be replacing the raid controller with Areca ARC1280ML.
     
  35. brokenarrow03

    brokenarrow03 [H]ard|Gawd

    Messages:
    1,655
    Joined:
    Nov 20, 2007
  36. axan

    axan [H]ard|Gawd

    Messages:
    1,935
    Joined:
    Nov 5, 2005
    ya it sux, basically wasted a week trying to get it to work. But the new raid card is on order and should be here Monday/Tuesday. Areca Tech assures me that since ARC1280ML is a sata card I won't have any issues.
     
  37. brokenarrow03

    brokenarrow03 [H]ard|Gawd

    Messages:
    1,655
    Joined:
    Nov 20, 2007
    But after dropping that kinda cash and it not working. You'd think they would do more for ya considering these are $1200+ cards...or Seagate considering 20 drives at ~$160 each $3k+ and no ETA for a fix?
     
  38. LittleMe

    LittleMe 2[H]4U

    Messages:
    2,977
    Joined:
    Feb 20, 2001
    You've got to consider that even so, how many people does Seagate really have running a setup like this? The people that will care the most are Areca and there is probably only so much they can do if it's the drives.
     
  39. brokenarrow03

    brokenarrow03 [H]ard|Gawd

    Messages:
    1,655
    Joined:
    Nov 20, 2007
    I agree. I would just expect a little more support from Seagate. Thats all...

    Did they refund your first card?
     
  40. vict0r

    vict0r Gawd

    Messages:
    1,013
    Joined:
    Jul 11, 2005
    what about the second xeon cpu? when are you putting it in ?