Prebuilt NAS or Custom Built PC? What?

[Gutspiller]

I've been asking around trying to figure out what exactly I need. I know how to built computers so building one isn't a problem.

I'm looking to build a remote backup server that will sit at my location and my web servers will send (rsync) backups to it daily, weekly, and monthly.

Talking with my host they've said this about what I will need:
"You will need raid 5 or 10 setup. If you provide shell or ftp access, we can enable backup on all 3 servers to be transferred to your backup server."

Is this possible with a prebuilt NAS product like this http://www.hothardware.com/Articles/Thecus_N5200BR_NAS_Server/?page=1

or is my only real choice to built a machine myself for what I need it for?



If I do need to go with a regular machine loaded with hard drives would a any mobo do as long as it had raid onboard or I had an add in raid card? I ask because I recently upgraded my home PC and I haven't sold my old mobo that is a P5W DH Deluxe, but either doesn't have raid or can only do it with 2 drives because it only has 2 SATA connections. Would this mobo work with a good raid card so that I can get the RAID 5 or 10 my host is recommending?

If so, what SATA RAID card should I go with?

Once I build this remote backup server I'd kind of like to not be bothered with it for a good length of time, so I'm thinking if my budget allows a 2TB of storage with that 2TB being mirrored so 4TB total.

I don't need 2TB now, but I figure this would keep me safe for a good long while unless the price is so large that 1TB backup with 1TB mirror is far cheaper and more space can be added without too much hassle of redoing the whole situation all over again as I will likely be getting heavy support from forumers in setting it up and a little help from my host, which I may have to pay extra for.

I don't know if a full blown PC is any worse than a smaller NAS system that does this as far as used power is concerned or not, or even if any of these smaller media NASes on are even capable of doing what I need it to do with allowing my host to set up my 3 web servers to send my backup data to it.

I'm currently paying them to handle the off-site backup, but it's costing me $100 a month so I'm looking to get off of that monthly fee ASAP as the connection here is doing nothing at night and it could be used for these backups with no extra expensive for bandwidth used at our location.

Can some people please direct me in the path I should take on this while still having something that is keeping my data safe, but that isn't too expensive. I was thinking of something that is nice and will last. I don't want to slack on price for lack of quality, but I don't want something that is hugely expensive for no reason or for features I don't need.

Thanks for any help!

 

[Ockie]

Have you considered your bottlenecks? Such as having a limited bandwidth or falling outside of your ISP quota?

If you are interested, I do cater in these areas, we offer dedicated server rentals that would suit your needs on 100mbit connections... also offer co-location options for you at a much more cost effective rate than $100 a month as you are currently paying, and you can use the servers for other tasks such as game servers, or heck, hosting your own websites for that matter.

 

[Gutspiller]

I will look more into that our work ISP allows it, but I think we would be well within our limits. As far as getting somebody else to do it, that is exactly what I am trying to get out of. I don't want the $100 fee at all. That is why I want to buy all the hardware and do it on a connection that is already being paid for and even though I'll check the limit, I truly doubt we're pushing our connection anything near what we are allowed.

That is why I'm doing it, because I see a way that we can more fully use that bandwidth and save money in another area.

 

[Ockie]

Oh okay, this is for business and you have the connection, then you should be set.

My recomendation would be a proper file server, not a nas box or any appliance.

What is your expected ROI time frame or your budget? Since this is a backup of an existing site, have you looked into WHS boxes? They are cheap and will do the tasks you need and are super simple to add more storage in the pool.

 

[Gutspiller]

What's a WHS box?

Do you have links so I can see some?

 

[Dangman]

What's a WHS box?

Do you have links so I can see some?

WHS = Windows Home Server

A couple of WHS boxes:
http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&DEPA=0&Description=mediasmart&x=0&y=0

 

[xenotype]

Those linked prebuilt boxes from the egg arent actually too bad as far as price/specs go. Current tech limits you to 4 tb, due to the formfactor of those particular models, but that will scale to 6tb as soon as whomever releases those 1.5 tb harddrives later this year

However, be mindful that your provider, the colo, might require a rackmount system ( ie, 1u, 2u, etc) which will greatly expand your options, but unfortunately, the cost as well.

For your money if you can use one of those newegg boxes, slap in some WD GP 1tb drives, and youre golden.

 

[Roliath]

If you build your own, buy a prebuilt make sure the drives are intended for a server environment. I would not use "home" drives in any NAS/backup solution.

 

[Gutspiller]

However, be mindful that your provider, the colo, might require a rackmount system ( ie, 1u, 2u, etc) which will greatly expand your options, but unfortunately, the cost as well.

For your money if you can use one of those newegg boxes, slap in some WD GP 1tb drives, and youre golden.

This backup server is actually going to be sitting at MY location not at my hosts. My webservers are at my hosts location and the backup server at my location. This will save me $100 a month and even more as the website data grows.

I have some spare parts like a mobo, power supply and case that I'm actually looking to use, so it seems like that might be more of an advantage to me if the prebuilt offer nothing "special".

If you build your own, buy a prebuilt make sure the drives are intended for a server environment. I would not use "home" drives in any NAS/backup solution.

After thinking a bit about what I need and why I'm leaning towards JBOD for my disk setup. I was thinking of using 2 Seagate Barracuda 7200.11 ST31000340AS 1TB drives. Those should be decent enough for this backup server shouldn't they?

Do I need a raid card if I want to use a JBOD configuration for the hard drives?

I was thinking of using the spare P5W DH Deluxe motherboard I have. It doesn't even have onboard raid so if that is something that is needed for even JBOD then I will need a raid card anyways. But then it comes down to is it better to buy a raid card for the P5W mobo or to get a new mobo that has the ability onboard?

 

[xenotype]

This backup server is actually going to be sitting at MY location not at my hosts. ...I have some spare parts like a mobo, power supply and case that I'm actually looking to use, so it seems like that might be more of an advantage to me if the prebuilt offer nothing "special".



After thinking a bit about what I need and why I'm leaning towards JBOD for my disk setup. I was thinking of using 2 Seagate Barracuda 7200.11 ST31000340AS 1TB drives. Those should be decent enough for this backup server shouldn't they?

Do I need a raid card if I want to use a JBOD configuration for the hard drives?

I was thinking of using the spare P5W DH Deluxe motherboard I have. It doesn't even have onboard raid so if that is something that is needed for even JBOD then I will need a raid card anyways. But then it comes down to is it better to buy a raid card for the P5W mobo or to get a new mobo that has the ability onboard?

Its usually best to get a dedicated card. This will allow you to migrate the RAID to another system if something happens to the system, as well as cards usually have vastly more features /support / drivers / etc then the motherboard.

I agree with your choice of drives, those are some fast / excellent drives.

 

[Gutspiller]

Is a raid card needed even for JBOD?

 

[xenotype]

Is a raid card needed even for JBOD?

Well, yes and no. Its not a normal feature for mobo's to have, its considered in the nebulae of RAID. If you want to do JBOD, and your mobo wont support it you will need a RAID card, you might be able to find this option on cheaper controller cards, but it will usually be bundled like: RAID1/0/JBOD

 

[Ockie]

Well, yes and no. Its not a normal feature for mobo's to have, its considered in the nebulae of RAID. If you want to do JBOD, and your mobo wont support it you will need a RAID card, you might be able to find this option on cheaper controller cards, but it will usually be bundled like: RAID1/0/JBOD

You can create a JBOD effect by using dynamic drives and binding them in a large expanded volume.

 

[xenotype]

ooooh yea. hehe, NTFS can do that now.

 

[Lazn_Work]

If I understand the situation correctly, by having the backups at your location I am sure your backups will count towards your bandwidth quota.. If this is an issue or not will depend on your quota and your traffic.

Also what kind of internet connection do you have? If you are trying to sync a couple TB each night over a few MB/s DSL or Cable you are going to be disappointed.

As for hardware, what is your budget, that's the real issue. To do it right you'd want to get a real disk to tape solution with off site secure storage of tapes. But I expect this is out of your budget.

You could consider the Adaptec Snap Storage servers as a kind of mid line solution.

If price is a real concern I would actually suggest a self built linux box running software RAID as that will be movable to any other hardware should the motherboard etc fry.

 

[/dev/null]

My suggestion for scalability, price and performance:

Linux box(es) with lustre FS with gig-e.

Start small & the thing scales like crazy.

This is what LLNL and ONL use in supercomputers. If you need help with this, pm me

 

[Gutspiller]

If I'm using rsync to keep the data synced between the webservers and the backup server, can the data be compressed?

I was really wanting this server to be Windows based, but some have noted the filename differences between Windows and the Linux that s on the webservers, namely the capital letters in filenames.

I was thinking that if the data was compressed that would solve the problem, but at the same time, that isn't possible if I'm not looking to download the entirety of the servers every night, right? I mean if it's compressed in a file it can't know what files have changed and only do those can it?

 

[sabregen]

Whore, Hoochie, Slut - search for them on google. there's plenty of links.

What's a WHS box?

Do you have links so I can see some?

 

[sabregen]

Keeping in mind that if you decide to go this route, and the host that is doing the software raid computations (which I might add, additionally increases your processing overhead...but that's another issue) fails, you will likely have significant downtime unless you have an identical host machine (hardware wise) to physically move the array into. My personal recommendation for your small implementation would be a decent SATA RAID card, and 4x1TB drives as a relatively cheap solution. RAID 10 would be a good recommendation, RAID 5 would also be fine. With RAID 5, you could use 750GB drives, which would be slightly cheaper. However, you can only sustain 1 total drive failure, whereas RAID 10 can sustain an entire disk subset failure, which does not necessarily imply a double disk failure. Double disk failures across both subsets would be catastrophic in RAID 10 as well. I would never use JBOD for anything more than scratch space, personally.

Ockie likely has a billion of these drives sitting in his personal box . He's stingy though.

You can create a JBOD effect by using dynamic drives and binding them in a large expanded volume.

 

[unhappy_mage]

If I'm using rsync to keep the data synced between the webservers and the backup server, can the data be compressed?

rsync can use compression on what it sends across the channel. Pass the -z flag to it, probably something like "rsync -az /var/www backup-host.corp.com:/data/web2" in all. Read the manual page (man rsync) and figure out what options you really need, but -az is a good start.

I was really wanting this server to be Windows based, but some have noted the filename differences between Windows and the Linux that s on the webservers, namely the capital letters in filenames.

Yeah, case sensitivity would be an issue if you put things on a standard filesystem. Linux handles this neatly, but ISTR there's a way to get NTFS to be case sensitive if it comes to that.

I was thinking that if the data was compressed that would solve the problem, but at the same time, that isn't possible if I'm not looking to download the entirety of the servers every night, right? I mean if it's compressed in a file it can't know what files have changed and only do those can it?

rsync transmits block checksums to decide what to send, then sends it. If you pass -z as above, everything it transmits is filtered through gzip. Thus you get incremental backups with compression over the wire. Give it a try before you put it in production, if the link is fast enough compression can slow you down... but you need to push at least 100 megabit before that's really an issue.

Keeping in mind that if you decide to go this route, and the host that is doing the software raid computations (which I might add, additionally increases your processing overhead...but that's another issue) fails, you will likely have significant downtime unless you have an identical host machine (hardware wise) to physically move the array into.

This is incorrect. Where did you read this? One of the advantages of software raid is that one can move to different hardware easily because all the state is stored on the disks and the OS can interpret that data regardless of which controller is used.

 

[Ockie]

This is incorrect. Where did you read this? One of the advantages of software raid is that one can move to different hardware easily because all the state is stored on the disks and the OS can interpret that data regardless of which controller is used.

While some of what you said is true, some of it is false. While software raid works good for its purpose, it does consume more system resources, whereas a hardware raid controller does not consume nearly as system resources as it features a lot of resource offloading capabilities.

Also, a lot of features is missing from software raid so you don't entirley get what you need with software raid. (IE Advanced levels of raid, battery backup, OCE, cache expansion, Out of Band Management, etc). You may also lack hot swappability with certain motherboards.

Now as for the disks being portable comment, while this is true, this can also be false becuase you may be able to move your drives to another system, but it has to be running a compatible operating system (read: same operating system), it also has to be a chipset on the board that is supported by the operating system, it has to have a controller with the functions you want. (Basically, you need a spare identical machine if uptime is an issue) Swapping your card out is much easier than swapping your entire system board out. Also with hardware raid you can run your controllers in parralel for maximum redundancy and failover... so now your card is no long a failure issue.

 

[unhappy_mage]

While some of what you said is true, some of it is false.

Fair enough. I don't agree with all of your reasoning, though.

While software raid works good for its purpose, it does consume more system resources, whereas a hardware raid controller does not consume nearly as system resources as it features a lot of resource offloading capabilities.

True, but for many usage scenarios CPU usage isn't a big issue. Using mirrors is cheaper now than ever before, and mirroring is practically free CPU-wise in software. RAID 5 isn't too bad these days---a single Opteron core can do something like 6 GB/s of XOR. That means unless you're managing to peg your storage and all of your CPUs at the same time, software RAID is a good match for you. If your application absolutely needs all the CPU it can possibly get, though, hardware raid is certainly a better option.

Also, a lot of features is missing from software raid so you don't entirley get what you need with software raid. (IE Advanced levels of raid, battery backup, OCE, cache expansion, Out of Band Management, etc). You may also lack hot swappability with certain motherboards.

"Advanced levels of raid"? What do you mean? With linux md, you can do raid 0, 1, 10, 4, 5, 6, or nest them in any combination---RAID 161, if you like. Monitoring is built in; configure and start a daemon (which takes all of three minutes) and it emails you when things go wrong. Expanding arrays works as well as it does on any controller, assembling broken arrays works, and you can even have it list the partial arrays it sees and what pieces of each it has. It's pretty hardware-like.

Battery backup can be done with an external UPS that you should have anyways. Set it up to shut down your system cleanly and you're all set. Cache expansion really isn't that big an issue unless you're bottlenecked by RAM; OSes will do a pretty good job of caching up to available memory, and generally have a better strategy for which blocks to keep in cache available.

And of course some motherboards don't support hot-swap. But addon controllers with no RAID functionality are a lot cheaper than full hardware controllers---$100 instead of $600.

Now as for the disks being portable comment, while this is true, this can also be false becuase you may be able to move your drives to another system, but it has to be running a compatible operating system (read: same operating system),

I don't see how this is an issue, honestly. You generally have a filesystem on your RAID arrays, and those generally aren't portable across operating systems anyways. The only exception would be NTFS from NT4 to 2k8, but anyone running NT4 for their RAID array is asking for trouble.

it also has to be a chipset on the board that is supported by the operating system, it has to have a controller with the functions you want.

Functions like... being able to read blocks from disks? I can't think of any other features that software raid requires.

(Basically, you need a spare identical machine if uptime is an issue)

When a hardware raid controller costs as much as a machine without one, it's not *that* big a deal to keep a spare around.

Swapping your card out is much easier than swapping your entire system board out.

If you have a removable motherboard tray, they're pretty similar in terms of difficulty. And for $600 savings, I can deal with a little hassle with a screwdriver.

Also with hardware raid you can run your controllers in parralel for maximum redundancy and failover... so now your card is no long a failure issue.

This isn't specific to hardware raid. SAS controllers do multipath under many OSes these days.

Finally, let me point out that I believe hardware RAID has its place. But it isn't needed for the home user. My software-driven array gets pretty close to maxing out gigabit in both directions, even though a few of the disks and the ethernet controller are on the same PCI bus. And I only have four disks, in two mirrored pairs.