Choosing File Storage/Backup Solution

N

n00zler

Limp Gawd
Joined
Nov 13, 2002
Messages
344
I'm in the middle of trying to figure out a file storage/backup solution. I've just about run out of space on my desktop, and I recently had a hard drive fail which has pushed me into getting this done quickly :)

I'm trying to decide which route to go. My first thought would be to go with a Drobo. Yes, I know its a bit expensive, and its kinda slow, but it's simple. While I'm capable of building a system from scratch, I'm working, going to school, and have a ton of other stuff going on, so this would be the easiest thing for me to do. I love that you don't have to use similarly sized drives, and that you only "lose" space equivalent to the largest drive.

Next would be to go the WHS route. This is nearly as simple as the Drobo, but would require me to build a system for it (I do not want one of the HP's). What I like is that it sounds pretty easy to set up automated backups from multiple machines (we have several laptops that I would like to have backed up as well as my main PC). I'm a bit wary that all really is, is RAID 1. But, I do like that you can add oddly sized drives.

Last would be to build a real file server using probably RAID 5. I like this idea because I feel like it has the most storage potential, I've got lots of options for OS's (freenas, a simple XP install, etc). The downside is that it would probably end up costing me the most (as I'd be likely to go all out). Also, it doesn't seem to be as easy to add space to later - I've got to use the same size drives, etc.

I'm looking to spend about $1500, less is always better, but I can go a bit more if need be. What would you all recommend? If you think WHS or file server, what build would you recommend? I'd like to maximize current storage, and also allow for later expansion (obviously Drobo doesn't really work here, but it would likely get replaced a few years down the road instead of added on to). Thanks!
 
i love my WHS system.... 3TB non raid, i use it for streaming movies via mymovies addin, torrents, RDP to all PCs when i am at work, stores all our pictures, movies, files and does backups.


i really like it, and the client console is very easy to use..... quick to setup and if you want there is a lot of function to be gained by tweaking and tinkering.... or leave it alone to do it MS approved job.
 
Well what I got out of this was you want mainly 3 things

1)backups
2)file storage
3)allow for expansion

First off Drobo is out, its slow, cant expand, and expensive....whats the upside?
So you have to build your own box. Depending on how much storage you want there are about a billion different ways to do it.

I would suggest WHS, as I am a huge proponent. It can do everything Server2k3 can and more.
I would use WHS over raid 5 because it allows for expandability....and I know someone is going to chime in and say but RAID cards can do OCE, but you have to use the same size drives.
What i really like about WHS over raid5 is that at any time I can pull a drive from my server and connect it to any other windows box and see whats on it.
WHS also has backups taken care of. It does it automatically and the restore process is a breeze.

While RAID5 will maximize your storage now, it will also cost a bunch now, and will not allow you to easily expand.


WHS FTW
 
So you have WHS installed on a single drive, and then are using it to manage a RAID 5 array? What is your rig like?
 
WHS will take all the drives you give it.... you can tell it to leave some alone, but it handles parity a semi raid 1.... it installs on 1 drive, give 20gb for the OS and then rest for DATA.
 
No im not using it manage a RAID5, I got rid of RAID5 and sold my controller to buy more storage and build my current server.

Lots of guys around here have switched and love it.

you choose what folders to make redundant in WHS it will not duplicate everything or anything for that matter unless you tell it to.
 
Ahh, gotcha - I misread your earlier post :) I've been hearing lots of good stuff about WHS, and it seems like it could be a much easier solution than the full on file server.

What would you recommend hardware wise for a WHS box?
 
if you dont need to transcode video:
cheap low power CPU, id go with a x2 something that is 45w
2gb of RAM
cheap motherboard with a lot sata ports and gigabit lan...
80% efficient PSU
a case that will allow for expansion. coolermaster makes some good cases, i have an antec p190


i have also seen the people build WHS in SFF boxes, just get one with 2 5 1/4" bays in the front to install a 3x2 hotswap in it for more storage.....

i like 3 drives of the same size to start with..... 3x750gb, 3x1tb, 3x500gb
 
$1500 is a good budget.

Case and power supply:
$290 - http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
$125 - http://www.newegg.com/Product/Product.aspx?Item=N82E16817703009
Total - $415

Motherboard, memory, processor
$100 - http://www.newegg.com/Product/Product.aspx?Item=N82E16813138128
$45 - http://www.newegg.com/Product/Product.aspx?Item=N82E16820231122
$65 - http://www.newegg.com/Product/Product.aspx?Item=N82E16819103298
Total - $210

Hard drives

(4) $105 - http://www.newegg.com/Product/Product.aspx?Item=N82E16822136151
Total - $420

WHS
$99 - http://www.newegg.com/Product/Product.aspx?Item=N82E16832116395

That leaves about $$250 for DVD, incidentals and shipping. This is a solid system, with a nice case, good quality power supply, good motherboard, memory, processor etc.

For my money, I would up for a tri-core phenom for about $85 (an additional $30) but this system will do just fine as a pure file server.

I have personally used all of these things except the case / power supply (which I WISH I had!).

You can build a quality system for $1500.
 
That Case and PSU are the ones I use in my server, and are awesome.

One thing I would change would be the HDD's
There is a new version of the WD green 1tb.....WD10EADS they are faster, and run at a lower wattage then the previous generation.
Also dont buy your HDD's from newgg. they do a crappy packing job and I have had too many newegg drives either DOA, or had to RMA them later.
 
Nitrobass,

The WD10EADS is a nice drive but it doesn't appear to be widely available, and is more expensive (new?). Are you using them?

I have been holding off on getting a couple of tb drives. ATM my WHS system is using a ton of old 320s that I yanked from one of my servers. Free is good. But I do need to set up some near-line backups for my client's SQL Server databases which are large, pushing a tbyte now.
 
jcolby said:
Nitrobass,

The WD10EADS is a nice drive but it doesn't appear to be widely available, and is more expensive (new?). Are you using them?

I have been holding off on getting a couple of tb drives. ATM my WHS system is using a ton of old 320s that I yanked from one of my servers. Free is good. But I do need to set up some near-line backups for my client's SQL Server databases which are large, pushing a tbyte now.
Click to expand...

Yes they are new.
No im not using them but i have read several reviews and gone over benches, and every single one of them has had better numbers then the EACS(1st gen) models. I'm holding out for 2tb drives, but if I were buying 1tb drives for my WHS I would buy the EADS because of the lower power consumption.
 
I just popped for a 1.5 tb drive to hold my "offsite" backups. I will need to do this a couple of more times to get multi-level backups.
 
jcolby said:
I just popped for a 1.5 tb drive to hold my "offsite" backups. I will need to do this a couple of more times to get multi-level backups.
Click to expand...

Yea I grabbed a couple the other day for off-site backups for my dads company. They were faster then I expected them to be.
 
If you use Windows and like to use RAID5 that means you have to buy an expensive RAID controller.

If you use FreeNAS or basically ANYTHING not based on Windows, you can use Software RAID5 with great speed and design. FreeNAS sports a nice raid5 engine that is able to beat Areca RAID5 in sequential transfers (read+write). It also allows you to use journaling to keep your data safe. So with just one box, you can store all your data. Its still no backup, but a second fileserver can always be bought and act as a network mirror backup, copying files every night then switching off, etc.

As for hardware, you can go quite cheap:

Socket AM2 motherboard with 6x onboard SATA
EUR 35,-

AMD Athlon 64 X2 3800+ SFF (2.0GHz; 35W; dualcore)
EUR 29,-

4GB DDR2/666 CL4 (2 modules of 2GB so 4GB)
EUR 35,-

WD Green WD10EADS 1TB drive (6 of them)
EUR 600,-

Subtotal: 700,- excl. casing & power supply & cables

That would give you a 6TB volume, and like 5TB in RAID5 (~4.4TiB i guess). With PCI-express you can expand, by inserting add-on controllers with 2 to 4 SATA ports. Good controllers to use under Linux/BSD:
JMicron JMB363
Silicon Image SiI-3132 (has some meta bugs. nothing serious)

They just cost 15 euro or so, have a PCI-express x1 connector and allow you to expand the number of disks. Even though your disks are connected to different controllers, you can still use them all for just one big RAID5 array.
 
Any reason to go GP over 1.5TB Seagate drives (with latest firmware)?

Eg
3 1TB GP drives is 3*5=15watts
2 1.5TB Seagate drives is 2*9=18watts

Exact numbers might be off but idea is there... the GP's power saving are cut by the fact you need 50% more drives. Then add in other saives like GB/$: 1TB@105$ vs 1.5TB@119$. And just overall you have less drives and save ports/bays.

I only buy GP drives cause they're quiet... but if you have a file server off in another room who cares.
 
Well if physical space is not an issue you would prob make up to difference in price after about a year of running them 24/7 but i dunno id have to do some math and my brain is fried right now from finals.
 
3x WD 10EADS Green 1TB = 3x2,9W = 5,7W
2x Seagate 1.5TB = 2x9W = 12W

So less than half when using the new WD Green 10EADS drives.The older 10EACS "GP" drives use 4,0W each. Other reasons to buy the WD Green might be higher expected lifespan due to less friction and less heat generation. You don't need to cool these. But you do need to cool regular drives or they run the risk of increased failure rate.
 
Thats not true....google did a study on hard drives and found some interesting things. including but not limited to high temps and high utilization did not mean higher failure rates.
 
You're refering to the closed Google hdd reliability report. First of all, they did not release detailed information allowing to validate the claims. Second, they use their disks in a much different way than the average consumer. One critical point is the rapid cooling/warming of disks allowing the metal to expand and contract. Rapid succession of this in realistic home-use cases should limit its lifespan. By putting it in an air-conditioned room like in Google's server parks, yields totally different experiences.
 
Okay if you want to get the 10EADS model you're then paying extra because no one is selling it for $105. I can't even find it for sale.

While heat could be an issue, its not if you have it in a file server or something.. I was kinda commenting on post with Norco chassis.

Seagate 1.5TB = 8.3Watt Idle = $129
GPold 1.0TB = 4.0Watt Idle = $105
GPnew 1.0TB = 2.9Watt Idle = $110 (This is too low I fear but just to be nice)

$/TB
Seagate = $86
GPold = $105
GPnew = $110

Watts/TB
Seagate=5.53
GPold=4.0
GPnew=2.9

Electricty cost per TB per year @ $0.09/kwh
Seagate=$4.36
GPold=$3.15
GPnew=2.29

Time for drive to be cheaper than Seagate assuming 24/7 operation
GP Old = 15.70 years
GP New = 11.59 years

That time period is crazy long, plus if you really want to pretend this in an investment you could always invest the $20-31 per TB savings by buying Seagate and the interest would destroy the electricity investment.

** Edit changed prices to reflect newegg's current price. I was going off BlackFriday's prices of $119 for Seagate and $99 for GPold.
 
Lowest prices i could find:

WD Green 1TB - 10EADS (2,9W)
http://www.heise.de/preisvergleich/a366846.html
EUR 85,42

WD GP 1TB - 10EACS (4,0W)
http://www.heise.de/preisvergleich/a269817.html
EUR 82,09

Seagate 1.5TB
http://www.heise.de/preisvergleich/a353918.html
EUR 124,48 (= 83 EURO for 1TB)

As you can see, the prices in my region differ from yours. A 1.5TB being just 9 dollars more expensive than a 1.0TB sounds like you're buying at the wrong place. Aside from the costs, i like cool and quiet drives. They should also be more reliable than 7200rpm disks.
 
I was using BlackFriday prices, I updated it to newegg but the numbers arn't much different. Its not buying at the wrong place, the 1.5TB Seagate drive is just real cheap in general. Newegg is about standard as you get for stores.
 
Enlightenment said:
If you use Windows and like to use RAID5 that means you have to buy an expensive RAID controller.

If you use FreeNAS or basically ANYTHING not based on Windows, you can use Software RAID5 with great speed and design. FreeNAS sports a nice raid5 engine that is able to beat Areca RAID5 in sequential transfers (read+write). It also allows you to use journaling to keep your data safe. So with just one box, you can store all your data. Its still no backup, but a second fileserver can always be bought and act as a network mirror backup, copying files every night then switching off, etc.

As for hardware, you can go quite cheap:

Socket AM2 motherboard with 6x onboard SATA
EUR 35,-

AMD Athlon 64 X2 3800+ SFF (2.0GHz; 35W; dualcore)
EUR 29,-

4GB DDR2/666 CL4 (2 modules of 2GB so 4GB)
EUR 35,-

WD Green WD10EADS 1TB drive (6 of them)
EUR 600,-

Subtotal: 700,- excl. casing & power supply & cables

That would give you a 6TB volume, and like 5TB in RAID5 (~4.4TiB i guess). With PCI-express you can expand, by inserting add-on controllers with 2 to 4 SATA ports. Good controllers to use under Linux/BSD:
JMicron JMB363
Silicon Image SiI-3132 (has some meta bugs. nothing serious)

They just cost 15 euro or so, have a PCI-express x1 connector and allow you to expand the number of disks. Even though your disks are connected to different controllers, you can still use them all for just one big RAID5 array.
Click to expand...

FTW.

Man, I just don't get all the WHS love on this forum. FreeNAS is great, and it's free. You can buy another 1TB drive for the price of a WHS license. Furthermore, FreeNAS is extremely efficient. Mine runs on a mini-ITX board with an embedded 1200mhz Celeron.
 
What kind of transfer rates do you get?
I ran freenas on the same hardware I run WHS and I got ~13MB/s....that blows
On WHS I get ~70MB/s
 
FreeNAS also allows RAID5 with write-back (just like Areca) but with pure software. This is something Windows doesn't offer. Windows & storage aren't meant for eachother. The misalignments is something Microsoft could not care less about. Fortunately, most servers do not run Windows. But home users can have a hard time switching to linux or bsd. Note that FreeNAS installation should be a breeze and all configuration is done via a web-interface. But it can't run additional software; only act as NAS.

Nitro: gigabit performance is something that heavily depends on your network. Virtually all windows installations should be hacked with a different .dll that allows proper parallellisation to achieve higher speeds. I'm not sure why you got such low numbers, but i'm getting 147MB/s here on a trunked gigabit line (actually two gigabit cards combined into one virtual 2Gbps card) which is not as fast as it could be, but fast enough. :)
 
I may try FreeNAS to start, and if I don't like it switch over to WHS. How much automation can I get with FreeNAS? How easy is it to set up automatic sync'ing of folders, etc?
 
FreeNAS comes with rsync, samba, ftp, nfs and others. Rsync is used to 'syncronise' two directories over the network, and can be fully automatized. Meaning that it can be configured to sync every night. Even shutting down after syncing, and powering up just before the sync.
 
Enlightenment said:
Windows & storage aren't meant for eachother. The misalignments is something Microsoft could not care less about. Fortunately, most servers do not run Windows.
Click to expand...

I dont know what world you are living in, but where I work...we run 99.99999% windows Servers....
BTW Server2K3 does SRAID5

Misalignments...you think people that are developing FreeNAS care about those? For instance the shitty implementation of SMB on BSD(which I believe was causing my slow network issues).

And yes I LOVE WHS I dont see what the problem with that is.
I used to run Server2K3, and it was great, but WHS and it serves my needs much better, and cheaper.
I use Linux for just messing around, and I can setup mdadm RAID5 and all that bs but its a lot more work to expand the array on that type of system opposed to using WHS where i simply install the disk, and click a button.
 
nitrobass24 said:
I use Linux for just messing around, and I can setup mdadm RAID5 and all that bs but its a lot more work to expand the array on that type of system opposed to using WHS where i simply install the disk, and click a button.
Click to expand...

That is the biggest draw to me. I like how easy it is to expand. Its much easier to buy a drive here and there when you need it than to save up and plan for each expansion in the array.
 
nitrobass: I don't have Gigabit at home, What can I say, I'm cheap. My router cost 11 bucks at Fry's. My FreeNAS build cost me $281 complete, 2x500GB running the FreeNAS software RAID1 was enough for my needs. Well ok the 128MB USB thumb drive it boots off of, I won free at a vendor lunch :D

FreeNAS does completely saturate my 100Mbit network, that much I can say. I could try it at work on gigabit, but I would have to do that on my own time, and just to prove a point on a message board, eh, I don't care that much.

We do have an OpenBSD box at work that runs SMB shares on the internal network, never had any complaints about the performance of it. shrug...
 
Well, i wrote expansion support for geom_raid5, meaning you would be able to just add disks to a RAID5 array on FreeNAS. But it's not finished yet. ZFS also does not allow expanding RAID-Z, so if you need expansion support you're kind of stuck with Hardware RAID like Highpoint RR23xx / Areca.

Windows & Storage.. well.. windows doesn't really provide any reasonable RAID-engine. RAID0 may be used. NTFS does not protect against write-back buffers so using RAID5 on Windows is a huge risk, unless you run a BBU (Battery Backup Unit). On Linux/BSD you don't need all that, allowing you to save up to 600 euro for controller & bbu.

But.. one major advantage of Windows is that it is familiar to you, and does not feature a steep learning curve.

PS With misalignments i meant stripe/filesystem misalignments due to the fact Windows uses a very strange offset to partition your disks; meaning that if used on RAID much of the performance potential is lost due to the way Windows creates partitions. On FreeNAS/Linux those are easily avoided by skipping the use of partitions on the RAID volume, just taking the bare drive. So you start at 0 which means the filesystem is perfectly aligned to the stripe blocks.
 
Enlightenment said:
Windows & Storage.. well.. windows doesn't really provide any reasonable RAID-engine. RAID0 may be used. NTFS does not protect against write-back buffers so using RAID5 on Windows is a huge risk, unless you run a BBU (Battery Backup Unit). On Linux/BSD you don't need all that, allowing you to save up to 600 euro for controller & bbu. .
Click to expand...

I worry more about multiple drive failures than battery backed write cache. We run RAID6 on our larger arrays to protect against that. I feel a little more comfortable with 2 drives for parity.
 
Question: can WHS handle multiple storage pools? Since WHS uses MBR instead of GPT, a single storage pool can only be up to 2TB. So, if I want more than 2TB, can I create more than 1 storage pool?

EDIT: also, how many drives can a WHS rig lose without failing?
 
brons2: you worry about multiple drive failure within a limited time (24 hours), which is very very uncommon. More likely all drives fail due to power problems or letting your case fall sideways; causing a head crash on multiple disks at once. RAID6 does not protect you against that.

RAID6 also does not protect you against lost buffers. A single crash could cause large buffers to be lost, carrying metadata for your NTFS filesystem. NTFS works great on HDD buffers, but not on write-back buffers like RAID5/6. If it fails, it can bring with it large portions of the filesystem becomming inaccessible, even those who you have not read or written to in a long time. NTFS also does not have any means of detecting subtle meta-data corruption. NTFS is old.. but at least its not FAT. :D

So you can have multiple redundancy but still a corrupt filesystem. Just know RAID can never replace a backup; those two act supplementary. I'd rather have two RAID0 arrays keeping a copy of eachother, than one big RAID6 array. That said, the latter situation is much simpler to administer than a nightly backup system.
 
n00zler said:
Question: can WHS handle multiple storage pools? Since WHS uses MBR instead of GPT, a single storage pool can only be up to 2TB. So, if I want more than 2TB, can I create more than 1 storage pool?

EDIT: also, how many drives can a WHS rig lose without failing?
Click to expand...

WHS has no realistic limitation to storage pool...if I remember correctly its in the Petabytes, or something ridiculous.

Your second question is slightly more difficult....It depends, how many disk you have, what do you have duplicated, what do you mean by fail, total data loss, partial data loss?

n00zler I would suggest you read the WHS whitepaper as it would answer just about every question you have and probably make you more knowledgeable then most people on its abilities and limitations.
 
You need GPT if you want to partition a volume larger than 2TiB. That simple. If WHS does not support GPT, that means maximum 2TiB partitions.
 
Sorry I should have clarified

WHS is Server2K3 with an application called WHS.....it fully supports GPT, as you can add a 20tb raid array to it if your heart desires. DE deletes the partition and creates a NTFS partition. It does not check to see what type of partition table you are using nor does it care.
 
WHS can take 1 drive going down.... even the OS drive.... and be rebuilt,,,,

the biggest selling point for WHS to the drive extender and easy image based backup.....

well and the live integration with a domain name.... which is really slick
 
You must log in or register to reply here.
Back
Top