Home Server Question Raid or Not?

[ComputerGeek]

Hello Guy's I have a question, I am building a Home Sever to back up my files and stuff on, and plan to have a main OS drive, and 2 1tb drives for now.

Here are the questions I had.

Should I do a Raid 1 or Should I just copy the files to each drive?

If I do a Raid 1 and the computer fails am I able to take one drive out and read the files off of it? Or do I have to take both out and setup a whole new Raid array and how do I save the files then?

Is it better to have the OS on the Raid Array? I like to have just my Files, on a Raid, is that smart?

What System Factors are better in a Home Server I have a dual core Amd Proc but I was wondering if it won't need all of that power, and I would be better off with a low power Single Core. Also, Is more memory better for a file server?

Also, Would I get better transfers with a 1000 Gigabit line or am I still limited by the Hard Drive Write speed? OR can I store the File/s in the Memory until they are written to the hard drive. Would an SSD as an OS drive Help?

Also, What OS should I use I have any Linux version available to me and also have windows server 2003.

I know this is a Ton of questions, Any Help would be great! Thanks!

 

[ComputerGeek]

Also, just a general question. What happens is there are some bad sectors on a drive when copying will windows read from the other drive so the end file is not corrupted.

 

[Dangman]

Should I do a Raid 1 or Should I just copy the files to each drive?

RAID 1 for simplicity's sake if you're just looking for data uptime. If you're looking for backup, keep the drives seperate. In fact, if you want backup, take one of those 1TB drives out, put it into an external hard drive enclosure, and once a week hook it up to the server, copy over files that you want saved, unhook it and store it somewhere safe. RAID by itself is not backup.

If I do a Raid 1 and the computer fails am I able to take one drive out and read the files off of it? Or do I have to take both out and setup a whole new Raid array and how do I save the files then?

It's the former assuming that whatever PC you're putting that drive in can read that file system.

Is it better to have the OS on the Raid Array? I like to have just my Files, on a Raid, is that smart?

Thats smart: Keep the OS off the storage RAID array.

What System Factors are better in a Home Server I have a dual core Amd Proc but I was wondering if it won't need all of that power, and I would be better off with a low power Single Core. Also, Is more memory better for a file server?

CPU wise, you should be fine with that dual-core AMD CPU. RAM wise, you should be fine with 1GB of RAM unless you're planning on using the server to do other things like virtual machines, encoding, etc. Or if you're using ZFS which requires a lot of RAM sometimes. Pretty much the important factors are the hard drives, how you're setting up the server, and your network,

Also, Would I get better transfers with a 1000 Gigabit line or am I still limited by the Hard Drive Write speed? OR can I store the File/s in the Memory until they are written to the hard drive. Would an SSD as an OS drive Help?

Many current 7200RPM and even some 5400RPM drives can fully saturate gigabit ethernet. So yes get gigabit ethernet capability if your motherbaord doesn't have any. Speaking of which, what mobo do you have?

Also, What OS should I use I have any Linux version available to me and also have windows server 2003.

Whatever Linux OS you're comfortable with should be ok for the server.

Also, just a general question. What happens is there are some bad sectors on a drive when copying will windows read from the other drive so the end file is not corrupted.

Unfortunately, one of the possible pitfalls of RAID 1 is that the corrupted file might end up on both drives.

 

[ComputerGeek]

Danny Bui;1038316073]RAID 1 for simplicity's sake if you're just looking for data uptime. If you're looking for backup, keep the drives seperate. In fact, if you want backup, take one of those 1TB drives out, put it into an external hard drive enclosure, and once a week hook it up to the server, copy over files that you want saved, unhook it and store it somewhere safe. RAID by itself is not backup.

I am just looking to use this as a Backup system to backup my other computers too, possibly to host media files like the .vobs off dvd's. Why keep them seperate? Also why put one in enclosure just for safety if there is a power surge? Or if someone steals both the Main and the Backup computer? "RAID by itself is not backup" Why?

Thats smart: Keep the OS off the storage RAID array.

Would the same apply if I am not doing raid? Also, should I get a Small SSD for the OS or would that be a waste in this aplication?

CPU wise, you should be fine with that dual-core AMD CPU. RAM wise, you should be fine with 1GB of RAM unless you're planning on using the server to do other things like virtual machines, encoding, etc. Or if you're using ZFS which requires a lot of RAM sometimes. Pretty much the important factors are the hard drives, how you're setting up the server, and your network,

I am just looking to use this as a Backup system, possibly to host media files like the .vobs off dvd's. I would possibly like to do wake on lan and access it remotely but I don't know how much of a security risk that poses. What is ZFS?

Many current 7200RPM and even some 5400RPM drives can fully saturate gigabit ethernet. So yes get gigabit ethernet capability if your motherbaord doesn't have any. Speaking of which, what mobo do you have?

I have a Asus A8n-sli deluxe came from a old Alienware system. Here's Two pic's. I also had a question about the Sata Ports. It's sort of weird there are two sets of sata ports one set is black and one red. Usually the Sata ports that I have seen would count up like this 1,2,3,4,5,6,7,8 however there are two sets on this board black that is 1,2,3,4, and red which is 1,2,3,4 I am just wondering if the red are used for something else, or only get activated after boot for like E-SATA or something, has anyone encountered this?

http://i359.photobucket.com/albums/oo37/Louis_Rubera/Computer_Geeks_Photos/IMG_0608.jpg
http://i359.photobucket.com/albums/oo37/Louis_Rubera/Computer_Geeks_Photos/IMG_0605.jpg

Whatever Linux OS you're comfortable with should be ok for the server.

I have another question about this but I will wait till I solve the other problems first.

Unfortunately, one of the possible pitfalls of RAID 1 is that the corrupted file might end up on both drives.

So, how do you deal with this? Do people not worry about it I mean if you save the file say off a SD card to your Hard drive on the main computer, and the main computers hard drive has some bad sectors, and then I backup my PC to this backup server which also happens to have some bad sectors, does the file end up with all that info lost? What can I do to prevent this? Or should I just not worry about it?

Thanks I know I have a lot of Questions.

 

[Red Squirrel]

With md raid and zfs being easily and freely available, I do nothing but raid at home now. Well the OS drives on most of my systems are still unraided, but I want to change that eventually. Though I may steer a slightly different direction and just use a SSD from now on for any OS drive.

Data is all on raid, and is backed up to single drives. My current solution is a drive dock, I just put the drive in it, turn it on, and run a script that backs up the correct folders based on the drive that's in there, so I can spread data across multiple smaller drives if I want to. These drives are put into a not so systematic rotation and I occasionally bring two of them to my PO box. I also have a rsync script that uploads the most important data over to my web host, but given my connection is not that fast, I can't be uploading GB and GB's of data. a single GB takes a few days to upload and during this time my internet is unusable. So yeah I only backup the most important stuff like my pins files, source code for major projects, etc. Pictures and stuff, I rely on the PO box backups. Even though they don't happen as often I figure not that much changes in a month anyway. If I go on a vacation or do something that produces very valuable data then I'll just run an offsite right after.

 

[Dangman]

Also why put one in enclosure just for safety if there is a power surge? Or if someone steals both the Main and the Backup computer?

Well, you don't leave the external drive powered on all the time. That minimizes the chance of a power surge getting to it. Again, once a week, you connect to the server, backup whatever data you really need backed up, disconnect it from the server and power strip, and store it somewhere safe. And yes, if someone happens to steal your main and backup computers, hopefully they didn't steal the external hard drive since you left the external hard drive in a safe place.

I am just looking to use this as a Backup system to backup my other computers too, possibly to host media files like the .vobs off dvd's. Why keep them seperate? "RAID by itself is not backup" Why?

RAID doesn't protect against accidental or malicioous deletion, corruption, removal, or replacement of data. A backup is usually a copy of the main data on a different medium than the main data. Technically the server that you're building is a backup actually. With that said, these days, a lot of people have a secondary backup for really critical data.

Hence why I recommended having the other drive in an external enclosure if you really want to keep your data safe.

Would the same apply if I am not doing raid? Also, should I get a Small SSD for the OS or would that be a waste in this aplication?

Yes. A SSD would be a waste in this application.

I am just looking to use this as a Backup system, possibly to host media files like the .vobs off dvd's. I would possibly like to do wake on lan and access it remotely but I don't know how much of a security risk that poses. What is ZFS?

ZFS Links:
OpenSolaris derived ZFS NAS/ SAN (Nexenta*, OpenIndiana, Solaris Express, napp-it)
Building ZFS Based Network Attached Storage Using FreeNAS 8
napp-it ZFS server appliance
Building your own ZFS fileserver

I have a Asus A8n-sli deluxe came from a old Alienware system. Here's Two pic's. I also had a question about the Sata Ports. It's sort of weird there are two sets of sata ports one set is black and one red. Usually the Sata ports that I have seen would count up like this 1,2,3,4,5,6,7,8 however there are two sets on this board black that is 1,2,3,4, and red which is 1,2,3,4 I am just wondering if the red are used for something else, or only get activated after boot for like E-SATA or something, has anyone encountered this?

The different colors denotes which SATA ports are controlled by the two SATA controllers you have on that motherboard. Since there's two different SATA controllers, the numbering scheme reflects that.

So, how do you deal with this? Do people not worry about it I mean if you save the file say off a SD card to your Hard drive on the main computer, and the main computers hard drive has some bad sectors, and then I backup my PC to this backup server which also happens to have some bad sectors, does the file end up with all that info lost? What can I do to prevent this? Or should I just not worry about it?

Yes the files can potentiall be corrupted and have no info. But usually there should be error message from the server end or the PC end that the data might be corrupted. Should say something like "CRC check" or something similar. To be honest, I'm a tad unclear on this myself but I don't worry about it as I usually have a 3rd copy of the data somewhere.

 

[ComputerGeek]

RAID doesn't protect against accidental or malicioous deletion, corruption, removal, or replacement of data. A backup is usually a copy of the main data on a different medium than the main data. Technically the server that you're building is a backup actually. With that said, these days, a lot of people have a secondary backup for really critical data.

OK, now I see why raid is useless in this situation. You've convinced me.

Hence why I recommended having the other drive in an external enclosure if you really want to keep your data safe.

OK, I got it now. I think I am going to leave it in for now as I don't have anything that nessasery right now, but I will soon. And when I do I'll get a 2tb external to back both of the 1tb up on.

ZFS Links:
OpenSolaris derived ZFS NAS/ SAN (Nexenta*, OpenIndiana, Solaris Express, napp-it)
Building ZFS Based Network Attached Storage Using FreeNAS 8
napp-it ZFS server appliance
Building your own ZFS fileserver

ZFS sounds interesting would you recommend it?

The different colors denotes which SATA ports are controlled by the two SATA controllers you have on that motherboard. Since there's two different SATA controllers, the numbering scheme reflects that.

OK are one set faster than the others? or should I put one drive on each? Also, I can attach one of these to an Esata port for Backups right?

Yes the files can potentiall be corrupted and have no info. But usually there should be error message from the server end or the PC end that the data might be corrupted. Should say something like "CRC check" or something similar. To be honest, I'm a tad unclear on this myself but I don't worry about it as I usually have a 3rd copy of the data somewhere.

When I transfer a flash card of video should not transfer the copied files form the main computer, but rather put the flash card in the backup server and copy them there? What backup copy I can just copy the data that's on backup server right? I am probably getting crazy here.

What would you do considering what I have? What Os would you run?


What do you say about remote access? Too Risky? If not a guide or something about what software to use and what not would be great.

Thanks!

 

[Dangman]

ZFS sounds interesting would you recommend it?

I would if you had enough drives for a proper ZFS array as well as lot of RAM. In addition, I don't know if ZFS compatible OSes will work with your hardware.

OK are one set faster than the others? or should I put one drive on each? Also, I can attach one of these to an Esata port for Backups right?

Unfortunately both controllers aren't that great. Both the Nvidia storage controller and the Silicon Image controller on that mobo is somewhat slow and not as reliable as current storage controllers. With that said, if you're using Linux, shouldn't matter where you hook up the drives.

When I transfer a flash card of video should not transfer the copied files form the main computer, but rather put the flash card in the backup server and copy them there? What backup copy I can just copy the data that's on backup server right? I am probably getting crazy here.

I have no idea what you're asking here. All of that doesn't make sense to me.

What would you do considering what I have? What Os would you run?

Again, use whatever Linux OS you're most comfortable with. Then google for a guide on how to turn that Linux distro in a server. Or just google "linux server". Should be a ton of links about that.

Though for my own server, I used the original Windows Home Server because of the drive pooling and the baremetal backup features.

What do you say about remote access? Too Risky? If not a guide or something about what software to use and what not would be great.

Enable it after you've fully familiarized yourself with whatever server OS you're using if you really need it.

 

[_Gea]

RAID doesn't protect against accidental or malicioous deletion, corruption, removal, or replacement of data. A backup is usually a copy of the main data on a different medium than the main data. Technically the server that you're building is a backup actually. With that said, these days, a lot of people have a secondary backup for really critical data.

Raid cannot help against a real disaster but if you use ZFS or sometimes btrfs together with a lot of snapshots and its self-healing filesystem due to real data checksums where you can go back to a valid data state even in case of accidential deletion, removal or overwrite of data. Silent data corruotion is also not a problem with ZFS and regular online data scrubbing to fix them. Raid+ZFS can help to improve data availability to a level where you need a backup extremly seldom.

The same with mirrors. ZFS knows which data is bad. It does not mirror bad data over good data like regular mirror-Raids can do.

 

[ComputerGeek]

I would if you had enough drives for a proper ZFS array as well as lot of RAM. In addition, I don't know if ZFS compatible OSes will work with your hardware.

How many drives do I need for a proper array? I only have 1GB of Ram but I could get more. I think my Proc is capable since its a dual core @ 2.0 ghz I believe. However, I have not used any of these, OpenSolaris, FreeBSD and FreeNAS which are the only supported OS's. Do you think I should stay away from it because I have never used those OS's or do you think that I'll pick one of those up quickly since I have used linux before.

Also, is ZFS really that great I am afraid Idk much about it, what so great about it, I know it has the protection against data corruption? Why sould someone choose this over a different setup? And who ussally use ZFS?

Unfortunately both controllers aren't that great. Both the Nvidia storage controller and the Silicon Image controller on that mobo is somewhat slow and not as reliable as current storage controllers. With that said, if you're using Linux, shouldn't matter where you hook up the drives.

OK I think I'll be using linux so Great.

I have no idea what you're asking here. All of that doesn't make sense to me.

Well after reading what I wrote I see why, lol. I was referring to the correct way to backup a flash card with video. I ussually just copy the card contents on to the main computer then copy (The Copied File on the Main Computer files unto the backup device.

Should I Instead copy the card contents once unto the Main computer and the read the card a second time to make the backup copy?

Again, use whatever Linux OS you're most comfortable with. Then google for a guide on how to turn that Linux distro in a server. Or just google "linux server". Should be a ton of links about that.

But are there any linux distros specific or better suited toward a home sever?

Though for my own server, I used the original Windows Home Server because of the drive pooling and the baremetal backup features.

What backup features do you like?

Enable it after you've fully familiarized yourself with whatever server OS you're using if you really need it.

Ok I'll do that.

 

[Dangman]

How many drives do I need for a proper array?

Like three I believe.

I only have 1GB of Ram but I could get more. I think my Proc is capable since its a dual core @ 2.0 ghz I believe. However, I have not used any of these, OpenSolaris, FreeBSD and FreeNAS which are the only supported OS's. Do you think I should stay away from it because I have never used those OS's or do you think that I'll pick one of those up quickly since I have used linux before.

I'm assuming that you should be able to pick up those OSes fairly quickly, especially with all the guides for those OSes out there.

Also, is ZFS really that great I am afraid Idk much about it, what so great about it, I know it has the protection against data corruption? Why sould someone choose this over a different setup?

Got a pros and cons here:
Pros:
+ Free
+ Fairly resilient against crashes, corruption, and such due to the copy-on-write feature as well as the snapshots feature.
+ Maintenance free in that the file system will periodically check and fix itself
+ Very flexible in terms of hardware and software changes.
+ Supports encryption of data

Cons:
- May be difficult to learn for a person who've never worked with BSD before
- Requires a large amount of RAM (4GB+ recommended)
- Cannot expand an existing RAID array with additional drives.

Notes:
* ZFS uses a different method of storage expansion:

What you cannot do, is expand an existing RAID-Z (RAID5) or RAID-Z2 (RAID6) array with one or more disks.
But, you can add new disks or RAIDs to an existing pool. So if you have a 4-disk RAID-Z, you can add another 4-disk RAID-Z so you have 8 disks. The second array would share free space with the first; in essence it would be a RAID0 of two RAID5 arrays. ZFS can expand this way.

What you can do, is expanding mirrors and RAID0's. In the example above that's what actually happened: a new array is RAID0-ed with the existing array. New created files will be written to both devices, for additional speed. Setting copies=2 would make files in that directory be stored on both RAID arrays; for extra redundancy.

And who usually use ZFS?

Pretty much anyone who has read enough about it, compared it to other platforms, and found that the ZFS setup meets their particular needs the best. In addition, read what _Gea wrote right above your last post. Covers additional and expands on some of the pros of ZFS. Not to mention that he's the author of one of the ZFS guides I linked to.

Should I Instead copy the card contents once unto the Main computer and the read the card a second time to make the backup copy?

If you can do that, yes. Best to have make a copy of the original source rather than a copy of a copy of the original source. Less chance of potential issues.

But are there any linux distros specific or better suited toward a home sever?

There is Amahi Home Server and UnRaid I believe. But I have not tried recent versions of those OSes yet. But I've done some research about UnRaid:
Unraid:
Links
http://www.lime-technology.com/
http://lime-technology.com/wiki/index.php?title=UnRAID_Wiki

Pros:
+ Allows use of different sized drives.
+ Should one or multiple drives die, only the data on those dead drives are lost.
+ If the data on the hard drive isn’t being accessed, the hard drive is spun down until needed.

Cons:
- unRAID Free has a limit of 3 drives (2 data, 1 parity).
- unRAID Plus has a limit of 7 drives (5 data, 1 parity, 1 cache)
- unRAID Pro has a limit of 21 drives (19 data, 1 parity, 1 cache)
- Costs $120 for unRAID Pro (21 drives) and $69 for unRAID Plus (7 drives)
- Relatively low write speeds without that cache drive.

Notes:
* “unRAID™ is similar to RAID-4 in that for every n hard drives, there are n-1 data drives, and a single fixed parity drive” - From unRAID website

I did use Ubuntu Server as my home server OS for a while but I ended up paying for WHS because I really liked the ease of use, the bare-metal backup and restore feature, and the drive-pool feature. But if I didn't need the last two features at the time and now, Ubuntu Server would probably have still been my server OS.

What backup features do you like?

The baremetal restore feature:
http://en.wikipedia.org/wiki/Bare-metal_restore

That has come in handy a few times already. Do note however that WHS v1.1, which is what I'm using, doesn't support 3TB drive natively and has some issues with certain 2TB drives. There is WHS 2011 but it doesn't have the drive pooling feature I'm fond of. But it's a fairly solid OS in its own right:
http://www.wegotserved.com/2011/04/07/windows-home-server-windows-home-server-2011-worth-upgrading/