Raid arrays for the beginner

[NukeULater]

I need some help venturing into the world of raid arrays. I am a photographer and due to the volume of images I take per year my hard drives fill relatively quickly. I currently have two 1.5TB Seagate Barracuda's as storage drives and my OS is on pair of 500GB Seagate Barracuda's in raid 0. The system specs are in my signature.

What I need is better, more reliable back up solution. I am going to be using 3TB Seagate Barracuda ST3000DM001's for the drives. I need advice as to what raid card and type of raid array I should use. I'm leaning toward raid 5 right now. I do have a free PCIe x16 slot which opens up some options.

Do I need to rebuild the raid array if I have to add drives to it? How flexible are the arrays? What type of raid should I use?

Any help is appreciated.

 

[grimster]

A single drive could be used for a backup, and only used when backing up or restoring stuff. There's no need to spend huge bucks on a backup, unless it's for something mission critical, and needs to instantly mirror the main copy. An external drive should be fine if you only need to back up occasionally. I would lean towards using RAID for a working volume, to gain some speed benefits.

 

[Jaxon_s]

I would honestly avoid raid and instead look into setting up a storage system running ZFS. Check out this thread:

http://hardforum.com/showthread.php?t=1573272

In addition to pooling multiple drives into one like raid, ZFS will make it a lot simpler to replace failed disks, upgrade capacity, and, most important of all, to maintain the integrity of your data. You could use this system as a file server in your house to keep your photos accessible on a day-to-day basis.

If I were a photographer, I definitely wouldn't sleep soundly with my photos stored in only one location. For a cheap backup of your primary system, you could consider just picking up a 4TB external drive, filling it up, and storing it somewhere off site like in a safe deposit box or at your friend's house.

 

[ripken204]

I don't even take many pictures and I have ~400GB worth of RAW images from the last few years.
You need to back them up in many places, immediately.

ZFS is nice but not really for a beginner, especially if you don't have a second computer/server to dedicate to storage.

How many 3TB drives to you plan on buying? 3 of them?

 

[Dangman]

What I need is better, more reliable back up solution.

RAID is not backup FYI. It doesn't protect against malicious or accidental corruption, deletion, infection, or relocation of data.

Backup is usually a copy of the data stored on a different medium other than the main source. I.e like an external hard drive that's not always connected, a NAS, file server, or even online backup. Not a RAID array on the sole data source. If you don't want to build a NAS, go ahead and grab a prebuilt one. Hit up www.SmallNetBuilder.com for NAS reviews and such. Should be able to find a NAS that fits your needs, skill level, and budget. If you're willing to build a NAS/File server, let us know and we'll come up with something.

I am going to be using 3TB Seagate Barracuda ST3000DM001's for the drives.

Unfortuantely, due to the fact that those are consumer grade drives, there's a good chance that your drives will not work with the majority of hardware RAID controllers out there. If you're planning on using Windows and RAID, then your best bet would be use enterprise grade drives or Hitachi and Samsung drives for better chances of compatibility. However, if you're willing to build a seperate PC for file serving/NAS duties, that definitely opens up more cost-effective options like ZFS as mentioend by Jaxon_s or Linux.

Do I need to rebuild the raid array if I have to add drives to it? How flexible are the arrays? What type of raid should I use?

If the hardware RAID controller, well most good ones anyway, supports Online Capacity Expansion or OCE, then no, you don't have to reformat the RAID array when you add drives to it. Not sure what you mean by that second question. As for what RAID to use, RAID 5 at a minimum but with that much data, RAID 6 is a viable option. You'll need a minimum of 3 drives for RAID 5 and 4 drives for RAID 6.

 

[brutalizer]

RAID is not backup FYI. It doesn't protect against malicious or accidental corruption, deletion, infection, or relocation of data.

ZFS protects against all of these. You just need to do a snapshot (which takes one second). Then you have taken a "photograph", a hard copy of the current state of the raid. And you can always go back to this snapshot, and back in time. You can have many snapshots. One guy had 64.000 snapshots, which consumed 4GB. Snapshots only store the changes, so they are cheap, fast and safe.

Each snapshot are mounted in Windows as a separate directory, so you can always go back to any snapshot (i.e. visit a different directory).

Snapshots are exactly like Apple's "Time Machine", but they dont cost anything to do, and you dont need a separate drive, you can apply them to the current zfs raid.

You can toy with Solaris / OpenSolaris / FreeBSD by running them virtually in the free software VirtualBox. Just install VirtualBox, download Solaris and install it inside VB.

Also, ZFS is coming to Mac OS X very soon:
http://tenscomplement.com/
They presented a beta version last month, or so.

 

[NukeULater]

Sorry I should have defined exactly what I needed at the beginning. I was short on time when I began this thread. I'm in desperate need of a larger storage drive on my desktop. As I mentioned, I have two 1.5TB drives already in the machine. One is used as a data drive and the other is used as backup. I know you wouldn't call that a backup because it is in one system. But it does not leave me dataless if one disk fails. ZFS is a little to much for me right now. This is the only modern computer in my house and I cannot afford to build a server and add more drives to my main system at this time. (All thanks to college...)

I don't even take many pictures and I have ~400GB worth of RAW images from the last few years.
You need to back them up in many places, immediately.

ZFS is nice but not really for a beginner, especially if you don't have a second computer/server to dedicate to storage.

How many 3TB drives to you plan on buying? 3 of them?

I was thinking about purchasing three of these drives. My image archive has been more than doubling in size per year. Last year was around 500GB, this year I'm looking at about 1TB. That is not including an of the design or video work that I also create. Because of this, single drives are becoming a hassle since the data is spread across them.

Unfortunately, due to the fact that those are consumer grade drives, there's a good chance that your drives will not work with the majority of hardware RAID controllers out there. If you're planning on using Windows and RAID, then your best bet would be use enterprise grade drives or Hitachi and Samsung drives for better chances of compatibility.

Thanks for informing me about that. I didn't realize that there was difference between desktop and server drives.

If the hardware RAID controller, well most good ones anyway, supports Online Capacity Expansion or OCE, then no, you don't have to reformat the RAID array when you add drives to it. Not sure what you mean by that second question. As for what RAID to use, RAID 5 at a minimum but with that much data, RAID 6 is a viable option. You'll need a minimum of 3 drives for RAID 5 and 4 drives for RAID 6.

After reading about the write hole that raid 5 and 6 have, would it be a better option to go with a simple raid 1 for the time being and later adding drives and moving to raid 10?

 

[ripken204]

After reading about the write hole that raid 5 and 6 have, would it be a better option to go with a simple raid 1 for the time being and later adding drives and moving to raid 10?

ZFS will get around the write hole and you are able to do the ZFS version of raid5/6
But do you have a computer you can run it from? You can't run it on windows.

If you did have a storage server and you share the files to windows to edit your images, it would be somewhat slow... Personally I have a RAID0 array on my desktop where I store my last year's worth of images. Editing images is fast this way. Then I copy the photos over to my file server.

 

[NukeULater]

ZFS will get around the write hole and you are able to do the ZFS version of raid5/6
But do you have a computer you can run it from? You can't run it on windows.

My secondary computer is an old overclocked Pentium 3. Which I am sure is not compatible with and not powerful enough to drive these arrays and new drives.

 

[ripken204]

definitely not, ZFS requires a decent amount of ram as well, you should really have at least 4GB.
with a P3 you probably can't even max out to 512mb