NAS or other options

[crazjayz]

Introduction:
So here's the story. The lab that I work at is looking for an easier, more reliable method of backing up data. Right now we use a pair of external hard drives to copy data from each computer that every researcher has. I have pitched the idea of a NAS or server-based data storage, and my co-workers think we should implement it. Our current situation, with each research saving data onto their personal computers, is 1) slow with regards to data sharing and 2) problematic when it comes to backing up data. Talking to my co-workers, we have settled on a 1TB storage solution, and right now we're leaning towards a NAS (Buffalo Terastation Pro II or Infrant Technologies ReadyNAS NV). With that out of the way, here are the questions:

1. Which of is more reliable?
2. Which is easier to use?
3. Would you suggest anything else? (ie: an older computer with a number of hard drives)

What We Need:
1. 500GB of drive space.
2. Some sort of backup/redundancy (which type of RAID would be best? 0? 0+1?)
3. Relatively small, quiet, and low power consumption
4. Easy to use interface
5. Low maintenance
6. Password protection at separate file levels
7. Must be accessible from any computer via username/password
8. Costs no more than $1000


Alright, so that's the situation. As usual, all comments are appreciated. Honestly, I can't say what's the major differences between the Terastation and the ReadyNAS, but right now, we're leaning towards the 1TB Terastation Pro II, only because of the price.

 

[Madwand]

Introduction:
So here's the story. The lab that I work at is looking for an easier, more reliable method of backing up data. Right now we use a pair of external hard drives to copy data from each computer that every researcher has. I have pitched the idea of a NAS or server-based data storage, and my co-workers think we should implement it. Our current situation, with each research saving data onto their personal computers, is 1) slow with regards to data sharing and 2) problematic when it comes to backing up data. Talking to my co-workers, we have settled on a 1TB storage solution, and right now we're leaning towards a NAS (Buffalo Terastation Pro II or Infrant Technologies ReadyNAS NV).

External USB2 drives are likely to be faster than the above consumer NAS boxes. See below for some performance data on consumer NAS boxes. Pay attention only to the large file size data, and even then understand that that data's somewhat polluted with cached performance information, so the real performance may be less than that given by the charts:

http://www.smallnetbuilder.com/component/option,com_nas/Itemid,190/chart,12/

If you're not getting good performance from external USB, here are some potential causes:

1. You're using USB 1.x, not 2.0
2. Your network/computers are only 100 Mb/s

(1) Should be easy to resolve with an add-in controller if needed.

You'd have to resolve (2) in any case before you could hope for any improvement from a NAS box.

 

[Orinthical]

1. Which of is more reliable?

Between server-based or NAS? Assuming both use RAID based storage with some sort of parity or mirroring, they are about the same. Warranties are usually longer on server hardware than on NAS devices but they cost more. It depends on what you want; If you want easy access to storage but nothing else, go with a NAS. If you want to do other things than storage, look at a server.

2. Which is easier to use?

A NAS device is generally easier to use.

3. Would you suggest anything else? (ie: an older computer with a number of hard drives)

A NAS device is probably your best bet IF you do not have a corporate IT department who could provide you with storage via a network share.

What We Need:
1. 500GB of drive space.
2. Some sort of backup/redundancy (which type of RAID would be best? 0? 0+1?)

RAID 5 will provide you a good mix of storage space and security, (750GB) while RAID0+1 will provide a good mix of performance and security (500GB). In your case, I'd say go with RAID5.

3. Relatively small, quiet, and low power consumption
4. Easy to use interface
5. Low maintenance
6. Password protection at separate file levels
7. Must be accessible from any computer via username/password
8. Costs no more than $1000

Alright, so that's the situation. As usual, all comments are appreciated. Honestly, I can't say what's the major differences between the Terastation and the ReadyNAS, but right now, we're leaning towards the 1TB Terastation Pro II, only because of the price.

Go for it. I wouldn't use "password protection" though, set up shares and usernames/passwords for each user. If the software supports locking down the server by MAC address or by IP range, I'd also look at doing that just to make it a bit more secure.

 

[crazjayz]

External USB2 drives are likely to be faster than the above consumer NAS boxes. See below for some performance data on consumer NAS boxes. Pay attention only to the large file size data, and even then understand that that data's somewhat polluted with cached performance information, so the real performance may be less than that given by the charts:

http://www.smallnetbuilder.com/component/option,com_nas/Itemid,190/chart,12/

If you're not getting good performance from external USB, here are some potential causes:

1. You're using USB 1.x, not 2.0
2. Your network/computers are only 100 Mb/s

(1) Should be easy to resolve with an add-in controller if needed.

You'd have to resolve (2) in any case before you could hope for any improvement from a NAS box.

I guess I wasn't clear on what I was referring to when I said that our current external hard drive method wasn't "fast enough" I'm not referring to the actually speed of the final transfer (which is USB 2.0), but the actual process of having each user connect the external, copy the data, and disconnect the external hard drive. We're tired of doing this and we would like our data to be saved at one location (not at each workstation) with some sort of redundancy and backup. But thank you for the comment, and I shall look into the link you posted. As far as file size goes, none of our files are larger than 20MB, but we have thousands of these "small" files. They just sort of add up.

And Orinthical thanks for your comment as well. You suggested RAID5, but honestly we don't need that much space right now, so I think we'll be going 0+1. As far as the whole username/pasword thing, it's not that we're terribly concerned with our data being extremely sensitive and classified, rather it's more of a worker-worker relationship thing. One of my co-workers was just asking if she could have her own folder with a password that was different than the main folder password. For example:


|Main folder (password = Lab)
_____|----- Co-worker Folder (password = Jane)
_____|----- My Folder (password = John)

That's what I was inquiring about. If some sort of setup like this is possible.

If anyone out there has a Terastation, PLEASE comment on your experience with it, it's ease of use, and the "privacy protection" of the folders, and anything else that my lab and I should possibly consider. Thank you!

 

[Haven]

PLEASE NOTE: RAID does not equal backup!! If the motherboard on the NAS box goes out you may lose your data.

What a NAS will do for you is give you a workgroup server to store data, backing the NAS box up to an external HD would still be prudent. I would also suggest taking that drive off site for line of business reasons.

If I were doing this, I would see if there is a way to back up the NAS box to an external HD, then get a safety deposit box at a bank and use two of these drives. One would always be at the bank, and the other would be at the office for backups. Once a week switch the two drives. This way if something happens to the building you can rebuild, and only be missing around a weeks worth of work.

 

[Madwand]

...thank you for the comment, and I shall look into the link you posted. As far as file size goes, none of our files are larger than 20MB, but we have thousands of these "small" files. They just sort of add up.

The Small Net Builder's "small file" performance results are highly misleading, so I'd like to steer your away from even looking at them significantly. The problem is that they're not actually measuring small file performance, but rather testing overall performance using very small files. This does not get very useful information -- rather, it measures something complex about how caching behaves when the files are small enough to fit entirely in various caches, potentially going down as far as the CPU cache in some cases. This is "interesting", but mostly useless in terms of interpreting a file server's performance -- because it bypasses the file server's drives.

For this reason, I'd encourage you to largely ignore anything but SNB's largest file size results -- these would exceed the RAM cache size on the server and then actually involve the server's drives and file system in the results.

Moreover, contrary to what SNB's results show, small files actually perform worse than large files, due to the file system, security, and connection overhead involved. It's more efficient to transmit a very large file than to transmit a number of small files, starting and stopping and performing administrative tasks for each. So the "large file" results are actually still more representative of small file performance than the "small file" results.

There are some instances in which server caching performance is relevant -- e.g. accessing the same relatively small files over and over across the network. OTOH, if this sort of performance is really important to you then (a) again, you should get out of consumer NAS box performance levels, as these are easily out-performed by affordable well-designed file servers (note also that SNB's "DIY file server" is not a well-designed file server IMO) (b) get a box where you can install a lot of RAM yourself, beyond the 64-256 MiB as is typical in consumer NAS boxes.

Finally, "RAID alone is not a backup". This means that if you put any original data on the file server, you should not rely on RAID alone to be its backup. You should consider further backup needs when planning a file server. You could re-use the external drive or enclosure for this purpose.

 

[crazjayz]

UPDATE:
Thank you very much Madwand and ianshot for you comments. ianshot, I know that RAID is not a substitute for backing up, that's why I say redundancy. We will still use our external harddrive for a true backup, but storing it in a safety deposit box is a little too intense for us.

Now, I think there might be a slight hitch in plans, and possibly in the function of this NAS/server/whatever thing. 1) It will need to be both Apple and Windows compatible and 2) it will be constantly used during work. I may have hinted at this before, but I don't believe I was very clear, so please let me try again. What we're looking for is some sort of external storage system that will be constantly accessed by users during work hours and also serve as a "backup". For instance, if we had a 4x250GB drive system, we may set up a RAID 0+1:
______
Drive 1|
Drive 2|
_______
Drive 3|
Drive 4|

This way, Drives 1 and 2 [set A, 500GB] are in RAID 0 for performance and Drives 3 and 4 [set B, 500GB] mirrors [set A] for redundancy. Day-to-day activities (opening, saving, editing, storing of files) will be only on [set A], the data on [set A] will be copied to [set B] but not accessed. This way, just in case Drives 1 or 2 fails, we have 3 and 4 to rebuild the data, which are untouched. Secondary to this will be an external drive (Drive 5) that will be used as a true backup, probably weekly, for our data.

A couple concerns still remain, which I personally don't know. Would RAID 5 have the same amount of "speed" and fault tolerance as a 0+1 (writing to both sets at the same time) or a dual 0 configuration (using [set A] during the day and copying data to [set B] every night). Finally, we have 5 researchers in our lab using the computers at any given time. If all 5 clients were accessing the NAS, reading and writing files, would something like the Terastation Pro II be enough, or should we just consider a true server?

 

[CaseyBlackburn]

If you can manage it, I'd get something like a Drobo and use a regular server to share it out. The Drobo in my eyes looks to be quite promising in terms of redundancy. The sad thing is it only has connection ports by the means of USB, so you need a computer to share it out. It is kinda costly especially since it doesn't include drives, but I think it's worth it.

 

[Madwand]

...we have 5 researchers in our lab using the computers at any given time. If all 5 clients were accessing the NAS, reading and writing files, would something like the Terastation Pro II be enough, or should we just consider a true server?

This depends a lot on the usage pattern details. If users are accessing 20 MB files individually at human speeds, then anything will do -- 20 MB across at 100 Mb/s will take around 2s, and that overall would be dominated by human time, not server time. And with 5 users, each 2s access would be over quickly, so there wouldn't be much likelihood of heavy concurrent access of the server.

At the other extreme, if users are accessing hundreds of 20 MB files rapidly/frequently, then you can bottleneck easily at the server and the network, and getting out of consumer NAS class would be worthwhile.

I'd want to study the usage pattern experimentally -- e.g. by setting up a desktop temporarily as a share point, and observing and logging accesses to that drive and network. Ideally, you'd get some meaningful feedback from a single user's experience as a starting point. The next step would be to evaluate concurrency / multi-user load behavior.

Note that a consumer NAS box will likely be no faster than a modern desktop, and moreover such a desktop has a good chance of being faster.

 

[Orinthical]

One note of clarification: RAID0+1 is not the same as 1+0, also know as RAID10. The Terastation in-fact offers RAID10 or 1+0, not 0+1, which is great if you can stomach the cost. RAID 10, after all, is the king of reliability but at a cost for both hardware and performance.

1) It will need to be both Apple and Windows compatible and 2) it will be constantly used during work. I may have hinted at this before, but I don't believe I was very clear, so please let me try again. What we're looking for is some sort of external storage system that will be constantly accessed by users during work hours and also serve as a "backup".

A NAS will offer that, so will a server. You're talking about five users - I'm sorry to say this is not an enterprise deployment here. You will not come close to pushing the limits of any well built NAS device out there. I would never recommend a NAS solution for a large workgroup or an enterprise as they simply don't offer enough performance to be a primary file server for large situations but they are ideal for small businesses and workgroups that need a centralized file storage system on the cheap. As for Apple compatibility, if these are OS X systems they can mount any shares you create using SMB so you'll be fine. The only thing you (probably) won't be able to do is use the "easy setup software" that the system comes with... but if you know how to connect to a file share in OS X, you should be fine without it.

This way, Drives 1 and 2 [set A, 500GB] are in RAID 0 for performance and Drives 3 and 4 [set B, 500GB] mirrors [set A] for redundancy. Day-to-day activities (opening, saving, editing, storing of files) will be only on [set A], the data on [set A] will be copied to [set B] but not accessed. This way, just in case Drives 1 or 2 fails, we have 3 and 4 to rebuild the data, which are untouched. Secondary to this will be an external drive (Drive 5) that will be used as a true backup, probably weekly, for our data.

That's not quite how RAID works, in any flavor. RAID1+0 (RAID 10) is a raw performance meets redundancy configuration where data is first mirrored and then striped across the two sets of mirrors.RAID5 works by distributing parity bits across all members of the raid set so that if any one of the drives fails, data can be rebuilt using the parity.

Some more on the subject:
RAID 5 vs 10: http://blogs.zdnet.com/storage/?p=131
RAID 10 (and every other level explained): http://www.acnc.com/04_01_10.html

A couple concerns still remain, which I personally don't know. Would RAID 5 have the same amount of "speed" and fault tolerance as a 0+1 (writing to both sets at the same time) or a dual 0 configuration (using [set A] during the day and copying data to [set B] every night). Finally, we have 5 researchers in our lab using the computers at any given time. If all 5 clients were accessing the NAS, reading and writing files, would something like the Terastation Pro II be enough, or should we just consider a true server?

Your budget is $1000 and for that price you're far better off going with a NAS device as a server with the amount of storage you're looking for will most definitely exceed that. I'm not talking about a home-brew thrown-together system here, I'm comparing NAS devices to real servers.

Whatever you decide on do not go RAID 0 / JBOD - even if you back up your data, you don't want the downtime and possible data loss you'll incur if you don't plan for some redundancy. RAID5 if you want performance while still being able to lose one physical drive. RAID 10 if you want an extra level of redundancy and can stomach a little less performance and a little less usable space. RAID 5 will probably yield somewhere around 690-700MB or formatted space while the RAID10 will give you 430-450.

 

[crazjayz]

This depends a lot on the usage pattern details. If users are accessing 20 MB files individually at human speeds, then anything will do -- 20 MB across at 100 Mb/s will take around 2s, and that overall would be dominated by human time, not server time. And with 5 users, each 2s access would be over quickly, so there wouldn't be much likelihood of heavy concurrent access of the server.

At the other extreme, if users are accessing hundreds of 20 MB files rapidly/frequently, then you can bottleneck easily at the server and the network, and getting out of consumer NAS class would be worthwhile.

I'd want to study the usage pattern experimentally -- e.g. by setting up a desktop temporarily as a share point, and observing and logging accesses to that drive and network. Ideally, you'd get some meaningful feedback from a single user's experience as a starting point. The next step would be to evaluate concurrency / multi-user load behavior.

Note that a consumer NAS box will likely be no faster than a modern desktop, and moreover such a desktop has a good chance of being faster.

For our day-to-day usage, it's mostly just opening individual files, editing, and saving them. Nothing in terms of bulk or speed, so I'll assume no bottlenecking. And yes, we're running on 100Mbit lines... maybe even 1000, but we're using CAT5 not CAT6 cables... Anyway, thank you very much Madwand for your help and rapid response to my questions.

Orinthical, your knowledge as been invaluable as well, as has your rapid responses. Thank you very much for the two links, it definitely helped me see what you were talking about. In looking at our needs, I feel, with the help of the comments here, that a 1TB Terastation Pro II setup as a RAID10 would be best for our lab. 430-450, even 400GB, is enough space for us, and the increased redundancy compared to RAID5 is a boon. I suppose I'll keep you guys posted on the final decision (determined by my boss), but since I'm the most knowledgeable on the subject (thanks to you guys), I'm pretty sure we'll settle on the Terastation. Once again, thanks to everyone who read and posted their comments.