Best Massive Storage Solution - NAS/Server/both?

B

bubbles

Limp Gawd
Joined
Dec 16, 2005
Messages
180
What do you guys think is the best storage solution for say 5+ TB of data, with room for expansion? This would not be an enterprise environment, just your home network.

The requirements:

  • Cheapest - $ per GB total cost.
  • Most energy efficient
  • Performance
  • Noise factor
  • Simplicity, convenience and organization
  • Expandability
  • Safety (RAID-5)

The options:

  • One big server (8-24 drives)?
  • A few smaller servers(8 drives)?
  • Hardware or software RAID?
  • One server and a bunch of NAS boxes?
  • No server and only NAS boxes?
  • One server and lots of eSATA enclosures?

Personally I would lean towards a server with an 8-port hardware RAID card and 8 drives in RAID5. The drives would depend on what the current $/GB sweet spot is. When you need to expand just build a new server. This seems like the most flexible and simplest solution to me. It could be the cheapest when you factor in the RAID overhead and use modest components.

What's your opinions?
 
bubbles said:
What do you guys think is the best storage solution for say 5+ TB of data, with room for expansion? This would not be an enterprise environment, just your home network.
Click to expand...
A file server. You could get something like this, which puts 12 drives in 3u. Then use a couple controllers like this. NAS boxes aren't flexible enough or big enough yet.
bubbles said:
Personally I would lean towards a server with an 8-port hardware RAID card and 8 drives in RAID5. The drives would depend on what the current $/GB sweet spot is. When you need to expand just build a new server. This seems like the most flexible and simplest solution to me. It could be the cheapest when you factor in the RAID overhead and use modest components.
Click to expand...

I'd lean toward something software-based you can run Solaris on. It's cheap, and you can run Raid-Z on it. It's like raid 5, but without the so-called "write hole" of raid 5. Performance is quite good for sequential transfers, and not terrible for non-sequential reads.
 
i would personally prefer a file server with hardware raid cards.

depending on budget, u can either buy workstation class or server class motherboards with PCI-X and PCIe slots. PCI-X hardware raid cards are cheaper.
 
naro said:
i would personally prefer a file server with hardware raid cards.

depending on budget, u can either buy workstation class or server class motherboards with PCI-X and PCIe slots. PCI-X hardware raid cards are cheaper.
Click to expand...

With the Solaris/raid-Z, you might be able to avoid buying controllers at all for your first set of disks. Intel and Silicon Image controllers are supported, and NVidia support is on the way. With some boards coming with 6 disks (like this one, for example) and room for 5 8-port controllers, you could easily put 46 disks on it. Spend $310 on the board, and then $300ish for each additional controllers, or spend $150 on a board and $600 for each controller - but I know which way I'd go. I threw together a system at Newegg that'd hold 12 drives out of the box (and 22 with a little more investment) for just over $1800. Add 12 1TB drives and a boot disk for $4100, and $500 worth of controllers, and you get 12 TB of capacity (before parity - after you'll probably get 10, depending how you work it) for about $6280 shipped. I'm not skimping on costs, either - this is a fully loaded system, with 4 gigs of memory, 8 Barcelona cores, a Supermicro board, a Corsair power supply, and a Lian-Li case. The memory is ECC.

You can do it cheaper, of course - get a consumer-level board, a single processor, and so on and so forth - but even if you knock off a thousand dollars (which might be doable, if you really pinch) your final cost per gigabyte is 52 cents instead of 63. I'd rather spend the extra money and get more room for expansion, bandwidth, processing power, and generally a more future-proof machine.
 
The problem with "large" storage is that the bigger you get, the more you have to lose. In order to compensate for potential lost, cost goes up. Therefore the whole concept of $/GB is moot IMO when it comes to the "storage" design.

Ask yourself some questions.

1. How much uptime do you want?
2. How much are you willing to spend (on average) per year keeping the box running?
3. What is your recovery plan for a complete box failure (you have a PSU which doesn't have an OV crowbar) and lose all the data?
4. How will you maintain data integrity and box health?
5. How much EFFORT per year are you willing to spend?

As you can see..my questions are different than yours.

I mean the cheapest solution is to just by a bunch of external HDD's on a USB hub and back them all up to DVD. However, while the cheapest...it is the most time intensive to maintain. The least time intensive but most costly would be two seperate RAID5 systems and different colo's where one mirrors the other with weekly tape backups.

Even on the front page of [H]OCP, some guy just lost 15 years worth of work because he kept his "backup" at his house. Bad move...very bad move.

I guess what i'm really trying to say is there is no "fits all" solution. Because honestly, that is what your requirements are leaning towards. When you say safety and RAID-5 in the same sentance...i'm not sure you understand the true purpose of RAID-5. May the correct way to answer your question is how much storage at what uptime. Backup is a not a backup unless you copy that data to another location.
 
Well it was more of a theoretical question just to see what people would go with. Right now I use a file server with a RocketRAID 1820a card and 8 SATA drives in RAID5. I use it mainly for movies and TV so it's no big deal if I lost the data. I also have about 10 old 250GB and 300GB PATA drives which i want to add to the server somehow. Then maybe expand further later on.

I said in the post it's for a home network, for hobby purposes so there is no need to start colocating mirror servers and getting high end equipment. Uptime - don't care. Is the cost of redundant PSUs worth my Twin Peaks episodes? No. Maintenence - thats the fun part. Running costs - cheap as possible.

As for RAID-5 and safety, it's safer than JBOD or RAID-0, but there can still be file system problems. I wouldn't pay $1000s of dollars just to backup 5TB of entertainment data. I wouldn't even bother backing that much data up on DVDRs. RAID-5 is better than nothing and it has the lowest overhead. I've had my server running for a couple years and never had a problem, unless 2 disks fail at the same time one day. I use the JFS file system under Linux.

After looking at the options now I think it would be easiest to start with a pretty simple server then add something like that Norco eSATA enclosure with software RAID when you need to expand. You could have 4 or 5 big arrays that way.
 
longblock454 said:
HighPoint has a new controller capable of 40 drives. That and 3 of those Norco edge boxes would be quite trick!
Click to expand...

That requires using port-multipliers though. Has anyone found them for less than $80?

unhappy_mage said:
Add 12 1TB drives and a boot disk for $4100, and $500 worth of controllers, and you get 12 TB of capacity (before parity - after you'll probably get 10, depending how you work it) for about $6280 shipped. I'm not skimping on costs, either - this is a fully loaded system, with 4 gigs of memory, 8 Barcelona cores, a Supermicro board, a Corsair power supply, and a Lian-Li case. The memory is ECC.
Click to expand...

That sounds like teh sex! I need a well paying job.
 
longblock454 said:
The Norco chassis has them built in..
Click to expand...

The linked Norco does not. It provides 12 HDDs and has three ports. The multi-lane cables combine four channels into a single cable. You would need a controller card that has three mini-SAS connectors or use a PM to get the extra four channels.
 
drizzt81 said:
The linked Norco does not. It provides 12 HDDs and has three ports. The multi-lane cables combine four channels into a single cable. You would need a controller card that has three mini-SAS connectors or use a PM to get the extra four channels.
Click to expand...

This one does. It's $800 instead of $640, though, which is pretty much the cost of two port multipliers. And much to my dismay, they broke the four sata ports out - it's not a single Infiniband connection.

As much as I dislike reading Newegg reviews, they're not encouraging in this particular case. "Flimsy", "Flaky", and similar words appear too many times to be a coincidence. It's a very nice concept, and if someone would build a disk-only chassis with a SAS expander in it, I might be impressed enough to buy one. If it had a disk layout like a Thumper tied to a pair of wide ports, say, I'd be at my boss's desk the next day to recommend one.
 
Actually on fleebay right now there are a few good used servers. One has a 16 bay SATA drive set with control card (supports RAID 6 and 50 as well), motherboard, processor, redundant supplies, memory, and cables. All that is needed is the HDD's. The RAID card alone MSRPs for $1000. My guess is when this goes it will go for about $1k when it is really worth nearly $3k new.

Search for supermicro on eBay...it might take a while, but server pulls in terms of case/drivebays/PSU is hard to beat.
 
Trepidati0n said:
I saw those...still too expensive. FBDIMM's...cringe. A base price of $3k is too high. I would easily pay $1000, maybe pay $1500...but $3k is GTFO to me.
Click to expand...

Yes, $3k is a bit on the high end. I'm just lucky enough to get this hardware and be able to utilize it, and get to put it into a production environment.

good luck in your findings!
 
Sniper|3d-R| said:
I've got several of these I'll be using, just need some software, and so far my testing, and research has found shitty backup software. :mad::(
Click to expand...

What medium are you planning to back up to? What OS will you be running?
 
I'm actually setting myself up to retire my RAID-5, 6 drive server. It has 6 300GB drives, a 3Ware 9500S-8 Sata controller, and the box runs on Windows 2003 Server Enterprise.

I'm tired of the noise, the heat and the maintenance of Windows.

I'm moving to NAS boxes.

I've got a QNAP TS-201 with two 320GB drives that I've had online for about 6 months and it's been flawless.

I just installed a new QNAP TS-209 Pro yesterday, with two 500GB drives.

I usually buy drives at the 'sweet spot'. 6 months ago, it was the 320's. This week, it was the 500's.

In a year, it will probably be the 1TB drives.

These QNAP boxes are gems. They are very small, built like tanks, and actually look great. They are Linux based and quite powerfull.

Someone mentioned that NAS boxes were not flexible enough. You should look at the TS-209 Pro.

It is a 12-in-1 server:

File Server
The basic purpose of File Server is to allow users to store and share files. Users often concern that their private data or confidential files would be accessed by others. TS-209 Pro allows server manager to create user ID and password, and define the authority and quota for each user. Thus personal data stored on TS-209 Pro is well protected.

FTP Server
Having troubles sending large files by e-mail due to the limited capacity? You can establish your own FTP server by using TS-209 Pro. You can manage the access authority of the FTP server and share your files with your friends or customers easily in a few steps.

Backup Server
Data backup may sound time-consuming and painful for most users. QNAP's Turbo Station now brings you the enterprise-class technology and make backup extremely simple and efficient! The most intelligent automatic backup software, NetBak Replicator, is provided for users to perform real-time synchronization or schedule backup from multiple PCs to TS-209 Pro.Enterprise users can also use their existing backup software (e.g. Arconics True Image, CA BrightStor ARCserver Backup, EMC Retrospect, etc.) to back up their important data to TS-209 Pro.

RAID-1 Mirroring with Hot-swap Design
What would you do to have the data back? TS-209 Pro supports RAID 1 Mirroring for non-stop service and easy maintenance. When data crash occurs, the administrator can be alerted by e-mail alarm and LED indication. The hard disk drive can be replaced by hot swapping without turning off the server. The intelligent data rebuilding capability allows users to replace a failed drive by hot-swap. This enables automatic data rebuilding to the new drive without suffering from any system downtime.

Remote Replication
Remote Replication provides a superior level of data redundancy. Effectively, Qnap Remote Replication is the most affordable way in the industry for data disaster recovery. Files can be backed up from TS-209 Pro to other QNAP NAS products in remote locations with or without encryption. Other advanced options such as data compression, block level incremental backup, and data synchronization save tremendous bandwidth and time for data backup.

Web Server
Owning a web server is no more a dream! With the dual support of MySQL and SQLite, SMB & SOHO users can easily set up their own web server. With the editable php.ini support, there is no need to go through all the hassles of setting up a web server as in a traditional Window PC based server. You can either develop programs by yourself or download a variety of open-source applications, such as online shops, forums, blogs, etc., over the Internet.

MySQL Server
TS-209 Pro is the most affordable choice as a standalone database server for system integrators. It can be applied flexibly in various deployments such as a database server of another web server in remote site or as an additional backup database server to achieve system redundancy.

Printer Server
You can share your printer device over the network by simply connecting a USB printer to TS-209 Pro's USB port and enabling the printer sharing function.

UPnP Media Server (with built-in TwonkyMedia Server)
TS-209 Pro is a perfect 2-bay media storage center and it works well with UPnP/DLNA (Digital Living Network Alliance) compliant media players. With the combination of TS-209 Pro and media player of your choice, you can play the photos and videos stored in TS-209 Pro directly to TV with remote control and without the presence of PC. In addition, you can listen to your favorite music and Internet Radio in the same way too.

Photo Sharing
Now you can share photos with your family and friends around the world by TS-209 Pro. By uploading your photos to TS-209 Pro, the server automatically generates a web-based photo album with thumbnails and details such as date, resolution, focal length, aperture, etc. You can browse and share the digital photos in a convenient way!

Download Station
TS-209 Pro serves as a continuous download machine without turning on your PC. TS-209 Pro supports download not just through BitTorrent but also through FTP and HTTP. Users do not have to worry that downloading torrent files, ftp, or thousands of pictures in http will heavily impact their normal PC workload because TS-209 Pro will take up the task professionally. Users can download directly through web browser or they can install Qnap's proprietary QGet download utility. The great web-based interface allows users to manage the download tasks anytime, anywhere! With QGet, users can even do multiple NAS download management.

iTunes Server
Music fans and Apple enthusiasts will be certainly rocked by the coolest music-sharing feature of TS-209 Pro! All your MP3 music can be collectively stored and streamed to Mac or Windows PC. Just turn on the iTunes program and enjoy all the music around your home network!

It also supports streaming to Xbox 360 and PS3.

It sure isn't the cheapest. The TS-209 Pro is $400 bare. Personally, I think it's worth every penny.

If you want the absolute most storage for the cheapest price, then I do think a server, with an ACCELLERATED Sata II card is the way to go, but a really good accellerated Sata II card worth it's salt is going to run you at least $500 alone.

But I've really become a believer in these QNAP boxes. Small, quiet, powerfull, does just about everything, and a snap to set up and maintain.
 
You must log in or register to reply here.
Back
Top