• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Crazy Storage Question

I

ibdoomed2

Limp Gawd
Joined
Jun 22, 2009
Messages
142
I was showing a friend my media center system (with a pathetic 12tb) and he was wondering if I could build one for him. He's and old collector of dvds and now blu and thinks he has in the neighborhood of 1500 movies and growing. With money being no problem we're wondering how crazy one could get.

Would this work?

Motherboard: http://www.newegg.com/Product/Product.aspx?Item=N82E16813151096
4 of these: http://www.cdw.com/shop/products/default.aspx?EDC=1785113
2 of these: http://www.cdw.com/shop/products/default.aspx?EDC=953632
along with the 6 onboard sata ports (subtract 1 for OS drive and 1 for BD) giving a total of 52 sata ports and 104tb of raw storage using the WD 2tb drives: http://www.newegg.com/Product/Product.aspx?Item=N82E16822136344


Would WHS choke?
 
pretty nutsy idea
how many cases you know who can hold 52 drives? :D
despite no $ limit, there probably is size/complexity limitations ...
if so what are they?
one case? unlimited case number? racks?
 
go with the least complex system... meaning the least number of drives to get to your goal, real Raid controller.
To many drives will start braking down on him eventually, than you will be stuck with maintaining him...
 
asgards said:
pretty nutsy idea
how many cases you know who can hold 52 drives? :D
despite no $ limit, there probably is size/complexity limitations ...
if so what are they?
one case? unlimited case number? racks?
Click to expand...

Not a one. I've seen some supermicro's that do 24 but I've never looked for anything bigger. We're still in the nutso stage so off the top of my head I'd just toss 20 in a norco and stack the rest on top. That's probably what google is doing =P
 
Assuming half his movies are blu-ray (conservative) he'd need about 25TB to store rips. You can easily fit that in a Norco 4020 filled with 2TB drives, although future expansion may be a concern. Depends on if your friend wants straight rips or if he'll do encodes/remove special features.

I don't think WHS would choke on that much storage, but I'm not sure what the limit is.
 
I don't think he nearly needs that much storage. But lets assume does.

You won't choke WHS and you wont have an issue with those controllers or that many drives.
 
My suggestion would be multiple Norco boxes. The only downside is multiple sources, but your front end player should be able to handle it fine.
 
Well like stated above you can fit 40tb in a single norco, which would be plenty of space for his movies.

So
Norco - $300
Mobo with onboard video, and at least 3 x8+ slots. - $150 MSI K9A2 Platinum comes to mind(4 x16 slots)
3x SM SASLP-MV8's - $300
20 2tb drives. $4500
WHS - $100
Then when you want to Expand just get a Chenbrok UEK SAS expander and put that into another norco with just a PSU and Drives.

This way when/IF you need more space you can add it to the same system easily using another case, and you dont have to buy mobo, ram, cpu ETC.

No WHS wont choke....its just Server 2003.
 
nitrobass24 said:
Well like stated above you can fit 40tb in a single norco, which would be plenty of space for his movies.

So
Norco - $300
Mobo with onboard video, and at least 3 x8+ slots. - $150 MSI K9A2 Platinum comes to mind(4 x16 slots)
3x SM SASLP-MV8's - $300
20 2tb drives. $4500
WHS - $100
Then when you want to Expand just get a Chenbrok UEK SAS expander and put that into another norco with just a PSU and Drives.

This way when/IF you need more space you can add it to the same system easily using another case, and you dont have to buy mobo, ram, cpu ETC.

No WHS wont choke....its just Server 2003.
Click to expand...

Good luck finding the Chenbro. Took me over a month to find the non UEK CK13601
 
I thought I read somewhere that Chenbro was coming out with new revisions of their SAS expanders and that's why the current ones have disappeared from the market, clearing out old stock. Can anybody confirm this?
 
Last edited:
epimetheus said:
I though I read somewhere that Chenbro was coming out with new revisions of their SAS expanders and that's why the current ones have disappeared from the market, clearing out old stock. Can anybody confirm this?
Click to expand...

Thats the word on the street.
 
nitrobass24 said:
Well like stated above you can fit 40tb in a single norco, which would be plenty of space for his movies.

So
Norco - $300
Mobo with onboard video, and at least 3 x8+ slots. - $150 MSI K9A2 Platinum comes to mind(4 x16 slots)
3x SM SASLP-MV8's - $300
20 2tb drives. $4500
WHS - $100
Then when you want to Expand just get a Chenbrok UEK SAS expander and put that into another norco with just a PSU and Drives.

This way when/IF you need more space you can add it to the same system easily using another case, and you dont have to buy mobo, ram, cpu ETC.

No WHS wont choke....its just Server 2003.
Click to expand...

Also, is the SM SASLP-MV8 compatible with the Chenbro SAS expanders? I don't recall seeing it on the compatible cards list.
 
I'll go ahead and confirm it. They're updating the whole line there.

As for price no subject build?
I can do 40TB usable RAID5 at around.. $24K I think it was, based on a hardware RAID5 build. I don't know how WHS feels about that; with that many drives, you really need hardware RAID, to handle the inevitable failures gracefully.

The recommendation for desktop boards, much less MSI, is just stupid. Money is not an object and we're talking 20TB+. Why are you wasting time on trash desktop junk? And ignoring future expansion on the system side, like DVR and such? Bleh.

So let's get underway. Bad case choices abound. Chenbro is good, but we'll be going with something much, MUCH bigger. Like the RM91250, a 9U 4x1650W fully redundant PSUs and 50 drive bays. Take a look. It's about $3800.
http://www.chenbro.com/corporatesite/products_cat.php?pos=24
Motherboard, why would you waste money on anything other than Supermicro? Figure he wants a 3-5 year usable lifespan out of the system, realistically. That means he needs serious expansion room overall. Because of the nature of the beast, we'll prioritize I/O and processor over memory. Memory isn't likely to get too silly. This leads us to the X8DTH-6F with dual socket 1366, QPI, onboard LSI 8 port SAS, and 7 PCIe8x Electrical. We could also use any other board in the X8DTH family. The design is beastly, and will handle as much disk as you can possibly throw at it with room to spare.
http://www.supermicro.com/products/motherboard/QPI/5500/X8DTH-6F.cfm
Processor doesn't matter - we'll say single 1.86GHz or thereabout and 6GB to start with. We need enough CPU to rip DVD and Blu-Ray, so any Xeon fits the bill. Memory we don't care; 2GB+ is fine. We go with 6GB to get triple-channel and not need to upgrade the memory for a good length of time.
RAID by Areca 1680i's and onboard. Dual high speed SAS 300GB in RAID1 for the OS/Landing, which will make life happy. Then we'll be stuffing in two Areca 1680i-24's.

That gives us a system good to 40TB+ with 1TB and 80TB+ with 2TB drives (when support is added by Areca) with a usable lifespan somewhere at 5+ years. It also allows us 5 PCIe slots for adding tuners for DVR or what have you. How about decode/encode accelerators and whatnot? Multiple independent HDMI output! Plenty of room. And let's say he exceeds 80TB; okay. Add a storage chassis in say 5 years with another 1680i using external connectors.

Overkill? Oh HELL yes. Deliciously upgradable? You betcha! Going to last him way longer than he's going to expect? I don't design 'em any other way. :)
 
AreEss said:
I'll go ahead and confirm it. They're updating the whole line there.

As for price no subject build?
I can do 40TB usable RAID5 at around.. $24K I think it was, based on a hardware RAID5 build. I don't know how WHS feels about that; with that many drives, you really need hardware RAID, to handle the inevitable failures gracefully.

The recommendation for desktop boards, much less MSI, is just stupid. Money is not an object and we're talking 20TB+. Why are you wasting time on trash desktop junk? And ignoring future expansion on the system side, like DVR and such? Bleh.

So let's get underway. Bad case choices abound. Chenbro is good, but we'll be going with something much, MUCH bigger. Like the RM91250, a 9U 4x1650W fully redundant PSUs and 50 drive bays. Take a look. It's about $3800.
http://www.chenbro.com/corporatesite/products_cat.php?pos=24
Motherboard, why would you waste money on anything other than Supermicro? Figure he wants a 3-5 year usable lifespan out of the system, realistically. That means he needs serious expansion room overall. Because of the nature of the beast, we'll prioritize I/O and processor over memory. Memory isn't likely to get too silly. This leads us to the X8DTH-6F with dual socket 1366, QPI, onboard LSI 8 port SAS, and 7 PCIe8x Electrical. We could also use any other board in the X8DTH family. The design is beastly, and will handle as much disk as you can possibly throw at it with room to spare.
http://www.supermicro.com/products/motherboard/QPI/5500/X8DTH-6F.cfm
Processor doesn't matter - we'll say single 1.86GHz or thereabout and 6GB to start with. We need enough CPU to rip DVD and Blu-Ray, so any Xeon fits the bill. Memory we don't care; 2GB+ is fine. We go with 6GB to get triple-channel and not need to upgrade the memory for a good length of time.
RAID by Areca 1680i's and onboard. Dual high speed SAS 300GB in RAID1 for the OS/Landing, which will make life happy. Then we'll be stuffing in two Areca 1680i-24's.

That gives us a system good to 40TB+ with 1TB and 80TB+ with 2TB drives (when support is added by Areca) with a usable lifespan somewhere at 5+ years. It also allows us 5 PCIe slots for adding tuners for DVR or what have you. How about decode/encode accelerators and whatnot? Multiple independent HDMI output! Plenty of room. And let's say he exceeds 80TB; okay. Add a storage chassis in say 5 years with another 1680i using external connectors.

Overkill? Oh HELL yes. Deliciously upgradable? You betcha! Going to last him way longer than he's going to expect? I don't design 'em any other way. :)
Click to expand...

Yea and spending 24k to backup 12k worth of movies is reasonable :rolleyes:

Also a Dual Nehalem board for a fileserver? AreEss cmon man seriously? were streaming movies not launching spacecraft.
There is absolutely no reason to for that kind of processing power for this.

Even ripping blurays is not compute intensive task. Your limited by disk speed.
Now if he wants to transcode them to another format then you would need processing power.....but again whats the point of transcoding if you have this much storage?

The only thing that transcoding will accomplish is degraded video quality, and wasted time.
Yea you can make the file sizes smaller.......BUT theres 40TB of space.
You would need 2000 full bluray ISO's before it even filled up more if got rid of the extras.

Let alone the electricity required for something like this.

Also Raid5 to gracefully handle drive failures?
50 disk Raid5 will not handle drive failures gracefully under any circumstances.
Do you know how long a rebuild like that would take? It would take DAYS
The chances for a URE or power failure or another drive failure are pretty good, and then your EFFd.

So for conclusion

My setup: 40TB ~$5K
AreEss Setup: 40tb ~$24k

Yea his disk IO will be higher, and sequential writes will be higher.....but who cares because HD movies use around 10MB/s peak and no matter what you are still limited to gigabit ethernet.
Whats next AreEss? does this guy needs to get run 40Gbps QDR IB throughout the house too:rolleyes:

O yea did I mention that my setup could run 24\7 for 38 years before the initial cost, and electricity cost would equal AreEss setup?
Assuming ~600w @ $0.10kwh
 
shabazkilla said:
Also, is the SM SASLP-MV8 compatible with the Chenbro SAS expanders? I don't recall seeing it on the compatible cards list.
Click to expand...

Well its based on the LSI 1068 Chip, so is the LSI84016E which is on the compatibility list so it would be safe to assume it would work.
 
LSI based sas-2 adapters should be available (+/-3 weeks) Sept 1st. Wait and see if they are in the same price range of current adapters.

1. A chassis for the disks; with a backplane that supports sas/sata; where each 4 disks = 1 mini-sas connector; pick a chassis that includes a sas expander.
2. A single controller that can drive many sas/sata disks.
3. One or two x4 sas cables to connect the pc to the storage array.
4. latest version of LSI megaraid manager, with the ssd guard feature (free).

Chassis and expander.
The chenbro cases usually have many configs. With/without power, backplane, etc. Some of the chenbro chassis include an expander. Some of the supermicro x24 2.5" chassis include an expander too. I'm sure that some of the supermicro 3.5" chassis also have this option.
Those really big cases have a lot of empty space inside. I'm sure you could find more room inside for even more disks, or a few dozen SSDs.

Controller.
You only need one controller, if it can drive many disks. A few LSI 1068 based controllers can drive many disks. The dell SAS 5/e hba, and any of the sas hba cards in LSI's "advanced connectivity line" can direct connect and drive over 100 disks via sas expanders. For raid cards, some cards based on LSI 1078 can drive many disks and create many raid 0,1,1+0, 5, 6, 50, 60 arrays via sas expanders. These include the perc 6/e, LSI 8888elp, LSI 8880em2

SAS-2
Supports 6GB per phy channel. But don't get too caught up in that. You would need sas-2 from end to end (disks cables backplane expander controller), and you still might only see a small benefit in speed. But you have to start somewhere. So if you were buying a controller card soon, I would at least look at sas-2 cards, that support many disks.

But on the other hand, you may see speed improvements right now. HP p410 p410i p411 raids cards already support sas-2 (firmware update). I have a p411/512MB. I have a perc 6/e 512MB (flashed with lsi firmware). When connected to the same storage (dell md1120, integrated sas-1 lsi expander), the p411 is quite a bit faster than the 1078 based card, on reads with block sizes from 0.5k - 32k. Then it falls down on 64k blocks. I'm hoping that this is something that gets fixed in firmware.

Hey! The LSI site now lists SAS-2 cards... Just there yesterday. Must be new info. I'll have to check them out.
 
nitrobass24 said:
Well its based on the LSI 1068 Chip, so is the LSI84016E which is on the compatibility list so it would be safe to assume it would work.
Click to expand...

It's not a LSI 1068 chip, it's a Marvell 6480 on the SASLP-MV8. 8x Cards with a 1068 chip include the LSI SAS3081E-R and the Supermicro USAS-L8i.
 
nitrobass24 said:
Yea and spending 24k to backup 12k worth of movies is reasonable :rolleyes:
Click to expand...

What you said rings true. A typical older movie or used movie can be had on amazon for about 4-5 bucks..... that translates to a full replacement on average of 6000 to 7500 dollars.

If you purchased movie collections on ebay, you can get the price even lower. Considering that it would be slim to none that you would lose your entire collection, that is something that needs to be thought about. If one movie or 100 scratched, that's just a fraction of the cost of this thing... much less it's electricity.
 
I saw something in stores a few years ago that was like dvd jukebox, in stereo component size. Held 200 dvds, auto loader, programable, menu, remote, about $500. I assume that since you are no longer handling the dvds, you wouldn't even get any scratches on them.

I want to correct what I wrote about the hp p411...
It falls down at 64k block size read/writes, if the bbwc is active for the logical disk.
With the on-card bbwc turned off, it scales nearly linearly, on raid 0. x2 sas 300gb = near double x1 sas 300gb, x3 = near 3x1, x4 = near 4x1.
 
nitrobass24 said:
Yea and spending 24k to backup 12k worth of movies is reasonable :rolleyes:
Click to expand...

You, sir, fail very hard at math. That's $24k for 40TB+ USABLE. That's not 12k worth of movies. That works out to 880 Blu-Ray Discs. If we presume an average price of $20 per Blu-Ray, that's $17,600 of Blu-Rays. (I need to re-run my numbers, they're old, but I think it's ~$18K actually.) Since there's a mix of DVD and Blu-Rays. So you cross a break-even. There's also a presumption that it's not going to just hold those things - it'll hold various files, backups, what have you.

Also a Dual Nehalem board for a fileserver? AreEss cmon man seriously? were streaming movies not launching spacecraft.
There is absolutely no reason to for that kind of processing power for this.
Click to expand...

Uh, that's why there's one 1.86GHz processor and not two 3.2's. Again; enough to rip and playback multiple at once, plus transcode. The QPI is there for the I/O capacity, transcoding and the like, and ability to expand the box to do things like stream multiple distinct HDMI outputs. If I could do it with less CPU, I would.

Even ripping blurays is not compute intensive task. Your limited by disk speed.
Now if he wants to transcode them to another format then you would need processing power.....but again whats the point of transcoding if you have this much storage?
Click to expand...

Uh. Yeah. You fail at understanding something VERY important.
This system is limited by your Blu-Ray drive, CPU, and disk in that order. Period. You're talking about disk speeds exceeding 150MB/s on the landing in RAID1, and well over 400MB/s on the RAID5's. Like I said, it's severe overkill. And it shows it. The thing's designed as a landing zone for large data sets that need to move quickly, typically. The difference is that here it's using SATA disks. Transcoding WILL be CPU limited exclusively.

The only thing that transcoding will accomplish is degraded video quality, and wasted time.
Click to expand...

Unless you're doing something like transcoding a bunch of DVDs to a format you can put on your Zune or the like. Or you're converting between file formats without changing codecs.

Yea you can make the file sizes smaller.......BUT theres 40TB of space.
You would need 2000 full bluray ISO's before it even filled up more if got rid of the extras.
Click to expand...

Uh, no. It's 880 presuming full 50GB per. And again, mixed load. Also, nobody said it had to go with both Arecas and full 1TB disk loadout. Disks just have to be done in RAID set multiples.

Let alone the electricity required for something like this.
Click to expand...

It's not that bad. The PSUs are 80+ Certified. Silver level, as I recall. Because there's little CPU draw, guess what? It's the same amount of power you're going to be using anyways. Sorry. Laws of physics don't change.

Also Raid5 to gracefully handle drive failures?
50 disk Raid5 will not handle drive failures gracefully under any circumstances.
Do you know how long a rebuild like that would take? It would take DAYS
Click to expand...

And do you know what I would do to anyone so completely incompetent as to attempt something the level of stupid required to try and put 50 disks in one array of ANY type?
Here's a hint. They would NEVER find the body. EVER.
I know what the hell I'm doing. I don't waste a lot of time explaining details, because I presume folks are going to do their homework on their own (AS THEY SHOULD) and see roughly what I'd do. It's not exactly obtuse - best practices are your friend.

Yea his disk IO will be higher, and sequential writes will be higher.....but who cares because HD movies use around 10MB/s peak and no matter what you are still limited to gigabit ethernet.
Whats next AreEss? does this guy needs to get run 40Gbps QDR IB throughout the house too:rolleyes:
Click to expand...

Actually, everything on mine will be faster, more reliable, and have a longer usable lifespan than yours. By double or better. I'd put that in writing, too. Mine also has a better usability range - it can store and feed multiple sources at the same time, it can serve as a total network fileserver, it can function as firewall, and still have resources to spare?
Oh, and not once did I not say it wasn't overkill. But he said "money is no object." So, that's what I recommend when money is no object. I could have built something a lot more expensive, easily.

O yea did I mention that my setup could run 24\7 for 38 years before the initial cost, and electricity cost would equal AreEss setup?
Assuming ~600w @ $0.10kwh
Click to expand...

Except your math is completely wrong there, but feel free to keep making completely invalid estimates there. None of your numbers are valid past 12 months because of energy markets, and your 600W versus 4x1650W claim is also a falsehood. And you're kind of ignoring a bunch of other factors that are required when attempting to calculate this sort of thing. (I should know, I've had to do it many times.) Nevermind the fact that your system is not going to be usable past 5 years, best case.

So, TL;DR version:
My system is better period, especially if you want to do more than just store movies, or want a very long usable lifespan.
The price was my old guesstimate, and is probably wrong. And doesn't need all 48 drives anyways.
It's overkill, like I said, so quit attempting to claim I didn't point that out three or four times.
 
simplesam said:
I saw something in stores a few years ago that was like dvd jukebox, in stereo component size. Held 200 dvds, auto loader, programable, menu, remote, about $500. I assume that since you are no longer handling the dvds, you wouldn't even get any scratches on them.

I want to correct what I wrote about the hp p411...
It falls down at 64k block size read/writes, if the bbwc is active for the logical disk.
With the on-card bbwc turned off, it scales nearly linearly, on raid 0. x2 sas 300gb = near double x1 sas 300gb, x3 = near 3x1, x4 = near 4x1.
Click to expand...

There's a few issues with the P411 though. I'd have to look at configs in more detail sometime, but I haven't got a P411 on hand to test and I can't reproduce on P600 family. I think it has to do with cache on top of cache issues (Controller+Drive Cache) and SATA behavior. SmartArray codebase is based on SCSI and presumes that when the drive confirms a completed write including cache flush, it isn't lying - except SATA always lies unless you disable cache.

Also, the DVD jukebox you speak of was a 100 DVD unit made by Sony. It has been replaced by a PC-based ripping system that according to most reviewers, completely sucks. Slow, less capacity, worse interface. You can get the Sony DVD jukeboxes used on eBay and the like for around $100 in working condition, last time I looked (about 3-4 months ago, I was looking to see if I could make them work for a backup system.)
EDIT: IIRC they use a Firewire interface, just as a heads up. But I may be misremembering the interface. It was whatever Not-USB interface Sony was pushing at the time they were made.
 
Last edited:
Just because money is not an object for someone, doesnt mean they would not want something because it wasnt the most expensive.

Practicality comes into play ya know.

Ive got a friend who has more money than god, but he doesnt buy the most expensive X just because he can.
That would be just silly.
 
His motivation is tri-fold. Coolness factor is huge (he can't wait to brag on the golf course) but there's also the laziness factor in not having to get up and hunt for a movie through the racks (and they aren't alphabetized like they should be, again laziness) and the frustration factor. With three young kids, there are disks laying all over the place in all kinds of states and the wife puts them away eventually. Then he gets a specific one he's in the mood for, opens the case, and discovers that it's not only in upside down, but completely covered in scratches, fingerprints, peanut butter, and who knows what else, so then he's got to find a replacement and doesn't get to watch it right then. Also, this guy is a doctor, so money isn't really a problem, we can't even begin to compete with his golf hobby. He understands technology and it's quirkiness, so problems aren't going to make him upset and he'll throw money at problems to make them go away.

Back to the real meat of the issue...

I was thinking server motherboards, the Tyan, but nothing way overkill and bleeding edge like i7, for stability reasons. We're definitely not transcoding, we want the best possible quality.

We've discussed storage systems at length and raid5 isn't even a consideration, raid6 or jbod are the two choices. Considering length of time for rebuilds and three drives (of possibly 50 all from the same 'lot') dying at the same time being possible, it comes down to inconvenience. Since these are backups of disks he owns, losing the data isn't that big a deal, it's just time to re-rip what's lost. So losing the entire array is far worse than just losing what's stored on a few drives and once rebuilds are taken out of the equation, then the odds of additional drives going at the same time is decreased. Building two for mirroring is being considered but he does have a touch of an environmental conscience.
 
Unless 50 disks are needed today, buying 50 disks all at once doesn't make a lot of sense. Buy only what you need or can fill in the next xx weeks/months.

Computer equipment gets faster/more capacity/cheaper/smaller at a very fast rate. Even if he makes loading the data his full time job, the disks will be worth half of what he paid before he fills 50tb. If your friend doesn't own a hotel with video on demand, Buy the disks as needed.

I don't think anyone would create a 50 disk raid 5. Raid 1+0.... maybe. But not raid 5. The point of the "many disk" raid controllers is: You can take those 50 disks and create 10 5-disk raid5's or a whole bunch of 4-disk raid10, and present those logical disks as jbod to the OS. For multi-purpose systems, you can create your own tiered storage. You could have 2 4-disk raid0's, + 8 4-disk raid10's and one 10 disk raid6 or raid50. All presented to the OS, as jbod.
 
ibdoomed2 said:
His motivation is tri-fold. Coolness factor is huge (he can't wait to brag on the golf course) but there's also the laziness factor in not having to get up and hunt for a movie through the racks (and they aren't alphabetized like they should be, again laziness) and the frustration factor.
Click to expand...

Well, a 9U server in the home rack with 40 or 80 TB of usable disk is definitely bragging rights.

Back to the real meat of the issue...

I was thinking server motherboards, the Tyan, but nothing way overkill and bleeding edge like i7, for stability reasons. We're definitely not transcoding, we want the best possible quality.
Click to expand...

Tyan is utter crap. (See our friendly Multi-processing forum on [H] or talk to Dan_D too.) Supermicro is pretty much it if you want server grade, and the board I linked is THE hotshit board. I mean seriously. That thing is absolutely insane. The one I got to test refused to break no matter how much I threw at it. I was able to CPU bind on everything. It's insane. I love it.
As for not transcoding, NEVER rule it out entirely. The need WILL come up when you don't expect it. And sure, there's other ways to do it. There's a whole ton of ways to do it. But NEVER say NEVER. I swore I'd never transcode, and instead have found that I actually DO transcode on occasion because I want to watch a movie on my Zune. Or it's a Blu-Ray but I'm taking it somewhere that only has a DVD player. It's gonna happen. But again, you've got way more CPU than you need pretty much everywhere, unless you're doing bulk transcodes. I definitely won't argue that. However, the Xeon overhead gives you the ability to do multiple independent stream decode and playback. Which is a great thing if you want a few TVs in the house to be able to access it.

We've discussed storage systems at length and raid5 isn't even a consideration, raid6 or jbod are the two choices. Considering length of time for rebuilds and three drives (of possibly 50 all from the same 'lot') dying at the same time being possible, it comes down to inconvenience. Since these are backups of disks he owns, losing the data isn't that big a deal, it's just time to re-rip what's lost. So losing the entire array is far worse than just losing what's stored on a few drives and once rebuilds are taken out of the equation, then the odds of additional drives going at the same time is decreased. Building two for mirroring is being considered but he does have a touch of an environmental conscience.
Click to expand...

Again, see my response to nitrobass. The fact is that you DO NOT AND WILL NOT PUT 50 DISKS IN A SINGLE ARRAY, UNDER PENALTY OF ... you know what? I'm going to need to come up with a new penalty. I mean seriously, that's worse than putting a Windows filesystem with 4k blocks on a 512k stripe.
Otherwise, the rest is mythical bullshit. Period. RAID5 and RAID6 rebuild times are identical, with RAID6 trending toward worse than RAID5 because of needing to wait on both parity disks. Either way, in the stated configuration, your rebuild time is going to be ~3-4 hours for 4+1 RAID5 or 8+1 RAID5, and about the same for 4+2 RAID6 or 8+2 RAID6. Given the controller arrangement, I'd actually trend toward RAID6 for a setup with 24 disks (at the cost of usable disk, of course), and RAID5 for a setup with 48 disks. The key point of rebuilds is this: rebuilds only work on the failed disk, not the array. That's why I call it mythical bullshit - larger arrays will never have longer rebuild times. Larger disks have longer rebuild times. A 73GB 15k in a DS4800 can rebuild in ~38 minutes, while a 300GB 10k will take ~80 minutes. So there's no fixed scaling, you have to go by estimates. 3-4 hours is being conservative; I have done 750GB rebuilds in <2hrs on multiple arrays. The only way to know the rebuild time for any given disk or array, is to do a rebuild and time it.

Another key point is that the fact that you can sustain N failures without interruption or major performance hit, where N is the number of arrays you have. This is another reason everyone who actually knows RAID will beat the crap out of you for putting 50 disks into a single array. It really is the pinnacle of bad configuration. RAID5 should never even exceed 9 disks.

The desire to NOT re-rip is also why hardware RAID is a critical component. Hardware RAID is your first and only line of defense against data loss from disk failure before going to tape or DVD backup. Which is the same as re-ripping. If we presume the drives are E7K's or F1's, you can figure around losing 2-3 drives out of 48 within the first 3 years for various reasons, with the number increasing to 1-3 drives per year past 3.5 years, guesstimating from experience. The failure rate is really quite low. You're not likely to see swaths of drives fail at once but paired failures are more likely with drives manufactured at the same time. It's the MTBF thing; they'll hit it at the same point. Typically I will order drives from 2-6 different vendors to insure against this, or at different points from the same vendor, or work with the vendor to ensure I don't have nothing but sequential serial numbers. As long as you cut that off at the head, the likelihood of data loss is extremely low with RAID3, 5 or 6. (RAID3 is a different setup, if he wants super bragging rights, BTW. PM me, 'cause there's proprietary goodies involved in RAID3 and it has to be explained before I'll consider quoting one out.) ;)
 
Last edited:
Speaking of R3....is FreeBSD the only way to implement that?
I havent seen anything else that I remember supporting it, or any raid controllers.
 
nitrobass24 said:
Speaking of R3....is FreeBSD the only way to implement that?
I havent seen anything else that I remember supporting it, or any raid controllers.
Click to expand...
Areca cards support RAID 3. There were also those god awful NetCell cards too, but they aren't worth mentioning.
 
First of all, probably 90% of those movies are DVDs unless he's bought every bluray to date and tossed his DVDs. A modest 10TB would probably hold the current collection.

Cheaper option for the blurays would be the new Sony ES megachangers and a little programming work from a HT installer. That's honestly the best option for picture and sound quality at the moment to boot. A good installer could probably integrate the changer into a frontend for MC and the bluray "folder" would actually be discs from the changer. I say probably because the changers just shipped this week, but unless sony locked out the serial port from controlling slot #'s it shouldn't be a problem since these are really just updated changers with a bluray drive catching the discs.

As for the second server, it would be off unless he's actively backing up to or restoring from the server so extra power usage would be minimal.
 
Hmm, a doctor who is lazy, gets frustrated, and wants to show off...

He would not be happy with the hurricane type sound coming from any of the cases listed here. He would need to talk pretty loud, while showing it to his friends. His wife probably wouldn't be too impressed with it either.

So expensive is ok, but you probably want it to be small and quiet. Like a stereo component. It's furniture.

The supermicro 2.5 x24 chassis is only 2u. One of them comes with two sas expanders (for redundancy), around $1,300. Get that case + no motherboard. Order and additional backplane and fit it in the case somewhere. Break off the spare expander and attach it to the other backplane. Remove the fans. Maybe add a larger/slower/quiet fan or two, if needed. Jumper two wires on the atx cable, so that it is "always on" if it has power. Maybe change to a quiet/low-power power supply, or just replace the ps fans.

Buy 4x 256gb ssd drives. You are only doing reads, so even the worst mlc ssd's are ok. Buy more SSD's as needed, taking advantage of price drop + rising capacity over the time to fill the disks. Dr's don't have much spare time anyway. Being able to store 150 - 200 movies would be a good start.

There you have a fast, nearly silent, 2u storage array, that can handle 48 disks (and might have room for more). If the SSD's are bought over time, it could have a capacity of ??.

As SSD capacity rises and you replace the older SSDs for him, you could do him a favor and "get rid of" the old SSDs for him. I'm thinking: "Oh these? Worthless. I guess you could use them as paper weights, if you want.".

Total start cost with new case and spare backplane, new 4x 256gb ssd, cables and sas hba off ebay: 4k - 5k. Total cost when filled ???, but probably less than you think.

Selling points: Silence. Durability of ssd vs hd in this application. The SSDs will pay for themselves in power savings, over the next 10 - 60 years.
 
Blue Fox said:
Areca cards support RAID 3. There were also those god awful NetCell cards too, but they aren't worth mentioning.
Click to expand...

LSI also supports RAID3 on a very limited number of cards, LSI/Engenio controllers running firmware 7.x support RAID3 but with stupid limitations, some SmartArrays support RAID3, and some older ServeRAID cards support RAID3.

RAID3 can absolutely decimate any other RAID when tuned correctly and using the correct hardware. The problem is that as opposed to RAID5 where you have 4+1 of the same disks? RAID3 you have D disks then you need P parity disks which are significantly faster than your Ds, because it's true single parity drive.

And people are ignoring the fact that one, SSDs are not proven reliable period. GET OVER IT. THEY ARE STILL A RELATIVELY NEW PRODUCT. ARGH.
Two, absolutely no system he builds with the storage capacity he wants is going to be installable in his entertainment center. Period. He's going to need a cabinet. There are options for soundproofing a cabinet properly, or he can just stick it in the basement somewhere and run cabling (which is much easier.)
Third, get a clue about the case before opening your mouth and proving you know nothing, simplesam. You cannot replace the PSUs in the Supermicro 2U's, you cannot change the fans, and they are among the loudest cases in the world because of the required airflow and static pressure to cool the system period.
But thanks for giving me the opportunity to point out that Supermicro doesn't have a chassis that's a viable option.

EDIT: Oh, found the changer I was thinking of. Sony Vaio VGP-XL1B2. Neat beastie in that it's reader/recorder, but apparently it's still some kind of misery from a labeling standpoint.
 
Last edited:
Sorry if I upset you AreEss. I think we are just approaching this from different angles.

I am just trying to offer ideas to someone who wants to build something on their own, while it sounds like you are trying for a consulting sale of some kind. I apologize if I am getting in your way, or making you angry.

I agree that SSD's are not a proven replacement for hard disks in an enterprise environment, or even as a boot drive in a laptop. But flash memory has been around for a while. Flash memory that is not written to very often is in many devices, and is proven tech. It is even in the raid cards listed in this thread.

In this case, we have "write once", read ?? times. I would think that flash memory would be safe/ok for this. If it's not, then any raid card you could recommend would be garbage too, as the raid card stores its config and firmware in onboard flash.

If static, replaceable movie files are not safe on ssd flash, I don't think I would trust the flash chip that holds the raid config, which will probably be written to more often than the flash chips in the ssd, which will hold only movies.

Regarding supermicro, fans, static pressure, system cooling, etc.
I do not own one of those cases. I just saw them last week. My interest in them is limited to the fact that they are available with a backplane and multiple sas expanders, at a reasonable cost. I am only looking at what you get, vs cost of individual components.

For myself, I wouldn't care about the noise. But some people would. While the supermicro chassis fans may not be directly replaceable, I'm sure that they are removable. Many system fans can be purchased, that would plug directly into various standard power supply connectors. As for replacing the PS, I admit that I have no clue on that.

The point I think you missed, is that there is no system in the chassis to cool. No MB, no cpu, no ram, no JBOSpinningD to cool. It would just be the PS which has its own cooling, backplanes, two 10w expanders, and dozens of relatively cool SSDs, of which only 1-4 or 1-8 would be actively reading/active at a time (for most raid configs). I don't think static pressure would come into play here. Slow/quiet fans should suffice.

If there are any factual, grammatical or semantic errors in this post which require your direct attention, I humbly apologize in advance. In the future, I will try to remember that I know nothing, before I post.
 
The power supply fans are extremely loud in those Supermicro cases. You cannot replace them and they even run when the system is off (as long as they are plugged into the wall). He is right in the fact that no Supermicro chassis is acceptable for such a configuration due to noise. I own a couple Supermicro systems (1U and 4U) and they are the loudest systems I own.
 
Blue Fox said:
The power supply fans are extremely loud in those Supermicro cases. You cannot replace them and they even run when the system is off (as long as they are plugged into the wall). He is right in the fact that no Supermicro chassis is acceptable for such a configuration due to noise. I own a couple Supermicro systems (1U and 4U) and they are the loudest systems I own.
Click to expand...

Well, that sucks.
 
Is it situation where the size or shape of the supermicro psu is proprietary and a semi-standard psu will not fit?

I own a few chenbro cases and have been able to install off the shelf power supply units. If this is not possible with supermicro, I will need to remember to avoid them.
 
simplesam said:
Is it situation where the size or shape of the supermicro psu is proprietary and a semi-standard psu will not fit?

I own a few chenbro cases and have been able to install off the shelf power supply units. If this is not possible with supermicro, I will need to remember to avoid them.
Click to expand...
Actually, no off the shelf power supply can replace the kind of stuff Supermicro includes. First off, you're not going to get redundant power supplies in normal ATX form factor (and if you find one, it's probably crap). Their power supplies are also extremely efficient (the majority of the new models are 93% efficient...try finding that elsewhere). Their quality is also excellent. All current Supermicro cases include power supplies anyway. There is a reason why Supermicro stuff is used for real servers and not just a bunch of pieces you buy off newegg...
 
I'm sure their quality is excellent, and their prices have come down over the last few years. But I have seen people mention that some of their adapter cards are proprietary, where the card is backwards/reversed.

I am wondering if this is the case with their psu's too, where a psu from a company like zippy wouldn't work. Either because of shape/size, or a non-standard atx connector.
 
simplesam said:
I'm sure their quality is excellent, and their prices have come down over the last few years. But I have seen people mention that some of their adapter cards are proprietary, where the card is backwards/reversed.

I am wondering if this is the case with their psu's too, where a psu from a company like zippy wouldn't work. Either because of shape/size, or a non-standard atx connector.
Click to expand...
The UIO cards are reversed for a reason. You can squeeze in more cards into their cases using a combination of normal expansion cards and UIO cards. For example, with a UIO compatible board, you can squeeze in 3 expansion cards in a 1U chassis (2 full height and 1 half height) or 7 expansion cards in a 2U chassis (4 full height and 3 half height).

As for power supplies, it is the form factor that is proprietary. They use normal connectors (as you can put non-Supermicro boards in their cases). There is no standard for redundant power supplies, so, it's not fair to single them out when every redundant power supply is also proprietary.

Thing is, look at any major manufacturer and they use more proprietary parts than Supermicro (Dell, HP, etc) in their servers.
 
You must log in or register to reply here.
Back
Top