• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Production server build advice request

D

djgizmo

[H]ard|Gawd
Joined
Dec 24, 2006
Messages
1,429
I'm now in a technical advisement role at a small business who distributes music (legally) to all the major retailers / streaming services. [rather not say who I work for, so please respect that].

They have about 6TB of catalog, however they've been renting a server in a Chicago data center with only about 1TB of space. (Which hosts their content portal).

problem A - no ability to deliver back catalog to new retailers.
problem B - running out space on current server to maintain day to day business operations. Every week they have to download 30-40GB of music to their local server just to maintain the status quo
problem C - I have limited knowledge on building large storage arrays. While I have several years of support desk experience and building 4-8TB arrays at home, they're looking to build out a 50+GB monster server to replace the rented server and probably a 30TB in house server.

However, I've done some research and I believe I have a plan of attack.

Data Pooling and protection (Flexraid)
22 Active Drives + 2 Hot spares (24 drives total)
I'd like to run 2 identical VMs. 11 drives on each. (unsure if that is even possible) The second VM will be save incremental backups of the first VM.
The primary VM will be RHEL or CentOS since that's what our current web server has run on and has done so with minimal glitches.


Here are the parts list I choose for the data center server.
I'm probably forgetting a few things, so please don't flame. I'm open to suggestions if anything is under or overkill.

http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=27166807 (2011 single socket build)
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=30179068 (1155 socket build)
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=27166867 (AMD C32 socket build)

**Updated the above to include 3 wishlists w/o drives** - This is to provide some perspective of the costs of the different systems with.
Yes, I know the SSDs in RAID 1 is overkill, but I'd rather have plenty of space for expansion of the VMs than not, and to reduce the likely hood the OS drive will fail.



Might be able to shave a few pennies off if I use camelcamelcamel/camelegg.com

This project isn't a emergency project, but if given the 'go' (which is likely due to the business continuity need), I need to be able to start purchasing the gear bit by bit and be able to have a full system ready to go by beginning of November for testing, so it can go into production in December.
 
Last edited:
Best advice I can give is to go with something OEM with North American technical support and 24x7 monitoring. From a performance standpoint you'll be able to deliver more bang for your buck with a custom build but long term TCO can be orders of magnitude higher in case of a failure.

Value up-time and reliability over all else even if it costs a fortune. Let the business decision makers veto that first. You may still end up building a custom box in the end but you've covered your ass by recommending something with a proven track record. Something no one-off custom box can deliver.

I had a professor tell me a story about IBM flying him, a senior level contract Linux engineer, to a companies doorstep same day to correct a system issue. The support call was placed just before business hours and the array was fully corrected by mid-afternoon. What's downtime worth?
 
With multiple Terabyte pools, you encounter some problems:
- Disk-errors on conventional filesystems like ext4, XFS or NTFS
You must do a offline filecheck to repair (or hope to, this can last days)

- Regain to service time on disk failures (with 24 disks expect multiple failures per year)
If you do not use realtime raid but a backup only solution like flexraid:
in case of a failure, you need to replace and rebuild the disk to regain access

- Silent data corruption
a problem of large pools, You will not be aware of until you find damaged files

- Read performance problems with multiple large parallel filestreams

If you cannot afford a professional solution for that (netapp etc),
you should use a solution based on ZFS with enough RAM (64GB+)
to cache most reads to overcome this problem.

ex: using stable releases: NexentaStor, Oracle Solaris 11 or OmniOS
(the last one is also free usable without support)

If you are not experiences, use a appliance distribution with Web-UI
and look for a distributor or someone that can give some level of support
and think of a backup/ failover solution at a second location in case of problems.
 
When you say "production" how many customers are you talking about?

If its more than 50, I think this sounds like the wrong way to go about it.
I think you want some hardware that isn't white-box.

Have you considered EC2? They support centos/RH I believe, and with their services you can scale out a lot faster.

It will also be much more robust than what you're putting together. It will also probably be expensive for that much storage.

I think flexraid is more for your home server than some sort of production server streaming a ton of music.
 
What concerns me most about this post (and please don't take this the wrong way, OP), is that they serve up 6TB of music, but have to buy parts piece by piece to fit a $6k budget? As previous posters said, this is a job for a proper hosted solution or good SUPPORTED array, otherwise you WILL be on call 24x7x365 and that sucks (been there, done that). Beyond that, other than storage capacity, what sort of performance out of the array do you need? i.e. How many clients hitting it at once and for what kind of traffic?
 
Thank you all for chiming in.

I'll try to answer your questions in reverse order.

Nate,
It's a small business. Literally we have 6 employees (including the owner), and 2 contractors. The owner only invests in technology when there is a critical business need. We foresee this business need, that's why we're in planning stages now. We're not streaming music, but we're delivering music to other services, like iTunes, Amazon, Google, etc.

As for performance, we're not shooting for records, just for space, reliability and expansion. 50MB/per throughput should be plenty.

As for clients, we have two types of clients. Content providers (Labels/Artists) and Retailers.
We probably have spurts of 30 content providers from around the world hitting the server at once for uploading music every couple days. The rest of the week time we probably have 10 content providers uploading at any given time. No more than 300 releases in a given week. (but each release usually has 2-5 tracks on average)

Then at weeks end, we send pack/prep the metadata and audio, and connect to all the Retailers FTP servers and delivery said packages one at a time. (delivery is automated).

Rinse and repeat.

hotcrandel,

Actually, I completely forgot about EC2 as an option. Didn't think they could go to high storage capacities. Just checked the website out and looks like they can go up to 48TB.
Since I'm unfamiliar with EC2 intricacies, does Amazon have a sales team I can call to discuss my situation?

_Gea,
Thank you for the word of caution.

Can you recommend some additional professional solutions? I recognize Netapp, but I'm not familiar with any other ones.

ZFS was a thought as well, but I haven't touched Solaris in over a decade, so I'm a bit intimidated. Plus I was under the impression that ZFS was difficult to expand.

ochadd,
Can you recommend any OEM's with monitoring and 24x7 support? We do value uptime above all else, and for the most part in the past 7 years, we've had a 99% uptime.
 
Oh wow. Just checked out EC2. I think my boss would poop himself hearing how much the yearly cost would be for the light utilization.

Year one costs on EC2 would be $21,000+ and $16k per year after that minimum. Ouch.
Don't get me wrong... I think it's a great value to the right EC2 client. Plus I don't see any easy way to seed the server.
 
djgizmo said:
ochadd,
Can you recommend any OEM's with monitoring and 24x7 support? We do value uptime above all else, and for the most part in the past 7 years, we've had a 99% uptime.
Click to expand...

I've only worked with Dell Equalogic SAN teams personally. They royally messed up initially on performance sizing but so did I. I thought they made it right after the fact with a massive discount on an addition. Sizing for space, as you are, is easier. They set it up while teaching me to manage it. Configured the device, the SAN network, even attaching ISCSI disks to the servers for nothing. The support since has been excellent. When my server room AC goes offline, which it has been doing weekly, Dell contacts me before it overheats. When a disk fails, Dell contacts me. If your business is located near a major metro area you can be guaranteed parts within 4 hours and possibly have a technician install it for you.

They'll propose a system after doing some data collection and it costs you nothing. They came on site and walked me through how the whole thing would work during the sales process and we're 120 miles from the nearest airport. They even talked me out of a Compellant array because of maintenance overhead even though I was dead set on one. It can't hurt to call them up.

We've been live for around a year and I love this thing.
Edit: When I was pricing ours they came in much cheaper than NetApp or EMC and with a badass 5 or 7 year warranty.
 
Last edited:
ochadd said:
I've only worked with Dell Equalogic SAN teams personally. They royally messed up initially on performance sizing but so did I. I thought they made it right after the fact with a massive discount on an addition. Sizing for space, as you are, is easier. They set it up while teaching me to manage it. Configured the device, the SAN network, even attaching ISCSI disks to the servers for nothing. The support since has been excellent. When my server room AC goes offline, which it has been doing weekly, Dell contacts me before it overheats. When a disk fails, Dell contacts me. If your business is located near a major metro area you can be guaranteed parts within 4 hours and possibly have a technician install it for you.

They'll propose a system after doing some data collection and it costs you nothing. They came on site and walked me through how the whole thing would work during the sales process and we're 120 miles from the nearest airport. They even talked me out of a Compellant array because of maintenance overhead even though I was dead set on one. It can't hurt to call them up.

We've been live for around a year and I love this thing.
Edit: When I was pricing ours they came in much cheaper than NetApp or EMC and with a badass 5 or 7 year warranty.
Click to expand...

Awesome info. Thank you. I just got off the phone with NetApp. Basically the provide 100% data peace of mind and reduced backup space. However for the space we're looking at... its 25-50k for the appliance. 25k 'might' be able to able to be swallowed... but 50k, not a chance. I did like the unlimited ability to keep expanding though, which was nice.

Do you have a contact at Dell you'd like to toss my way? Referrals are a mans best friend. Dell small business has always treated me right for the most part (for my personal home business) so I'm glad Dell is still treating people right at the mid tier level. The fact that they taught you how to maintain and connect it is a good perk. I'm in Tampa so, a 4 hour support window is awesome as well.
 
As a Dell/Equallogic/Compennent reseller, You won't find an Equallogic for less any that in the same capacity. As long as you know what you understand what youre getting into with a Whitebox solution then go for it. Personally, I run FreeNAS for my environment at home and if it all crashes, I don't lose any $$, just inconvienence.

Assuming that Whitebox route, I'd stick with a major distrobution that you can call on support with incase of an issue, FreeNAS (supported by IXsystems) or Nexenta would probably be the best bet. Or you might consider an appliance-based solution. An 8 bay QNAP or Synology with 3TB enterprise drives for right around 4K.
 
10-15k is the hardware for 50TB, the difference to 20-50k is software and support.
While you can get similar features in most cases for free with ZFS on
SuperMicro or Dell Hardware. OS support may be a problem if you need that.
(beside Nexenta, OmniOS and Oracle that offer ZFS OS support)
 
djgizmo said:
Oh wow. Just checked out EC2. I think my boss would poop himself hearing how much the yearly cost would be for the light utilization.

Year one costs on EC2 would be $21,000+ and $16k per year after that minimum. Ouch.
Don't get me wrong... I think it's a great value to the right EC2 client. Plus I don't see any easy way to seed the server.
Click to expand...

$20K is about 200 hours of mid level labor. Can you maintain a 50TB system in house for less than 200 hours of labor a year.?

I think even EC2 is a good deal. (I am not suggesting you use EC2, just that the price is reasonable for what you hope to get.)
 
Nate7311 said:
As a Dell/Equallogic/Compennent reseller, You won't find an Equallogic for less any that in the same capacity. As long as you know what you understand what youre getting into with a Whitebox solution then go for it. Personally, I run FreeNAS for my environment at home and if it all crashes, I don't lose any $$, just inconvienence.

Assuming that Whitebox route, I'd stick with a major distrobution that you can call on support with incase of an issue, FreeNAS (supported by IXsystems) or Nexenta would probably be the best bet. Or you might consider an appliance-based solution. An 8 bay QNAP or Synology with 3TB enterprise drives for right around 4K.
Click to expand...

Thank you for the insight. Well, the one thing that NetApp appliance that I liked is the deduplication/compression technology for snapshots. To be able be able to store snapshots of 6TB of data in less than 1TB is great benefit as our catalog will only grow each year. We're expected to add another 1-2TB by year end, and probably another 4TB next year alone, so we need room for growth. The rep for NetApp said they usually reserve 20% for snapshots, and he's said he's never seen anyone use that full 20%. So even if we reserved 25% for snapshots, in a 24TB setup, we would still have about 18TB of space. Figure 10% of OS overhead, and another 10% for parity we'd have 14+TB of usable space. It would solve quite a few problems. However, I think we'll need to go with the 3TB drives in stead of the 2TB as starting with 36TB sounds a lot better after all is said done with the overheads / snapshots.

I'm scheduled to talk to the local NetApp vendor tomorrow morning so I'll provide more of an update then.
 
GeorgeHR said:
$20K is about 200 hours of mid level labor. Can you maintain a 50TB system in house for less than 200 hours of labor a year.?
Click to expand...

Well, if that is mid-level, then I'm in the wrong business. (aren't we all). To answer your question honestly though, I'm not so much worried about the maintenance, as I'm the initial setup and configuration. I do suspect it'll take me about 200 hours for the initial setup if we do decide to build our own.

Smartly, one of our vendors mentioned if we're going to go collocation, we should do it in our immediate metro area (Tampa). While I don't love our are for Data Center services, if something did break, I could be to the data center in less than an hour (even mid day with traffic) and start repairing things on site.
 
Be wary of the compression/deduplication claims. In the first place, they can be a huge hit to performance. Second, the Dedup is typically conditional on matching identical blocks of data. So unless you are keeping a ton of copies of data, the Dedup will be a gimick and nothing more. Snapshots technology is nifty, but its usefullness depends on the ease of accessing the versioning. Based on the fact that you are entertaining NetApp, might as well go through a demo with an Equallogic partner as well. They don't quite have all of the bells and whistles, but tend to be more straight forward to use. If you let us know what region you are in, I'm sure someone here can put you in touch with a rep that won't waste your time.
 
Nate7311 said:
As a Dell/Equallogic/Compennent reseller, You won't find an Equallogic for less any that in the same capacity. As long as you know what you understand what youre getting into with a Whitebox solution then go for it. Personally, I run FreeNAS for my environment at home and if it all crashes, I don't lose any $$, just inconvienence.

Assuming that Whitebox route, I'd stick with a major distrobution that you can call on support with incase of an issue, FreeNAS (supported by IXsystems) or Nexenta would probably be the best bet.
Click to expand...

Hey Nate, I just checked out Nexenta, it sounds A LOT like a NetApp replica. Thanks for the tip.

Sigh, I eliminated FreeNAS earlier on because I thought I wanted to just extend RHEL to incorporate FlexRAID. As this takes shape, it looks like I'll have to run 2 servers. A dedicated Web Server and a dedicated DAS/NAS. Sigh, the more I learn the more I have to learn. This is going to end up as another this VS that vs XYZ thread, and that... well sucks

Besides the 'SnapRaid' comparison page, is there a spreadsheet that lists all the features of the different options side by side? I get that everyone and their brother believes that ZFS is the next slice of bread, but, there are so many options out there its over whelming.

I have information overload. I feel like I'm back at square one again. I'll check back in the morning.
 
djgizmo said:
Hey Nate, I just checked out Nexenta, it sounds A LOT like a NetApp replica. Thanks for the tip.

Sigh, I eliminated FreeNAS earlier on because I thought I wanted to just extend RHEL to incorporate FlexRAID. As this takes shape, it looks like I'll have to run 2 servers. A dedicated Web Server and a dedicated DAS/NAS. Sigh, the more I learn the more I have to learn. This is going to end up as another this VS that vs XYZ thread, and that... well sucks

Besides the 'SnapRaid' comparison page, is there a spreadsheet that lists all the features of the different options side by side? I get that everyone and their brother believes that ZFS is the next slice of bread, but, there are so many options out there its over whelming.

I have information overload. I feel like I'm back at square one again. I'll check back in the morning.
Click to expand...

Separation of Web/Application server and Storage is probably better in the long run anyway. And allows for more flexibilitywith both pieces. And given the favor to the Whitebox route, In your shoes, I'd go with a presetup Distro rather than a roll your own scenario where support is virtually non-existent.

Next question for this scenario. What are you doing for Backup? :D
 
I'd say if you were comfortable with doing this before then there wouldn't be any problems. But just going through your hardware choices I'm finding less than ideal parts.

First the 155 socket has limited upgrade-ability across the board (and won't be around for too much longer). For new builds if you are going Intel then you should be using 2011.

Second, if you are doing VM work that i3 isn't going to cut it. (I would separate this out anyway).

Third, that SSD probably isn't needed and if you want one you want two of them to mirror. You'll need this guy by the way with that case. MCP-220-00043-0N

Fourth, WD Reds are typically used for long term storage, backups, or if power, money, or throughput takes precedence over IOPS.

Finally while FlexRaid is great for home use. You should be using either ZFS, MDADM+LVM, or a hardware solution before getting to FlexRaid. They all perform better and there's no point in leaving performance on the table if you don't have to.

With all of that said you might be better off going with a storage appliance or server from an OEM. If you don't know what stuff is then you will make mistakes...costly ones. If you have a position where they don't care. Great! But if you are not then you should either take a lot of time going through the hardware and what you plan on doing or just order an appliance.
 
Last edited:
GeorgeHR said:
$20K is about 200 hours of mid level labor. Can you maintain a 50TB system in house for less than 200 hours of labor a year.?

I think even EC2 is a good deal. (I am not suggesting you use EC2, just that the price is reasonable for what you hope to get.)
Click to expand...

I do not agree in a lot of use cases.
Mostly a usual storage server is no more complicated than any other computer.
It has more disks, maybee you are unfamiliar with the OS and there are some
special things to care about but that is similar to any other specialiced computer -
unless you can avoid some hardcore features like high availability or clustering.
Even a Wlan router can be as complicated or as easy.

Today there are many smaller and larger/very large enterprises that use OpenSource
or lowcost solutions without support and 4 hour Service Level Agreements.
You need some IT skills and you should not be in a situation where you cannot allow
a downtime or have the inability to fix a failure yourself due to limited personal resources.

You should als not be in a situation, where you can say - nobody got ever fired due to
a decision for IBM mainframes, regardless the costs and the question wether it was the best solution.

Back to storage:
If your enterprise is quite small, downtimes are not extremely critical and you have some usual
IT skills and want to learn some basics, you can build a stable whitebox storage server with similar
features like highend boxes for a fraction of the money. That is what I do (university use).

Usually you buy two regular 24bay Supermicro Storage server from stock (one of these
with the X9 Mainboard series that we all use) with 32-64 GB RAM and 3 x IBM 1015 Controller, add
16 x 4TB Sata disks for a dual Raid-Z2 config, buy some as hot-spare from a distributor you trust,
In the best use case, you buy a third system as coldspare/testsystem.

Then download my napp-it to Go (this is a ready to run USB image), plug it in and your storage server
is ready to use with OmniOS, a stable ZFS distribution and a Web-UI (no config, no setup needed).

needed actions:
- mirror USB sticks (via Web-menu)
- create a datapool (via Web-menu)
- share filesystems (via Web-menu)
- management (via Web-menu)

The price:
about 3000 Euro for the hardware (case, mainboard, CPU, HBA, RAM)
plus 16 x 4 TB disks (Raid-60 similar config) = 48 TB usable, with Seagate Constellations 4 TB=5600 Euro
Software is basically free (you can buy OS support and some extensions)

= about 20k for two fully equipped 50 TB boxes (usable capacity after Raid-60) + one spare system without disks.
(1 Euro=1,30 USD)
 
djgizmo said:
Oh wow. Just checked out EC2. I think my boss would poop himself hearing how much the yearly cost would be for the light utilization.

Year one costs on EC2 would be $21,000+ and $16k per year after that minimum. Ouch.
Don't get me wrong... I think it's a great value to the right EC2 client. Plus I don't see any easy way to seed the server.
Click to expand...

You say that, and on paper it looks a bit more intimidating. But the fact is you don't have to manage storage, replace hard drives, rebuild the thing when it crashes, or replace power supplies when if they go back.
You don't have to procure bandwidth and have excellent networking equipment on site, worry about the HVAC failing, worry about power outtages and battery backups. And EC2 can do backups for you right then and there to glacier. no fuss.

And not to mention that you can scale out at anytime you want.

Do you already have a full blown server room with HVAC? Fast internet connection? Firewall? Power infrastructure?

I'm not saying you can't do it in-house, but its not wise to build your own ship, when for a bit more money you can get on the titanic of web-hosting. I will work on this analogy but you get the idea.

The list of benefits goes on and on.
 
Thank you everyone for the continuos feedback. I'll be speaking with Dell today about a few options. Equalogic & Nexenta.

Personally, I'd like to take on the challenge on building a solid whitebox as we need one for in office anyways.


kac77

First the 155 socket has limited upgrade-ability across the board (and won't be around for too much longer). For new builds if you are going Intel then you should be using 2011.
Click to expand...

Thank you for the heads up about 2011 socket. I know the 1155 has limited upgrade ability, but really, if this box can serve data in and out, how much upgrade ability will we need in the next 3 years?

With that said, I really haven't thought about going the AMD route, but since this is looking like I'll need to build out a white box solution in house, wouldn't an AMD route save some money?

Second, if you are doing VM work that i3 isn't going to cut it. (I would separate this out anyway).
Click to expand...
We're not doing any VM work currently, but the cpu I chose was a Xeon E3, not i3.
Third, that SSD probably isn't needed and if you want one you want two of them to mirror.
Click to expand...
Yea, I'm aware the SSD isn't needed. Just wanted to reduce the chance of the OS drive failing. I did mean to add a 2nd SSD (to mirro) to the list, so thank you!

Fourth, WD Reds are typically used for long term storage, backups, or if power, money, or throughput takes precedence over IOPS.
Click to expand...
Yep, this is exactly what we need. That is why I choose the WD Reds. We need more long term storage than anything.
-----------------------------

Nate7311
As for backup, that's still a plan in progress depending which route we want to go. Originally, as mentioned in the 1st post, I was going to run ESXi, run 2 VMs. 1 VM as our main webserver and main data pool, and the 2nd as a backup repo to have it protected.

If we go with building the whitebox using a pre set distro (like Nexenta)... Should I build literally a separate box for the webserver or just virtualize everything? (3 VMs as it stands.)

Personally, I'd like the challenge of trying something new, but I realize that if I make a mistake, I'm the one on call.
 
hotcrandel said:
And not to mention that you can scale out at anytime you want.

Do you already have a full blown server room with HVAC? Fast internet connection? Firewall? Power infrastructure?

I'm not saying you can't do it in-house, but its not wise to build your own ship, when for a bit more money you can get on the titanic of web-hosting. I will work on this analogy but you get the idea.

The list of benefits goes on and on.
Click to expand...

You make some valid points. We don't plan on hosting the server in house, only building it in house and then hosting our webserver and whitebox in a datacenter in town.
 
djgizmo said:
Thank you for the heads up about 2011 socket. I know the 1155 has limited upgrade ability, but really, if this box can serve data in and out, how much upgrade ability will we need in the next 3 years?
Click to expand...

Well that board is short on RAM slot for starters. So if you want more RAM you'll be forced to use more expensive sticks to go higher sooner. In addition to that, going with the 1155 means you won't have any upgrade path in 3 years. Once Intel EOL's Xeons they tend to die off quite quickly. Comparatively if you plan ahead you can have that server last you for much longer than 3 years. You have a decent sized chassis. There's really no reason why you couldn't get 5 years or more out of it if you plan accordingly.

djgizmo said:
With that said, I really haven't thought about going the AMD route, but since this is looking like I'll need to build out a white box solution in house, wouldn't an AMD route save some money?
Click to expand...
For 2P AMD makes sense. For single socket I probably would stick with Intel. There's nothing wrong with going with AMD in this space but you'll burn more power. Having said that, it's not that big of a deal since the TDP on the server parts are lower than their desktop counterparts.

djgizmo said:
We're not doing any VM work currently, but the cpu I chose was a Xeon E3, not i3.
Click to expand...
Yeah I know I just call the E3's i3's because they are so castrated. :)
 
Going AMD can possibly make a lot of sense if you are using software that can and needs to scale to multiple processors but your software doesn't have to be licensed per processor.
 
Thank you for the feedback.

Yea I don't foresee us using any software that is has to be licensed per processor. Literally a LAMP server, a NAS solution (FreeNas and Nexenta seem to be decent choices), and segregated VM for backups.

As for the E3's being castrated... That's why I swapped out the original E3 for the Xeon E3-1230 V2, which does have hyperthreading. Our processing needs aren't very high, and don't expect to go very high any time soon.

Either way, anything would be an upgrade wise to the current server. Right now the one we're renting is a Dell PowereDge 2950 with Dual Intel 5420's, which when EOL back in 2010!


However, I do see your points about the 1155 being a deadend at some point. Like everything, I have to pitch several options to the boss and nudge him in the right direction. For the most part, he's been 100% supportive and has confidence in my suggestions, however he pays the bills.
 
The big question is will the the HBAs work in a virtualized environment?
 
I heard something about that in my research (about Spotify). My company delivers a ton of dance of music there.

I counted today, we deliver to over 65 store/streaming services. We're lucky we only transmit 4TB of data per month. (From my calculations, I need to plan to have about double the bandwidth if we plan to start sending out back catalog to recent retailers).

I suspect that we're going to go the ZFS route (Nextstore or FreeNAS) for the data center server, and probably FlexRaid for the in house server. The reason why I'm leaning towards FlexRaid, is that we don't need to have all the drives spinning at once, which would be nice considering how small the office is.

Not sure which to test first. FreeNas, Nexenta, or Flexraid.
 
How critical is uptime for you? What happens if two disks crash, can Flexraid handle that? ZFS can handle three disks crashing, via raidz3.

And if you have lot of data, it is much easier to administrate much data via a big pooled storage of say 20TB and directories, instead of having many 2TB disks. "Let me see, where did I put that rock band, was it on disk K: or was it on disk M:? Or have I moved it back to disk O:? Now my rock music disk N: is full, so which other disk should be rock music, maybe disk T:?". Instead you just have some directories in ZFS, "/rockmusic/IronMaiden" etc. And you dont have to move data back and forth to different disks as in flexraid.
 
brutalizer said:
How critical is uptime for you? What happens if two disks crash, can Flexraid handle that? ZFS can handle three disks crashing, via raidz3.

And you dont have to move data back and forth to different disks as in flexraid.
Click to expand...

You choose the number of parity disks in Flexraid, theoretically allowing you to handle n failures.

Also, there is a pooling solution with FlexRaid. Not quite as advanced as the balancing with ZFS, but it works.
 
If it can handle redundancy well, and if it can pool storage, then I see no reason to use ZFS instead. But I concur that you should try both first. Theory is one thing, practice in real life another. But to me it seems that flexraid might do well in this scenario. For media you dont need the heavy bit protection that ZFS provides, it does not matter if a bit is flipped. No one will notice if one pixel is grey instead of blue.
 
You must log in or register to reply here.
Back
Top