Need Advise: Updating storage RAID array

[KaptainBlaZzed]

I have a 7 year old RAID 6 array on a PERC 6i with 8 x 2TB 5400RPM Drives. In this config i get 10.9TB of usable space.

I currently have this array 85% full and have also had a couple of disks die in the past couple of months.

I have already purchased a used LSI MegaRAID 9266-8i card and need help with the drive selection.

This array is used for my personal storage of movies/tv shows/music. The content is streamed to my xbox's.
The server also runs BOINC 100% of the time.
I have never had a speed problem with the array and the new raid card will be quite a bit faster than the old card.
So HD speed is not really a concern, i am interested in space.

I have been looking at the Seagate Archive drives in 8TB. (they are listed as a compatible drive for my new raid card)

I know the 8TB drives will take a long time to rebuild if one fails, but uptime is not critical.
I have a duplicate copy of the array on my media server split across 4 x 4GB drives.

I thought about getting the smaller 5GB or 6GB drives, but the price difference is negligible.

Give me your feedback on this plan.

 

[Brian_B]

Actually I was thinking about doing something similar with my RAID5 in a similar situation, I am wondering the same thing, since those 8TB drives are so inexpensive.

 

[SomeGuy133]

Actually I was thinking about doing something similar with my RAID5 in a similar situation, I am wondering the same thing, since those 8TB drives are so inexpensive.

i think the chance of a failed rebuild is too high for 8TB on RAID 5 from waht other members were saying It needs to be RAID 6 or 3rd gen file system

 

[Brian_B]

Going from 4x3tb to 4x8tb I could change from RAID 5 to 10 and still come out a good bit ahead in storage space

 

[SomeGuy133]

Going from 4x3tb to 4x8tb I could change from RAID 5 to 10 and still come out a good bit ahead in storage space

that would be pretty damn fast. I would still start looking at third gen file systems

BTW are the 8TB still those archive drives? IO heard its a no no to use those as an actual drive.

 

[KaptainBlaZzed]

that would be pretty damn fast. I would still start looking at third gen file systems

BTW are the 8TB still those archive drives? IO heard its a no no to use those as an actual drive.

From what i have read, they are limited on the write speed but not on the read speed. So in my case i write the file once and read it every once in a while. (sometimes never)

Do you have an link to an article talking about not using the archive drives as daily/actual drives?

 

[SomeGuy133]

From what i have read, they are limited on the write speed but not on the read speed. So in my case i write the file once and read it every once in a while. (sometimes never)

Do you have an link to an article talking about not using the archive drives as daily/actual drives?

what you just decribed was basically archive. I recall some weird mumber of 150GB of writes a day or 150TB a year or something. I might be confusing something else. Trimlock I am sure could answer this. (or one of the other more fluent big data guys)

 

[izx]

I know the 8TB drives will take a long time to rebuild if one fails, but uptime is not critical.
I have a duplicate copy of the array on my media server split across 4 x 4GB drives.

Unless you're going to just 16TB, you'll have to expand your backup array too.

I thought about getting the smaller 5GB or 6GB drives, but the price difference is negligible.

Well, that's because the 8TB's SMR technology is cheaper than regular 5/6TB drives and it's slower.

I wouldn't recommend the 8TBs for RAID-5 at all. Their sustained write speed is around 30 MB/sec, which would slow rebuilds dramatically, increasing the chance of a second drive failing during the rebuild. For RAID-6, perhaps, if you can afford to have all important data backed up. I believe the 8TBs are much better suited to being part of your new backup array.

I recently upgraded my 10x2TB RAID-6 array (6 years old) to 10x4TB, using the Seagate 4TB 5900rpm (ST4000DM000) drives. Yes, they're Seagates but these particular ones are well-reviewed for RAID use on the forum and I got them cheap from externals (about ~$100/drive). If I had some more money, I'd have preferred the Hitachi 4TB NAS drives, which is what I'd recommend for your array, if you're comfortable with simply doubling your space for now.

 

[izx]

what you just decribed was basically archive. I recall some weird mumber of 150GB of writes a day or 150TB a year or something. I might be confusing something else. Trimlock I am sure could answer this. (or one of the other more fluent big data guys)

Yeah, they're rated for workload (=total reads+writes) of 180TB/year. OP doesn't have to worry about this.

 

[diizzy]

Be careful about the "Archive drives", they do not work as your average HDD (SMR etc) . As far as consumer drives goes I'd say Toshiba by far and HGST (Hitachi) after that especially when going for LSI HBAs/RAID cards.

 

[izx]

As far as consumer drives goes I'd say Toshiba by far and HGST (Hitachi) after that especially when going for LSI HBAs/RAID cards.

I'd put HGST first unless cost is an overriding priority. I recently purchased two Toshiba 6TB externals and two Toshiba X300 6TB internals. The drives in the externals were the "older" MD04ACA600 series. By all accounts they should be equivalent, but the MD04s are slower at sequential read/writes by about 20 MB/s (peak ~185 vs X300 at ~207). Funnily, the warranty is better on the external (3 years) compared to the internals (2 years), although the internals were certainly more expensive. Using them in a 4-drive RAID5 atm, have endured badblocks torture just fine. They do run a little warm though, but that's expected at 7200rpm.

 

[SomeGuy133]

I'd put HGST first unless cost is an overriding priority. I recently purchased two Toshiba 6TB externals and two Toshiba X300 6TB internals. The drives in the externals were the "older" MD04ACA600 series. By all accounts they should be equivalent, but the MD04s are slower at sequential read/writes by about 20 MB/s (peak ~185 vs X300 at ~207). Funnily, the warranty is better on the external (3 years) compared to the internals (2 years), although the internals were certainly more expensive. Using them in a 4-drive RAID5 atm, have endured badblocks torture just fine. They do run a little warm though, but that's expected at 7200rpm.

apples to oranges there it sounds like. You sound like your comparing 5400/5900 to 7200

What is warm? I don't even have my case sealed so my HDDs are getting crap air flow and they idle 32-35 C. That will drop when the case is actually sealed too.

Max under stress load was 39C and again that'll be lower when case is sealed and fans actually work like they are supposed to.

So what is warm? IIRC I remember people talking about google released some papers saying cold hard drives failed more often and running them too cool was bad. (was a few to a number of years ago)

EDIT: max is 37-40C depending on drive. I assume that happened when I did a 2x bad sector test that took 2 or so days of straight read/writes. Or when I was trying I/O tests on it.

 

[izx]

apples to oranges there it sounds like. You sound like your comparing 5400/5900 to 7200

What is warm? I don't even have my case sealed so my HDDs are getting crap air flow and they idle 32-35 C. That will drop when the case is actually sealed too.

Well, warm here is relative of course -- the Toshiba's will go up to 40-41C at full load, in a properly ventilated Norco 4224 case. In comparison, my old Hitachi 2TB 7200rpms maxed out around 36-37C in the same case/slots. A companion Seagate 6TB 7200rpm (enterprise), part of the same array (3 Toshibas, 1 Seagate) will max at 37C.

I haven't done any semi-serious comparisons for temperature, and in absolute terms, 41 C isn't hot per se. I am envious of the 5900rpms in a different array in the same case which max out at 32-33C and idle well under 30 C. Apples to oranges, as you said though

 

[SomeGuy133]

Well, warm here is relative of course -- the Toshiba's will go up to 40-41C at full load, in a properly ventilated Norco 4224 case. In comparison, my old Hitachi 2TB 7200rpms maxed out around 36-37C in the same case/slots. A companion Seagate 6TB 7200rpm (enterprise), part of the same array (3 Toshibas, 1 Seagate) will max at 37C.

I haven't done any semi-serious comparisons for temperature, and in absolute terms, 41 C isn't hot per se. I am envious of the 5900rpms in a different array in the same case which max out at 32-33C and idle well under 30 C. Apples to oranges, as you said though

yea in that seagate thread someone was talking about how high temps dont matter til the extreme. I do wonder what that is. 60C? 80C? it was in some google article or something. I never bothered to look becuase it is really not note worthy to me since I don't ever worry about high temps. Also that 41C is with the case lid off so air flow is total crap. I bet those tmeps will drop a lot once it is closed but again I don't care til it is 50s or 60s C and then i am still not even sure if that is even note worthy.

 

[Stereodude]

I'm also in the process of building a new NAS / "server" right now myself. I currently have a PERC-6i with 8x2TB 5400RPM Samsung drives. It's pretty much full and I also have some concerns about the number of hours on the drives after 5+ years of running 24/7.

The new NAS / "server" is going to have a IBM ServeRAID M5016 (LSI MegaRAID 9265-8i equivalent) + HP SAS Expander so I can support more than 8 drives for online capacity expansion down the road if/when I need it. I'm planning to start with 6 or 8 7200RPM Toshiba Enterprise SATA drives in RAID-6. Specifically, the 6TB MG04ACA600E model. Assuming no FedEx problems I should get the first two today for testing. I plan to make sure they will play nice with the expander and the IBM/LSI RAID card before buying another 5 to 7 of them.

I ruled out SMR drives and decided to stick with PMR drives. My workload is definitely more reading than writing getting that it's mostly serving media to other devices in the house, but I have no idea how to predict when the SMR slowdown will hit when writing data to the array or how much has to be written before it gets slow. I don't want to dump a Blu-ray image to it and have the copy get really slow halfway through. I also decided to go with Enterprise class HDDs this time around for a few reasons.

1) They have an order of magnitude better Non-recoverable Error Rate vs. consumer drives. 1 error per 10^15 bits read vs. 1 error per 10^14 bits read. With 6TB drives you've basically got a 50% change of getting a unrecoverable bit during a rebuild or other operation that reads the whole drive. The Enterprise drive is only 5%.
2) They're not that much more expensive than consumer drives and I guess I've become less price sensitive about protecting my data.
3) They have features to better ensure data integrity like ECC on the cache.
4) Taking point 3 one step further, the Toshiba drives I chose further have have flash memory to back up the cache in the event of the drive losing power. The backEMF from the motor and platters spinning down generates power which is to used to copy any unwritten data from the drives cache to the flash on the drive where it will sit protected until power is restored. The IBM ServeRAID controller I'm using has the same feature, but uses a supercapacitor to provide power for the copy operation.

 

[izx]

Yeah, the Toshiba Enterprises are a great choice. Where did you get yours from? I've also become a believer in spacing out drive purchases over some time and different suppliers to ensure they're not from similar batches.

 

[Stereodude]

Where did you get yours from?

I ordered from Wiredzone.

I've also become a believer in spacing out drive purchases over some time and different suppliers to ensure they're not from similar batches.

I haven't gone to those lengths. I don't think it really matters much. Further, I haven't found another source with pricing similar to Wiredzone on the enterprise Toshiba drives. Paying more for the same drives doesn't make sense to me when they're probably all drop shipping from the same warehouse / distributor.

 

[KaptainBlaZzed]

I ordered from Wiredzone.


I haven't gone to those lengths. I don't think it really matters much. Further, I haven't found another source with pricing similar to Wiredzone on the enterprise Toshiba drives. Paying more for the same drives doesn't make sense to me when they're probably all drop shipping from the same warehouse / distributor.

Damn, they have good prices on those drives. I might just have to get some.

 

[Stereodude]

Damn, they have good prices on those drives. I might just have to get some.

In general, their pricing on enterprise drives is quite good. However, some of the individual drive pricing doesn't make a lot of sense. Like a 4TB 7200RPM Toshiba with 512e sectors is $171.20. The same drive with 4Kn sectors is $230.98. That's an extra $59.78 for 4Kn. But, at 6TB the price delta between 512e and 4Kn is only $5.16 ($266.99 / $272.15).

 

[KaptainBlaZzed]

Update:

I went with 8 x 6 TB Toshiba Enterprize HDD's in RAID 6.
It took about 12hrs to initialize the array and another 24 hrs to copy 10 TB of data to it.

I also replaced a 4TB backup drive to a 8 TB seagate Archive HDD.
they are cheap, but writing to them is very slow. it took 26 hours to write 4TB to it. (large movie files)
I am glad i decided against putting these in an array as it would have take 4 to 6 days to get it initialized and get all the data transferred to it.