Building 8TB Storage server

[ARH]

Hi

I want to build the same system like Ockie was build in Galaxy projects, our goal is to rich 8TB of storage with sixteen 500GB HDD with components which is available in my country.

I want to use this Chassis :
http://www.green-case.com/products/case/server/case.php?model=powerful

with two RAID controller card from Promise:

http://www.promise.com/product/product_detail_eng.asp?product_id=156

with these cards I could make two RAID 6 array each with 8 HDDs. But I have no idea about how much of my storage dedicated to Parity bits of RAID 6 ?

I want to buy ASUS M2N32 SLI Deluxe motherboard featured to PCI-E X16 and one PCI-E X4 that I can install my Graphic card and two PCI-E RAID card on it.
also there are other choice like ASUS Crosshair or M2N32 WS Professional or Gigabyte M59SLI-S5, what is your suggestion about motherboard ?

For the processor I am thinking to AMD Athlon 64 X2 6000+, this processor have very good performance/price ratio in my county.

And for the PSU , I have ordered SevenTeam 1KW EPS Power supply.

http://www.seventeam.com.tw/front/detail.asp?lang=eng&ID=144

I want to use Windows Server 2003 64Bit with NTFS. are there any limitation in partitioning of drives spaces ? what is your suggestion for partitioning tool of this TB storage ?

I have seen some RAID controller have a Back-up battery, Promise EX8350 also have optional battery pack… what is the functionality of back-up battery in RAID controller?

What I should do for increasing the reliability and durability of such a huge computer?


I would appreciate of your answers or suggestions.



Note: for choosing the best storage solution I have already opened this topic, but for the economic reasons I can't

 

[Cyrilix]

RAID 5 = 1, RAID 6 = 2.

 

[unhappy_mage]

I want to use this Chassis :
http://www.green-case.com/products/case/server/case.php?model=powerful

That only has room for 8 drives. What will you do with the other 8? Are you planning to have a separate boot disk?

with two RAID controller card from Promise:
http://www.promise.com/product/product_detail_eng.asp?product_id=156

Those cards tend to run hot, and I've seen mixed reviews of them. Are Areca products available in your country? What country is it? I might be able to find more information if I can be specific with my searching.

with these cards I could make two RAID 6 array each with 8 HDDs. But I have no idea about how much of my storage dedicated to Parity bits of RAID 6 ?

Each array would have the capacity of N-2 disks, where N is the number of disks in that array. In your instance, you'd get the capacity of 6 500GB disks in the array, or 3TB per array.

I want to use Windows Server 2003 64Bit with NTFS. are there any limitation in partitioning of drives spaces ? what is your suggestion for partitioning tool of this TB storage ?

You'll need to make a GPT partition table if you want it all in one volume: GPT FAQ.

I have seen some RAID controller have a Back-up battery, Promise EX8350 also have optional battery pack… what is the functionality of back-up battery in RAID controller?

It keeps cached data alive when the power goes out unexpectedly. This can keep your array from being corrupted. You'll almost certainly want this, and if you expect the array to perform well, you'll need this.

What I should do for increasing the reliability and durability of such a huge computer?

Backup battery would be a good idea.

 

[Tolyngee]

Backup battery would be a good idea.

http://shop1.outpost.com/product/3493453?site=sr:SEARCH:MAIN_RSLT_PG

http://www.provantage.com/belkin-f6c1500-tw-rk~7BELZ036.htm

If you can buy all that hardware, but can't afford the above, then well, you just deserve to lose your data or something...

 

[ARH]

That only has room for 8 drives. What will you do with the other 8? Are you planning to have a separate boot disk?

yes I have ordered to customize it for 20HDD+1DVD-Rom

Those cards tend to run hot, and I've seen mixed reviews of them. Are Areca products available in your country? What country is it? I might be able to find more information if I can be specific with my searching.

Unfortunately there aren't any other famous Brand in my country, so I have to use this card and haven't other choices .
I scare to tell you about my nationality, you could think I am living in a under development county.

You'll need to make a GPT partition table if you want it all in one volume: GPT FAQ.

Thanks for the link.
I don't want to gather it all in one volume , maybe 2 partitions for 3TB of storage will be enough, did Ockie use the same solution for partitioning his storage? I haven't see any thing in her topic.

 

[unhappy_mage]

Unfortunately there aren't any other famous Brand in my country, so I have to use this card and haven't other choices .
I scare to tell you about my nationality, you could think I am living in a under development county.

What about ordering it from a shop that ships internationally?

Thanks for the link.
I don't want to gather it all in one volume , maybe 2 partitions for 3TB of storage will be enough, did Ockie use the same solution for partitioning his storage? I haven't see any thing in her topic.

Anything over 2 TB needs GPT. Ockie isn't using RAID, so he has a bunch of smaller volumes that are less than 2 TB each. Thus he doesn't have to worry about GPT, yet.

 

[Ockie]

Hi
I want to build the same system like Ockie was build in Galaxy projects, our goal is to rich 8TB of storage with sixteen 500GB HDD with components which is available in my country.

YES! I have a following!! haha jk!

I want to use this Chassis :
http://www.green-case.com/products/case/server/case.php?model=powerful

Okay. Sounds good, especially if you are going to mod it.

with two RAID controller card from Promise:

http://www.promise.com/product/product_detail_eng.asp?product_id=156

I would recomend that you do not use these controllers, for the reason that they are more expensive than other options and they are going to be a PITA for wiring (the location of the sata ports are weak). You should look at something like the highpoint 2340, then you can have 16 drives on one controller, clean wiring, good connection to the controller (no wiggling cables if they come loose, and cheaper. You can dump your entire array on one controller and have space if you ever wanted to expand later on.

Just find a vendor that would ship to your country.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816115031

with these cards I could make two RAID 6 array each with 8 HDDs. But I have no idea about how much of my storage dedicated to Parity bits of RAID 6 ?

I'd worry about this once you have the card and have decided which Raid you wish to use.

I want to buy ASUS M2N32 SLI Deluxe motherboard featured to PCI-E X16 and one PCI-E X4 that I can install my Graphic card and two PCI-E RAID card on it.
also there are other choice like ASUS Crosshair or M2N32 WS Professional or Gigabyte M59SLI-S5, what is your suggestion about motherboard ?

I'd go with the M2N32-SLI, it's a great board for this application and is solid. There is also the Intel version professional board, which actually has 4 PCIE x16 slots (physical). But the M2N32-SLI is a good choice sir

For the processor I am thinking to AMD Athlon 64 X2 6000+, this processor have very good performance/price ratio in my county.

This processor is beyond overkill. Unless if you are intending to offload controller work onto the processor (not the case, since these are good controllers), you won't need something this powerful. I'd rather spend a little more in another area or add more storage and get a lower processor.

Really anything in this department will work, get yourself a cheap dual core.

And for the PSU , I have ordered SevenTeam 1KW EPS Power supply.

http://www.seventeam.com.tw/front/detail.asp?lang=eng&ID=144

Sounds good.

I want to use Windows Server 2003 64Bit with NTFS. are there any limitation in partitioning of drives spaces ? what is your suggestion for partitioning tool of this TB storage ?

What unhappy_mage said.

I have seen some RAID controller have a Back-up battery, Promise EX8350 also have optional battery pack… what is the functionality of back-up battery in RAID controller?

Backup battery is to keep RAID configs alive as have been explained above. I'm not too much of a stickler in this, but it's a good thing to have. For a home system, probably not too much of a MUST have option. If it's not much more, try to get that.

What I should do for increasing the reliability and durability of such a huge computer?

Keep it up to date with patches and antivirus. Thats pretty much all you can do

 

[ARH]

hah
I find one company in my country who sell 3war and LSI products , I have seen promise EX8350 reviews, there are some disappointing issue in reviews.
what is your suggestion for RAID controller ?

 

[ARH]

I would recomend that you do not use these controllers, for the reason that they are more expensive than other options and they are going to be a PITA for wiring (the location of the sata ports are weak). You should look at something like the highpoint 2340, then you can have 16 drives on one controller, clean wiring, good connection to the controller (no wiggling cables if they come loose, and cheaper. You can dump your entire array on one controller and have space if you ever wanted to expand later on.

Just find a vendor that would ship to your country.

http://www.newegg.com/Product/Product.aspx?Item=N82E16816115031

Yes, this is a good choice but I haven't seen it in my country, I am trying to contact Highpoint for finding local resellers.

This processor is beyond overkill. Unless if you are intending to offload controller work onto the processor (not the case, since these are good controllers), you won't need something this powerful. I'd rather spend a little more in another area or add more storage and get a lower processor.
Backup battery is to keep RAID configs alive as have been explained above. I'm not too much of a stickler in this, but it's a good thing to have. For a home system, probably not too much of a MUST have option. If it's not much more, try to get that.

This server have a tremendous load in some times of the day , when we are running collective application. Also Athlon 64 X2 6000+ is cheap here, cheaper than one 750GB HDD.

Backup battery for promise RAID controller is $119


Thanks for your answer

 

[unhappy_mage]

hah
I find one company in my country who sell 3war and LSI products , I have seen promise EX8350 reviews, there are some disappointing issue in reviews.
what is your suggestion for RAID controller ?

I don't recommend the 3ware cards. The LSI cards might be a good buy or not; which cards are available from them?

Yes, this is a good choice but I haven't seen it in my country, I am trying to contact Highpoint for finding local resellers.

I won't judge you if you tell me what country you're from, I promise. Have you seen the list of distributors on Highpoint's website?

This server have a tremendous load in some times of the day , when we are running collective application. Also Athlon 64 X2 6000+ is cheap here, cheaper than one 750GB HDD.

Backup battery for promise RAID controller is $119

You need the battery backup, simple as that. It will let you use your cache in the faster write-back mode, as explained by this blog post. This mode is usually locked out when there is no battery backup present, to prevent data loss in the event of power failure.

 

[Ockie]

This server have a tremendous load in some times of the day , when we are running collective application. Also Athlon 64 X2 6000+ is cheap here, cheaper than one 750GB HDD.

Wait, you are saying "we" and saying tremendous load...

Is this for a business? What exactly will this server be doing? If this is for a business or focused for a business purpose, then my entire recomendation changes.

 

[Farva]

If this is for business use, I would recommend an Areca card instead of the Highpoint.

 

[ARH]

I don't recommend the 3ware cards. The LSI cards might be a good buy or not; which cards are available from them?

.

Currently I don’t know, I just know that a company in my country sold this cards, I was tell to that company and he told me RAID card salesman goes to journey and not be right back until Sunday.

I won't judge you if you tell me what country you're from, I promise. Have you seen the list of distributors on Highpoint's website?

Thanks, you are magnanimous, I am Iranian. I was send an email to sales of Highpoint and Areca, but there are no hope to give reply.

 

[ARH]

Wait, you are saying "we" and saying tremendous Is this for a business? What exactly will this server be doing? If this is for a business or focused for a business purpose, then my entire recomendation changes.

Yes Ockie , this server will be dedicated to Application+Web+Storage server of a digital library, currently we use the same configuration for this job with out terabyte storage and simply connect our HDD to the motherboard (about 1TB) , but book collection grows and we are needing a storage solution for this job.

I was discussed about possible solution in this topic and you are posting on it, but these solution didn't match to our budget, we were try to import some product needed for this solution but this work give lots of time and not currently possible.

Currently our server based on Athlon 64 3000+ (socket 754) work well and in the most of the time of day, CPU usage limits to 10-20%.

But in some time when we wants to add thousands of book , CPU usage of the application we write for this job limits to 90-100% , this work done at 1-2 hour.

Hey... are you really thinking I need this Computer for home usage?

 

[Ockie]

Thanks, you are magnanimous, I am Iranian. I was send an email to sales of Highpoint and Areca, but there are no hope to give reply.

Yeah, you are screwed. You are not going to get many vendors who will ship to there. I am surprised you guys can get promise products.

Yes Ockie , this server will be dedicated to Application+Web+Storage server of a digital library, currently we use the same configuration for this job with out terabyte storage and simply connect our HDD to the motherboard (about 1TB) , but book collection grows and we are needing a storage solution for this job.

Yeah, scratch the highpoint in this case. You want Arcera and 3Ware.

Currently our server based on Athlon 64 3000+ (socket 754) work well and in the most of the time of day, CPU usage limits to 10-20%.

But in some time when we wants to add thousands of book , CPU usage of the application we write for this job limits to 90-100% , this work done at 1-2 hour.

Oh okay.

Hey... are you really thinking I need this Computer for home usage?

Well this is the hardforums People do crazy things... such as myself!

 

[Farva]

Well this is the hardforums People do crazy things... such as myself!

Yeah, like the Worklogs and the Storage Showoff Thread.

 

[unhappy_mage]

Thanks, you are magnanimous, I am Iranian. I was send an email to sales of Highpoint and Areca, but there are no hope to give reply.

It looks like this eBay auction will ship to Iran for about $43. It is expensive - $1048 USD compared to about $930 USD where I would probably buy it - but they do ship internationally.

 

[ARH]

It looks like this eBay auction will ship to Iran for about $43. It is expensive - $1048 USD compared to about $930 USD where I would probably buy it - but they do ship internationally.

Wow, what a great card, I really surprised when I read the specification.
We could afford this card, because two Promise EX8350 RAID Controller cost similar (each one sells around $470 here) But we can't wait and we can't trust … I think the delivering process can take up to 6 weeks or more and we can't trust to any one with current political situation of our country.
Hey guys be gratitude for your possibilities.

 

[ARH]

Yeah, like the Worklogs and the Storage Showoff Thread.

I will be a new member of Storage Showoff Thread as soon as possible , lets this work finish.

 

[unhappy_mage]

Wow, what a great card, I really surprised when I read the specification.
We could afford this card, because two Promise EX8350 RAID Controller cost similar (each one sells around $470 here)

It is indeed rather nice Here's another possibility for getting it: call Yang Ming International at 1-800-526-8650 and ask them. They don't take online orders, but they do ship internationally. WebConneXXion also carries the card and ships internationally.

But we can't wait and we can't trust … I think the delivering process can take up to 6 weeks or more and we can't trust to any one with current political situation of our country.
Hey guys be gratitude for your possibilities.

No kidding!

 

[ARH]

I have read Promise EX8350 user review in newegg, I think there is good card and it is enough for me, some ones report poor performance and compatibility, I think their problems related to their platform and configuration not to Card.


Tomshardware reviewed this card and compare its performance to Adaptec and Areca 1220, but no reported for any problem issue with this card.

It seems I haven't any choices for RAID controller, Still waiting for company news until sunday

this review from one buyer of Amazon is noticeable:

First mistake was having the IT Tech at the time build critical mission servers himself, second mistake was to use this card. Card is very picky with what motherboards it chooses to work with. The final straw with this garbage is losing the RAID 5 and not being able to delete the logical drive from the BIOS.

Promise support is less than helpful, if they answer you at all.

This might be fine for at home setup, but for anything business related, go buy a name brand. With the time I spent trouble shooting this card, we would have saved money doing so.

 

[ARH]

Good news our chassis and PSU was Customized...
I was think this work take more times, Thanks to efforts of Mr.Omrani from GREEN Company and their cooperators from Seventeam Company in this picture :

this is our chassis:

and this is the rear view, as you could it is tall and spacious :

this chassis have 8 HDD slot in stock, we try to mod in for up to 20 HDD slot, as you can see in this picture , we install two HDD cage each with 5 HDD slot in the CD-ROMs Slots and above the PSU :

in front of the cage inserted in CD-ROMs slots we install one 120mm and one 80mm fan for cooling the drives.

in the back of the cage inserted in above the PSU we install one 120mm fan.

for the 8 HDD inserted in the stock HDD slot, the chassis have 3 fans.

Seventeam 1000watt PSU also moded to give me 10 SATA cable and 10 Molex cable , also the 5v-3.3v combine power of it improved from 250watt to 350watt.
this mod take placed under supervision of SevenTeam Technical manager who came to IRAN for business reasons.
you can see PSU cables in this image :

and finally, this picture shows fans cabling :

 

[elec999]

Just curious why not use Areca cards.
Thanks

 

[ARH]

Just curious why not use Areca cards.
Thanks

I am limited to only products available in my country and I can't purchase any think aboard for the political and time reasons.


I was email to sales of Areca company and this is their reply :

Hi

We do not have reseller in Iran. Sorry.....

Would you contact with our USA or Germany distributor to buy what you need ??

Bets Regards,
Polly Lee
Areca Technology Corporation
www.areca.com.tw
TEL: 886-2-87974060 ext: 232

 

[Damodred]

Hello

As a small sidenote, you do not need to use GPT if you update to SP1 or later with Windows Server 2003.

 

[Sparkyy]

Actually you need to use GPT if you want to go over the 2TB mark, SP1 only enables 32bit Server 2003's to use GPT, prior to SP1 for Server 2003, you could only use 64 bit processors.

 

[duostrike]

Actually you need to use GPT if you want to go over the 2TB mark, SP1 only enables 32bit Server 2003's to use GPT, prior to SP1 for Server 2003, you could only use 64 bit processors.

It would also seem that Vista supports GPT though I wouldn't recommend putting it on a fileserver though..
http://www.maximumpc.com/article/a_reason_to_finally_use_vista

 

[ARH]

Thank you friends
I want to use Windows Server 2003 X64 , and use GPT , GPT have some benefits over MRT that if I don't really need it, worth to pay it

mainboard and CPU will be come soon. be ready ...

 

[Damodred]

really? Im not so sure about that, altho its seldom I know what Im doing completely...

I made an array three days ago of 9x500gb in Windows Server 2003. I converted them to dynamic disks and then created the array. Formated with NTFS and thats about that. No problems. First I tried to convert from Dynamic to GPT which worked out but then I lost a whole 1,2gb compared with regular,...uhm...non GPT. So now its just NTFS.

 

[ARH]

really? Im not so sure about that, altho its seldom I know what Im doing completely...

I made an array three days ago of 9x500gb in Windows Server 2003. I converted them to dynamic disks and then created the array. Formated with NTFS and thats about that. No problems. First I tried to convert from Dynamic to GPT which worked out but then I lost a whole 1,2gb compared with regular,...uhm...non GPT. So now its just NTFS.

Damodred
what is a dynamic disk? I see it in disk manager but I think this is not relate to my work.
as I could understand the only cons of GPT tabling was consume more larger space than MBR, is it true ?
what is the number of your volumes when configuring GPT ?

thank you

 

[Sparkyy]

You lost 1 or 2 GB by using GPT? I have a 2.27 TB RAID 5 right now and when I converted it from Basic to GPT it shows I only lost like .05 GB, I switched it between Basic and GPT just enough to see the change and it was so minor compared 2.2 TB that I hardly noticed any space lost.

Maybe because it went to Dynamic then GPT?

 

[Damodred]

Yea most likely.
The reason I actually saw it was because I use software raid5 (hehe dont kill me) and in the diskmanager, 127MB on each disk except the first one was unused with GPT while every MB was used with dynamic. Dunno if it has to do with the way GPT system is stored on the first disk or something, regardless I decided not to use it. Its a very minor difference here as well but I thought I might just as well need that last GB later on

 

[ARH]

Yea most likely.
The reason I actually saw it was because I use software raid5 (hehe dont kill me) and in the diskmanager, 127MB on each disk except the first one was unused with GPT while every MB was used with dynamic. Dunno if it has to do with the way GPT system is stored on the first disk or something, regardless I decided not to use it. Its a very minor difference here as well but I thought I might just as well need that last GB later on

I agree with Sparkyy, I is hardly noticeable spcase.

What is your Storage throughput performance under software RAID 5 ? are you test it ? would you mind test it with HDD Tune or HDD Tach .

 

[Damodred]

Read is as always good, cant find a single computer that can single handed write as fast as it reads. Im copying a lot of data on it right now from my other two arrays (2x 8x300gb) and speed is slightly over 2gb per minute over a gbit network, if I calculate right (which I seldom do) I get slightly above 36MB/sec. These numbers are about the same if I write from or to the arrays, they all have the same speed more or less.

Tomorrow when hopefully everything has been copied I can give out some HDD Tach pics or Sisoft Sandra.


EDIT: I cant make HD Tach or HD Tune to do arrays so I used a program called ATTO Disk Benchmark
http://members.home.nl/rvandesanden/ATTO benchmark.html

Systemdrive:

Raid5 array consisting of 9x500gb

Im not sure its reliable, it seems to be quite high on both the single system drive and the raid5. If someone is up for it, please make your own test and compare it with my results.

 

[ARH]

Raid5 array consisting of 9x500gb

Im not sure its reliable, it seems to be quite high on both the single system drive and the raid5. If someone is up for it, please make your own test and compare it with my results.

ooooO your RAID 5 array OVERKILL theoretical SATA II bandwidth.
I will test ATTO as soon as possible , but this results aren't trusty.
why you couldn't use HDD Tune or HD Tach ? I just familiar with these tools

 

[Damodred]

As I said, with both of the programs I can only test individual drives and not arrays. If someone knows how to make it work on arrays let me know and I'll test it, they are both still installed on my system Regarding ATTO, it might show burst speed which might explain the high results.

 

[ARH]

Hi

finally I have buy other component of system :

First, because price of the Intel Core 2 Quad dropped in my country and now it would be affordable , I decide to make a risk and come in Intel platform, yes finally I am buy Q6600:

and for it's mainboard , I buy ASUS Striker Extreme :

yes, this is too huge mainboard and I dont need some of its features , but there aren't any other choice for me because the only this motherboard available here features 3 PCI-E X16 slot.
I have searched all of stores for the mainboard. but I can't find any thing , this board is on of the best mainboard in the world, but not for my usage.

also I have buy 4GB Corsair DDR2-800 Memory :

and promise EX8350 RAID controller :

and HDDs and other necessary things .


Lets look at my Moded Chassis :

I have wrote detail of adding 2 custom HDD cage in this post

this is overall look of the Chassis (back door is open) and PSU bag:

this Chassis have 10 fan (plus PSU fan) and this is It's air follow scheme:

we have installed an anti-dust filter in front of upper input air flow:

and carefully route fan cables :

I am going to Build this Server, Tommorow