Help me build a dedicated folding box

[dreadwing]

I have emailed and chatted with several of you on IRC and figured I'd make a thread to get some build advice on my next dedicated folding rig.

I currently have 4 machines that I am folding on that produce about 65K PPD in total (all SMP WU's). My goal would be to get somewhere in the range of 120-200K PPD on a single machine.

Here are the requirements, so please let me know what you think:

- Ability to run ESXi 5.0 with VT-D (IO Passthrough of my M1015)
- Cheap but FAST DDR3 Ram with a minimum of 32+ GB
- 100K+ minimum, prefer to be close to 200K PPD.
- low heat and power CPU chips using a 2 processor motherboard.
(I am open to AMD 4P setup but figured I'd start with something mid-range for now. If there are deals on a 4P setup, I would consider that)
- What cases/psu's would you suggest. Something as small as possible that can still fit an e-atx and yet looks decent.
- Any additional tips on performance for my existing machines. So far I have been running on Win7-64 with virtualbox using ubuntu 10.10 and linuxforge images. I have not OC any of my systems so some advice on improving my CPU and RAM speed would be great.

I am also interested in the SR-2 motherboard for it's OC abilities but need to know if the motherboard supports VT-D so I can pass-through my SAS controller. If anyone has esx running on theirs, I'd like to get some feedback.

Thanks again to all that have been so helpful in getting me started and answering questions.

 

[Kendrak]

Not sure on the VT-D, but the SR-2 is what you want.

A dual hex will do 200k ppd with propper WU and OC.

Also, if you catch a deal here, one should be able to grab a combo for ~$1300 or less depending on the sanity of the seller and what is coming with it.

THB, when you are ready, I would put a WTB thread up in the FS section (if there isn't a SR-2 listed) and see if anyone was thinking of parting with theirs.

 

[dreadwing]

I've seen some mention about the L5640, is that the chip to pair with the SR-2? Also, when getting ram, what chips would you pair with that? Is there a certain brand/type that work well with folding and can be OC?

I will take your advice and post on WTB thread on here if I decide to go the SR-2 route. Just need to make sure I can do pass-through on it. If anyone has one, can they check their bios and see what virtualization options are available?

 

[Kendrak]

I will look when I get home.

And yes, the l5640 are a common chip to pair with SR-2s.

RAM is RAM as long as yor timings/speed are good (1600 or higher).... and a quality name so sticks won't start going bad on you. Then it is a game of find the bad stick and you have 6 to chose from.

 

[dreadwing]

Thank's Kendrak. I have a bunch of extra corsair vengeance LP chips I can use.

Is there a better option than the L5640?

 

[DooKey]

Thank's Kendrak. I have a bunch of extra corsair vengeance LP chips I can use.

Is there a better option than the L5640?

There are better options, but that drives the cost up. The spicy l5640s have a 18x turbo multi which means most times you'll only get around 3.6ghz out of them.

 

[dreadwing]

3.6 GHZ seems fast enough. At that speed what temps are you seeing on full load? Is there a certain HSF that you use or suggest?

I noticed they were going for $350 on eBay. What better options would you consider and how much does that raise the cost?

 

[musky]

First question - what is the budget?
Second question - what is the final goal (ppd and ppd/W)?

If 120K-200K ppd is the final goal, there are a few ways to go. The best option depends on the up-front budget. I kinda like:

2P G34 SM board - $350
6166 HE 12 core - $325 x 2
CAS7 7 1333 memory - $32 x 4
Some HSFs - $30 x 2
Some PSU (this thing will be well under 300W, so anything will do) - $50

That comes in at under $1250, somewhere around 155K ppd with overclock, somewhere around 250W, 620 ppd/W (very respectable for a 2p), and somewhat upgradable (you could swap the 6166Hs for 6172/6174/6176/6180/even IL at some point.) Also, E-ATX should be much easier to stuff in a case.

If that is too much money, then let us know the budget.

 

[Tobit]

2P G34 SM board - $350
6166 HE 12 core - $325 x 2
CAS7 7 1333 memory - $32 x 4
Some HSFs - $30 x 2
Some PSU (this thing will be well under 300W, so anything will do) - $50

I endorse this .config. I may do something similar soon to go with my other 2P's. I don't think I want to venture into 4P... well, I do.. I just don't want to invest that kind of money.

 

[dreadwing]

$1500-2000 though budget is somewhat open as I will be willing to pay more if the performance is worth the extra cash. I would like the ability to start out and upgrade/add as needed.

Closer and past 200K PPD would be ideal or at least the ability to at some point.

Just note that the I will need to use this as an occasional ESXi 5.0 lab when not folding. Just need to make sure what ever option I go with will support pass-through of sas devices to a VM. However, I will be primarily using this as a dedicated folding server.

What are some smaller cases that would accommodate that e-atx board?

First question - what is the budget?
Second question - what is the final goal (ppd and ppd/W)?

If 120K-200K ppd is the final goal, there are a few ways to go. The best option depends on the up-front budget. I kinda like:

2P G34 SM board - $350
6166 HE 12 core - $325 x 2
CAS7 7 1333 memory - $32 x 4
Some HSFs - $30 x 2
Some PSU (this thing will be well under 300W, so anything will do) - $50

 

[musky]

$1500-$2000 gets you into the 4p realm - at least for a motherboard. The nice thing about the 4p boards is that you can run them with any number of procs, so you wouldn't need the huge investment up-front. The down-side is that they are significantly more expensive. Since you are looking for SAS, you become even more expensive. If you don't mind zero overclock potential, the Tyan 4p board may be your best bet. It has 2 x SAS ports and is really a nice board for a server board.

As for a case - the HAF 932 or X will hold the 4p board. Many cases will hold E-ATX.

$1500-2000 though budget is somewhat open as I will be willing to pay more if the performance is worth the extra cash. I would like the ability to start out and upgrade/add as needed.

Closer and past 200K PPD would be ideal or at least the ability to at some point.

Just note that the I will need to use this as an occasional ESXi 5.0 lab when not folding. Just need to make sure what ever option I go with will support pass-through of sas devices to a VM. However, I will be primarily using this as a dedicated folding server.

What are some smaller cases that would accommodate that e-atx board?

 

[Kendrak]

$1500-$2000 gets you into the 4p realm - at least for a motherboard. The nice thing about the 4p boards is that you can run them with any number of procs, so you wouldn't need the huge investment up-front. The down-side is that they are significantly more expensive. Since you are looking for SAS, you become even more expensive. If you don't mind zero overclock potential, the Tyan 4p board may be your best bet. It has 2 x SAS ports and is really a nice board for a server board.

As for a case - the HAF 932 or X will hold the 4p board. Many cases will hold E-ATX.

This is an great suggestion.

 

[dreadwing]

It does sounds like a pretty compelling argument to go for a 4P system. If I get a 4P board, I will probably start with 2 CPU's and upgrade from there. What CPU do you suggest? I want to get a decent processors that will be good from the start. Should I buy chips off eBay and should I try to find ES's?

Also, I do not need SAS on the motherboard. I have a m1015 raid Card that I would like todirectly pass to a VM in ESXi. Do you know if that is supported with this setup.

On 4P system, what parts would you recommend as far as CPU, Mobo, Ram... And what do you project the final investment to run/

I rather pay a little more on something that will last awhile.

Thanks!

$1500-$2000 gets you into the 4p realm - at least for a motherboard. The nice thing about the 4p boards is that you can run them with any number of procs, so you wouldn't need the huge investment up-front. The down-side is that they are significantly more expensive. Since you are looking for SAS, you become even more expensive. If you don't mind zero overclock potential, the Tyan 4p board may be your best bet. It has 2 x SAS ports and is really a nice board for a server board.

As for a case - the HAF 932 or X will hold the 4p board. Many cases will hold E-ATX.

 

[Zink]

Those 6166 HE look really good. They will be much cooler than new 32nm chips like the Opteron 6272 and almost as fast.

 

[musky]

Should you buy chips off eBay? Yes, you should. If you are patient, you can usually find some good deals.

Should you try to find ES chips? Well, yes, but they are very rare and usually very expensive. I don't think I would go into it planning on ES chips.

CPUs - anywhere from a 6164HE to a 6180, depending on what you want to spend. I would go with 12 core chips myself, although you could go 8 core for quite a bit less (sub-@$200 each.) I would not go with Interlagos - they are more power-hungry than the Magny Cours chips. 6172/6174 may be the sweet spot price/performance. 6166HEs can usually be found for around $300 each, and are not bad at all (370K ppd in a 4p configuration stock with lousy memory.) Again, it is all about how patient you are. You could most likely buy 6174s today for $500 each, or you can sit on auctions and try to snipe them for $400 or less. Just let us know what you are trying to get, as many of us watch everything on eBay and try to snipe them ourselves...

Board - depends on your cooling plans. SuperMicro is the way to go if at all possible. The newer version can be had for $635 or so new, but the socket layout is horrible. The older ones have a more reasonable socket layout, but are also around $75 more expensive. Look on the SuperMicro site for one with the features you need, then check for the best price. Note that you will most likely have to wait a couple weeks, as most (all?) places actually drop-ship from SM. SM will allow you some overclocking (12.5% is fairly common.)

Memory - 1333 CAS 8 or 7, 1.5V. The big things are 1.5V or less and as tight timings as you can find/afford. 1333 CAS 7 and 1600 CAS 8 are pretty common. You do not need ECC memory.

Cooling - Dynatron A5 or A6, CoolerMaster Hyper 212+, water. Choices are limited, and it all depends on your noise tolerance. You are also more limited with the newer SM boards because of the socket layout - CM 212+'s will not work on them. The CM 212+'s will need to be modded regardless if you use them - R-Type has a nice write-up somewhere on how to mod them.

Final investment - depends completely on what CPUs you get and how patient you are. It also depnds on whether or not you absolutely have to overclock - I'll have a Tyan board for sale shortly in the $500 range. The range would be roughly $1500 (low end 8-core MC chips, a used/open box Tyan board, and cheap 1333 CAS 9 memory) to well over $3000 (high end 12-core chips and nice memory) or well over that if you find ES chips.

 

[Kendrak]

Musky has spoken.

 

[musky]

Musky has spoken.

Is it sad that i could probably price out an entire 4p G34 system to within $25 without looking up a single component?

 

[Tobit]

Is it sad that i could probably price out an entire 4p G34 system to within $25 without looking up a single component?

No, you are a Dovahkiin... it's expected.

 

[dreadwing]

Musky,

Thank you so much for the info and taking the time to write this out. I will plan to go ahead with the 4P system and buy parts as I find them available. I will check on IRC as well when bidding on eBay. I have had good experiences with SuperMicro and will check out their site to get some board options.

Based on your response I will start picking up 6174's ram, and HAF 932 or X case as I find them at the right price. What PPD do you estimate with just 2 chips until I upgrade to the additional 2. This should make for quite a nifty folding box right?