• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

CPU/GPU Render Server/Workstation

M

mastermike707

Weaksauce
Joined
Jul 25, 2008
Messages
93
I am looking to build a server with the best possible CPU performance with at least a single GTX 680.
1) What will you be doing with this PC? Gaming? Photoshop? Web browsing? etc
Adobe After Effects Rendering in Classic engine (CPU only); Later on possibly 3D Ray Tracing (not priority)
2) What's your budget? Are tax and shipping included?
$3k-5k; yes.
3) Which country do you live in? If the U.S, please tell us the state and city if possible.
US; Mesa, Arizona
4) What exact parts do you need for that budget? CPU, RAM, case, etc. The word "Everything" is not a valid answer. Please list out all the parts you'll need.
CPU, RAM, Case, PS, Video Card, SSD, HDD, optical drive.
5) If reusing any parts, what parts will you be reusing? Please be especially specific about the power supply. List make and model.
nothing.
6) Will you be overclocking?
If possible, yes.
7) What is the max resolution of your monitor? What size is it?
24" 1080p.
8) When do you plan on building/buying the PC?
One to two weeks or less (depending on advice here)
9) What features do you need in a motherboard? RAID? Firewire? Crossfire or SLI support? USB 3.0? SATA 6Gb/s? eSATA? Onboard video (as a backup or main GPU)? UEFI? etc.
Nothing specific.
10) Do you already have a legit and reusable/transferable OS key/license? If yes, what OS? Is it 32bit or 64bit?
Yes, don't worry about OS.

I am completely open to multiple systems, single system is not a requirement. I want the best possible price for performance in CPU compute power. This also means I am definitely willing to overclock.
 
Just to be clear: Your planned software can take advantage of a multi-PC render farm?
 
Buy two of this PC and you should be ok:
$750 - 2 x AMD Opteron 6234 12 Core CPU
$420 - Asus KGPE-D16 Dual Socket G34 AMD SR5690 SSI EEB Motherboard
$110 - 2 x Kingston 8GB ECC Registered KVR1333D3Q8R9S/8G DDR3 1600 RAM
$102 - Crucial M4 CT128M4SSD2 2.5" 128GB SSD
$70 - Seagate Barracuda ST1000DM003 1TB 7200 RPM SATA 6.0Gb/s Hard Drive
$16 - Samsung SH-224BB SATA DVD Burner
$60 - 2 x Dynatron A1 Socket G34 Amd Opteron 6000 1u Heatsink CPU Cooler
$115 - Seasonic M12II 650 SS-650AM 650W Semi-Modular PSU
$105 - Norco RPC-450B 4U Case
---
$1748 shipped

So two of the above setup will come in around $3500 and gets you 48 cores. If you can increase your budget by another $200 or so, you can add get a total of 32GB of RAM per system:
$225 - Kingston 4 x 8GB ECC Registered DDR3 1600 RAM

EDIT: Actually you can drop the DVD burner from the second PC to save a little cash. Maybe the SSD as well from the second setup.
 
Last edited:
As an Amazon Associate, HardForum may earn from qualifying purchases.
 
Last edited:
I honestly wouldn't know the first thing to do with those case mods. They also look extremely time consuming to make. However, space shouldn't be a problem, so feel free to recommend multiple computers. I looked through the parts you suggested and was immediately surprised by your recommendation for an AMD processor. While that setup has a lot of cores aren't FP intensive work loads going to suffer a performance loss because of AMD's design? How well would some over clocked ivy bridge 6 cores perform? Also please include a single GTX 680 in the build for gpu acceleration of after effects.
 
mastermike707 said:
While that setup has a lot of cores aren't FP intensive work loads going to suffer a performance loss because of AMD's design?
Click to expand...
Possibly but the extra cores may/will make up for that.
mastermike707 said:
How well would some over clocked ivy bridge 6 cores perform?
Click to expand...
Compared to the 48 core setup I linked to above? I'm honestly not sure. AFAIK, no one has done a direct comparison of an overclocked Core i7 3930K CPU VS two AMD Opteron 6234 CPUs with the application you're talking about. And to thrown another wrench into things, I was also planning on recommending a 40 to 48 core AMD Vishera render farm as an alternative to that Opteron. It's a given that on a core for core and clock for clock basis, Intel will outperform their AMD counterparts. But I'm not entirely sure how the two Core 7 3930K (12 cores, 24 threads) would perform against a quad Opteron 6234 setup (48 cores) or five to six AMD FX-8120 setup (40 to 48 cores) in a render farm for Adobe After Effects.

Then again, now that you are requiring that the setup includes a GTX 680, that does change my proposed setup quite a bit.
mastermike707 said:
Also please include a single GTX 680 in the build for gpu acceleration of after effects.
Click to expand...
Hmm, gonna have to rethink the whole plan then. Bump up this thread in 24 hours if I don't reply.
 
mastermike707 said:
How well would some over clocked ivy bridge 6 cores perform?
Click to expand...

There are currently no Ivy Bridge CPUs with more than four physical cores (due to the fact that all current Ivy Bridge CPUs use Socket LGA 1155, which is limited to only quad-cores to begin with, and not Socket LGA 2011). You meant to say Sandy Bridge-E 6-cores? If so, then just one CPU costs as much as or more than two of the AMD Opteron CPUs that Danny recommended. What's more, the SB-E Xeons can't be overclocked at all by any means other than raising the BCLK strap (and even then, there is no guarantee that a particular SB-E Xeon can even handle overclocking at all). With two Sandy Bridge E Xeon CPUs with decently high clock speeds, expect to spend $2000, if not more, just for the CPUs (the cheaper 6-cores are clocked so slow that they might as well be Celerons).

Finally, if you're talking about overclocking, none of the server CPUs can handle overclocking much if at all. And the most popular overclocked CPUs are ill-suited for your purpose because whatever purpose you're using the system for requires ECC RAM, which those highly overclockable CPUs that you were talking about do not support at all.
 
After playing around on newegg, I think something like this build looks to be more what I was looking for:

Workstation:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=26095007
$1890

I would then buy 2 more systems like that with no GTX680 (-$570) a smaller SSD (-$60), no HDD (-$70), lower wattage PS (-$20?), and smaller case (-$60). This puts me at around $1110 per extra render system. I would buy 2 of these to go with the workstation.
I would overclock all the core-i7's to 5GHz.
So we end up around $4110 for the set. This gives me 18 sandy bridge E cores at 5GHz. If we assume AMD's cores are 60% as powerful as Intel's core (I have been unable to find many benchmarks for this application as well) then your AMD setup gives us roughly 2.4GHz*48 cores*.6 = 69.12 while my Intel setup gives roughly 18*5=90. I realize this result is highly dependent on the number used as an estimate, but I am having trouble doing anything more than theorycraft for this application.

Another option is two 6-core Xeons in a dual server board, but due to the increased pricing for RAM (~$100) and for motherboard (~$200) while decreased clock speed (5GHz to 2GHz) it seems like an unattractive option.

I am somewhat unfamiliar with AMD CPU model numbers and generations; is Vishera the most recent architecture?

By the way, E4g1e there is no requirement for ECC ram. Consumer systems are fine!

Please let me know your thoughts on the build and please suggest another system if you feel my assumptions are off base.
 
So your budget is now $4000+?

mastermike707 said:
After playing around on newegg, I think something like this build looks to be more what I was looking for:

Workstation:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=26095007
$1890

Please let me know your thoughts on the build and please suggest another system if you feel my assumptions are off base.
Click to expand...
I'm about to increase the cost of that setup but for good reasons though:

Motherboard: IMO, it kinda defeats the big advantage of X79 if you go with a socket 2011 mobo with only four RAM slots. Since it's socket 2011, you might as well go full bore and get the 8 DIMM motherboards so that you can get up to 64GB of RAM later on. My recommendation would be this mobo:
$310 - Asus Sabertooth X79 Intel X79 ATX Motherboard

RAM: That RAM is a tad overpriced. I recommend getting four of this similar performing but cheaper Corsair RAM instead:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820233366

SSD: The regular 840 isn't as good as its Samsung 830 predecessor or the Crucial m4 series either. As such I recommend the slightly cheaper Crucial M4 256GB:
$171 - Crucial M4 CT256M4SSD1 2.5" 256GB SSD

If you want something better, you're gonna have to step up to the Samsung 840 Pro SSD.

PSU: That PSU is a total piece of crap. It won't be able to provide its rated 750W under real-world conditions. Not to mention that you really don't need a 750W PSU for your setup. I recommend this PSU instead:
$94 - SeaSonic M12II 620 Bronze 620W Modular PSU

HSF: Considering that you're planning on overclocking the Core i7 3930K, I recommend getting a far better HSF considering the extra cores and therefore extra heat compared to Intel quad-cores. Not to mention that your stated goal is 5Ghz so that will require some of the best cooling you can get. I recommend getting the Noctua NH-D14 SE2011:
$90 - Noctua NH-D14 120mm and 140mm HSF
mastermike707 said:
I would then buy 2 more systems like that with no GTX680 (-$570) a smaller SSD (-$60), no HDD (-$70), lower wattage PS (-$20?), and smaller case (-$60). This puts me at around $1110 per extra render system. I would buy 2 of these to go with the workstation.
Click to expand...
Some notes:
1) You still need to buy a seperate GPU since the 3930K does not come with onboard video. With that said, video cards are relatively cheap:
$30 - PowerColor Go! Green AX5450 1GBK3-SH Radeon HD 5450 1GB PCI-E Video Card

2) Generally, the ~$60 SSDs aren't cost effective since their larger brothers are often not that much more and therefore offer a significantly higher bang for the buck value. As such I recommend the 128GB Crucial M4:
$110 - Crucial M4 CT128M4SSD2 2.5" 128GB SSD

3) I recommend this PSU:
$58 - Antec NEO ECO 520C 520W PSU

4) As for the case, I recommend checking out the Corsair 200R:
$60 - Corsair Carbide Series 200R ATX Case

mastermike707 said:
I would overclock all the core-i7's to 5GHz.
Click to expand...
5Ghz is too optimitic of a goal IMO. I'd go with the more conservative figure of 4.5GHz.

mastermike707 said:
So we end up around $4110 for the set. This gives me 18 sandy bridge E cores at 5GHz. If we assume AMD's cores are 60% as powerful as Intel's core (I have been unable to find many benchmarks for this application as well) then your AMD setup gives us roughly 2.4GHz*48 cores*.6 = 69.12 while my Intel setup gives roughly 18*5=90. I realize this result is highly dependent on the number used as an estimate, but I am having trouble doing anything more than theorycraft for this application.
Click to expand...
The problem is that you don't add up clock speed like that since the CPUs won't be accessed or used in a way where that can happen. Remember that we're talking about parallel computing here. In order to do a correct comparison, we need actual updated performance figures which are unfortunately severly lacking at this point in time.

mastermike707 said:
I am somewhat unfamiliar with AMD CPU model numbers and generations; is Vishera the most recent architecture?
Click to expand...
Correct.
mastermike707 said:
(I have been unable to find many benchmarks for this application as well)
Click to expand...
Unfortunately we're gonna have to run with year old information in this case since there aren't any updated articles, reviews, benchmarks, or even forum threads regarding AMD's Vishera CPUs and the application you're talking about. So stick with Intel in this case. Seems to be what everyone, a year ago, is aiming for when it comes to render systems for Adobe.

With that said and considering that I spent quite a bit of time setting up this list, if I had found info supporting AMD's Vishera, I would have recommended this AMD Render Farm for your original $3500 budget:
Master (x1):
$210 - AMD FX-8350 CPU
$130 - ASUS M5A99X EVO AM3+ AMD 990X AM3+ AMD 990X ATX Motherboard
$120 - 2 x Corsair CMV16GX3M2A1333C9 2 x 8GB DDR3 1333 RAM
$470 - eVGA SuperClocked+ 02G-P4-2684-KR GeForce GTX 680 2GB PCI-E Video Card
$110 - Crucial M4 CT128M4SSD2 2.5" 128GB SSD
$70 - Seagate Barracuda ST1000DM003 1TB 7200 RPM SATA 6.0Gb/s Hard Drive
$90 - Corsair 650TX V2 650W PSU
$60 - Corsair Carbide Series 200R ATX Case
$25 - Cooler Master Hyper 212 Plus HSF for LGA 1366 and LGA 1156
----
Total: $1285 shipped


Slave (x 4):
$180 - AMD FX-8320 CPU
$85 - ASRock 970 EXTREME3 AM3+ AMD 970 ATX Motherboard
$60 - Corsair CMV16GX3M2A1333C9 2 x 8GB DDR3 1333 RAM
$30 - PowerColor Go! Green AX5450 1GBK3-SH Radeon HD 5450 1GB PCI-E Video Card
$70 - Seagate Barracuda ST1000DM003 1TB 7200 RPM SATA 6.0Gb/s Hard Drive
$45 - Rosewill REDBONE U3 Red
$45 - Corsair 430CX V2 430W PSU
$25 - Cooler Master Hyper 212 Plus HSF for LGA 1366 and LGA 1156
----
Total: $540 shipped

Network gear:
$54 - Netgear GS108 8 Port switch

-----------
Total: $3499 shipped

With the above setup, you would have gotten 40 cores of AMD's new Vishera CPUs. Each setup would have been overclockable but the slave systems would be limited to a mere 4Ghz overclock due to the motherboard's inability to handle more than that. However the master system would have been able to OC to 4.5Ghz..

Unfortunately all of that is moot since there's little to no widely available info showing how well the Vishera CPUs would do in this kinda of situation.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
After reading through your recommendations and reading some more about intel and amd general cpu performance, it appears that Vishera's single threaded performance can be lacking, while I realize that the whole point of building a "render farm" is to take advantage of parallelism, I would not be surprised if after effects workload looks something like the visual studio workload seen on this page:
http://www.anandtech.com/show/6396/the-vishera-review-amd-fx8350-fx8320-fx6300-and-fx4300-tested/3

I looked through your amd systems and decided that an ivy bridge slave build around $500 to $600 would serve better than an expensive sandy bridge E slave. Here is what I came up with (based on your AMD slave):

Slave:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=21119452
Cost is rougly $615 (ram has $10 promo and PS has $5 promo)

I also reworked my master based on your suggestions. I decided to go with the Corsair H60 for CPU cooler instead of the Noctura you recommended because of the lower price and similar performance. I didn't opt for the board with eight slots as 32GB should be overkill and any more would just be double overkill.
Master:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=26095007
Cost ends up at $1900

For this system I am paying a premium of approximately $475 compared to the Ivy Bridge slave (Mobo+CPU) for the extra two cores and hyperthreading. That's almost the cost of another whole Ivy Bridge slave; do you think I would be better off basing the workstation off Ivy Bridge?

By the way the budget is $3-5k or $3000 to $5000.


Edit: One more thing, the AMD setups, while they are overclockable, they are already close to maximum clocks while the Intel CPUs in these builds should have much more headroom (I expect 4.4GHz+ from stock 3.8GHz). So clock for clock the Intels are going to perform very well.
 
Last edited:
If you're not going to use the extra DIMMs, then yeah just go for an Ivy Bridge setup. Have the master use the 3770K so that you can get some added performance from the extra threads.

As for the slave systems, that mobo doesn't OC that well long term IIRC. I'd spend the extra cash for a better overclocking mobo:
$140 - Asus P8Z77-V LK Intel Z77 ATX Motherboard
 
Danny Bui said:
As for the slave systems, that mobo doesn't OC that well long term IIRC. I'd spend the extra cash for a better overclocking mobo:
$140 - Asus P8Z77-V LK Intel Z77 ATX Motherboard
Click to expand...

Where did you hear that? All I've seen are glowing reviews about that board and ones like it. That Asus board looks like it has less OC hardware and has less features than that AsRock.
A review on the Extreme4 shows good OC performance: http://www.overclockersclub.com/reviews/asrock_z77_extreme_4/6.htm
 
mastermike707 said:
Where did you hear that? All I've seen are glowing reviews about that board and ones like it. That Asus board looks like it has less OC hardware and has less features than that AsRock.
A review on the Extreme4 shows good OC performance: http://www.overclockersclub.com/reviews/asrock_z77_extreme_4/6.htm
Click to expand...

Ahh this is what I was referring to:
http://hardforum.com/showpost.php?p=1039278834&postcount=16

Substandard VRMs/Mosfets according to Mercuri, one of the posters here on the forums.

And that's just the Extreme4. As shown by previous HardOCP reviews of AsRock mobos, the quality difference between two mobos of the same line can be vastly different.
 
Thanks so much for that link, I can look through all of the boards to find a good OCer. Will look through the list and post an update later.
 
Danny Bui said:
How willing are you to do a case mod like these?:
http://coyoteridgeobservatory.blogspot.com/2012/03/clusterf-24-core-linux-cluster-in.html
Click to expand...

The ClusterF case mod was time consuming but it was a great project. I really enjoy making things.

Unfortunately the project it was built for didn't fully materialize so I'm going to have to sell it either whole or break it up and re-case the nodes. Before the project was canceled the slave nodes were upgraded with an AMD 5450 video card and a 250GB HD.
 
Sorry, I hate to necro this thread, but I am getting ready to build this again. Buyer wanted to wait until Haswell, but then it was impacting his workflow because of slow rendering times. So now here is my planned parts:

Workstation $1600:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=22359445
+ SanDisk Extreme 240GB for $170

+NZXT Phantom White

Slave $800:
http://secure.newegg.com/WishList/PublicWishDetail.aspx?WishListNumber=21119452
+ SanDisk Extreme 120GB for $100

Please note that now, Cinema 4D performance is also a target as it is taking a significant amount of time.

Looking mainly to improve cost-effectiveness and towards motherboard/power supply choices.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
Drop the RAM: That AMD RAM will not work with Intel CPUs. Not sure what AMD did exactly but I've never seen anyone successfully use AMD branded RAM with an Intel setup. Go with this RAM instead:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820145357

Drop the SSDs: They're an older set of SSDs and there are faster and of higher quality SSDs. Go for this SSD for the slaves:
http://www.amazon.com/dp/B009NHAEXE/?tag=extension-kb-20

For the main, go for the higher quality Samsung 840 Pro 256GB SSD. The motherboards are fine. The Slave PSUs are fine as well. I'm hesitant in recommending that PC P&C PSU considering that it's owned by OCZ and therefore may have gotten OCZ's rather lackluster customer support. I recommend this higher quality Seasonic instead:
http://www.newegg.com/Product/Product.aspx?Item=N82E16817151118
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
I've had really good luck with the Corsair CX430. I have them in 8 systems. Half of them render full time. I'm also using the 212 cooler and the CPU barely gets warm.
 
You must log in or register to reply here.
Back
Top