Editing/Engineering Build: Cosmos II - 3960X Sandy Bridge-E Platform

[SonataSys]

This workstation is designed for heavy post-production editing and software engineering, with some occasional gaming.

This preview video summarizes the chosen components and build design.

Cooler Master Cosmos II Tower
Enermax MaxRevo 1350W Power Supply
MSI Big Bang-XPower II Intel LGA 2011 X79 Mainboard
Intel Core i7-3960X Extreme Edition Sandy Bridge-E 3.3GHz LGA 2011 CPU
3-Way SLI EVGA GTX TITAN SuperClocked Signature 06G-P4-2793-KR
Corsair Dominator GT 32GB (4 x 8GB) DDR3 1866Mhz (PC3 15000) 1.5v CAS 9-10-9-27 SDRAM
RAID-0 Boot Set: Two Samsung 830 Series 512GB Solid State Drive Array (1TB)
RAID-0 Data Set A: Two Western Digital RE4 2TB SATA 3.0Gb/s 3.5" 7200 RPM Disk Array (4TB)
RAID-0 Data Set B: Two Western Digital RE4 2TB SATA 3.0Gb/s 3.5" 7200 RPM Disk Array (4TB)
RAID-5 Archive Set: Four Western Digital Green 2TB 5400 RPM Drobo External Disk Array (6TB)
Noctua NH-D14 SE2011 6-Heatpipe CPU Heatsink
Two Pioneer BDR-207DBKS Blu-ray burners

Note: The original Radeon HD 7970 CrossFireX configuration is being replaced by EVGA GeForce GTX Titan video cards in a 3-way SLI configuration.


The following operating systems are used to develop and test software on various platforms, native and virtual machines:

Navtive - Windows 7 Professional SP1 (64-bit)
Oracle VM VirtualBox - CentOS Linux
Oracle VM VirtualBox - Solaris
Oracle VM VirtualBox - Ubuntu Linux
Oracle VM VirtualBox - Mac OS X Server
Oracle VM VirtualBox - Windows XP


The following engineering software is used for development work:

Storage: Oracle, MongoDB and Cassandra databases
Back-end: WebLogic, JBoss and Tomcat containers, Apache, Spring/CXF web services
Front-end: HTML5, GWT, JSF, Grails and Struts
Languages: Java, JavaScript, Groovy, Gradle
IDE: IntelliJ IDEA, Eclipse, DreamWeaver, AppCode


The following editing software is used for post-production work:

Adobe Master Collection CS6
Nero Ultra Edition
Handbrake


Here are the boxing photographs:

 

[SonataSys]

...

 

[Druneau]

What kind of engineering software will you be running on this impressive hardware? (didn't see them mentioned in the video).

Very nice parts and pictures.

Edit: just re-read, you specify software engineering. I guess those were covered in the video!

 

[SonataSys]

What kind of engineering software will you be running

Since there's never enough space to list everything in a video, here's a comprehensive list of engineering software in use. A subset of this is running in the background at any one time, including multiple, rather large server instances with long up-times (hopefully)!

With this new workstation, I fully expect to support both my engineering and post-production work streams at the same time, unlike before. To achieve this requires a lot of memory to keep everything resident and avoid swap delays. Adobe's CS6 suite supports the GTX Titan and all 6GB of VRAM provided. Combined with a 1TB RAID0 SSD boot/scratch set and two 4TB RAID0 disk sets, I expect excellent rendering/encoding results with significantly reduced wait times...


Servers:

Apache
Tomcat
WebLogic
JBoss
GlassFish
Oracle 11g
MySQL
MongoDB
Cassandra
Lucene
Solr


Client-side:

HTML 5
JavaScript
Backbone
JQuery


Server-side:

Java
GWT
JSF
Spring
Struts
Seam
REST Web Services (CXF)
SOAP Web Services
Groovy
Gradle


Development Toolset:

IntelliJ IDEA
Eclipse
Adobe Dreamweaver
Adobe Flash Builder
Adobe Flash Professional
TeamCity

 

[XXshadowwaterXX]

what is your reasoning for buying gaming cards instead of workstation cards?

 

[SonataSys]

what is your reasoning for buying gaming cards instead of workstation cards?

My 3D rendering tasks are rather limited in size/complexity, mainly involving short sequences in Adobe AfterEffects and touch-ups using Photoshop, not really into AutoCAD or Maya projects. Most of my post-production work involves editing video footage and cleaning up bad sound. My workflow bottleneck has never been related to GPU performance: it has always been inadequate RAM, slow disks, a congested bus and pegged CPU cores during endless encoding runs. I hope to resolve those four primary source issues with this build.

Actually, the performance gap really isn't that big between higher-end gaming cards with lots of VRAM and their workstation counterparts. Recent gaming cards with lots of VRAM can even outperform workstation adapters in some of the simpler rendering modes, but fall behind again dealing with heavier, specialty 3D modes. For example, even in AutoCAD an average gaming card like the Radeon HD5870 can render lighter wireframe and flat shaded models just as well as most workstation cards. Another example is 3dx Max rendering, where there's very little difference between workstation and high-end gaming cards, regardless of manufacturer or model. Some of the heaviest rendering modes, including 3ds Max, have architectures that bind the graphics pipeline heavily to the CPU, which prevents them from taking full advantage of any GPU, workstation or gaming.

Besides, I also enjoy playing a few games now and then-- at 2560 x 1600 with high frame rates!

;->

Regardless, I would certainly advise a nice workstation adapter (or two) for those who spend the majority of their life rendering large 3D models in Maya or AutoCAD...

 

[XXshadowwaterXX]

i was just asking because we have master came and things of that nature at the school i go to and some of the parts we machine with that software looks terrible when we use one of my friends very high end gaming rigs with two 6990's but when we use the computers at school with FirePro V5900's it runs like a dream. but then again you arent useing cad software so i could so how that wouldnt be useful

 

[SonataSys]

i was just asking because we have master came and things of that nature at the school i go to and some of the parts we machine with that software looks terrible when we use one of my friends very high end gaming rigs with two 6990's but when we use the computers at school with FirePro V5900's it runs like a dream. but then again you arent useing cad software so i could so how that wouldnt be useful

It's all about optimizing whatever you spend the most time doing. For those who spend the majority of their time working with large 3D models in packages like Maya or AutoCAD, then a nice workstation card is worth the extra cost. It's not a good idea, however, if you spend most of your time doing something else, like playing BF3 or Crysis. For example, I have a Quadro card in my workstation at work, and it does render 3D content extremely well but it delivers sub-par frame rates in most games compared to higher-end gaming cards.

For those who spend about half their time in both worlds and cannot afford two systems dedicated to each work stream, I would probably go with a workstation card and live with the slower frame rates outside of the 3D modeling arena.

Another option I have not personally explored but have heard works: download and install "unlocked" workstation video drivers for your gaming card, then switch back to the default gaming driver whenever you're done modeling and want faster frame rates while gaming. The hardware itself is usually very similar in both workstation and higher-end gaming cards, with the biggest performance delta being the driver software installed. It's at least something to consider trying before springing extra bucks for a workstation adapter...

 

[SonataSys]

The stock case fans are history, except for the 200mm front intake fan. For slightly better performance and less noise, all case fans (except the rear exhaust) are from Enermax using pull-through silicon mounts to reduce vibration:

• Enermax T.B. Silence 120mm x 25mm "Twister Bearing" 3 Manual Speed Fan (UCTB12A)
• Be quiet! Shadow Wings 140mm x 25mm PWM rear case fan (BL027)
• Nexus Silicon Fan Mounts (SFM-1000)

Most storage components will be isolated in the lower chamber, so the middle cage is just in the way, disrupting air flow into the main chamber. Therefore, that cage is also going away.

 

[SonataSys]

The Enermax T.B. Silence and Be Quiet! Shadow Wings fans are now installed. The silicon mounts are very easy to work with. The Cosmos II has plenty of room on top to route fan cables directly to the front controller, with no need to expose them to either side of the case.

Installing the SSD and disk drive RAID0 arrays also went quickly and without mishap. The drive caddies in each storage unit operate smoothly, and both units can be removed from the lower storage area with ease, no tools needed.

I consider this lower secondary storage area the heart of the system, because I/O throughput has the single biggest impact on my editing productivity. Having insufficient or slow storage/scratch space, or inadequate memory capacity dramatically impedes the editing work stream.

 

[Druneau]

120mm fans look like 80's in that case! Very nice case.

 

[SonataSys]

120mm fans look like 80's in that case! Very nice case.

The Cosmos II is heavy and does take up a lot of space, but it provides great clearances throughout the case. It is very easy to work with and nothing feels cramped or crammed in...

 

[SonataSys]

How do I describe MSI's outrageously named, outrageously looking Big Bang-XPower II mainboard?

Despite its overblown design, the board is actually very well thought out otherwise. Connection points and headers are in the right places. The over-clocking features and numerous SATA ports are wonderful. The board's larger form factor provides good clearances, especially between the seven expansion slots.

Regardless, I freely admit the whole "military class" theme is sort of creepy, campy, nerdy, etc.

What can I say?

I can only confess being a sucker for Gatling guns! I don't even own a gun, but I have always been fascinated by them:

 

[SonataSys]

The Cosmos II is so spacious, even Noctua's enormous NH-D14 almost appears normal in size. Pretty much everything put into this case recedes and appears smaller than it really is.

Clearances are considerable between the heatpipe towers and the rest of the case, especially the front and rear, so I decided to add a third BeQuiet! 120mm Shadow Wings PWM fan on the back side of the tower closest to rear exhaust fan. Noctua (located in Austria) quickly processed my request for extra fan mounting hardware, and shipped it Priority Mail, no charge. I am very impressed!

Unfortunately, clearances between the towers and the Corsair Dominator DRAM modules was poor, and I was forced to remove the heat sinks on top of the DRAM modules.

The BeQuiet! fans replace the stock Noctua hardware. Essentially, at full-RPM, the Noctua fans are slightly better performers but also slightly louder. I decided to give up a little CFM to gain a little reduction in decibel level. If performance turns out to be an issue, I can always bring back one or both Noctua fans.

 

[SonataSys]

The Enermax power supply is rather small given its modular design and power rating. The clip attachment above the power chord adapter is a nice touch, safeguarding inadvertent pulling of the plug. The roomy lower chamber isolates the power and storage components; this hardware should remain cool with three dedicated 120mm intake fans, along with adequate passive exhaust on the other side and rear of the case.

Two Radeon 7970s is probably overkill, especially since I don't game much, but when I do I prefer maximum resolution at 2560 x 1600 whenever possible. This pair from XFX should do the job right. I selected XFX because I wanted slightly lower noise and cooling numbers compared to stock hardware. I didn't want to risk installing after-market cooling myself, so I decided to go with XFX and its Double Dissipation and Ghost air cooling approach.

When under heavy gaming load, these cards will be the source of significant heat. To help keep them relatively cool, the huge Cosmos II and expansive MSI mainboard are key factors, allowing plenty of clearance for air flow around/between the CrossFireX configuration. Two rear exhaust slots remain open on either side of both video cards. Also, two 120mm fans mounted on the side case door intakes cool air directly into the GPU space.

This solution is probably the most that can be done without resorting to liquid. Essentially, the entire build is an experiment to determine what kind of performance can be attained with a purely air-cooled configuration.

 

[SonataSys]

The Cooler Master Cosmos II Achilles heel is the fan controller, which causes many fan models to tick, sometimes with loud clicking noises whenever fan speeds are changed. I'm beginning to think people who simply switched to a third-party controller took the right path. However, I decided to take Cooler Master's free replacement offer, and after a three-week wait an entirely new head/controller assembly arrived.

As a back-up measure, I also decided to go with Enermax manual-control case fans, which override and regulate fan speed locally with a small adjustment switch-box. If the Enermax fans misbehave with this Cooler Master controller, the plan is to keep the Cosmos II controller set on high for maximum power, and then manually regulate each fan down based on fan role/location/performance requirements.

It is a shame the case's fan controller is so problematic, given its otherwise excellent design. I probably should have just went with an entirely different fan controller...

Before totally removing the original Cosmos II head/fan controller assembly, I decided to wire up all eight Enermax TB Silence (manual speed control model UCTB12A) 120mm case fans to determine how the manually controlled hardware performs with the problematic fan controller. Several fan makes/models emit clicking or humming noise as speed adjustments are made with the controller. To my surprise, the Enermax fans worked perfectly with the original fan controller. No issues whatsoever moving from low, medium or high speed settings.

I soon realized the replacement fan controller Cooler Master sent me recently may not be necessary, given the good results encountered with the original hardware. However, I decided to proceed with moving to the latest fan controller since it should provide even better fan support, and the fan/led and mainboard cabling is now entirely black and that fits my color preference as well. I was a bit concerned The God of Irony would strike, and I would discover issues with the latest fan controller and the Enermax fans.

Thankfully, I can report both the original and new fan controller work extremely well with Enermax TB Silence manual-control fans.

 

[SonataSys]

I am thankful the build has progressed steadily and smoothly. The workstation is now online and kicking. Every chosen component arrived safely and performed as expected, with no returns, RMAs and zero regrets. Even the problematic fan controller in the Cosmos II turned out to be a non-issue with the Enermax case fans. Moreover, the workstation just survived an overnight burn-in at stock speeds, although technically the XFX Black Edition 7970 cards do ship with mild GPU and memory over-clocks.

So, how old school am I? Well, how about analog temperature gauges? That's right:She wears glasses!

My overall impression is astonishment. The performance delta between my prior workstation and this behemoth is massive. Windows 7 Professional boots in thirteen seconds with stock settings. The Samsung 830 striped set simply rips through all application and scratch storage requests.

Launching Photoshop CS6-64 takes place in two eye-blinks, with every window and floating panel appearing in less than two seconds. The Adobe OpenCL graphics engine manipulates dozens of massive RAW images in memory with ease, instantly rendering 3D effects, rotations, etc.

Running six cores with 32GB of RAM enables instantaneous switching between several Adobe editing and affects packages (Premiere, Encore, After Effects, etc.) that are all configured at maximum performance setting levels, even while working with very large video and photo projects. Performance within each application is superb. The Western Digital RE4 striped arrays transfer several hundred GBs of HD footage at nearly twice the throughput of my prior rig, providing uninterrupted editing of two HD timelines, each assigned a dedicated RAID0 set. Final encoding times should drop dramatically using this workstation!

Given the traditional air-cooled approach with high-performance components, idle state temperatures are amazingly good across the board. The room temperature is 22C. Inside the Cosmos II, the main upper chamber is one degree warmer at 23C. The lower storage chamber is a half-degree higher at 22.5C. The XFX Double Dissipation and Ghost air cooling technologies idle both video cards at 30C with a standard 20% fan level, rather astonishing considering both cards ship with over-clocked processors (1000 MHz) and VRAM (1425 MHz). Equally impressive, perhaps even more so, the Noctua NH-D14 heat-pipe and BeQuiet! PWM fan configuration do an amazing job quietly cooling the 3960X, with all six cores idling between 28C and 37C, an average core temperature of exactly 31.8C. Although the BeQuiet! fans provide a 1-db decrease in fan noise, the trade-off is they push about 10% less air compared to the Noctua fans. I suspect if I switched back to the stock Noctua fans the average core temperature would drop by about 10 percent to below 30C.

 

[SonataSys]

The following benchmarks were taken with the following overclocks:

• CPU from 3.3 to 4.0 GHz
• Dominator SDRAM from 1600 to 1866MHz

The XFX 7970s in crossfire remain at 1000MHz for the GPUs and 1425MHz for each board's 3GB of VRAM.

At idle, the average CPU core temperature rose from 31.8C to 38.8C, a 7C degree increase. The average idle temperature rose 22% after the CPU clock frequency was adjusted up 20%, a moderate and entirely acceptable rise for the increased clock rate.

Other good indicators that the air cooled configuration is working well include:

• The XFX crossfire idle state temperatures remains stable at 30C.
• The upper case chamber temperature barely increased from 23C to 23.8C.
• The lower case chamber temperature did not change.

The average core temperature rose to 65C while running Prime95, a figure well below (by 26 degrees) the 91C throttle-down threshold. This represents good performance from a purely air-cooled solution.

The Samsung 830 RAID0 SSD array rocketed through the HD benchmark, with an average transfer rate over 1000MB/s. The two Western Digital RE4 RAID0 disk arrays both performed four times slower than the SSD array, but each provides four times the capacity. They provide a sustained average transfer rate well above 200MB/s, which means multiple video/audio timelines can be edited simultaneously without suffering I/O bottlenecks, painful interruptions and annoying stuttering while editing.

The performance numbers drop considerably for the external (Drobo) RAID5 array, but this trusted archive is used for longer-term storage via overnight back-ups, no real-time usage.

 

[SonataSys]

3960X CPU @ 4GHz
R7970 GPU @ 1050MHz
R7970 VRAM @ 1500MHz

 

[SonataSys]

Despite recent AMD driver improvements, the XFX Radeon 7970s in CrossFireX are rather loud under heavy load, and a bit of stutter remains in some games. For example, Battlefield 3 multi-player (64) pegs both GPUs at maximum quality rendering at 2560x1600. Simply put, the minimum frame rates at this level just aren't good enough. Overclocking the cards simply generates even more heat, more stutter and produces even higher noise levels.

Moreover, it is taking Adobe way too long to fully support Radeon hardware in Premiere Pro and other CS6 applications. The lack of hardware video editing support for Radeon 7970 is ridiculous, and I'm just tired of waiting.

Therefore, the 7970 hardware is being replaced with three EVGA GeForece GTX Titan video cards, which should resolve these issues and leave a good amount of headroom for even higher multi-planel resolutions and next-generation gaming engines coming down the pike:

 

[fullvietFX]

Why such a mild CPU overclock?

 

[SonataSys]

Why such a mild CPU overclock?

Essentially, the 3960X handles everything just fine without any overclock. The only reason I'm running moderately higher than stock is because it's so trivial to enable the OC Genie feature on the MSI Big Bang XPower II mainboard.

Rendering nearly an hour of HD video footage in twelve minutes flat is the most taxing task I perform, and it's the only thing that even approaches pegging all twelve 3960X threads.

Everything else is simply a breeze at 4GHz.

The only other task that pushes the system in any serious/sustained way is playing BF3 with 64 players at 2560x1600 and high quality. The 3960X is only 36% utilized under these conditions, so it's not really even breaking a sweat. The only bottleneck in the system are the cross-fired 7970s-- they are pegged!

Perhaps some day, whenever future applications and/or games force me to, I will actually have to seriously overclock the CPU and GPUs. Whenever that day arrives, I will probably opt to re-configure the entire existing setup using water instead of air prior to pushing everything higher up near the melting point.

 

[SonataSys]

All three GTX TITAN cards have arrived, shipped from two different sources: NewEgg and Amazon. NewEgg has a single-unit-per-household limitation but Amazon doesn't, so one card came from NewEgg and the other two were sourced from Amazon.

I apologize for not removing the cards from their anti-static bags, but I'm not going to do that until right before I actually install the hardware this weekend.

My initial impression is just how heavy the card feels, very solid. The packaging by EVGA is top-notch, with the card surrounded on all sides by thick, custom-fitting foam. There is no doubt the boxing will remain in storage just in case this hardware needs to be transported again.

The warranty registration process with EVGA is somewhat involved, but the website is nicely done and everything worked smoothly. I decided to extend the warranty to 5 years and include advanced 2nd-day RMA service to ensure prompt cross-shipping and minimize down-time.

 

[DraginDime]

That's just awesome. I'd love to have a couple Titans, so if you don't like those 3, you can just send 2 of them to me.

Amazing looking rig, can't wait to see those GPU's in there. Keep up the good work

 

[SonataSys]

That's just awesome. I'd love to have a couple Titans, so if you don't like those 3, you can just send 2 of them to me.

Amazing looking rig, can't wait to see those GPU's in there. Keep up the good work

Sure, and do you need the spiffy EVGA t-shirt (XL) thrown in as well?

;->

The CrossFireX/SLI migration work takes place this weekend . . . and I will update the thread with more content.

 

[SonataSys]

Thankfully, the migration away from CrossFireX to 3-Way SLI went smoothly. After uninstalling the AMD Catalyst driver and shutting down, the former XFX Double D Black Edition Radeon 7970 cards were removed.

The only complicating factor when installing the three TITAN cards was providing a third PCIE power cable with 6 and 8-pin connectors from the Enermax PSU. The MSI Big Bang XPower II mainborad and Cosmos II case are both very large, offering excellent clearance between components. Air flow is also good, especially in the GPU area. There is plenty of room to work inside the case and it does not look overly crowded even after installing all three TITAN cards.

I am hoping this configuration will provide good thermal performance. Benchmarks will be taken next after the initial NVIDIA driver setup and EVGA Precision configuration process is completed...

 

[SonataSys]

The new TITAN 3-way SLI setup has lowered temperatures in nearly every way compared to the former HD7970 CrossFireX configuration in terms of overall Cosmos case, CPU and GPU temperatures. The primary reason for this drop is a major difference in GPU cooling design: The TITAN design pushes most heat out the back of the case, but the XFX Double D/Ghost design exhausts most of the heat into the upper main chamber.

Two upper/lower Cosmos 2 chambers are monitored for temperature. The main upper case chamber temperature is 22.4C at idle, 1.4C above the ambient room level. This is .5C lower than the CrossFireX setup, which was 1.9C above ambient. The smaller, lower chamber temperature is 21.5C at idle, .5C above ambient; the CrossFireX setup produced exactly the same performance. Under 100% folding load (3960X CPU and all three GPUs) the upper chamber temperature is 23.5C, 2.5C above ambient. This is 4.2C lower than the CrossFireX setup, which was 6.7C above ambient.

At idle, the 3960X CPU averages 38.2C across all 6 cores with the new TITAN setup, 52.8C away from Tj Max. This is .4C lower than the former CrossFireX setup, where the CPU idled at 38.6C with a Tj Max delta of 52.4C. Under 100% folding load, the new TITAN setup averages 64C across all 6 cores, 27C away from Tj Max. This is 5C lower than the CrossFireX setup, where the CPU was fully loaded at 69C with a Tj Max delta of 22C.

At idle, the TITAN setup averages 30.3C across all three video cards. This is .3C higher than the CrossFire setup, which averages 30C between the two HD 7970s. This area was the only measurement which favored the CrossFireX configuration. The slight rise in average idle temperature can be accounted for by the slightly higher idle clock rate (324MHz versus 300MHz) and more restricted card clearances in the 3-way SLI configuration.

Under 100% folding load, the TITAN setup average 59.3C across all three cards clocking at an average boost rate of 1040.6MHz . This is 6.2C lower than the CrossFireX setup under the same load, averaging 65.5C between both HD7970s while clocking at 1000MHz. This is impressive given the tighter 3-way card configuration and the slightly higher TITAN clock rate.



TITAN 3-WAY SLI SETUP 3960X @ 4GHZ


Case Temperatures

• Room Ambient: 21C
• Idle Upper Chamber: 22.4C
• Idle Lower Chamber: 21.5C
• 100% Load Upper Chamber: 23.5C
• 100% Load Lower Chamber: 22.0C

CPU Idle/Folding@100% Temperatures

• Freq: 4GHz
• VID: 1.3260v
• Tj Max: 91C
• Idle: 38.2C
• Idle Power: 69W
• Idle Tj Max Delta: 52.8C
• 100% Load Average: 64C
• Load Power: 149W
• Load Tj Max Delta: 27C

Idle GPU Temperatures

• TITAN-1
• Idle: 33C
• Idle Clock: 324MHZ
• Idle Voltage: 887MV
• FAN: 34%
• TITAN-2
• Idle: 32C
• Idle Clock: 324MHZ
• Idle Voltage: 875MV
• FAN: 32%
• TITAN-3
• Idle: 26C
• Idle Clock: 324MHZ
• Idle Voltage: 875MV
• FAN: 30%
• TITAN-AVERAGE
• Idle: 30.3C
• Idle Clock: 324MHZ
• Idle Voltage: 879MV

GPU Folding@100% Load Temperatures

• TITAN-1
• Load: 59C
• Load Clock: 1019MHz
• Load Voltage: 1161MV
• TITAN-2
• Load: 64C
• Load Clock: 1058MHZ
• Load Voltage: 1161MV
• TITAN-3
• Load: 55C
• Load Clock: 1045MHZ
• Load Voltage: 1161MV
• TITAN-AVERAGE
• Load: 59.3C
• Load Clock: 1040.6MHZ
• Load Voltage: 1161MV

HD7970 CROSSFIREX SETUP 3960X @ 4GHZ


Case Temperatures

• Room Ambient: 22C
• Idle Upper Chamber: 23.9C
• Idle Lower Chamber: 22.5C
• 100% Load Upper Chamber: 28.7C
• 100% Load Lower Chamber: 23.2C

CPU Idle/Folding@100% Temperatures

• Freq: 4GHz
• VID: 1.3260v
• Tj Max: 91C
• Idle Power: 70W
• Idle: 38.6C
• Idle Tj Max Delta: 52.4C
• Load Average: 69C
• Load Power: 150W
• Load Tj Max Delta: 22C

Idle GPU Temperatures

• R7970-1
• Idle: 30C
• Idle Clock: 300MHZ
• R7970-2
• Idle: 30C
• Idle Clock: 300MHZ
• R7970-AVERAGE
• Idle: 30C
• Idle Clock: 300MHZ

GPU Folding@100% Load Temperatures

• HD7970-1
• Load: 67C
• Load Clock: 1000MHz
• HD7970-2
• Load: 64C
• Load Clock: 1000MHz
• HD7970-AVERAGE
• Load: 65.5C
• Load Clock: 1000MHz

Next, 3D benchmarks and gaming performance will be measured...

 

[SonataSys]

The TITAN 3-way SLI configuration scored a 3DMark 11 result of P25177. That's 9,495 points higher than the former CrossFireX setup's score (P15682).

Link Test: http://www.3dmark.com/3dm11/6491299

Score P25177
Graphics: 38241
Physics: 13382
Combined: 11125
Graphics Test 1 167.4 FPS
Graphics Test 2 198.9 FPS
Graphics Test 3 252.9 FPS
Graphics Test 4 111.2 FPS
Physics Test 42.5 FPS
Combined Test 51.7 FPS

 

[DraginDime]

Those Titan's look good in there. Nice 3Dmark numbers too. I always prefer reference coolers, even though other solutions run cooler. I just like pushing that hot air outside of my case. It works well to warm up my room in the winter also.

Oh and you can keep the shirt

 

[SonataSys]

Those Titan's look good in there. Nice 3Dmark numbers too. I always prefer reference coolers, even though other solutions run cooler. I just like pushing that hot air outside of my case. It works well to warm up my room in the winter also.

Oh and you can keep the shirt

I gave the EVGA shirts, mouse pads and stickers to my nephews, but they wanted the GPUs instead, of course! I won't be able to get near this system whenever they come over to visit...



The switch to TITAN/SLI has been painless and has far exceeded my expectations in terms of power, temperatures, noise levels and performance-- especially when the system is under heavy load, whenever playing Crysis 3 or BF3 multi-player at hi-res.

 

[SonataSys]

I use BONIC to crunch the SETI@Home project. The system receives 58,000 credits per day now.

Here's an interesting observation: Each TITAN crunches 1% of a cuda-fermi work unit every second, so three work units are finished off every minute-and-a-half. When it comes to crunching a work unit, a single TITAN Kepler GPU is about fifteen-times faster than a single 3960X thread. However, the 3960X has 12 threads munching on 12 work units at the same time...

Here is the BONIC start-up log:

4/29/2013 7:40:43 AM | | Starting BOINC client version 7.0.64 for windows_x86_64
4/29/2013 7:40:43 AM | | log flags: file_xfer, sched_ops, task
4/29/2013 7:40:43 AM | | Libraries: libcurl/7.25.0 OpenSSL/1.0.1 zlib/1.2.6
4/29/2013 7:40:43 AM | | Data directory: C:\ProgramData\BOINC
4/29/2013 7:40:43 AM | | Processor: 12 GenuineIntel Intel(R) Core(TM) i7-3960X CPU @ 3.30GHz [Family 6 Model 45 Stepping 7]
4/29/2013 7:40:43 AM | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 sse4_2 popcnt aes syscall nx lm vmx tm2 dca pbe
4/29/2013 7:40:43 AM | | OS: Microsoft Windows 7: Professional x64 Edition, Service Pack 1, (06.01.7601.00)
4/29/2013 7:40:43 AM | | Memory: 31.95 GB physical, 79.88 GB virtual
4/29/2013 7:40:43 AM | | Disk: 906.10 GB total, 488.78 GB free
4/29/2013 7:40:43 AM | | Local time is UTC -7 hours
4/29/2013 7:40:43 AM | | VirtualBox version: 4.2.4
4/29/2013 7:40:43 AM | | CUDA: NVIDIA GPU 0: GeForce GTX TITAN (driver version 314.22, CUDA version 5.0, compute capability 3.5, 4096MB, 4096MB available, 4989 GFLOPS peak)
4/29/2013 7:40:43 AM | | CUDA: NVIDIA GPU 1: GeForce GTX TITAN (driver version 314.22, CUDA version 5.0, compute capability 3.5, 4096MB, 4096MB available, 4989 GFLOPS peak)
4/29/2013 7:40:43 AM | | CUDA: NVIDIA GPU 2: GeForce GTX TITAN (driver version 314.22, CUDA version 5.0, compute capability 3.5, 4096MB, 4096MB available, 4989 GFLOPS peak)
4/29/2013 7:40:43 AM | | OpenCL: NVIDIA GPU 0: GeForce GTX TITAN (driver version 314.22, device version OpenCL 1.1 CUDA, 6144MB, 4096MB available, 4989 GFLOPS peak)
4/29/2013 7:40:43 AM | | OpenCL: NVIDIA GPU 1: GeForce GTX TITAN (driver version 314.22, device version OpenCL 1.1 CUDA, 6144MB, 4096MB available, 4989 GFLOPS peak)
4/29/2013 7:40:43 AM | | OpenCL: NVIDIA GPU 2: GeForce GTX TITAN (driver version 314.22, device version OpenCL 1.1 CUDA, 6144MB, 4096MB available, 4989 GFLOPS peak)

 

[refraxion]

Whats the speed comparison for you going from sli to tri sli? Trying to justify a third titan purchase dor 1440p..

 

[SonataSys]

Whats the speed comparison for you going from sli to tri sli? Trying to justify a third titan purchase dor 1440p..

Sorry, I don't have hard numbers for you because I didn't benchmark one TITAN or TITAN SLI-- I just went for 3-way SLI immediately.

I am fairly certain, however, that two TITANS will easily average 80 FPS or more in a 2560 x 1440 setup with ultra rendering quality. I'm averaging 130 FPS in a 2560 x 1600 3-Way SLI configuration now.

;->

To me, the only reason for 3-way TITAN SLI is to handle insane resolutions while averaging 70 FPS with ultra rendering quality in multi-panel 5760 x 1200 or 1440 configurations.

If you have plans (like I do) of upgrading to a multi-panel environment within the next year or so, then I would go for 3-way TITAN SLI. Otherwise, TITAN SLI is the way to go...

 

[refraxion]

Sorry, I don't have hard numbers for you because I didn't benchmark one TITAN or TITAN SLI-- I just went for 3-way SLI immediately.

I am fairly certain, however, that two TITANS will easily average 80 FPS or more in a 2560 x 1440 setup with ultra rendering quality. I'm averaging 130 FPS in a 2560 x 1600 3-Way SLI configuration now.

;->

To me, the only reason for 3-way TITAN SLI is to handle insane resolutions while averaging 70 FPS with ultra rendering quality in multi-panel 5760 x 1200 or 1440 configurations.

If you have plans (like I do) of upgrading to a multi-panel environment within the next year or so, then I would go for 3-way TITAN SLI. Otherwise, TITAN SLI is the way to go...

Hmm, thats pretty awesome... I wouldnt' mind getting more frames for my setup. Which games do you play that you avg 130? Also, a side question.. how did you take off that HDD cage that is directly in front of the 200mm fan?

 

[SonataSys]

Hmm, thats pretty awesome... I wouldnt' mind getting more frames for my setup. Which games do you play that you avg 130? Also, a side question.. how did you take off that HDD cage that is directly in front of the 200mm fan?

Here's a decent 1/2/3-way TITAN SLI review that basically comes to the same conclusion I did:

http://www.guru3d.com/articles_pages/geforce_gtx_titan_3_way_sli_review,20.html

I'm averaging 130 FPS in BF3 (2560 x 1600 with ultra rendering quality) multi-player with 64 gamers online.

FPS can vary a bit from map to map and also depends on how frequently I die a violent/explosive death. The more players, more explosions and more gunfire results in a lower average FPS.

I like to say this build is "totally stock", but as you noticed that is not 100% true.

In fact, for the sake of slightly better airflow and thermal performance within the case, I actually did use my dremel to drill-out the rivets holding a particularly bulky front cage/frame in place.

It was impeding smooth airflow directly towards the video card area:

That was my one and only case modification, and I have no regrets doing so.

One task on my "to-do" list is to try different front-intake fans, instead of the stock 200mm Cooler Master fan, including even 140mm units from Noctua, for example.

However, since my temperature measurements at idle and under load are rather excellent, I'm not feeling any urgency to undertake this particular line of experimentation...

 

[SonataSys]

After several hours of testing, I eventually settled on the following overclock configuration for all three TITAN cards using EVGA Precision.

This configuration provides a very stable average overclock of 269MHz across all three TITAN video cards while gaming or folding under heavy load:

Note how the bottom card operates about 10C cooler under heavy load compared with the other two cards.
For comparative purposes, a stock TITAN from EVGA clocks at 837MHz with a boost of 876MHz:


EVGA Stock TITAN

• Base Clock: 837MHz
• Boost Clock: 876MHz

The installed SuperClocked edition TITAN cards from EVGA clock higher by 39/52MHz respectively:

EVGA SuperClocked TITAN

• Base Clock: 876MHz (39MHz > Stock)
• Boost Clock: 928MHz (52MHz > Stock)

Under heavy gaming and 100% folding load the highest sustained and stable clock rate was 1123MHz, which is 286MHz above stock.
The overall average improvement across all three TITAN cards was 269MHz.

EVGA SuperCLocked TITAN - LOAD

• TITAN-1: 1084MHz (247MHz > Stock)
• TITAN-2: 1123MHz (286MHz > Stock)
• TITAN-3: 1110MHz (273MHz > Stock)
• TITAN-AVERAGE: 1106MHz (269MHz > Stock)

Below are two screen shots of BONIC Manager in action, along with Precision metrics while playing BF3 with 64 other online players at 2560 x 1600 using ultra rendering quality.

I haven't seen FPS drop below 100 yet:

 

[signalpuke]

nVidia mentioned they had limited support for the 600 series on the x79 platform, are you seeing any problems with the Titans?

 

[SonataSys]

nVidia mentioned they had limited support for the 600 series on the x79 platform, are you seeing any problems with the Titans?

I have experienced zero issues thus far, but it's only been a few weeks. In that limited time frame, this system has been through several 3DMark 11 stress tests, three days of continuous folding at 100% utilization, and around six hours of heavy BF3 and Crysis 3 gaming. I have encountered nothing but tremendous performance, astonishingly quiet operation and good thermal numbers.

I was not aware of NVIDIA's limited support for 600-series cards on x79 platforms, and I'm not sure what this has to do with TITAN. The GTX TITAN is based on an entirely different Kepler GPU, hardware that's normally part of NVIDIA's Tesla K20 product line. Further, NVIDIA has already emphasized that TITAN is not part of the 600-series, and it doesn't represent the start of the next GeForce series either. TITAN simply "stands alone".

Actually, the need for a strong X79 platform and higher-end CPU are clearly expressed in nearly every TITAN review I have encountered, especially in 2/3/4-way SLI configurations. For example, every Guru3d.com review involving TITAN mentions the need for an X79 platform; all of their TITAN reviews use the same X79 motherboard found in my workstation: MSI Big Bang XPower II.

GTX TITAN Review

GTX TITAN Overclock Guide

GTX TITAN 3-Way SLI Review

 

[refraxion]

What is your crysis 3 peformance like for single player?

 

[signalpuke]

Are the titans showing pcie 3?
http://1pcent.com/?p=257