SR-2 Optimization Thread

[zero2dash]

does anyone here use their SR-2 rig as their workstation?

I was thinking about it today, I was planning on using my SR-2 as my main workstation (mostly browsing the internet, some TF2...) or would that be a bad idea?

[raises hand]
My SR-2 is my primary system. Pretty much the only system I use at home. (occasionally the q9450 but less than 2%....usually all I do on that is update and restart my 8800gt client)

No performance problems for me. I even let it re-encode mp3s while folding.
Only time I really shut anything down is my second 260 while using Photoshop because of the gpu acceleration, otherwise it runs like crap. (Ps I mean)
All my other work (Illustrator, InDesign, Acrobat Extended) all work fine, no performance hit while folding.

Only other time I shut down anything is if I'm gaming...depends on the client though. I usually try games with clients running and see if they're playable. Most things have been so far. (not really pushing 3d gaming....but stuff like Beat Hazard, Worms Reloaded, Max & The Magic Marker all run fine)

 

[may i be worthy]

does anyone here use their SR-2 rig as their workstation?

I was thinking about it today, I was planning on using my SR-2 as my main workstation (mostly browsing the internet, some TF2...) or would that be a bad idea?

I agree with zero2dash and Sazan, I think you would be fine. FAH is super sensitive - light web browsing, typing, etc can take me from 160,000ppd to under 145,000ppd. I make sure nothing like taskmanager, temp monitors etc is left running when I am not actually on the machine. But the roulette of the work units you draw make a much bigger difference...

My real problem is, as Sazaneyes indicated (I must whine about it memorably) is that for work I will be doing 3D rendering, that uses 100% of every core, that can run from minutes into days. Not a lot of point folding during the longer renders. And rebooting for software updates etc is negligible- even 15 minutes is 1% of a day I always say.

Gaming - I can game fine Far Cry 2 setting affinity of FAH to up to 22 of 24 cores. PPD takes a hit a lot more than 2/24 - they are fighting for memory etc but absolutely no need to stop folding.

Running your main rig folding is actually a pretty good idea for power usage, compared to as running a whole second desktop machine.

Mine will be my primary PC. As soon as I finish overclocking tests this weekend, I'm going to start installing software and copying data from my old desktop. Currently trying to find max stable BCLK. I keep getting BSODs when running LinX @ 20GB, so I'm reducing BCLK by 2MHz each time -- down to 184 MHz now..

184? What settings/components are common to the previous system? This sounds like the problem was the board or the ram settings, not the CPUs... that's like the unluckiest baseclock limit going around right now.... the only thing that springs to mind was maybe your RAM is like my first ever OCZ - simply unstable at 20G IBT @ 1066 no matter what.

But very interesting on the CPU swap / temps evening out - I had thought about that and wondered if it might make a difference, but I think for me the SR2 is the biggest difference between temps - both rigs behave exactly the same way. Hopefully tomorrow, couriers willing, I will finally have a third data point to share But if I get a bigger temps difference, I will swap them now for sure, thanks.

 

[404 - File not found]

198BLCK 1.225vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1066 9-11-9-28-2T B2B:8 1.65v mchstrap:auto **STABLE**
198BLCK 1.225vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 9-11-9-28-2T B2B:8 1.65v mchstrap:auto **reboots after 30min**
UPDATE:
198BLCK 1.225vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 10-12-10-29-2T B2B:8 1.65v mchstrap:1600 *Stable 1+ hour IntelBurn
200BLCK 1.250vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 10-12-10-29-2T B2B:8 1.65v mchstrap:1600 *Stable 30 min IntelBurn
202BLCK 1.300vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 11-13-11-29-2T B2B:8 1.65v mchstrap:1600 **reboots after 30min**
200BLCK 1.250vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 7-9-7-26-2T B2B:8 1.65v mchstrap:1600 **freeze in IntelBurn
200BLCK 1.250vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 8-9-8-28-2T B2B:8 1.65v mchstrap:1600 **freeze in IntelBurn
200BLCK 1.225vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 10-12-10-29-2T B2B:8 1.65v mchstrap:1600 **reboots after 30min**
200BLCK 1.250vcore 1.3vtt 1.75cpu pll 1.4ioh 1.125ich RAM-- 1333 10-13-10-31-2T B2B:8 1.65v mchstrap:1600 testing...


Other settings:
numa disabled
PCIE 101
IOH QPI 0 Signal -85
IOH QPI 1 Signal -16
c state, acpi t state disabled
Rest default

 

[jebo_4jc]

you are pushing 1600mhz RAM above close to 2000mhz. Have you tried loosening timings at all?

 

[404 - File not found]

you are pushing 1600mhz RAM above close to 2000mhz. Have you tried loosening timings at all?

they are very loose. Rated to run at 6-9-6. Now trying MCH-1600.

 

[404 - File not found]

8749mhz stable!!

.... I wish.

 

[FLECOM]

.... I wish.

now I am sad

 

[FLECOM]

No performance problems for me. I even let it re-encode mp3s while folding.

I am not worried about folding affecting the performance of the machine, I am worried about affecting the performance of the folding lol

I am just considering if I should use my i7 920 dedicated folder as my workstation and leave the SR-2 as a dedicated folder or if I should just use the SR-2 as my workstation

 

[jebo_4jc]

I am not worried about folding affecting the performance of the machine, I am worried about affecting the performance of the folding lol

I am just considering if I should use my i7 920 dedicated folder as my workstation and leave the SR-2 as a dedicated folder or if I should just use the SR-2 as my workstation

with that option, I suggest using the i7 920 as your workstation.

With the bonus system, any performance degridation is amplified in terms of lost PPD.

For that reason, I'm trying to figure out a way to move my WHS to my SR2 (possibly via VM on top of Windows 7) and use my single socket system as my main box.

 

[SazanEyes]

184? What settings/components are common to the previous system? This sounds like the problem was the board or the ram settings, not the CPUs...)

184 BCLK was stable overnight (LinX 20GB) with CPU multi x12 and memory at 2:6 to eliminate variables. The only parts in common with the last board are the CPUs. Mobo, RAM, HSF, PSU, etc. are all new. I really think (and I've read some hints elsewhere) that one or both of my X5650s has a weak memory controller onboard, or something else that's limiting my overclock. That said, my limit on my first SR-2 was 175 BCLK with these CPUs, and that board easily hit 200 BCLK with the L5640s. I'm currently trying to get the CPUs to run at full speed (>4GHz) @ 184. I'm not there yet but I still have VCore headroom.

My MCH is set to 1600 to get 11-11-11-29 auto timings, which seems to help at the 2:8 divider. I noticed that the mobo was choosing those timings sometimes, so I wanted to force them. I knew these 4GB DIMMS would cause some issues, but I'll play with LinX after I'm done overclocking to verify which settings give the most performance. The only other concern so far are temps (I think partially due to VTT 1.375) but I'll work on lowering voltages after I figure out how far I can get at relatively high volts.

edit: Forgot to mention that occasionally I would boot with only 20GB instead of 24GB. Loosening the timings seems to have solved that.

 

[Jmtyra]

Fellow SR-2 peeps -- what RAM set (or brand) ended up working best with the SR-2?
Going to get another soon, hopefully, and trying to decide what RAM to get.

Would presume 1600+, 1.65v and lowest CAS that I can find...

 

[jebo_4jc]

Not sure about "best", but my Corsair XMS3 1600 Cas 8 have been flawless AFAIK

 

[tived]

Wow, you guys are amazing!
Just read through 33 pages (yes, it did take a few days!)

Expecting my SR-2 anytime soon. Had to buy in the US for it to be shipped to Australia (No local stock :-( ) anyway, PerformancePC.com has shipped one, so fingers crossed one will be here this week.

So far I only have my case Lian Li 2120 black/silver internal and an Antec 1200w PSU.
CPU's 2x X5650 (hoping with a little help to get close to 4Ghz)

Don't really have the biggest budget here, but need to make this workstation as i need more speed.
So I would appreciate all the advice you guys have to make this a successful mean machine

This will mainly be my workstation for photo related work. Folding on my current 3 machines but they are only small (2x Q6600 and 1x 2x285Opty).
I am not sure yet as what to get for ram, I would like to fill it with as much as possible min 24GB but 48 would be great (in particular with 6x 8GB sticks) but which ram??
Is ECC not essential anymore?

Although Photoshop doesn't take full advantage of Cuda, but other programs I use such as PTGui and Autopano Giga, Helicon Focus.
Any advantage of getting two graphics cards, given I will mainly be doing photo related work? No, games - at least not for now.

One of the biggest bottlenecks in PS or almost any photo related work is the hard drives, so this will be an area where I need to put in some extra attention
Has anyone played around with any of the PCIe SSD cards? OCZ does some, are they any good? currently for hard drives I am thinking of 60GB SSD sandforce based drives and WD Black Caviar for storage.

Down the track get a PCIe Raid controller and get more disks for bigger arrays.



thanks guys - fantastic information and dedication of you guys!

Henrik

 

[Jmtyra]

Not sure about "best", but my Corsair XMS3 1600 Cas 8 have been flawless AFAIK

Good to hear...I thought there was an issue with the XMS modules on the SR-2 but I couldn't remember. My small 4gb set works ok, too.

 

[jebo_4jc]

Wow, you guys are amazing!

Thanks. Glad the thread helped. Stick around and ask questions. We're all still learning too!

This will mainly be my workstation for photo related work. Folding on my current 3 machines but they are only small (2x Q6600 and 1x 2x285Opty).

For which team? Obviously, Team 33 has the best users

CPU's 2x X5650 (hoping with a little help to get close to 4Ghz)

I am not sure yet as what to get for ram, I would like to fill it with as much as possible min 24GB but 48 would be great (in particular with 6x 8GB sticks) but which ram??

Is ECC not essential anymore?

ECC is not essential. I think MIBW might have 48GB in one of his SR2s. I'll let one of "those" guys answer the Big RAM question. I myself have only used Corsair 2GB sticks (which I referenced one post above) but they have been great.

One of the biggest bottlenecks in PS or almost any photo related work is the hard drives, so this will be an area where I need to put in some extra attention

Has anyone played around with any of the PCIe SSD cards? OCZ does some, are they any good?

currently for hard drives I am thinking of 60GB SSD sandforce based drives and WD Black Caviar for storage.

Yes, they are good. Anandtech (whom I consider "the" authority on SSDs, they have a lot of good articles there) just looked at OCZ's Revo X2 and found it to be great, but they said "There’s no performance advantage over you running four of your own SF-1200 based SSDs in RAID-0. The RevoDrive x2 is pretty much a four drive SF-1200 SSD on a stick for those who want simplicity. " So, you can get four sandforce drives or the Revo X2, whichever you prefer, and whichever works out best for price.

Any advantage of getting two graphics cards, given I will mainly be doing photo related work? No, games - at least not for now.

no, just stick with one GPU. I think the GTX470 is fully supported by the new Adobe GPU accelerated software. That might be a good bet.

Good luck and have fun

 

[404 - File not found]

About the hard drive, if it's only for photo work, I assume you only need high sequential t. r., consider going 4+ x WD 640GB short stroke raid 0 array. (that gets ~400mb/sec read and write).
Cheaper, plus.. long term writing/erasing a lot off ssd is not good.

 

[SazanEyes]

OK, after more tweaking I really seem to be temp-limited at the high end (hitting TjMax @ 184), so I backed off on voltage and BCLK a bit. I really think one of my X5650s (the "hot" one) is a dud. Here's where I am now:

X5650s @ 3916 MHz
BIOS ver. A50
BCLK 178
VCore 1.35V
VTT 1.35V
IOH 1.35V

RAM: G.SKILL Ripjaws 24GB
DDR 1066 (2:8 ratio)
1.5V
MCH 1600 for 11-11-11-29 1T at auto settings
(stock speed is 9-9-9-24 1.5V)

Turbo enabled
NUMA enabled (I couldn't POST with it disabled, but I haven't tried it again with the loose timings -- the problem is having to reset CMOS if it doesn't work)

Peak temp 90C, average ~86C during LinX, F@H should be 5-10 degrees cooler

These settings sucessfully did 10 LinX 20GB passes (in 2 hours 3 minutes). I want to try a few other benches to confirm stability. I'm going to see if I can lower the volts a little more for lower temps, too. I may also try some faster fans or dual fans on the TRUEs. I have fan clips, but no extra fast fans.

BTW, I'm pretty happy with the G.SKILL Ripjaws. There are limited choices in 4GB DIMMs, but these seem to be OK. I think I'll leave the Corsair Dominator in the other SR-2, where they should be able to take advantage of a higher BCLK with the L5640s.

 

[jebo_4jc]

About the hard drive, if it's only for photo work, I assume you only need high sequential t. r., consider going 4+ x WD 640GB short stroke raid 0 array. (that gets ~400mb/sec read and write).
Cheaper, plus.. long term writing/erasing a lot off ssd is not good.

I don't know. Modern SSDs are pretty darn resilient. I think this fear is a bit over hyped aorund the internet IMHO.

Also, Intel's research indicates SSDs are about 35-45% faster in Photoshop than 15k RPM Seagate HDDs. So I think it's very safe to say SSDs blow HDDs out of the water in Photoshop.
Source: ftp://download.intel.com/design/flash/NAND/extreme/Photoshop_CS4_Performance_Comparison.pdf

 

[404 - File not found]

Have you tried playing with the Back to Back CAS setting?
Also, why not more vdimm? Not same g.skill kits, but I'm running mine at 1.65v instead of the rated 1.35v, it does help a lot.

The big problem with the bios is the Auto ram settings keep changing. On Auto some ram timings are too tight, whatever the BLCK. I manually entered some settings, now and I'm stable at 2000mhz with numa disabled.

 

[404 - File not found]

I don't know. Modern SSDs are pretty darn resilient. I think this fear is a bit over hyped aorund the internet IMHO.

Also, Intel's research indicates SSDs are about 35-45% faster in Photoshop than 15k RPM Seagate HDDs. So I think it's very safe to say SSDs blow HDDs out of the water in Photoshop.
Source: ftp://download.intel.com/design/flash/NAND/extreme/Photoshop_CS4_Performance_Comparison.pdf

A lot of BOINC crunchers over at xtremesystem lost their ssd within months / a year. Keep in mind that was for second generation (vertex 1 or before).

 

[may i be worthy]

@ Sazan - I cant think of anything more to suggest, just good luck buddy.

Fellow SR-2 peeps -- what RAM set (or brand) ended up working best with the SR-2?

I only have personal experience with 2 types, some OCZ gold OCZ3G1600LV6GK that was so horrific I bear the mental scars to this day, and the Kingston KHX1600C7D3K3/6GX I have now - it might not be the best available, but having bought 8 kits across 3 SR2s I know the damn stuff works - good to at least DDR 1890.

Wow, you guys are amazing!
Just read through 33 pages (yes, it did take a few days!)

Expecting my SR-2 anytime soon. Had to buy in the US for it to be shipped to Australia (No local stock :-( ) anyway, PerformancePC.com has shipped one, so fingers crossed one will be here this week.k

Welcome - another Aussie!

Yeah, my latest SR2 took 2 weeks to come in - always "2-3 days" away

I think MIBW might have 48GB in one of his SR2s. I'll let one of "those" guys answer the Big RAM question.

No - I just have 12 x 2GB of the kingston above in my main workstation - 12 DIMMs is not ideal, but I have it working perfectly now, and I hear 4GB kits have their own issues...

Anyhoo, housekeeping out of the way, SR2#3 is up and running.

Prelim overclock is pretty much identical to the x5660, I have 4.3Ghz working immediately, but 4.4 looks like hard work. So if any of you were were on the fence... it is very much diminishing returns. But as I got these for x5660 prices, I couldn't not try could I?

It is also a squealer! heaps of motherboard noise - never heard a peep from the other two.

 

[SazanEyes]

Have you tried playing with the Back to Back CAS setting?
Also, why not more vdimm? Not same g.skill kits, but I'm running mine at 1.65v instead of the rated 1.35v, it does help a lot.

Now that I have a good understanding of the limits of my system, I'm going to get into the smaller tweaks, using LinX to check for real-world improvement. I'll probably have to back off on the OC a bit for stability. IntelBurnTest failed at max RAM, and my temps are still high, so I seem to be right on the edge. I haven't tried pushing more RAM volts, but so far I've been dealing more with CPU issues than RAM issues. I think I can get back to CAS9 without too much trouble. The nice thing about Auto settings is it lets the mobo decide where you need some help. I'll try to lock down the settings once I get the final results.

The big advantage to the L5640 (and apparently the L5639) seem to be the lower temps. You can really push voltage through them without too many issues. The X5650s seem to get really hot in comparison. Then again the X5650s, etc., make up for it with the higher multi. Since I'm not putting them underwater anytime soon, I think by backing off a bit I should be able to get back to cool temps at 1.3V Vcore. I'm hoping to be able to disable NUMA as well.

The biggest issue with 4GB DIMMs is lack of selection. Nearly all of the DDR3-1600 models are CAS9, and some can be very expensive. The big problem with lots of memory in general is that it seems to put more load (and generate more heat) on the memory controller integrated into the CPU. I'm definitely seeing higher socket temps on my X5650s with 24GB than I did with 12GB, and the temps go up as you run LinX with more RAM. There is some wisdom in minimizing RAM in a dedicated folder rig.

 

[Jmtyra]

I only have personal experience with 2 types, some OCZ gold OCZ3G1600LV6GK that was so horrific I bear the mental scars to this day, and the Kingston KHX1600C7D3K3/6GX I have now - it might not be the best available, but having bought 8 kits across 3 SR2s I know the damn stuff works - good to at least DDR 1890.

Thank you, MIBW...this gives me a few choices as to what to buy. Seems Kingston and Corsair play nice with the SR2.

It is also a squealer! heaps of motherboard noise - never heard a peep from the other two.

That's odd...what type of noise? Like the caps or something?
IMHO, but I'd be a bit nervous with something like that coming from my SR-2

 

[10e]

Thank you, MIBW...this gives me a few choices as to what to buy. Seems Kingston and Corsair play nice with the SR2.


That's odd...what type of noise? Like the caps or something?
IMHO, but I'd be a bit nervous with something like that coming from my SR-2

It's probably VRM noise. My Gigabyte GA-X58A-UD3R makes enough that I can hear it naked, but can't hear it when the case is closed. Alternatively my SR-2 doesn't do any of this.

It's like a raspy squeal, similar to what video cards make, but with more of a grating undertone to it.

 

[jebo_4jc]

I would RMA the squealing mobo.

 

[Jmtyra]

It's probably VRM noise.

Is that the "Voltage Regulator Module"?
What's your take...keep or RMA? Just curious!

 

[10e]

Is that the "Voltage Regulator Module"?
What's your take...keep or RMA? Just curious!

Yep, it's the Voltage Regulator Module(s)

If it bothers you, RMA it. If not, keep it. It could just be a batch of boards with different VRM modules causing the squeal. It's generally not a predictor of failure from what I know.

 

[10e]

I dunno if this will help you, but I lowered my temps by a couple degrees celsius on both CPUs by lowering CPU PLL for each to 1.725v from 1.80V.

BTW, LinX is more RAM dependent on folding, meaning it may show you a great improvement when RAM speeds are raised, but it may not transfer to a greater speed folding.

For example at 4.045 Ghz (184x22) my folding is faster but my LinX Gflops are slower than 3.9Ghz >195x20 by a bit. This is due to slightly lower RAM speeds.

Now that I have a good understanding of the limits of my system, I'm going to get into the smaller tweaks, using LinX to check for real-world improvement. I'll probably have to back off on the OC a bit for stability. IntelBurnTest failed at max RAM, and my temps are still high, so I seem to be right on the edge. I haven't tried pushing more RAM volts, but so far I've been dealing more with CPU issues than RAM issues. I think I can get back to CAS9 without too much trouble. The nice thing about Auto settings is it lets the mobo decide where you need some help. I'll try to lock down the settings once I get the final results.

The big advantage to the L5640 (and apparently the L5639) seem to be the lower temps. You can really push voltage through them without too many issues. The X5650s seem to get really hot in comparison. Then again the X5650s, etc., make up for it with the higher multi. Since I'm not putting them underwater anytime soon, I think by backing off a bit I should be able to get back to cool temps at 1.3V Vcore. I'm hoping to be able to disable NUMA as well.

The biggest issue with 4GB DIMMs is lack of selection. Nearly all of the DDR3-1600 models are CAS9, and some can be very expensive. The big problem with lots of memory in general is that it seems to put more load (and generate more heat) on the memory controller integrated into the CPU. I'm definitely seeing higher socket temps on my X5650s with 24GB than I did with 12GB, and the temps go up as you run LinX with more RAM. There is some wisdom in minimizing RAM in a dedicated folder rig.

 

[may i be worthy]

Squealing is totally gone this morning... but then I am folding, rather than doing 2 instances of IBT trying to kill the thing... I had to wait 2 weeks to get this rig together, the idea of RMA'ing - I imagine would take weeks. And the P6900s will all be gone by then. No, until it fails, dis is MINES!

@sazan - I never thought to look at socket temps between my 24GB SR2#1 and 12GB SR2#2 - but then one is in a case, the other nekkid, so you would expect 1 to be hotter. What Uncore are you running?

But if you find x5650's hot... not at all happy about the power use of the x5680s, SR2#2 uses 570watts folding, SR2#3 is using 680watts! I have to check there aren't other variables... been a while (EDIT - now 635 watts - changed power supply to Enermax 1020) since I measured SR2#2. But I don't think I fully thought through the implications or power draw. If the x5680 ends up getting no tangible benefit to the x5660 in overclock, I might just have added to my power bill.

@Jmtyra - yes, I went through so much drama and testing to get the ram working, it was a no brainer to simply get the same stuff, and not have to start over. First SR2 took me months to OC and get right with ram. Second took a few weeks. Third was folding 6 hours after first boot.

@10e - another great tip on the CPU PLL volts - I will try that out for sure... dunno when - it is raining fast new bigadv now. But I would need to redo stability testing... you are running basically at stock aren't you?

 

[FLECOM]

with that option, I suggest using the i7 920 as your workstation.

With the bonus system, any performance degridation is amplified in terms of lost PPD.

For that reason, I'm trying to figure out a way to move my WHS to my SR2 (possibly via VM on top of Windows 7) and use my single socket system as my main box.

argh, now with these new WU's I don't know if I want to tie up my i7 920 either lol

maybe I will go back to my C2Q machine since that barely gets any PPD

 

[may i be worthy]

I see a netbook in your future

 

[FLECOM]

na Core2 machines are completely worthless as far as PPD, I turned off my QX6700 machine a while ago... any decent GPU client outperforms a C2Q running SMP on the newer units

 

[10e]

@10e - another great tip on the CPU PLL volts - I will try that out for sure... dunno when - it is raining fast new bigadv now. But I would need to redo stability testing... you are running basically at stock aren't you?

I was running CPU PLL at stock, but I've seen no stability issues surface since lowering it to 1.725V in E-Leet..

As always your mileage may vary, but so far after a full day no issues on my end.

 

[FLECOM]

ok so a new drama in the saga of my SR-2 rig... I decide to remove my GTX470 since I am going to make this a dedicated folder... put a GeForce 8500 in it and now I can't overclock at all... no matter what I do I am stuck at 2.27ghz..

I was running 200x18 3.6 @ 1.35v for days without issue... now I cant even OC to 134mhz x18... it always reverts and boots up at 133mhz

tried a different video card also (quattro 295)

 

[jebo_4jc]

hard reset your BIOS? reseated RAM?

 

[FLECOM]

yes and yes

tried a geforce 6600 and same result... cannot overclock at all

 

[jebo_4jc]

same GPU slot?

 

[FLECOM]

yes first slot

 

[SazanEyes]

All I can say is, "that's weird." I have an 8800GS in my L5640 SR-2 with no overclocking issues.

Have you tried putting the 470 back in?

 

[FLECOM]

no I cant really without a lot of work since it was in the water loop and I removed it... temps on the CPU's are low and everything seems fine but it just wont oc

I think I am getting ready to give up on this thing... its been nothing but problems from day 1... every time I reboot it its a giant fight to get it to do anything