Linux versus Windows - bigadv folding results

[Patriot]

I threw it on the dual hex mid WU, resumed without issue. But understand caution.....

H.

already paused it once... (blk 100 > 106 )each time multiplies risk...
It normally doesn't happen... but every once and a while I lose a wu because of it... lost one this afternoon in the lab...paused and restarted 2... thankfully the one at 98% picked back up.... the one at 40% failed... oh well...it happens...

 

[APOLLO]

already paused it once... (blk 100 > 106 )each time multiplies risk...
It normally doesn't happen... but every once and a while I lose a wu because of it... lost one this afternoon in the lab...paused and restarted 2... thankfully the one at 98% picked back up.... the one at 40% failed... oh well...it happens...

I don't know your setup but can't you back up the WU when you make setting adjustments midway through by stopping it and saving? If not, I'd just wait for the WU to complete. No need to take unnecessary risk.

 

[Patriot]

I don't know your setup but can't you back up the WU when you make setting adjustments midway through by stopping it and saving? If not, I'd just wait for the WU to complete. No need to take unnecessary risk.

yes you can... but no saying that when you pause it that the checkpoint is good... 89% now...

 

[hspeirs]

Apollo,

Linux WU's can be fragile. Ctrl-C'ing one under windows is normally safe, doing so on Linux - it's a lottery.

H.

 

[Patriot]

Apollo,

Linux WU's can be fragile. Ctrl-C'ing one under windows is normally safe, doing so on Linux - it's a lottery.

H.

when I had to restart all of the clients to fix the machineId issue I lost one wu out of 17 folders....

today 1 of 2 lol...

 

[APOLLO]

Apollo,

Linux WU's can be fragile. Ctrl-C'ing one under windows is normally safe, doing so on Linux - it's a lottery.

I remember that from the A2 days but was under the impression things had changed when the A3 core was released..

 

[[H]ecklerKoc[H]]

And it'll save you ~$100 per box on OS cost.

Who pay for Windows when you can eval it over and over?

 

[Patriot]

I remember that from the A2 days but was under the impression things had changed when the A3 core was released..

client hangs on starting mid wu are much more common than loosing a wu...

I pause and startup wu all the time on the server and rarely loose them.... even on early stepping es...
but those of you who have overclocked systems seem to have a higher incident....

right now I am overclocked on a new arch that makes me once again a noob...

 

[APOLLO]

right now I am overclocked on a new arch that makes me once again a noob...

Possibly, but this team was built upon the shoulders of pioneers like yourself, so that's a good thing.

 

[Patriot]

Possibly, but this team was built upon the shoulders of pioneers like yourself, so that's a good thing.

106.2 is the max for this setup on this bios...

http://www.amdzone.com/phpbb3/viewtopic.php?f=521&t=138463
this might help drop a few seconds... but 2 pauses and starts is enough for being at 92% lol....

 

[may i be worthy]

Hey Patriot,

Have you tried Tear's affinity setting wrapper? It shaved around 10 sec/frame on a 6901 on the dual hex.

H.

Oh hells yeah - 23 seconds better.

SR2#3 @4.3Ghz: 6901 benchmark file:

Frame time without - NUMA Disabled:
9:54 = 197,700 ppd (windows= 11:12)
Frame time with wrapper - NUMA Disabled:
9:53
Frame time with wrapper, NUMA Enabled:
9:30 = 210,000ppd

Finally broke the 200k barrier. So now we are up to +46,000ppd and 28% better PPD than windows.

Full instructions and download here BTW: http://www.amdzone.com/phpbb3/viewtopic.php?f=521&t=138463

 

[musky]

...scrambling to install this now...will post results momentarily...

 

[may i be worthy]

@Musky - heh - which rig to do first? which RIG?! aaahhh!

Also a great note from Tear at that link:

CAUTION: FahCore_a5 is known to be problematic
   at user-induced shutdowns (see here). To be on a
   safe side make a backup of complete client directory
   before hitting Ctrl+C
   To tell whether checkpoint was written correctly check
   the size of work/wudata_XX.ckp file (where XX is the
   current slot number). It should be 75160 (for core A5).
   If it's not -- better switch to backed up directory.

 

[musky]

Simple enough to install....now to wait on frame times...

*queues Tom Petty*

The waitng is the hardest part...

 

[musky]

First full frame - 11:13
Reference average on this unit - 11:32 (a P6901)

Not quite as big of a jump as MIBW, but close. According to the calculator, that is a 6.7K jump in ppd.

Edit: Frame #2 - 11:09 - up to an 8.2K ppd gain...I am liking this....

 

[musky]

Here is what we have so far
Box - unit - old time - new time
believer - 6901 - 11;32 - 11:09
cowgirl - 2686 - 11:31 - 11:14
jackpot - 2686 - 11:36 - 11:16
topraider - 2685 - 19:37 - 19:04
thunderbeast - 6901 - 11:34 - 12:40

The first three are dual hex SR-2s @ 3.6GHz running Ubuntu Desktop. The fourth (topraider) is a dual X5550 @ 2.93 GHz running Ubuntu Server. The last one is also a dual hex SR-2 @ 3.6GHz running Gentoo.

The theory is that when Tobit set the Gentoo box up, he did not compile in NUMA support since we weren't using it at the time. Since I turned on NUMA for this wrapper, that is causing the slowdown. Tobit is adding NUMA support as I type this, so I suspect the numbers on the last box will come back to being in line with the other SR-2s.

The dual X5550 box is really impressive, even though it only equates to a ~3K ppd increase. Over 30 seconds/frame is nothing to sneeze at.

Edit: Tobit's "demon tweak" adding NUMA support to the Gentoo kernel brought frame time down to 11:08, as expected.

Gotta love a 30K+ ppd increase doing nothing but software tweaks...

 

[may i be worthy]

So it looks like you get a frame time reduction of about 3.4%, vs my 4.0% and 5.5%. Just checking you have c-states enabled?

I just did Sr2#2 as well - 9:35 - and the numbers now line up freakishly in proportion to Sr2#3.

SR2#3 - 117.79% faster than windows = 127.81% more PPD
SR2#2 - 117.84% faster than windows = 127.84% more PPD

The frame times are amazingly consistent
SR2#3 - 0:09:31 0:09:31 0:09:30 0:09:30
SR2#2 - 0:09:35 0:09:35 0:09:35 0:09:35

I can't remember having 4 identical frames in a row before.

 

[Tobit]

musky needs to give my back my SR-2.. I stress the "give" part as I, sadly, am broke and why I sold it to him in the first place. Glad to see tear's affinity wrapper is working out well and kicking myself for not trying it sooner. I need to add it to two of my systems here.

 

[Zink]

It would be amazing if there is a "Notfred's" type OS for VM or headless produced that includes the wrapper and all ideal tweaks. Having frame times consistent to the second with one install would be slick. Pande could always convince someone to do it and release an official bigadv OS for hardcore folders... It's too bad 1p systems already have good scheduling, I wish I could get the wrapper boost on my i7 systems.

 

[Patriot]

so after you had me turn numa off on all my rigs....
great...
numa is in rhels kernels already....
arg... just need to pop on tears wrapper

 

[Jmtyra]

Just now seeing this (thanks MIBW!!) and I'm LOL'ing a bit that numa is now back in the game.
I'll have to tinker with this and try it out tomorrow.

Good info guys!

 

[may i be worthy]

Twas Musky that did the hard yards - don't miss his Linux for dummies recipe here:
http://hardforum.com/showpost.php?p=1036935472&postcount=259

I was floundering until he put that up, now I have 2 rigs running Linux. Even converted my laptop to Ubuntu for giggles - and it works great. (vista install only took 2 years to overflow c: partition and develop unfixable networking problems - damn you to hell MS and Dell)

 

[Jmtyra]

Ah, OK. Well my appreciation and thanks goes to both of you guys, in addition to many others, who have been extremely [H]elpful with stuff like this.

Thanks folks

 

[may i be worthy]

Ok, mostly through the first live work units with Tears affinity wrapper.

Hmm, SR2B takes a minute or two to start/resume after ctrl-c to set oneunit flag. No biggie.

Start:

[11:14:32] Project: 2686 (Run 0, Clone 6, Gen 86)
[11:14:32] 
[11:14:32] Assembly optimizations on if available.
[11:14:32] Entering M.D.
[COLOR="Cyan"][11:14:39[/COLOR]] Mapping NT from 24 to 24 
[COLOR="Orange"][11:16:04[/COLOR]] Completed 0 out of 250000 steps  (0%)
[11:25:54] Completed 2500 out of 250000 steps  (1%)
[11:35:34] Completed 5000 out of 250000 steps  (2%)

Resume:

[00:58:36] Project: 2686 (Run 0, Clone 6, Gen 86)
[00:58:36] 
[00:58:36] Assembly optimizations on if available.
[00:58:36] Entering M.D.
[00:58:42] Using Gromacs checkpoints
[COLOR="Cyan"][00:58:44][/COLOR] Mapping NT from 24 to 24 
[COLOR="orange"][01:01:07][/COLOR] Resuming from checkpoint
[01:01:13] Verified work/wudata_06.log
[01:01:13] Verified work/wudata_06.trr
[01:01:14] Verified work/wudata_06.xtc
[01:01:14] Verified work/wudata_06.edr
[01:01:14] Completed 210188 out of 250000 steps  (84%)

SR2#3 however, gets right to work:

[11:19:35] Project: 6901 (Run 20, Clone 4, Gen 11)
[11:19:35] 
[11:19:35] Assembly optimizations on if available.
[11:19:35] Entering M.D.
[COLOR="Cyan"][11:19:41] [/COLOR]Mapping NT from 24 to 24 
[COLOR="Cyan"][11:19:43][/COLOR] Completed 0 out of 250000 steps  (0%)
[11:29:10] Completed 2500 out of 250000 steps  (1%)

I wonder why the difference.

But the great news is the consistency of frame times. If the question is "is running Ubuntu GUI a problem" the answer is no.

SR2#3 on a 6901:

Just look at that - average frame time 9:28. BUT - only a variance of between 9:26 and 9:29 - basically a mere one second under or over average, with a handful 2 seconds under.

If I crop the y axis scale you can see that clearly. Unbelievable

I see zero performance reason to run without a GUI.

 

[Jmtyra]

Good to know, as the GUI makes using Linux easier for nubs like me.
Used musky's guide for the 2nd SR-2 and grabbed Tears wrapper. We'll see how things go...

Now I just need to work on tightening up my ram timings hehe

 

[musky]

Oh hells yeah - 23 seconds better.

SR2#3 @4.3Ghz: 6901 benchmark file:

Frame time without - NUMA Disabled:
9:54 = 197,700 ppd (windows= 11:12)
Frame time with wrapper - NUMA Disabled:
9:53
Frame time with wrapper, NUMA Enabled:
9:30 = 210,000ppd

Finally broke the 200k barrier. So now we are up to +46,000ppd and 28% better PPD than windows.

Full instructions and download here BTW: http://www.amdzone.com/phpbb3/viewtopic.php?f=521&t=138463

With a new install, you need to actually download the core(s) before running the installer (step 6 in the linked guide.) You need the core available before the wrapper can "wrap" it.

Also, don't forget to enable NUMA. If you don't, you will not see any gains with this wrapper.

 

[musky]

If the question is "is running Ubuntu GUI a problem" the answer is no.
[snip]
I see zero performance reason to run without a GUI.

This is exactly what I am seeing. I shut the GUI off on all of mine, and am seeing identical frame times. Logic tells you that not running gdm and the other background processes associated with the GUI should improve performance at least a little. Actual number have not backed that up, however.

 

[may i be worthy]

This is exactly what I am seeing. I shut the GUI off on all of mine, and am seeing identical frame times. Logic tells you that not running gdm and the other background processes associated with the GUI should improve performance at least a little. Actual number have not backed that up, however.

Yup, case closed. I have come to the conclusion that a GUI used to have a performance hit, but does not bother a 24 thread monster like a SR2 very much, when not interacting with the machine.

The penny dropped while running the system monitor on my dual core 2 year old laptop running ubuntu. "idle" CPU is a lot higher than on the SR2 - simply because of the size of the task and the number of cores available.

Likewise on Windows Idle in taskmanager used to be about 1% on older machines, and these days reads 0% - even with the bloated crappy newer taskmanager from Vista onwards.

Finally a GUI is not the hit it used to be. It also means there is almost no limit to the amount of shit windows will try to run at the worst possible times. I wish it had a control panel option for "Performance mode - check if you actually intend to use this PC to do something" but that is a gripe for another day.

 

[Patriot]

Yup, case closed. I have come to the conclusion that a GUI used to have a performance hit, but does not bother a 24 thread monster like a SR2 very much, when not interacting with the machine.

The penny dropped while running the system monitor on my dual core 2 year old laptop running ubuntu. "idle" CPU is a lot higher than on the SR2 - simply because of the size of the task and the number of cores available.

Likewise on Windows Idle in taskmanager used to be about 1% on older machines, and these days reads 0% - even with the bloated crappy newer taskmanager from Vista onwards.

Finally a GUI is not the hit it used to be. It also means there is almost no limit to the amount of shit windows will try to run at the worst possible times. I wish it had a control panel option for "Performance mode - check if you actually intend to use this PC to do something" but that is a gripe for another day.

I use one of my dual 2ghz no ht machines in gui on a regular basis... just grabbing and dd or cp drivers to usb keys... if I do that all day it adds ~30s /frame avg ...

 

[scutzi128]

Pre-requisites:
1. Make sure that (where appropriate)
  - NUMA is enabled,
  - node interleave is disabled,
  - ACPI SRAT is enabled

In this step how do you do these things/verify that they are set correctly?

 

[10e]

In this step how do you do these things/verify that they are set correctly?

NUMA settings can be found in the BIOS on any multi-cpu setup, but for single CPU it will not show up.

ACPI SRAT is a setting on AMD multi-cpu boxes, as is Node interleaving. Node interleaving ON sets NUMA to OFF on these boxes.

Only the NUMA setting is prominent on the SR-2s. Here is that info:

http://hardforum.com/showpost.php?p=1036264773&postcount=241

 

[Jmtyra]

In this step how do you do these things/verify that they are set correctly?

Depends on what system/BIOS you're using. For the SR-2, it's NUMA which can be found under Power Mgmt --> ACPI Config --> Advanced ACPI Config --> NUMA Support.

Edit: 10e beat me to the punch!

 

[Vediovus]

NUMA settings can be found in the BIOS on any multi-cpu setup, but for single CPU it will not show up.

ACPI SRAT is a setting on AMD multi-cpu boxes, as is Node interleaving. Node interleaving ON sets NUMA to OFF on these boxes.

Only the NUMA setting is prominent on the SR-2s. Here is that info:

http://hardforum.com/showpost.php?p=1036264773&postcount=241

That explains why I didn't see an option to turn NUMA on/off on my board. I'll have to try and remember that next time I play around.

 

[scutzi128]

NUMA settings can be found in the BIOS on any multi-cpu setup, but for single CPU it will not show up.

ACPI SRAT is a setting on AMD multi-cpu boxes, as is Node interleaving. Node interleaving ON sets NUMA to OFF on these boxes.

Only the NUMA setting is prominent on the SR-2s. Here is that info:

http://hardforum.com/showpost.php?p=1036264773&postcount=241

Thanks for the info I am only running a single hexcore (970) so I guess I don't need to worry about that. I will report back with what difference if any the wrapper makes.

 

[Patriot]

Thanks for the info I am only running a single hexcore (970) so I guess I don't need to worry about that. I will report back with what difference if any the wrapper makes.

correct... but I would still use tears wrapper for your 12 thread system...
if your on linux...
if your not... then I would suggest it lol....

 

[Zink]

Is there any reason it would give a boost on a 1p system? Even a 5% boost would be useful.

 

[may i be worthy]

I thought it was only of any use on multi socket systems, but of course it can't hurt to try it.

 

[musky]

Is there any reason it would give a boost on a 1p system? Even a 5% boost would be useful.

While I haven't tested it, it should not make any diffference on a single proc system.

 

[10e]

Depends on what system/BIOS you're using. For the SR-2, it's NUMA which can be found under Power Mgmt --> ACPI Config --> Advanced ACPI Config --> NUMA Support.

Edit: 10e beat me to the punch!

Ninja'D!

Is there any reason it would give a boost on a 1p system? Even a 5% boost would be useful.

It might. I haven't tried it yet on my i7 980x but I'm gonna do that NOW and restart the client.

Alternatively I've seen a decent reduction in time-per-frame stats on my SR-2 and my AMD (Skankbox48 ©) after I added Tear's wrapper.

TPF on p6901 on the SR-2 with x5650s at 3.9Ghz (195x20) and RAM at 9-9-9-24-1T @ 1560 is now down to 10:30 for 180K PPD.

The evil AMD is up to about 308K PPD with t-p-fs of 7:20 - 7:24. Now to overclock!

Thank Dog for Tear. He's the man.

Update: Kraken has cut about 7 to 8 seconds off my p2685 TPF running on my i7 980x so it may be something to look at

 

[Activate: AMD]

So running with the Kraken wrapper is ok on GUI'd up ubuntu? I've got only rudimentary linux knowledge from about 3 years ago so something relatively painless would be ideal.