• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Linux versus Windows - bigadv folding results

musky

[H]ard|DCer of the Year 2012
Joined
Dec 14, 2009
Messages
3,154
Since bigadv is now available for Linux, I figured we needed a good comparison. If you are currently using Windows, should you switch to Linux? The rumor mill is in full force - I have heard it is faster, and I have heard that it is slower. I wanted to put the rumor to rest and get some real data to go with.

The machine:
There is no sense pussyfooting around - if you are going to test, test big! My test machine is, you guessed it, a dual hex SR-2. This particular one is running at 3.618 GHz with 6 x 1 GB Kingston HyperX PC1600 running at 1608, 8-8-8-20-1T. It is actually my slowest dual hex machine, but it is also the most consistent. It has been churning out 130K - 135K ppd for several months very consistently without a reboot or crash. It also was the first box I switched over to the new 6.34 Windows core, so I have numbers from a couple of the new P6901 units running the Windows client.

Windows 7 Ultimate x64 was what was originally on this machine. The first 6901 it ran under Windows averaged 12:56/frame. The second averaged 12:59/frame. The variance was around +/- 10 seconds, so I am pretty confident that these times are very representative of what I would continue to see under Windows. The times are also right in line with what I was seeing on P6900s and P2686s, which also makes me think that these times would hold over time.

I installed Ubuntu Server 10.10 on this machine last night. My recent history with various flavors of Linux has not been good at all, but since I was feeling adventurous and wanted to get some data to the masses, I went ahead and installed it. The install itself went without issue – basic server with SSH and samba, nothing special. On first boot, I saw a problem – the pciehp daemon was logging errors, two every few seconds, displaying to the main screen and filling it up. On a hunch, I SSH’ed to it from a different machine and no longer saw the errors. They were still being logged and displayed on the main terminal, but were at least out of site from my remote terminal. Since I could actually work on the machine and everything seemed to function correctly, I went ahead and configured samba and set up F@H. Googling the errors being logged was not too helpful. A big thanks to Dsee from our IRC channel for helping me troubleshoot. While he was unable to solve the problem, he got me on the right track. At this point it was getting late and I wanted to see some numbers, so I fired up the client. It downloaded an A5 core and a P6901 work unit and started without issue. Success!

The first few frames were slower by around 15 seconds. Running top showed me that this pciehpd process would jump up and grab some CPU time every so often – not at all good on faster bigadv machines. Overnight the frame times settled in at 13:14 average, not bad, but slower than Windows. Since I was still not convinced that the pciehpd daemon and all of the errors it was logging weren’t affecting my frame times, I started troubleshooting again this morning. Several hours and a complete kernel recompile later, I was no longer logging the errors. I started F@H back up and now top was showing virtually nothing using CPU time except the F@H core. This is about as good as it is going to get.

The results: 10 frames after the kernel recompile, frame times are at 12:58. This is virtually identical to my Windows times. I don’t plan on changing anything on this machine any time soon, so we will see if the frame times hold up. It looks like they will – all of the frame times so far are within 5 seconds of each other, so it look pretty consistent.

Summary – you aren’t going to gain anything switching to Linux, but you also aren’t going to lose anything. I was hoping for a more conclusive result, but I am happy with the fact that we now have a valid non-Windows choice for folding boxen.
 
whew. Honestly I was afraid you were going to say OMG LINUX IS THE BEST EVER!!!!

I will continue to fold under Windows in peace.

(this post will be followed by Linux geeks telling musky he used the wrong distro and obviously he's doing it wrong and OMG LINUX IS THE BEST EVER!!!)
 
(this post will be followed by Linux geeks telling musky he used the wrong distro and obviously he's doing it wrong and OMG LINUX IS THE BEST EVER!!!)

I am sure you are right. Even Linux geeks hate Ubuntu. I do get a little "geek cred" for using a CLI server distro, and I am technically running a custom kernel now. Even so, it is still Ubuntu.

If anyone want me to try a different distro or has some kernel tweaks I should apply, let me know how to do it and I will post the results.
 
lol kudos on server distro...

I will throw it on one of my dualies tuesday and see if the numbers are any different between distros... I have noticed a difference in frame time between windows and linux so I am curious... I have heard the kernel argument... and heck... if using rhel5u3 is faster than rhel5u6... thats what I will do...
 
The rumor is that SLES 10 SP2 is the fastest out-of-the-box. The problem I had is that it doesn't configure the network adapters on an SR-2 out of the box, and I have no idea what i am doing in Suse. I am at least somewhat familiar with Ubuntu, and could probably fake it with Debian or other Debian-based distros.
 
The rumor is that SLES 10 SP2 is the fastest out-of-the-box. The problem I had is that it doesn't configure the network adapters on an SR-2 out of the box, and I have no idea what i am doing in Suse. I am at least somewhat familiar with Ubuntu, and could probably fake it with Debian or other Debian-based distros.

I can walk you through it... its pretty easy... and no even on network installs it doesnt config the network till sp4

but I hate sles...
 
And it'll save you ~$100 per box on OS cost.

Yeah, but it becomes a total bitch if you have your foldings rigs connected via wireless to the router. With Windows 7, it just autodetects everything which is so nice.
 
Yeah, but it becomes a total bitch if you have your foldings rigs connected via wireless to the router. With Windows 7, it just autodetects everything which is so nice.

certain wrappers make it easy as pie... but meh
 
Yeah, but it becomes a total bitch if you have your foldings rigs connected via wireless to the router. With Windows 7, it just autodetects everything which is so nice.

Ubuntu auto detects wifi and connects for ya ;)

even does it out of the box, you need to realize Ubuntu is made for the masses, and not the ultra "nerd", so everything *now* is setup out of the box so it appeals to a larger demographic


great info here musky, i plan on setting up my SB with a variation of Ubuntu once they dam boards get back in stock
 
Ubuntu auto detects wifi and connects for ya ;)

even does it out of the box, you need to realize Ubuntu is made for the masses, and not the ultra "nerd", so everything *now* is setup out of the box so it appeals to a larger demographic


great info here musky, i plan on setting up my SB with a variation of Ubuntu once they dam boards get back in stock

I will have to do some testing to see if there is an "easy" to setup linux that gives better than windows performance...

I will have to do some distro to distro comparisons...
 
There have been a couple of times when Tobit, Freaky (or others) have done custom builds of Linux that were supposed to really help with the efficiency of the SMP WU's, but nobody ever produced one that they would openly share with the rest of us. I know I inquired a couple of times with fellow members with a willingess to test and run those builds, but always got put off. :(

It would be great to have an open thread where those of us who want to learn more about tweaking Linux could ask questions and get some advice. I've been entertaining the thought of running Linux as my primary OS on the main boxen, but just don't feel like I would ever be able to get the proper amount of support to feel confortable tweaking it and learning the things I want to in order to get the most out of it...
 
There have been a couple of times when Tobit, Freaky (or others) have done custom builds of Linux that were supposed to really help with the efficiency of the SMP WU's, but nobody ever produced one that they would openly share with the rest of us. I know I inquired a couple of times with fellow members with a willingess to test and run those builds, but always got put off. :(

It would be great to have an open thread where those of us who want to learn more about tweaking Linux could ask questions and get some advice. I've been entertaining the thought of running Linux as my primary OS on the main boxen, but just don't feel like I would ever be able to get the proper amount of support to feel confortable tweaking it and learning the things I want to in order to get the most out of it...

well.. I loaded ultimate edition (built off ubuntu) as much as I can enjoy hc linux distros... sometimes I like ones that just work...

Ultimate edition is built for speed demons... dunno how well it works...
Installed the latest on my Q6600 build last weekend and honestly have not touched it since...

Kinda in the middle of spring cleaning my room lol...

go ahead and start a thread if you want... I will comment as I am capable...
 
I'll be switching my windows boxen to centos 5.5 ASAP with the linux bigadv news.

Do we need to have the 6.34 version for linux bigadv? I think I still have some older clients running.
 
Until someone says they got one to upload, I would hold off switching. I just lost my first one - it hit 100%, gave me the FINISHED_UNIT message, and hung.

Code:
[01:25:35] - Writing 100192115 bytes of core data to disk...
[01:25:51] Done: 100191603 -> 96469982 (compressed to 10.5 percent)
[01:25:51]   ... Done.
[02:06:33] - Shutting down core
[02:06:33] 
[02:06:33] Folding@home Core Shutdown: FINISHED_UNIT

Folding@Home Client Shutdown.


When I restarted the client, it deleted the WU:

Code:
[02:07:44] Folding@home Core Shutdown: BAD_FILE_FORMAT
[02:07:44] CoreStatus = 6F (111)
[02:07:44] + The core could not recognize the format of the provided work file.
[02:07:44] Deleting current work unit & continuing...
[02:07:44] - Preparing to get new work unit...
[02:07:44] Cleaning up work directory
[02:07:44] + Attempting to get work packet

No idea what appened. The machine didn't lock up or reboot or anything. We'll see if this one fares better.

And yes Mtnduey, you need the newest client.
 
whew. Honestly I was afraid you were going to say OMG LINUX IS THE BEST EVER!!!!

I will continue to fold under Windows in peace.

(this post will be followed by Linux geeks telling musky he used the wrong distro and obviously he's doing it wrong and OMG LINUX IS THE BEST EVER!!!)

I still think he was talking bout me...
could have been due to my Linux FTW threads over at OCW a few years back...
 
The windows core used to be a fair bit slower.
Seems that has changed.
 
The windows core used to be a fair bit slower.
Seems that has changed.

or the linux has slowed down to match it...

I will be playing with different distros both old and new to try and figure out which is true...

they seem evenly matched... though I have seen lower tpf since I switched back to linux... perhaps windows 8 alpha is slower than 7 lol...
 
thanks for the info, I've switched my existing linux clients over to 6.34 and they're now running bigadv

Time to see if my secret weapon is going to be available *evil grin*

LET THE WAR BEGIN!!!!
 
I will have to do some testing to see if there is an "easy" to setup linux that gives better than windows performance...

I will have to do some distro to distro comparisons...

specifically ill be doing Mythbuntu ;) If it doesnt work well ill be on that speed one you talk about and just install MythTV server onto that ;)
 
Time to see if my secret weapon is going to be available *evil grin*

LET THE WAR BEGIN!!!!
Hopefully, you and all the other hitherto non-bigadv Linux guys can get this team back to the top spot...!! :cool:
 
Hopefully, you and all the other hitherto non-bigadv Linux guys can get this team back to the top spot...!! :cool:

I be doing my best :)

800k for today so far...
 
Hopefully, you and all the other hitherto non-bigadv Linux guys can get this team back to the top spot...!! :cool:

I dont know if I can take the big guns off their projects but all the D I can spare is going for the [H]

there are a couple strange and kooky weird things I'm working on that I hope work in the proof of concept stage.
 
Patriot and I were comparing times on bigadv jobs on dual hex core Opterons - he was getting 10 minutes a frame better than I was, with him using a slightly slower machine. After I tweaked and adjusted memory/NUMA settings I got it down to an 8 minute difference - but my times were still way to slow.

So I switched from Windows to Linux:

2 * Opteron 4180 @ 2.6Ghz, SSD drive: on Windows 2008 R2 - 6901 @ 30:22 / frame

2 * Opteron 4180 @ 2.6Ghz, Sata Laptop Drive: on OpenSuse 11.2 - 6901 @ 21:59 / frame

PPD Works out as 36803 vs 59751

62% more ppd using linux rather than Windows 2008 R2

I was stunned.

H.
 
That 6901 I mentioned earlier went through without a hitch. Got my points for it at the 2am CST stanford update.
 
UPDATE: (this is a big one)

For some reason, this machine has gone nuts on frame times. It did this on the last unit for several frames as well. This one has been consistent through 64% though
Code:
Frame #		Time
0	-	
1	-	0:11:40
2	-	0:13:51
3	-	0:12:42
4	-	0:11:39
5	-	0:11:40
6	-	0:11:24
7	-	0:11:37
8	-	0:11:38
9	-	0:11:37
10	-	0:11:25
11	-	0:11:37
12	-	0:11:35
13	-	0:11:40
14	-	0:11:36
15	-	0:11:25
16	-	0:11:38
17	-	0:11:38
18	-	0:11:35
19	-	0:11:25
20	-	0:11:33
21	-	0:11:33
22	-	0:11:32
23	-	0:11:25
24	-	0:11:34
25	-	0:11:32
26	-	0:11:31
27	-	0:11:32
28	-	0:11:24
29	-	0:11:31
30	-	0:11:30
31	-	0:11:30
32	-	0:11:24
33	-	0:11:30
34	-	0:11:30
35	-	0:11:29
36	-	0:11:25
37	-	0:11:29
38	-	0:11:30
39	-	0:11:30
40	-	0:11:24
41	-	0:11:30
42	-	0:11:30
43	-	0:11:29
44	-	0:11:29
45	-	0:11:24
46	-	0:11:29
47	-	0:11:28
48	-	0:11:30
49	-	0:11:24
50	-	0:11:29
51	-	0:11:29
52	-	0:11:28
53	-	0:11:24
54	-	0:11:30
55	-	0:11:30
56	-	0:11:43
57	-	0:11:23
58	-	0:11:29
59	-	0:11:29
60	-	0:11:30
61	-	0:11:29
62	-	0:11:24
63	-	0:11:30
64	-	0:11:30

Remember tha this is the same machine that was turning just under 13 minute frames under Windows. It equate to a 25K ppd increase over Windows. We'll see if this one will upload.
 
UPDATE: (this is a big one)

For some reason, this machine has gone nuts on frame times. It did this on the last unit for several frames as well. This one has been consistent through 64% though

Remember tha this is the same machine that was turning just under 13 minute frames under Windows. It equate to a 25K ppd increase over Windows. We'll see if this one will upload.

Nice improvement. Hope this is the real deal, and the WU's prove to be consistantly stable...
 
Nice improvement. Hope this is the real deal, and the WU's prove to be consistantly stable...

I have a second dual hex on Linux now as well. This time, I went with the simpler Ubuntu Desktop install. I am running a 6701 right now, and it is slow as dirt. I think that there is some post-install processing going on, because I appear to only be using 23 of the 24 cores right now. Hopefully, it will straighten up and start using all cores here in a bit. That would explain what i saw yesterday - 1 1/2 minutes longer frame times for a while before it kicked in and dropped to what appears to be a "normal" frame time.

More to come...if the desktop version produces similar results, anyone can install it as Wfeather mentioned earlier.
 
Ubuntu auto detects wifi and connects for ya ;)

even does it out of the box, you need to realize Ubuntu is made for the masses, and not the ultra "nerd", so everything *now* is setup out of the box so it appeals to a larger demographic


great info here musky, i plan on setting up my SB with a variation of Ubuntu once they dam boards get back in stock

I might give Ubuntu another try, but I couldn't get my wireless card to get installed with Ubuntu or Win XP 64. Both my folding rigs are running a Netgear Rangemax wireless card.
 
I might have to find a Linux distro that would handle my WHS duties.
Posted via Mobile Device
 
I might have to find a Linux distro that would handle my WHS duties.
Posted via Mobile Device

FreeNAS ?


@amdgamer - it pretty much covers almost all the available wifi cards out there :D
 
I will start playing around with different distros... I have no fears... I want to max out the team ppd... musky might do some serious damage with the new hexcore and the right distro... it looks like it finally picked up...

@musky
I wonder if the filesystem was still initializing or something... what filesystem type are you using...
 
Last edited:
...musky might do some serious damage with the new hexcore and the right distro...

Heh, like he's not already doing a lot of damage? :cool: I loaded up Mint 10 64bit on a i7-950 / 3.6Ghz and am getting about 35:10 minutes per frame on a bigadv 6901. Might have to clock that up some more and put it under water if I continue though...I usually only run standard A3 SMP's on everything but the SR-2.

Can anyone tell me a reliable way to monitor temps under Linux?
 
:cool: I loaded up Mint 10 64bit on a i7-950 / 3.6Ghz and am getting about 35:10 minutes per frame on a bigadv 6901. Might have to clock that up some more and put it under water if I continue though...I usually only run standrad A3 SMP's on everything but the SR-2.

Can anyone tell me a reliable way to monitor temps under Linux?

http://www.lm-sensors.org/wiki/iwizard/DistroPackageInstall
Heh, like he's not already doing a lot of damage? :
Yeah but I am pretty sure he doesn't want to give up first place in team ppd...
 
Heh, like he's not already doing a lot of damage? :cool: I loaded up Mint 10 64bit on a i7-950 / 3.6Ghz and am getting about 35:10 minutes per frame on a bigadv 6901. Might have to clock that up some more and put it under water if I continue though...I usually only run standard A3 SMP's on everything but the SR-2.

Can anyone tell me a reliable way to monitor temps under Linux?

lm-sensors as Patriot said, and gkrellm if you like a GUI front end for it.

Ax, can you check your resource monitor and see if you have all cores pegged? I am not able to with regular Ubuntu Desktop.
 
I will start playing around with different distros... I have no fears... I want to max out the team ppd... musky might do some serious damage with the new hexcore and the right distro... it looks like it finally picked up...

@musky
I wonder if the filesystem was still initializing or something... what filesystem type are you using...

Whatever the default is - ext4?
 
Back
Top