![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Linux versus Windows - bigadv folding results
- Thread starter musky
- Start date
lol... So am I ... so am I....
and I am glad for those who used only windoze to have seen the light... knew I wasn't completely crazy...
I am gonna be adding a sandy at home for bigadv...
if only those mobos will come back in stock... super excited...
gentoooooo
@ musky what are you running your ram at? what config on the dualy x5550 box... I have some dual 2.93 quads and I am getting more like 20-21min 6901...
and I am glad for those who used only windoze to have seen the light... knew I wasn't completely crazy...
I am gonna be adding a sandy at home for bigadv...
if only those mobos will come back in stock... super excited...
gentoooooo
@ musky what are you running your ram at? what config on the dualy x5550 box... I have some dual 2.93 quads and I am getting more like 20-21min 6901...
I have the board set to force PC1333, but I don't think it actually is. I think it runs PC1066 triple channel. About the only other thing I have set differently than stock is that I disabled NUMA. Other than that, it is nothing special. It is also still holding at 19:51/frame for 41 frames now. This is with Ubuntu Server 10.10 with the BFS patch and no other tweaks.
well I have yet to implement bfs patches... thats next up on my list...
@tobit have you had any trouble with drivers after patching... it is technically changing the kernel and initrd... I will have to pay close attention to make sure the raid drivers are included in the patched initrd...
Nope.
Sometimes we just don't need to know.......
Masochist.
Not sure what it is about ext4, XFS, and JFS (the three file systems I've last experimented with) that causes the lag. It sure would be nice to figure out why it behaves like this.
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Morning all! Well, SR2#2 folded overnight fine and did several SMP units, but bigadv seems to be rare as hens teeth...
Musky or anyone got a captured 6901 for me? Bueller... Beuller?
Musky or anyone got a captured 6901 for me? Bueller... Beuller?
Morning all! Well, SR2#2 folded overnight fine and did several SMP units, but bigadv seems to be rare as hens teeth...
Musky or anyone got a captured 6901 for me? Bueller... Beuller?
Running one right now. I'll see if I can grab it for ya mate. Just have to get Samba rocking on this now.
So far not impressed with Gentoo on my SR-2. Frame times seem almost identical with Windows 7 on it. No system changes except to enable NUMA, which is a fair bit quicker by my estimates based on a 710x unit I had earlier than ran a lot quicker with NUMA enabled.
On the AMD 48-core the frame times on un-optimized, non BFS patched Ubuntu 10.10 server are still under 8 minutes after running for three days. This is still nearly 3 minutes TPF faster than Windows on p6901.
I guess this might mean that we will see great benefit on AMD based systems than Intel ones?
Morning all! Well, SR2#2 folded overnight fine and did several SMP units, but bigadv seems to be rare as hens teeth...
Musky or anyone got a captured 6901 for me? Bueller... Beuller?
I haven't tried to run it, but here:
http://www.avid-edge.com/fah/linux6901.zip
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
You beauty! Thanks mate.///
(Not calling you beautiful - that would be weird. It is an Aussie thing.)
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Update - SR2#2 - Ubuntu 10 Musky Mix. Running Summer Overclock at 4.255Ghz.
(Windows average p6901 - 11mins 22sec)
Ubuntu DFS kernal: p6901. Numa disabled. Captured p6901 from Musky.
10:53
11:09
10:55
Benefit - 3.5% to 4.4%.
That was running nocpulock=0. For giggles, I ran nocpulock=1
10:53
11:08
10:53
11:04
Amazing to see how closely it tracked the prior run - second frame longer etc.
So far 4% faster is nothing to sneeze at, but well short of where I was hoping to be. Testing other tweaks now...
(Windows average p6901 - 11mins 22sec)
Ubuntu DFS kernal: p6901. Numa disabled. Captured p6901 from Musky.
10:53
11:09
10:55
Benefit - 3.5% to 4.4%.
That was running nocpulock=0. For giggles, I ran nocpulock=1
10:53
11:08
10:53
11:04
Amazing to see how closely it tracked the prior run - second frame longer etc.
So far 4% faster is nothing to sneeze at, but well short of where I was hoping to be. Testing other tweaks now...
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
USB drive. Is that a bad idea?
PS: disabling c-states takes frame times to 11:30 - Tobit woz right.
PS: disabling c-states takes frame times to 11:30 - Tobit woz right.
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
I forgot - the whole interface is very laggy on Ubuntu - fedora and the others were snappy as anything.
CPU folding reports solid 2400%. Nary a dip in the graph.
EDIT: oops, no sooner I said that and a 10 second spike down of ragged mid frame - that was c-state disabled, going back to best settings and I will run the CPU mon the whole frame.
CPU folding reports solid 2400%. Nary a dip in the graph.
EDIT: oops, no sooner I said that and a 10 second spike down of ragged mid frame - that was c-state disabled, going back to best settings and I will run the CPU mon the whole frame.
The lag is probably from the fact that you are finally using 100% CPU. When you have a full core not being used, the UI is going to be faster. That being said, I am not seeing lag at all, on the 970 or the SR-2s
For the USB thing, in theory, it shouldn't matter. Mine are all on actual drives, though. Just looking for variables...
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Yes, the lag doesn't bother me so much - but it does happen under no CPU load which is interesting. But I did catch some CPU events - this is mid frame:
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
UPDATE: Fresh install of Musky Mix Ubuntu to a hard drive, not USB stick.
(Windows average p6901 - 11mins 22sec)
Ubuntu DFS kernal: p6901. Numa disabled. C-states enabled. Captured p6901 from Musky.
10:18
10:17
Benefit - 10.5% frame time = 16% better PPD
161,000 ppd up to 186,700 ppd. That is a fair bit better.
Testing continues..
(Windows average p6901 - 11mins 22sec)
Ubuntu DFS kernal: p6901. Numa disabled. C-states enabled. Captured p6901 from Musky.
10:18
10:17
Benefit - 10.5% frame time = 16% better PPD
161,000 ppd up to 186,700 ppd. That is a fair bit better.
Testing continues..
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Worth a shot - Thankfully The Folder Formerly Known As Tobit mentioned it, and my quick testing showed it does make a significant difference. (5% frame times, greater PPD)
As to WHY the hell an idle state should be even an issue when running all cores 100% for days on end... above my pay grade.
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Just plugged my new Ubuntu HDD install into my fastest rig SR2#3 - this is a bit different as it is the only rig running with turbo off - the x5680 has simply so much multi it is not needed.
At 4.3Ghz (205 x 21):
[11:18:08] Project: 6901 (Run 11, Clone 6, Gen 5)
[11:18:08]
[11:18:08] Assembly optimizations on if available.
[11:18:08] Entering M.D.
[11:18:15] Mapping NT from 24 to 24
[11:18:16] Completed 0 out of 250000 steps (0%)
[11:28:15] Completed 2500 out of 250000 steps (1%)
[11:38:13] Completed 5000 out of 250000 steps (2%)
[11:48:11] Completed 7500 out of 250000 steps (3%)
Gotta love cracking the 10 minute barrier - 9:58 per frame. = 195,700ppd.
If I can scrape another 9 seconds off I will crack the 200,000 ppd barrier.
This is about 12% better times than windows = 19% more PPD.
At 4.3Ghz (205 x 21):
[11:18:08] Project: 6901 (Run 11, Clone 6, Gen 5)
[11:18:08]
[11:18:08] Assembly optimizations on if available.
[11:18:08] Entering M.D.
[11:18:15] Mapping NT from 24 to 24
[11:18:16] Completed 0 out of 250000 steps (0%)
[11:28:15] Completed 2500 out of 250000 steps (1%)
[11:38:13] Completed 5000 out of 250000 steps (2%)
[11:48:11] Completed 7500 out of 250000 steps (3%)
Gotta love cracking the 10 minute barrier - 9:58 per frame. = 195,700ppd.
If I can scrape another 9 seconds off I will crack the 200,000 ppd barrier.
This is about 12% better times than windows = 19% more PPD.
Regular GenKernel. Tried out what you had written to Lord_of_Doors earlier here:
http://hardforum.com/showpost.php?p=1036931044&postcount=238
But I received an error saying "no /etc/portage/package.keywords" file which I found strange. Is it supposed to be in /etc/portage.... or /usr/portage....? I can't remember if there is an /etc/portage in the "live" or chrooted environment. That was like three days ago
and I'm over 40 and have a touch of ADHDHDDDDDD or whatever they call it. Can't remember So right now I'm using GenKernel. I might have to do a full re-install, or change my CFlags and reboot because the ~amd64 keyword keeps masking out the alternative kernel sources and I do remember taking their advice and using the "core2" keyword in CFlags instead of amd64 earlier on because these are Xeons.
Probably dumb on my part to use genkernel and core2 CFLags, but that's how one learns, and this is more exciting than my job right now which has me learning a 20 year O/S again. That's right, Novell
Just plugged my new Ubuntu HDD install into my fastest rig SR2#3 - this is a bit different as it is the only rig running with turbo off - the x5680 has simply so much multi it is not needed.
At 4.3Ghz (205 x 21):
[11:18:08] Project: 6901 (Run 11, Clone 6, Gen 5)
[11:18:08]
[11:18:08] Assembly optimizations on if available.
[11:18:08] Entering M.D.
[11:18:15] Mapping NT from 24 to 24
[11:18:16] Completed 0 out of 250000 steps (0%)
[11:28:15] Completed 2500 out of 250000 steps (1%)
[11:38:13] Completed 5000 out of 250000 steps (2%)
[11:48:11] Completed 7500 out of 250000 steps (3%)
Gotta love cracking the 10 minute barrier - 9:58 per frame. = 195,700ppd.
If I can scrape another 9 seconds off I will crack the 200,000 ppd barrier.
This is about 12% better times than windows = 19% more PPD.
Hey I run my x5650s without turbo. Is this NUMA on or off? What was it running in Windoze, 10:45 or something like that? Very nice. If you're running with BFS it might get a bit faster.
EDIT: My bad, you stated 11:22 up there. And NUMA still off. ZERO for TWO today
Nice improvement!
Last edited:
/etc/portage since it's a config file. /etc/portage doesn't exist by default so you have to create the directory first if it doesn't exist.But I received an error saying "no /etc/portage/package.keywords" file which I found strange. Is it supposed to be in /etc/portage.... or /usr/portage? I can't remember if there is an /etc/portage in the "live" or chrooted environment. That was like three days ago
Code:
mkdir /etc/portage
echo "sys-kernel/zen-sources ~amd64" >> /etc/portage/package.keywords/etc/portage since it's a config file. /etc/portage doesn't exist by default so you have to create the directory first if it doesn't exist.But I received an error saying "no /etc/portage/package.keywords" file which I found strange. Is it supposed to be in /etc/portage.... or /usr/portage? I can't remember if there is an /etc/portage in the "live" or chrooted environment. That was like three days ago
Code:
mkdir /etc/portage
echo "sys-kernel/zen-sources ~amd64" >> /etc/portage/package.keywordsThat is why you need to add the package.keyword shown above.
Ignore the fact that it says amd64. In the Linux world, amd64 applies to both AMD and Intel. As long as your cflags are correct,you are fine. Your CFLAGS should be:
Code:
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CXXFLAGS="${CFLAGS}"may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Hey I run my x5650s without turbo. Is this NUMA on or off? What was it running in Windoze, 10:45 or something like that? Very nice. If you're running with BFS it might get a bit faster.
EDIT: My bad, you stated 11:22 up there. And NUMA still off. ZERO for TWO today
Heh - too many distros/machines/configs. 11:22 was SR2#2... SR2#3 went from 11mins12sec to 9mins58sec.
Ubuntu Musky Mix - BFS kernel. NUMA off in BIOS, c states on.
WHY did nobody tell me BFS stands for http://en.wikipedia.org/wiki/Brain_Fuck_Scheduler!! Legendary.
Now how do I disable NUMA in this kernel do you think? Just to be sure.
Heh - too many distros/machines/configs. 11:22 was SR2#2... SR2#3 went from 11mins12sec to 9mins58sec.
Ubuntu Musky Mix - BFS kernel. NUMA off in BIOS, c states on.
WHY did nobody tell me BFS stands for http://en.wikipedia.org/wiki/Brain_Fuck_Scheduler!! Legendary.
Now how do I disable NUMA in this kernel do you think? Just to be sure.
What do you expect from somebody who sounds like he's of Greek descent? We're a twisted lot.
You'd have to recompile the kernel from source but, since you are running a pre-compiled binary, you don't have the source tree with the BFS patch. I'm not sure where to begin on telling you how to do it on Ubuntu. It's also easy to break if you don't know what you are doing and I don't think you'll gain anything based on other reports I've read.
may i be worthy
[H]ard|DCer of the Month - December 2010
- Joined
- Aug 17, 2010
- Messages
- 1,186
Agreed - in a good way. I am from Melbourne - the city with the third largest Greek population in the world.
Ok, good enough for me. Now that I think about it, I am pretty sure I am getting the benefit of NUMA disabled, so lets not go insane with power and FUBAR everything.
But I gotta say, this 19% better PPD in software tweaks is addictive.
/etc/portage since it's a config file. /etc/portage doesn't exist by default so you have to create the directory first if it doesn't exist.
That is why you need to add the package.keyword shown above.Code:mkdir /etc/portage echo "sys-kernel/zen-sources ~amd64" >> /etc/portage/package.keywords
Code:CHOST="x86_64-pc-linux-gnu" CFLAGS="-march=core2 -O2 -pipe" CXXFLAGS="${CFLAGS}"
Yeat that's what my CFLAGS = "" is set to. Thanks ToBit. I'll check it out. I didn't even think to just create it. Was a bit bleary eyed from reading the darn Handbook three times over.
Still not impressed with the gains on the SR-2 compared to SkankBox48 but this is fun.. so I'll continue down this path. Linux is fun. Haven't used any *nix skills since I became Solaris certified in 2000 LOLz.
Once you have a kernel with BFS, you will see better gains. When you configure the kernel, make sure you set the following:
General Setup ->
Select BFS
Zen-Tune Profile = Server
Processor Type and Features ->
Processor Family = Core 2/newer Xeon
Preemption Model = No Forced Preemption (Server)
Timer Frequency = 100 Hz.
I also do a bunch of stuff such as removing all drivers except for what I need, removing AMD support (since it's an Intel box), etc., etc. Just be careful and you'll be fine.
Thanks Tobit. The SR-2 will be set to fold for team 33 (at least until Chimp Challenge 2011) when I'm finished re-tweaking the kernel and adding your above parameters. Hopefully I'll see some gains.
I was hoping to get her close to 180K PPD, Pretty insane if you add it to the 280K PPD I'm getting from SB48. 460K PPD out of two boxes is pretty stupid.
When I'm done with that I'll try and see what I can get out of my hex cores with either Gentoo (Gentooobit mix) or Ubuntu Desktop using the Meerkat Musky cocktail
EDIT: Working on the kernel now. I mistakenly didn't add the -qt4 to the USE flags earlier so I had to add it to /etc/make.conf, then run "emerge --update --deep --newuse world" and then did an "emerge samba" to get rid of the qt4 dependency alerts. Dependencies can be painful but I think I'm getting the hang of this.
Neat stuff. I figured I'll learn the hardest distro so that the other ones are easy. Additionally because I'm running a "mini-linux" Android phone I'll learn something about that too
Last edited:
I hope I can get rhel5 working with the ck patches...
no installer like for ubuntu... oh boy... lol
I am curious as to what improvements I will get... as I didn't really have any performance problems like musky ...
no installer like for ubuntu... oh boy... lol
I am curious as to what improvements I will get... as I didn't really have any performance problems like musky ...
Not you too! I always wondered about my fellow Canadian with the handle and talents of a Greek God!!!!!!!!!!!!!!!!
LOLz.
LOLz.
You got that right...
Seems decent to me. It's faster than I ever got on my Windows based 4.1Ghz i7 920 with RAM at 1624 and 8-8-8-24-1T speeds and GPUs OFF, by nearly two minutes time-per-frame with -smp 8