Bigadv SMP Client Slowdown ???

[tjmagneto]

For the second time now my L5640 decided to take a more leisurely pace when folding a bigadv. Sometime during the middle of a 6900 the frame times jumped to over 30 minutes per frame instead of 23 minutes per frame it cruises along at.

I checked task manager and the CPU utilization was at 9% ?????? It gets even better as for whatever reason when I restarted the client I noticed in the messages the log reported -np 0. Huh?

During my troubleshooting I noticed that Windows Update was downloading updates w/o installing so I changed it to notify me. Another thing I changed was putting the additional client parameters back into the configonly in stead of passing them in from a shortcut which had worked fine in the past. Also I'm wondering if an unstable overclock would cause this? I guess if all else fails I can re-install the client. Any other ideas?

 

[may i be worthy]

I have one machine that needed more vcore - occasionally a bigadv would result in long frame/hung frame. Bizarrely this would often happen after a cool change, rain, and high humidity!

Once I raised vcore a few notches problem solved.

How are things now - and was the frame time increase consistent or random?

 

[TheMTZ]

I also got this weird PPD change sometimes. WU's start with like 45k PPD and then finish at around 39k PPD. Temperature and humidity is no issue because this box is running in a dedicated server room with AC and all that fancy stuff.

 

[tjmagneto]

I have one machine that needed more vcore - occasionally a bigadv would result in long frame/hung frame. Bizarrely this would often happen after a cool change, rain, and high humidity!

Once I raised vcore a few notches problem solved.

How are things now - and was the frame time increase consistent or random?

Things seem fine for now. The WU finished without a problem but I took a 12k point hit when it was turned in. Currently it's working on a 2685 and frame times look appropriate. I also kicked up the vcore a notch.

Funny you should mention the weather as both incidents happened with a weather fluctuation.

 

[Patriot]

Things seem fine for now. The WU finished without a problem but I took a 12k point hit when it was turned in. Currently it's working on a 2685 and frame times look appropriate. I also kicked up the vcore a notch.

Funny you should mention the weather as both incidents happened with a weather fluctuation.

you on a ups... I have had trouble with overclocks at lans when I was too close to the edge... the minor difference in power at the wall can make a difference... storms can do that...

 

[tjmagneto]

you on a ups... I have had trouble with overclocks at lans when I was too close to the edge... the minor difference in power at the wall can make a difference... storms can do that...

All my computers are on UPS.

 

[may i be worthy]

you on a ups... I have had trouble with overclocks at lans when I was too close to the edge... the minor difference in power at the wall can make a difference... storms can do that...

I never thought of that... fascinating.

My issues were never during proper storms, just cool changes and a bit of rain. It is puzzling becuase the added humidity and dropped temps make air cooling easier, thus stress on the system lower.

I never thought about the implications to the power grid supplying the juice...

 

[tjmagneto]

I think I'll go back into my logs and see if there are any sort of pattern involving the two slowdown events.

 

[tjmagneto]

After review of the logs I think I confused the thing after a reboot. It says -np 0 (not 12) and then maps the number of threads from 1 to 1 (again not 12). This happened after a shutdown of the client and a reboot.

[14:30:34] - Calling '.\FahCore_a5.exe -dir work/ -nice 19 -suffix 03 [B]-np 0[/B] -checkpoint 15 -verbose -lifeline 896 -version 634'

[14:30:34] 
[14:30:34] *------------------------------*
[14:30:34] Folding@Home Gromacs SMP Core
[14:30:34] Version 2.27 (Mar 12, 2010)
[14:30:34] 
[14:30:34] Preparing to commence simulation
[14:30:34] - Ensuring status. Please wait.
[14:30:44] - Looking at optimizations...
[14:30:44] - Working with standard loops on this execution.
[14:30:44] - Previous termination of core was improper.
[14:30:44] - Files status OK
[14:30:48] - Expanded 26616012 -> 32949201 (decompressed 123.7 percent)
[14:30:48] Called DecompressByteArray: compressed_data_size=26616012 data_size=32949201, decompressed_data_size=32949201 diff=0
[14:30:49] - Digital signature verified
[14:30:49] 
[14:30:49] Project: 2685 (Run 1, Clone 15, Gen 78)
[14:30:49] 
[14:30:49] Entering M.D.
[14:30:55] Using Gromacs checkpoints
[14:30:56] [B]Mapping NT from 1 to 1 [/B]

 

[DeFex]

Windows was probably doing stuff, if one cpu takes longer to do its part all the other ones have to wait for it. Frozen winamp instances are the worst!

 

[Patriot]

that is bizarre

 

[APOLLO]

Ok, I'm having this problem on one machine and the only thing that seems to be helping for a while is stopping and restarting the client. The WU is a P2685 using the A5 core. Never seen anything like this and the only thing this machine is doing besides -bigadv is a Fermi GPU client.

 

[tjmagneto]

It happened with a 2685 and a 6900 for me. For a while several frames took 3+ hours to complete.

Does your FAHlog show the "-np 0" and "Mapping NT from 1 to 1" as in my earlier posting?

 

[APOLLO]

It happened with a 2685 and a 6900 for me. For a while several frames took 3+ hours to complete.

Seems like a corrupt series of WUs to me. I had a couple of frames over an hour but not that long.

Does your FAHlog show the "-np 0" and "Mapping NT from 1 to 1" as in my earlier posting?

No, I couldn't find anything like that in the log. On second look, yeah sort of. I found a few entries where it reported "Mapping NT from 1 to 1" but not the first part.

 

[tjmagneto]

I found a few entries where it reported "Mapping NT from 1 to 1" but not the first part.

The "Mapping NT from 1 to 1" means only one thread is folding. The log for my L5640 are currently showing "Mapping NT from 12 to 12" for the last and current WUs so everything appears fine for now. Why this even happened is still beyond me so I'm still keeping an eye on it.