P2686 problems

[APOLLO]

One of my -bigadv machines is having problems with P2686 WUs. I had 2 of these WUs crash without apparent reason, even at stock frequencies. I can't continue running -bigadv on this machine without identifying the problem. It only seems to affect this particular WU and no other. Anyone heard about a specific -bigadv WU series that has weird issues?

 

[may i be worthy]

nope. Can you capture it? Once the dust settles I can run it with asknet=no and see if crashes for me.

But given that I can think of nothing that would make me happier right now than to wake tomorrow to see two 2686's running, this would be a horror outcome.

EDIT: actually early P2682 had issues did they not?

 

[APOLLO]

I never had problems with the A2 -bigadv WUs except for the A2 version of the P2684 that everyone experienced problems with. As far as the P2686 is concerned, I no longer have the WU and reverted to standard SMP until I get an idea what is causing the crashes. It could just be this particular run of the project.

 

[may i be worthy]

(Sent you a PM with a test WU...)

This is not just since the NUMA disabled trick is it?

 

[sirmonkey1985]

nope. Can you capture it? Once the dust settles I can run it with asknet=no and see if crashes for me.

But given that I can think of nothing that would make me happier right now than to wake tomorrow to see two 2686's running, this would be a horror outcome.

EDIT: actually early P2682 had issues did they not?

yeah the 2682's had problems.. they were taken down, reworked and turned into 2692 or 89 or something like that..

 

[zero2dash]

I've had a few problems lately, I don't believe it's WU specific but I've had several "client-core communications error: error 0xc0000029" crash and flushes on a bigadv or two, I believe a 2685, and a few regular a3's as well.

After googling it I just went ahead and fresh installed smp2 and so far it's been fine, so it could have been some carry over from running the same exe setup from the "old machine" (i7) to my SR-2.

 

[musky]

Every issue I have had recently has been related to memory problems. That would be where I would start.

 

[may i be worthy]

I pranged my test 2686 unit with the "client-core communications error" mentioned above, but that was because I was faffing around testing at 4.4Ghz

So it can be an overclock problem as well as a memory problem. But more likely a memory issue, as Musky said.

 

[amdgamer]

One of my -bigadv machines is having problems with P2686 WUs. I had 2 of these WUs crash without apparent reason, even at stock frequencies. I can't continue running -bigadv on this machine without identifying the problem. It only seems to affect this particular WU and no other. Anyone heard about a specific -bigadv WU series that has weird issues?

2686's have been dead solid for me. Given how rare bigadv has become lately, I wonder whether there are a few bad ones that have been released recently. Given your luck of drawing a disproportionate percentage of 2684's, maybe you are a magnet for this type of stuff.

yeah the 2682's had problems.. they were taken down, reworked and turned into 2692 or 89 or something like that..

The 2682's were actually pretty solid. From what I understand, they were A2 work units and the few that were left after Linux bigadv was discontinued were switched over to work with the A3 core and renamed 2692.

 

[musky]

The 2682's were actually pretty solid. From what I understand, they were A2 work units and the few that were left after Linux bigadv was discontinued were switched over to work with the A3 core and renamed 2692.

They were re-worked when they were renamed 2692. The original 2682 under A3 had serious issues with 24 cores and memory requirements.

 

[amdgamer]

They were re-worked when they were renamed 2692. The original 2682 under A3 had serious issues with 24 cores and memory requirements.

Ahh. I didn't realize that. I didn't even know that the 2682's were there for A3's as I don't think I managed to draw any of the A3 ones before Linux bigadv got axed.

 

[SazanEyes]

I'd look at memory issues too. Try running LinX with as much memory as you can, and see if it's successful. I have no issues with P2686s.

 

[10e]

They were re-worked when they were renamed 2692. The original 2682 under A3 had serious issues with 24 cores and memory requirements.

Tear proved anything above 16 threads tried to allocate more than 3GB of memory, and due to F@H being a 32-bit binary, would basically run out of memory regardless of how much RAM you had, or whether you were running 64-bit O/S..

Strangely enough, the original A3 2682s never failed on my Opteron 6168 duallie box. Don't ask me why.

 

[APOLLO]

It's not memory issues, it's the WU itself. I received another P2686 on a different machine that crashed. So, 2 different systems proven stable since the inception of the A3 core happen to crash on the same WU project? I'm not touching these systems until I get another project to determine if the issue is more widespread.

WU details: Project 2686, Run 3, Clone 14, Gen 23

 

[Tobit]

Apollo, you mentioned receiving at least two bad 2686 WUs. Do they both have the same trajectory (same run/clone/gen) or are they both unique?

It might be worth reporting them on the foldingforums to see if others have completed them successfully. If you don't want to do it, PM me and I can do it on your behalf.

 

[APOLLO]

Apollo, you mentioned receiving at least two bad 2686 WUs. Do they both have the same trajectory (same run/clone/gen) or are they both unique?

I think they were very similar, but not entirely sure. I was intending to go through the logs later this evening and find the exact RCG of the problem WUs.

It might be worth reporting them on the foldingforums to see if others have completed them successfully. If you don't want to do it, PM me and I can do it on your behalf.

I don't post at the FF or any other DC forum anymore but this one. I'll let you know as soon as I find out.

 

[lehmann]

funny you metion that i had one crash yesterday as well but figured it was my fault too much OC. but the machine had been rock solid.

 

[APOLLO]

I finally went through the logs. I was a bit surprised that the failed WU I first encountered was the same WU that both systems received on different days: Project 2686, Run 3, Clone 14, Gen 23. This WU crashed at 0% no matter how many times I restarted it on either system.

The second WU that crashed is a different RCG. This WU failed at 11%: Project 2686, Run 3, Clone 7, Gen 26.

I'll reemphasize that both systems are rock stable and have been since A3 SMP commenced. I've had no problems with any other -bigadv WUs or regular SMP for that matter. These WUs crashed even at stock frequencies.

 

[may i be worthy]

Thanks for posting that - I guess the blessing is that they failed very early rather than after absorbing a day of CPU time.

I was grumpy at having 5 of 7 bigadvs since the drought broke being 2684s, but maybe that isn't so bad.

 

[APOLLO]

Thanks for posting that - I guess the blessing is that they failed very early rather than after absorbing a day of CPU time..

This happened several times altogether. Each time it occurred the system went idle for many, many hours, and two systems were affected by bad WUs. Total estimated lost processing time calculated for both systems due to crashed WUs is at least a full day. The crash I spotted today alone caused the machine affected a minimum 12 hours of processing time. This is ending a pretty bad week for me. The system that experienced the most crashes probably hasn't uploaded a -bigadv WU in what must be close to 5 days, if not longer.

BTW, thanks for sending me the file. I'm going to test one of my systems anyway sometime this weekend. Couldn't hurt to see how it does.

 

[SazanEyes]

I haven't had any problems with P2686 on either my SR-2 (with either set of Xeons) or i7 980X box. The most recent one completed was Run 4, Clone 13, Gen 31.

When WUs fail immediately, it's usually some sort of incompatibility with the system. As described above, the P2682s would fail immediately on dual-hex systems because the client couldn't allocate enough RAM for all of the threads. What are the specs of your systems that aren't working, as far as CPU, amount of RAM, and Windows version (32/64bit)? Maybe they have something in common with each other that's different from the rest of us with no issues.

 

[Tobit]

Apollo, I've heard back from the moderator regarding the WU that failed twice at 0% - he has flagged it as a bad WU.

Thanks for the report. Counting Apollo's experience plus the two EUEs in the database, I'll call it a bad one.

The WU (P2686,R3,C14,G23) has been reported as a bad WU.

 

[APOLLO]

Thanks, Tobit, it is very much appreciated.

 

[APOLLO]

What are the specs of your systems that aren't working, as far as CPU, amount of RAM, and Windows version (32/64bit)? Maybe they have something in common with each other that's different from the rest of us with no issues.

They are both Core2 dual Harpertown systems running Win 7 64-bit with 4 GB of RAM. Since it's the same RCG, it's unlikely to be another kind of issue such as architecture. I never encountered WU failures with a CPU client due to architecture differences from other systems. Both machines have been stable and both have completed P2686 WUs prior to the crashes.

 

[amdgamer]

Wow, so you really are a magnet for bad and pesky work units!