Points System for SMP2 work units

[Tobit]

Dr. Kasson has published information about the new points system for the upcoming SMP2 work units:

The SMP2 Core A3 work units mark the debut of a new points system. We have been testing the key element of this system--early completion bonuses--in the bigadv work unit program. Please refer to this document with for a more detailed explanation of the points system. We are also changing our benchmark system over to a Core i5. Points have been calibrated against previous benchmarking setups, as described below.

Introduction
Points are a key aspect of distributed computing, such as Folding@home (FAH), as it both works to indicate to donors how much they have contributed, as well as foster friendly competition between donors that has always been an essential part of distributed computing. Folding@home’s point system is based on the concept of a benchmark machine, i.e. a particular class of hardware which we use as a standard to define how many points a given calculation should get. The choice of this benchmark machine can have implications for points for donors. Moreover, how we use this benchmark is important.

Our benchmarking philosophy tries to balance two elements: keeping a system reasonably simple (both for donors and for the FAH team to calculate) as well as keeping points in alignment with scientific value of a given calculation. Indeed, donors will optimize their machines (eg choice of hardware, choice of clients, etc) based on points, so it is important that points awarded be reflective of the scientific gain.

While our basic benchmark idea is pretty simple, this document is fairly long in order to give donors full details about how we have chosen the benchmark machine as well as giving detailed information of this machine and how this could impact points for donors.

Benchmark philosophy
Our philosophy is pretty simple: we would like to standardize benchmarks to a single machine and standardize and simplify the bonus schemes now employed. Bonuses have played a key role in aligning points with science and we will continue to use them. For example, returning work units (WUs) promptly can be very important for the science we’re doing, so we provide bonuses for this, especially with the high performance clients.

Machines used in comparison
We chose a 2.2 GHz E6600 as the prototype dual-core machine and a Q6600 at either 2.4 GHz or 3.2 GHz as the prototype quad-core.
The new benchmark machine is a Core i5-750 with Turbo Mode off. We compare single-core performance to the old benchmark machine, a 2.8 GHz Pentium 4.

FAH Projects used in the comparison
We base comparisons to the single-core benchmark machine on projects 4442 and 6315, comparing single-core speed on the 2.8 Ghz Pentium 4 to ideal quad-core speed on the 2.6 Ghz Core i5 machine.
We base comparisons to quad-core machines on project 2671.
We base comparisons to dual-core machines on project 6012.

Results.
Machine: Performance relative to Core i5:
P4 2.8 0.098 (on project 4442)
P4 2.8 0.12 (on project 6315)
E6600 0.30
Q6600-3.2 1.1
Q6600-2.4 0.82

Based on these multiplicative speed factors, we can project ppd output based on either the A1 or the A2 benchmarking standards.
Machine: A1 ppd: A2 ppd:
E6600 521 1663
Q6600-3.2 1933 6172
Q6600-2.4 1450 4629


Bonus point formula
Briefly summarizing our bonus formula, the bonus is applied for users who have a passkey, have successfully returned at least 10 bonus-eligible WU's, successfully return >=80% of assigned WU's, and return the WU before the preferred deadline. Bonus points do not apply to partial returns.

Our bonus formula calculates final points as follows:
final_points = base_points * max(1,sqrt(k*deadline_length/elapsed_time))
Note that the max(1,...) ensures that final_points are never lower than base_points.

We can convert this formula to points per day as follows:
ppd = base_ppd * speed_ratio * max(1,sqrt(x*speed_ratio)),
where speed_ratio is the machine speed relative to the Core i5, and x = k * deadline_length.

Parameter determination
If we set the new quad-core base ppd to 1024 and the parameter x to 30, we get the following results:

Machine: projected ppd:
E6600 903 (greater than A1, less than A2)
Q6600-3.2 6456 (greater than A2)
Q6600-2.4 4628 (approximately equal to A2)
P4 171 (on project 4442)
P4 228 (on project 6315)

Explanation of x parameter
We may vary the deadline length between projects (some projects require fast completion and thus have short deadlines). Each project has an associated k parameter that controls the bonus points yield. We standardize k as follows:
x * speed_ratio = k * deadline_length / elapsed_time
since we can express speed_ratio as Core_i5_time / elapsed_time:
x * Core_i5_time / elapsed time = k * deadline_length / elapsed_time
therefore:
x *Core_i5_time = k * deadline_length
solving for k, we obtain:
k = x * Core_i5_time / deadline_length
and since x is set to 30,
k = 30 * Core_i5_time / deadline_length, where Core_i5_time is the time to complete a work unit on our Core i5 benchmark machine.

Summary
According to our projections, this new benchmarking standard will result in points yield for a 2.8 GHz P4 that is slightly above the typical uniprocessor values, points yield for a 2.2 GHz E6600 that is greater than typical A1 core yields but less than typical A2 core yields, points yield for a 3.2 GHz Q6600 that is greater than typical A2 yields, and additional points yield rewards for faster systems. The crossover point between A3 and A2 ppd in speed falls approximately at a 2.4 GHz Q6600.

 

[Elledan]

I love ze math~

 

[404 - File not found]

they lost me on the second part lol

 

[Zero82z]

Summary: If you want to max your PPD, run a2 workunits on dual-core CPUs, and run a3 units on quad-cores and up.

 

[EvilAlchemist]

Summary: If you want to max your PPD, run a2 workunits on dual-core CPUs, and run a3 units on quad-cores and up.

I am getting more PPD with SMP2 A3 on my dual cores.

SMP1 (A1 & A2) will be gone soon after A3 is released.

 

[Tobit]

A1 will be gone first, A2 will probably hang on for a bit longer but will ultimately be going away as well. There is no -bigadv yet for A3 and -bigadv is very useful to Stanford right now.

 

[Zero82z]

I am getting more PPD with SMP2 A3 on my dual cores.

SMP1 (A1 & A2) will be gone soon after A3 is released.

Based on Kasson's benchmarks it looks like dual-cores don't do as well with SMP3. But I trust your numbers .

 

[EvilAlchemist]

I am not sure where he got that dual core number to be honest.
His E6600 is running at 2.4Ghz and mine are slightly higher clocked.

That small Mhz difference does make the bonus play in my favor but i would not think that much.

Both my Laptop and my WHS both are getting 2000 PPD - have been for weeks

T9300 45nm @ 2.5 Ghz & Xeon 3075 65nm @ 2.66 Ghz

 

[Zero82z]

So about the same as a2 units then?

 

[EvilAlchemist]

So about the same as a2 units then?

From what I remember, A2 got slightly less in native linux vs windows A3 but to make the answer easier, yes about the same. (Been so long since I had it configured like that)

 

[EvilAlchemist]

2nd Reminder - Your 3rd Party Software needs to be able to calcuate K factor

FAHmon does not currently support this.
HFM.net and FAHspy (most current verions) do.

Refer to release noted to enable SMP2 ppd calcuations on your software.

 

[Elledan]

Will FahMon be updated soon?

 

[Tobit]

Doubtful, the last update was in June.

 

[Elledan]

Doubtful, the last update was in June.

Figures. I installed FahSpy earlier and will probably keep both applications running for the time being. I must admit to liking FahMon more in some respects (UI mostly)

 

[zero2dash]

A1 will be gone first, A2 will probably hang on for a bit longer but will ultimately be going away as well. There is no -bigadv yet for A3 and -bigadv is very useful to Stanford right now.

That's all I needed to hear. Sticking with VM -bigadv then myself.

Figures. I installed FahSpy earlier and will probably keep both applications running for the time being. I must admit to liking FahMon more in some respects (UI mostly)

Same here. FahMon is a nice looking little app. I've tried HFM.net and FahSpy and neither is as visually appealing as FahMon (despite FahMon not showing bonuses); I keep going back to it and uninstalling the others.

 

[Elledan]

Same here. FahMon is a nice looking little app. I've tried HFM.net and FahSpy and neither is as visually appealing as FahMon (despite FahMon not showing bonuses); I keep going back to it and uninstalling the others.

Yeah, FahSpy just looks primitive and unfinished whereas HFM.net is just very slow with UI updates (.Net app?) which makes it a pain to use.

Also, HFM.net gives an unknown core for two out of three GPU clients, even though they run the same cores. This is stupid >.>

 

[Tobit]

Unknowns = new/beta project it doesn't already know about because it is not listed on psummary. It isn't a HFM.net problem.

 

[Elledan]

Yeah, it works now for one more GPU at least. Thanks for informing me, wasn't aware there were different psummary files.

Got a P5784 on the second 9800GX2 core still listed as 'Unknown client type'.

 

[The_Heretic]

I glanced at mine this morning and all the GPUs were running "unknown"

 

[Tobit]

Got a P5784 on the second 9800GX2 core still listed as 'Unknown client type'.

P5784 or P5748? If the later it's because P5748 doesn't exist on any psummary page. This has been pointed out to the developer of the project for over a week now but it has not been addressed yet.

 

[Elledan]

That's because P5784 doesn't exist on any psummary page. This has been pointed out to the developer of the project for over a week now but it has not been addressed yet.

Ah okay, didn't know. Sorry