Ideas for new point system

P

perrosnk

n00b
Joined
May 2, 2013
Messages
4
Hello everyone!
I am very new at the distributed computing world.
I was wondering if there is any way you can think of, to make a unified point system. For example, is there a way of real time measurement of the CPU or GPU work done, in flops or whatever else, and then get points based on flops or any other measure?
Is there a way of ensuring that nobody will be able to cheat the system?
 
There is always some way to cheat any system. Just give people time and they will exploit it.
 
Maybe I'm missing your point perrosnk. I get using encryption, but that doesn't fix cheating. It is only one piece of it. For example... BOINC runs benchmarks on your CPU so that it knows how to claim credit. These numbers can shift wildly from system to system and depending on what else is running in the background. Someone could/can manipulate this which throws off potential points. (this is the dumbed down version obviously) Now, since BOINC and FAH are two different worlds in the DC universe, I'm pretty sure you get what I'm saying. So, there needs to be several parts of the equation addressed.
 
Last edited:
I am not sure how one could cheat, you need to process the Wu which takes time and is validated at standford prior to getting points. Perhaps I am missing something?
 
Gilthanis said:
For example... BOINC runs benchmarks on your CPU so that it knows how to claim credit.
Click to expand...

A fundamental piece that FAH is missing. Stanford makes up for that by punishing all because of the few that do cheat.
 
What kind of cheating are we talking about?

Spoofing your system to get work units you are not supposed to obtain?

You only get points when the WU is completed. What am I missing?
 
I think the suggestion was about shifting performance measurement to the client [?]

That would never work. Any metric taken at client's end can be tampered with.

FAH has been using work-based rewards for quite some time and nothing better
has been invented yet...
 
I agree with scotty and Pocatello; I don't think there is any cheating because PG gives points to only completed and validated work. It's theoretically possible to fool them but I think that would be possible only if you're a pissed-off ex-PG member :D jk

I think PG's approach is the best here. Not only because it's easy to control and validate, it is also fair. Well, assuming the points benchmarking system is fair.

So, if your question was about a fair benchmarking system. That's a big, and endless issue to solve. I believe PG is doing their best in terms of that, given their limited manpower and monetary resources. To be honest, I rather have them work on actually curing the diseases, than finding that optimum points system to appease the ever-unhappy group of folders...
 
Any of the various DC projects currently running have a performance measurement that their points are based off of ..... and short of hacking databases I dont think there is a way to "cheat" the system for points...... So what are you really after here mate?
 
Except that F@H pulls numbers out of their @$$ and says this is worth this much. I personally quit folding F@H for that reason. If they said a flop was worth 1 point didn't matter if it was a cpu or gpu that processed the flop, then we would have an even based system. Now you as the folder could choose to either run high wattage systems or low wattage systems, but you atleast know you are getting 1 point per 1 flop.

But right now F@H at home says I will give you 100 point per flop if you run Nvidia gpu. If you run AMD gpu you get 25 points per flop. If you run multi cpu we will give you 50 points per flop. Single cpu system will net you 4 points per flop.

Now you can say but this uses more power or whatever. The point is they have no standard and are so f 'd up it is not funny.
 
dcds1 said:
Except that F@H pulls numbers out of their @$$ and says this is worth this much. I personally quit folding F@H for that reason. If they said a flop was worth 1 point didn't matter if it was a cpu or gpu that processed the flop, then we would have an even based system. Now you as the folder could choose to either run high wattage systems or low wattage systems, but you atleast know you are getting 1 point per 1 flop.

But right now F@H at home says I will give you 100 point per flop if you run Nvidia gpu. If you run AMD gpu you get 25 points per flop. If you run multi cpu we will give you 50 points per flop. Single cpu system will net you 4 points per flop.

Now you can say but this uses more power or whatever. The point is they have no standard and are so f 'd up it is not funny.
Click to expand...

As they've said many times, raw FLOPs do not tell the whole story. Until the beta core 17, GPUs could only do implicit solvent calculations, which are less scientifically valuable to them than explicit solvent calculations, which CPUs have been able to do for a while. Hence, GPUs earned fewer points per FLOP than CPUs. With the new core, which allows for explicit GPU work, this imbalance should be rectified.

There's also the issue of not all FLOPs being equal. An Nvidia GPU may be able to do a certain complex calculation in 2 cycles, while it takes an AMD GPU 4 and a CPU 15. That's going to throw a wrench in 1 FLOP = x points calculations, and further explains why they don't base the points solely on that.

I don't think they'd disagree that their prior AMD GPU cores were rather underperforming, though. The new core really improves on that front and allows for much more accurate apples-to-apples comparisons and compensation.
 
Ok guys, this is NOT a discussion on the merits of the current F@H point scheme. Lets keep it away from the flames there ;)
 
"shifting performance measurement to the client" is exactly what I meant.
Is there any possible way you could imagine of doing so?
 
I don't know about FAH's client, but that is how BOINC kind of does it. Unfortunately, this allows cheaters to run modified clients that basically lie about the performance. If someone modifies the client, they can trick it to try and claim much higher then it deserves. Each BOINC project handles their points differently based on their thoughts and opinions and try to catch those manipulating the system. Some do a good job and some don't. So, it would be a bad idea to let the peasants have more control over such matters.
 
Well in the future they plan to include a benchmark tool inside the client, but this will only be used to determine what WUs you get.

And like others have said each FLOP doesn't gain a equal amount of science.

Oh and the old GPU cores were a complete mess, that has been fixed in core 17.
 
Stop feeding the troll...

Ask yourself, what problem is it proposing to fix?

Why would Stanford ever use a metric that is not explicitly tied to actual production?

Let the discussion die, or move to one of the other forums where he (as a 1 day old user) posted the same question.
 
perrosnk said:
"shifting performance measurement to the client" is exactly what I meant.
Is there any possible way you could imagine of doing so?
Click to expand...
Without ensuring the client-side-score can't be tampered with I can't see it working...

In other words, write any benchmarking program, I'll find a way to make it produce _arbitrary_ scores.

One would need to resort to Trusted Computing concepts to make it _really_ work.

But, even so!
Imagine we have guaranteed that client-side-score is accurate. What good does it really do?

MD simulation software tends to utilize certain hardware platforms better than others.
How do we address that?

You may have a ride with 500hp motor. That sure is powerful.
Though all that power is of little use if you have open diff and shitty rubber :)
 
Quisarious said:
Stop feeding the troll...

Ask yourself, what problem is it proposing to fix?

Why would Stanford ever use a metric that is not explicitly tied to actual production?

Let the discussion die, or move to one of the other forums where he (as a 1 day old user) posted the same question.
Click to expand...

That's very rude, and cannot find a reason why you said that.

Anyway, I did ask the same question on another forum in order to increase the chance of getting a solution. To which problem?

I believe that all projects on boinc and the fah have the same scientific value. I hope you would agree on that. But till now there is no way to compare a user's contribution between all these projects.
 
perrosnk said:
I believe that all projects on boinc and the fah have the same scientific value.
Click to expand...
1) Opinions on relative values between any projects vary greatly.
2) Opinion != Fact.

tear said:
You may have a ride with 500hp motor. That sure is powerful.
Though all that power is of little use if you have open diff and shitty rubber :)
Click to expand...
Mmmm...burnout...

http://www.youtube.com/watch?v=unwujSVpZwA

Of course, 2,200 HP (2,800 w/Nitrous) and 5,600 ft/lb of torque render the locker/rubber irrelevant.:D

http://www.youtube.com/watch?v=-mB8Fj-ZO2Q
 
Last edited:
perrosnk said:
That's very rude, and cannot find a reason why you said that.

Anyway, I did ask the same question on another forum in order to increase the chance of getting a solution. To which problem?

I believe that all projects on boinc and the fah have the same scientific value. I hope you would agree on that. But till now there is no way to compare a user's contribution between all these projects.
Click to expand...
Are you saying that you want boinc and F@H to share the same point system?
If so you need to complain to boinc because F@H was here first.
Also all the different DC apps are run by different groups and they have better things to do than to make sure their point system matches up with other DC groups.

Also scientific worth is in the eye of the beholder.
I personally will not waste one thin dime of electricity on SETI.
I suspect the reaction you are getting from some people is (me included) is that you are a noob and come in and want to reinvent the wheel.
We are not the people who set the point system, so you are harping at the wrong people.
You need to e-mail Pande group and the heads of the other groups with your concerns.
 
Last edited:
I see nothing wrong in discussion per se...

But I guess important question was raised -- what is the "problem" we're trying to address?

A "fair" point system *within* FAH? A hypothetical unification of BOINC+FAH? Something else?
 
They can't unify the BOINC projects, what makes you think they could compromise with FAH? And FAH tried the BOINC client in its early days and decided to go another way.
 
Hehe Boink has never had a fair / even points system within itself nor do I think it ever will. Hell want to do a comparrison of Boinc just run World Comunity Grid and see how many points you get on some of the projects they run within the Boinc system, then run the same projects within the Boink system and you will be credited allot less points than what WCG gives you.

People regularlly switch projects within the Boinc system depending upon the PPD given for a particular project and a given hack that can increase that projects PPD. I would not ever expect F@H to join in with Boinc in a points system. There would be no reason or incentive to do so while some believe F@H point system is messed up IMHO it is by far the best system out there. It clearley works the way it was desighned to, if you want more people to fold a given project you just value it at xx points if that value is high enough the people will choose to invest in the equipment to run it if it is not they will choose not to. (Simple and efficent) that is all that is needed. ;)
 
AFAIK, the points system at FAH reflects the kind of H/W they need running. They give big points for the stuff they need more of, and lesser points for stuff they have plenty of.

Or at least that's how it SHOULD work.

It's the only signal we have as to how can help the most.
 
I apologize if I was rude. Since you didn't articulate what problem you saw, and you posted the exact same post on FoldingForum (which has nothing to do with BOINC), I assumed you were talking about FAH.

There are are a few FAH donors that have made it their life's work to complain about the FAH point system, and the vagueness of your post made me think you were just trying to stir the pot.
 
Grandpa_01 said:
Hehe Boink has never had a fair / even points system within itself nor do I think it ever will. Hell want to do a comparrison of Boinc just run World Comunity Grid and see how many points you get on some of the projects they run within the Boinc system, then run the same projects within the Boink system and you will be credited allot less points than what WCG gives you.

People regularlly switch projects within the Boinc system depending upon the PPD given for a particular project and a given hack that can increase that projects PPD. I would not ever expect F@H to join in with Boinc in a points system. There would be no reason or incentive to do so while some believe F@H point system is messed up IMHO it is by far the best system out there. It clearley works the way it was desighned to, if you want more people to fold a given project you just value it at xx points if that value is high enough the people will choose to invest in the equipment to run it if it is not they will choose not to. (Simple and efficent) that is all that is needed. ;)
Click to expand...

Different projects have different points given to them, but at least the work in the projects give the same points. I have confirmed this from running an E-350 all the way to an 1075T. All I care about is that a job I do pays the same no matter if I have a computer that is running slower or faster.
I ended up stopping F@H when I noticed that points given for a same work unit were different depending on the system, example would be a work unit from 6709. Now just because my daughters 955 @ 4Ghz took 30 min to an hour longer to complete the same work unit that my 1075T at 3.3 Ghz would, there is no reason to award less points. And no I am not talking ppd but actual points for the work unit.
And as for Boinc projects giving different points for work unit, F@H is no different. Show me any 2 projects in F@H that gives the exact same points. The points very all over the place.
 
QRB is what your complaining about

it's not going to go away
Stanford has a reason to award quick return bonus to WU's
the faster it gets done the quicker it can dole out new work and the sooner a project research gets done

you have to remember their researchers have time limits and they have competitors (others doing similar research) so the faster they can get it done the better
you don't want another researcher getting credit before you or to run out of time before your research grant runs out run out of time for your doctorate thesis to be submitted

Also awarding quick return rewards those with better equipment and encourages those with older equipment to invest in new hardware

if you want to argue over how QRB and some projects get awarded more than others - that is up to stanford and the research they do - which project is more important to them ... the more important -more points
 
QRB, just like in real life.

Who is worth more and deserving of decent pay:
A plumber who fixes your toilet in a few hours
A plumber who takes a week to fix your toilet

I know which one I would pay more.
 
IMHO, points are nothing more than a digital carrot for us mules to chase. They have no value or use for the donor. They are simply used to create competition between users and teams in order to produce more and faster results.

Sometimes there is a big carrot on the end of the stick; or if you are known to produce faster results, you get two or three bonus carrots at the end for a heavy load. As long as the points are relatively consistent for equal work, it is all good. It is only when someone gets a small carrot and someone else gets big carrot for the same load and completion time that things get unsettled.

I would prefer the scientists stay focused on the science, but sometimes wish they had some intern doing better benchmarking for some of the projects to keep the mules content.
 
You must log in or register to reply here.
Back
Top