What is "aggressive discarding?"

[rflcptr]

I read the Storage@home: Petascale Distributed Storage pdf, and had a question concerning folding@home.

"For example the Folding@home distributed computing projectis currently running over 210 TFLOPS (actual) and even with the aggressive discarding and compression of generated data, it has resulted in 150 terabytes (TB) of data. This is growing at over 2 TB per month and all this data needs backup and some limited distribution."

http://www.stanford.edu/~beberg/Storage@home2007.pdf

What do they mean by "agressive discarding?" How much data is thrown out? What qualifies this data as unnecessary or unusable?

 

[Pocatello]

I've never heard of that term.

My friend Google was not much help.

 

[DriveEuro]

Isn't that when folders delete their work folder and queue.dat file to get a new WU? This was a big issue back in 05 or so with the DGGromacs

 

[Xilikon]

I'm thinking it has more to do with the Stanford staff discarding useless generated data from users. Agressive discarding : discarding more than normal but even with that, data growth is still very big.

 

[Zero82z]

I'm thinking it has more to do with the Stanford staff discarding useless generated data from users. Agressive discarding : discarding more than normal but even with that, data growth is still very big.

+1, that's what it sounds like. Since the F@H project is very big and has a great deal of redundancy, they can afford to be very meticulous when it comes to deciding the reliability of each piece of data and whether or not it should be kept.

 

[EvilAlchemist]

Remember that the SMP & the newer GPU2 clients have been used to verify the results of the uni processor client's results and vice verse.

Keep data from both sets would take more storage so only one data set may be kept instead of both.

One theory .. to add to the others here.