How Retractions Hurt Scientists' Credibility

[FrgMstr]

While I have looked to MIT over the years to answer some incredibly difficult questions and callings, I am not so sure this one was that tough to figure out. Saying something is fact, then saying you were wrong, will make every statement of fact you put forth before come into question.


"The question we’re asking is: Do retractions trigger, at an individual level, something like an infection mechanism, where the retracted author is being punished and discredited for being dishonest or just incompetent?” says Alessandro Bonatti, an associate professor at the MIT Sloan School of Mangement and a co-author of a new paper detailing the study. “We find that yes, there is such a mechanism in place, and it operates through citations."

Of course the fact that the MIT news office spells "Mangement" incorrectly is icing on the cake. Are we being trolled? Hell, I even looked it up because I of course called my own spelling of it into question.

 

[Merc1138]

LOL, wow.

I think the bigger issue they're having is that they're so used to being "right" all the time(I mean really, normally folks wouldn't do too much doubting of things that come out of MIT) that they can't handle the real world implication that yes... when you're wrong and have to make a correction it starts to make people doubt you.

 

[bigdogchris]

I can totally understand why a retraction would hurt someone's credibility. Going through the scientific process it should be apparent whether your data is sound or not, and if you should even publish it.

It also demonstrates why it is vital to the scientific process for peer review. Only after a thorough peer review process (for papers that are worth peer review) can someone even begin to claim something as a legitimate theory (facts). Even well established scientist like Steven Hawking must go through the peer review process before people accept his work.

I think the biggest retraction of all time was the guy who claimed vaccines cause autism.

 

[Captindecisive]

The guy who claimed vaccines caused autism never did legitimate work or study either; he admitted he was bribed to falsify evidence at some point. In his case, he should be in prison indefinitely for all the people who have either died, gotten sick from preventable diseases, and the return of some diseases that were essentially gone.

Agreed on the patterns as mentioned above also. If you're going to publish something that will see the public at all, you need to be very thorough and careful.

 

[oROEchimaru]

Ever make a mistake in IT? I once mistyped a word in an email and the citrix engineers wouldn't look at the issue effecting all patients in the hospital because they didn't believe me all 30 citrix servers were down since I typed Citrixx instead of Citrix... they said I had no credibility. They refused to look at the fact that the entire EHR (electric health care system) was down until I had to pull the CSO and CIO in to get them off of facebook, fox news etc. lol

 

[Jagger100]

Why would I cite someone whose citation may become invalid, undermining my paper. I guess if you know the reason for the retraction, but still in almost all cases, the author is accountable for the validity of his assertions.

 

[kju1]

Ever make a mistake in IT? I once mistyped a word in an email and the citrix engineers wouldn't look at the issue effecting all patients in the hospital because they didn't believe me all 30 citrix servers were down since I typed Citrixx instead of Citrix... they said I had no credibility. They refused to look at the fact that the entire EHR (electric health care system) was down until I had to pull the CSO and CIO in to get them off of facebook, fox news etc. lol

Yup I deleted an entire cloud once. I owned up right away, shut it down, and hand edited the name node to undo the delete. Took me hours to recover. People hated that the service was down but I actually was thanked for acting quickly.

Earlier in my career I blew away an exchange server by being careless at 2 am in the morning after rebuilding and restoring it from a hack. Turned a 12+ hour outage into a 24 hour one. Also had good support from my customers on that since I owned up to the mistake...

My mistakes have been rare but I have always owned up to them and then learned from them as well (i.e. put procedures in place to ensure they dont happen again).

 

[SmokeRngs]

I have no doubt a retraction can hurt credibility. However, different things need to be taken into account. Was the retraction an honest mistake or something such as a small detail which was overlooked? Does the person have a history of retractions? Was the retraction made because the premise was bullshit from the get go and known as such?

I don't think any retraction should be an immediate loss of credibility but the reasons for the retraction can make a big difference.

 

[jardows]

I agree with those who want to look at the nature of the retraction. If someone posts a story that says "Eating popcorn causes Cancer" with the idea we should never eat popcorn again (I happen to be eating some popcorn while writing this), then posts a retraction story stating they realize the study was flawed, and it now OK to eat popcorn with minimal health impact, that would be a serious blow to the credibility of all who were involved with publishing the study.

There is a more serious issue at hand here - that is the trust we are asked to place in the infallibility of our scientists. If the "Scientific Community" comes to a "consensus" about something, we are not to question that. Whether it be about nutrition, climate, sociology, origins, etc., the "science" is "established" and any divergence must be silenced, ridiculed, and persecuted. With this level of trust we are asked to place in our scientists, it is no surprise that retractions may be seen as a dangerous threat to that trust.

 

[jbc029]

Retraction sucks, yes. What sucks worse? Being discredited. People make mistakes.

My mistakes have been rare but I have always owned up to them and then learned from them as well (i.e. put procedures in place to ensure they dont happen again).

Damn right. That's how you build a better rep. Displaying accountability and reliability. If your mistakes are rare and you work to remedy them, without bullshitting your stakeholders, it ends up being a net positive. You actually start to build a reputation of not only the person who can get it done, but the person who can objectively get it back on track.

 

[SmokeRngs]

I agree with those who want to look at the nature of the retraction. If someone posts a story that says "Eating popcorn causes Cancer" with the idea we should never eat popcorn again (I happen to be eating some popcorn while writing this), then posts a retraction story stating they realize the study was flawed, and it now OK to eat popcorn with minimal health impact, that would be a serious blow to the credibility of all who were involved with publishing the study.

There is a more serious issue at hand here - that is the trust we are asked to place in the infallibility of our scientists. If the "Scientific Community" comes to a "consensus" about something, we are not to question that. Whether it be about nutrition, climate, sociology, origins, etc., the "science" is "established" and any divergence must be silenced, ridiculed, and persecuted. With this level of trust we are asked to place in our scientists, it is no surprise that retractions may be seen as a dangerous threat to that trust.

That's one of the big problems with using words such as "consensus". I can probably go out and find a consensus of people who believe the "theme" colors of HardOCP and Hardforum are purple, green and white. However, that doesn't mean they know what they are talking about or even that they're lying. Which they obviously are, of course.

 

[M76]

This is what science is about. You come up with a theory, and then later come up with a better one.

Nothing is final or ultimate, it's only the best answer we came up with until that point in time.

That's what makes science exciting. It's great to come up with a new theory, but it's also progress if you prove that a theory is wrong. I'd be worried if there were no retractions, because that would mean science has turned into a religion.

So nope, the people who think retractions discredit science, are the ones who don't know how science works.

 

[DocFaustus]

So nope, the people who think retractions discredit science, are the ones who don't know how science works.

I agree completely and see this all too often. It reminds me of Asimov's Relativity of Wrong essay.

 

[kju1]

Damn right. That's how you build a better rep. Displaying accountability and reliability. If your mistakes are rare and you work to remedy them, without bullshitting your stakeholders, it ends up being a net positive. You actually start to build a reputation of not only the person who can get it done, but the person who can objectively get it back on track.

Thats what most people just dont get in all fields (though I see it most often in IT). I firmly believe this is one of the major reasons why I have been consistently referred to as "top talent" and all that garbage. I know for a fact there are people who are better at *nix than I am at work...and yet everyone looks to me to be the lead because of my reputation.

For my new employees I try to impart this as a way to set themselves up for success but unfortunately it seems like people think making mistakes = getting fired. For the record: I have never fired someone for making a single mistake. I have fired someone for hiding their mistakes or failing to improve after multiple "lessons".

 

[Logan321]

There is no truth in science. There is only speculation of fact based on available data. As the data set increases, so does the likelihood that the speculation approaches truth. Flawed data yields flawed speculations. Also, scientists are humanand make mistakes. Sometimes it's "oops, I have a digit or decimal place wrong" or "oops, I lied for money". It's the latter that affects credibility.

 

[Full Otto]

Making mistakes is human. Identifying your mistakes, owning them, and rectifying them efficiently is godly. So yea, the circumstances make all the difference. A retraction due to a recent discovery or some unforeseen aberration that is quickly caught, sure, kudos. A retraction due to faulty logic or over abstraction of the data will have me looking at your body of work with some doubt.

 

[Jagger100]

This is what science is about. You come up with a theory, and then later come up with a better one.

Nothing is final or ultimate, it's only the best answer we came up with until that point in time.

That's what makes science exciting. It's great to come up with a new theory, but it's also progress if you prove that a theory is wrong. I'd be worried if there were no retractions, because that would mean science has turned into a religion.

So nope, the people who think retractions discredit science, are the ones who don't know how science works.

I believe a retraction is not because the underlying theory is wrong. I think when someone say's retraction, they mean, someone fucked up the data, fucked up a calculation, or had some invalid assumptions. I think error in claims of the paper. Not that someone decided they can disprove a theory. If that was a case, you would do a second paper. The exception is fhe underlying theory the paper was supporting was debunked conclusively just before publishing, which would be rare.

 

[M76]

There is no truth in science. There is only speculation of fact based on available data. As the data set increases, so does the likelihood that the speculation approaches truth. Flawed data yields flawed speculations. Also, scientists are humanand make mistakes. Sometimes it's "oops, I have a digit or decimal place wrong" or "oops, I lied for money". It's the latter that affects credibility.

There are no absolute truths, but sane and useful descriptions of reality can only come from science. Scientific theories as measured on how well they describe the observable reality. And it's important to apply the theories well. For example general relativity works well on macroscopic levels, but it doesn't describe what's going on a quantum level, does that mean the theory is wrong? Of course not. It just means it's not the whole picture, only what is observable on a macroscopic level.

 

[M76]

I believe a retraction is not because the underlying theory is wrong. I think when someone say's retraction, they mean, someone fucked up the data, fucked up a calculation, or had some invalid assumptions. I think error in claims of the paper. Not that someone decided they can disprove a theory. If that was a case, you would do a second paper. The exception is fhe underlying theory the paper was supporting was debunked conclusively just before publishing, which would be rare.

Mistakes are made sometimes, it happened before, and it will happen in the future. There were results published that later turned out to be inconclusive or wrong going back all the way, it's not a new thing. That's why you have peer review. Every scientific theory is verifiable by third parties. They're not assertions of truths. If a mistake is made it will be revealed sooner or later, that's no reason to dismiss science.

 

[Prisoner849]

Some of this can be shouldered by the explosion of 'journals' which are not properly set up to assist researchers, many of whom work in relative isolation and rely on the publishing process for peer review. This can result in an idea gaining credibility or being questioned prior to being published which would avoid retracting and discrediting.

As it stands, things get published without due process and problems ensue. Its a systemic problem in which researchers are both victim and perpetrator; they should be more selective about where they publish, but are under pressure from those paying their bills to get something out there. Nobody is innocent, but some more culpable than others in the chain.

 

[RealBeast]

It happens in all fields.

The entire foundation of multimodal postoperative pain control was retracted after it was determined that the author had forged the data for 21 major studies, and even added coauthors without their knowledge. At the time nobody noticed that every study he did was a success for his sponsor (like Pfizer among others), none ever failed to be clinical statistically significant. $$$$

Despite that retraction about 10 years ago, a large percentage of anesthesiologists (including very recently trained physicians) still use the study recommendations in their daily practice.

http://www.anesthesiologynews.com/P...Rocks-Anesthesiology-Community/12634/ses=ogst

 

[Loop242]

This is funny!
http://blogs.discovermagazine.com/neuroskeptic/2017/07/22/predatory-journals-star-wars-sting/
A number of so-called scientific journals have accepted a Star Wars-themed spoof paper. The manuscript is an absurd mess of factual errors, plagiarism and movie quotes. I know because I wrote it.

Inspired by previous publishing “stings”, I wanted to test whether ‘predatory‘ journals would publish an obviously absurd paper. So I created a spoof manuscript about “midi-chlorians” – the fictional entities which live inside cells and give Jedi their powers in Star Wars. I filled it with other references to the galaxy far, far away, and submitted it to nine journals under the names of Dr Lucas McGeorge and Dr Annette Kin.

 

[kju1]

That's a typo I think. They spell it correctly in the header caption. Ironic typo, given the context of the study.

I dont care if its a typo. That kind of mistake destroys credibility imo. You should have had an editor look over it or at the very least used a damn spell checker. Its a PROFESSIONAL release it should not contain any typos.

I have this same issue with the news.

 

[sfsuphysics]

LOL, wow.

I think the bigger issue they're having is that they're so used to being "right" all the time(I mean really, normally folks wouldn't do too much doubting of things that come out of MIT) that they can't handle the real world implication that yes... when you're wrong and have to make a correction it starts to make people doubt you.

The thing is that science isn't about being "right", you have an idea, you test it, and if it works out great you publish, if it doesn't you hopefully don't falsify your reports because fuck this I just need to publish once to get my doctorate.

 

[bman212121]

Despite that retraction about 10 years ago, a large percentage of anesthesiologists (including very recently trained physicians) still use the study recommendations in their daily practice.

Garbage in, Garbage out. When it gets really scary is when someone else is using that work as the basis for their own. And someone could give you a straight face and tell you with 100% certainty their data is accurate because it is, but it's still wrong because the data they based their data off of was incorrect.

 

[Derangel]

It's a human thing and I don't think we are getting away from it any time soon, but I can tell you that scientists are not the only ones suffering from it.

The Intelligence Community has their own language that supports this issue. Some background first;

As an Intel analyst, I would need to make scheduled reports on observed activity, usually every 12 hours back in the old days with something really special getting immediate attention. So every 12 hours I would prepare for a briefing that went something like;

"During the last 12 hours, through RADAR intercept, we observed blah blah blah. Now that's the easy part, just reporting what someone else reported is pretty safe and I am just adding it all up and filtering out the shit that didn't matter. Now comes the hard scary part, I have to go out there and tell the boss that based on what I have seen, this is what I expect to see in the next 12, 24, and 72 hour time frames. I have to become predictive.

Now I do not actually have a crystal ball and if I tell the boss that the enemy is going to conduct a river crossing at a specific point within the next 24 hours, I better fucking well be right. The problem is, I can see every indication but the enemy reacts to things as well and they may have intended to cross that river exactly on schedule, and changed their mind because for whatever reason they couldn't do it anymore or don't need to any more. And now I look like a dumbass cause the enemy didn't stick to their script.

So how do I keep from looking like a dumbass? I used the special words;
Possible
Probable
Likely
Could
May
Might

There is a pattern in here somewhere right?

But scientists don't always do crystal balls do they? Still, they could take some lessons from the Intel guys but I am sure it would really take the WOW !!!! out of "DNA shows Dinosaurs descended from Beagles" claim.

Here is the big problem: Proper scientific papers don't read like "DNA shows Dinosaurs descended from Beagles". The paper would show the relevant data, how the tests were run, explaining the correlation and set it up as a scientific theory, supporting every bit of that theory along the way. The proper conclusion would really be something akin to: "Recent DNA evidence points to an ancestral link between Dinosaurs and Beagels" or something to that effect. The media are the ones that twist the words to create sensational headlines and take every tiny little thing as 100% solid fact, instead of properly analyzing the data and presenting that analysis as a possibility over a stated fact of the universe. Modern media is little better than click-bait.

 

[Delicieuxz]

Retracting false information is the only good thing to do. And reputations due to lying and faking accuracy are not deserved, and harm the credibility of all sciences and those working in them.


https://arstechnica.com/science/201...4-cancer-research-papers-contains-faked-data/

https://arstechnica.com/science/201...0-journals-appointed-a-fake-person-as-editor/

https://arstechnica.com/science/201...windle-patients-is-what-hooked-big-investors/

https://arstechnica.com/science/201...aunches-website-to-shame-pharma-greed-sleaze/

https://www.nature.com/news/over-half-of-psychology-studies-fail-reproducibility-test-1.18248

http://www.bbc.com/news/science-environment-39054778

http://www.cbc.ca/news/health/clinical-trial-results-not-published-1.3865785

http://www.theglobeandmail.com/opinion/social-psychologys-credibility-crisis/article29184360/

http://www.huffingtonpost.com/johan...l-make-you-question-everything_b_9484744.html

http://www.cbc.ca/news/health/vaginal-atrophy-analysis-1.3786547

http://www.iflscience.com/health-an...ng-research-downplaying-its-effect-on-health/

https://www.theguardian.com/commentisfree/2009/jun/13/bad-science-cocaine-study

http://www.cbc.ca/news/opinion/opioids-chronic-pain-1.4216530

http://scienceblogs.com/insolence/2...y-crisis-in-biomedical-research-2017-edition/

http://www.vox.com/2016/3/22/11278760/war-on-drugs-racism-nixon

http://www.cbc.ca/news/health/stem-cell-science-hype-1.3582223

 

[oROEchimaru]

Well all I jinxed myself posting here... an hour later I dropped a table in SQL lol F ME

 

[lcpiper]

Here is the big problem: Proper scientific papers don't read like "DNA shows Dinosaurs descended from Beagles". The paper would show the relevant data, how the tests were run, explaining the correlation and set it up as a scientific theory, supporting every bit of that theory along the way. The proper conclusion would really be something akin to: "Recent DNA evidence points to an ancestral link between Dinosaurs and Beagels" or something to that effect. The media are the ones that twist the words to create sensational headlines and take every tiny little thing as 100% solid fact, instead of properly analyzing the data and presenting that analysis as a possibility over a stated fact of the universe. Modern media is little better than click-bait.

I agree, the media today are terrible. The model may be good for profits but in the long run I don't think it's good for business.

 

[knowom]

Don't you know all scientists are right 101% of the time? Especially when money is involved...like climate change or anything NASA related! How dare you question their authenticity on such matters they are SCIENTISTS obviously they are right all the time!! They would never dream of faking a moon landing for money...lets head to Mars forget about sending a live unmanned drone shuttle to the moon what people don't know what won't hurt them!!! CGI we've got new planets to produce from miles away!!!! God and scientists are cut from the same tapestry of bs as far as I'm concerned....