Beware: This Blog Apparently Causes Academic Fraud

(Digital Vision)

Way to scapegoat, Chronicle of Higher Education!

An article about a Dutch psychologist accused of faking his research data wonders if academic fraudsters are responding to the wrong incentives:

Is a desire to get picked up by the Freakonomics blog, or the dozens of similar outlets for funky findings, really driving work in psychology labs? Alternatively—though not really mutually exclusively—are there broader statistical problems with the field that let snazzy but questionable findings slip through?

All I can say is that if you’re an academic who’s willing to fake your findings just to get on this blog (!!!), then you have a dangerously distorted sense of reward versus risk. Yes, a blog mention may indeed result in heightened exposure — but to risk your academic integrity for that? Anyone that desperate for attention might be best served pursuing good p.r. the old-fashioned way: cash bribes!

That said, it is worth noting that a piece of eye-grabbing research often does get massively exposed and then, if corrected, the correction tends to gather dust. Consider David Freedman‘s excellent Atlantic piece about this problem in medical research.


Nanno

"the reforms they advocated for in their paper—namely, fuller descriptions of research protocols, and more tolerance of imperfections in initial papers. When the data are supposed to support a thesis perfectly, the incentives to cut corners increase."

Says it all.

Joshua Northey

Some of this could be solved with a less "star" based academic culture. People who "discover" things that would have been "discovered" in the next year by someone else are often heralded as singular geniuses unique in their time, rather than someone straightforwardly building upon a lot of per-existing work.

If the rewards for being a year faster were lower the incentive to fudge would be reduced. Granted you don't want to make it easier on nonperforming academics , but right now getting some flashy article picked up and then sorting out the finer points of everything later (or not even) can be a huge personal windfall.

I would love to see more restraint (or even explicit protocols) in science journalism/journals surrounding how sensationalized results can be, and how much unwarranted speculation based on the results is permissible.

So much research these days is presented as "OMG huge fact about the world X!". When if you pick at it a bit you find the truth is "We found small data point A. Assuming we didn't make an error this allows us to assume B. If you take B and combine it with a bunch of random assumptions I made you get sensationalist fact X."

Read more...

robin marlowe

Dear Joshua:

Seems to me every scientist wants to be a winner. That is understandable. That said, Not everyone can i.e., has the (je ne said qua) to come up with an `original' idea. The question is what will make it a bit easier for that person to step up to the mount. The answer- simply put-- I had to have the idea, then test it or figure it out and then come to the realization that it is my idea! Notice- of course- the past tense. It has taken me 48 years to get here.
Robyn Ann Goldstein, 2012. All Rights Reserved. No two words of this text may be used without the author's permission.

As for the star stuff- I do find it amusing. But really- my only thoughts right now are of finding the " inner strength" to finish up-- So if the blog has appeared to cause people to make fraudulent claims - well is not that the stuff of which science is about- until the person who made the discovery gets it right.

So Rex- I understand your need to rou tinize the process of making a discovery- the trouble is= you cannot. what you are describing is the research that follows . As I see it, we are in somewhat of a holding period akin to the "calm before the storm. Seems to be that this is unavoidable. As for Dubner et.al. reading my stuff- they already have. Just not the book yet. No one else has read it as it now stands.

Read more...

Rex McClure

Saying the "fraud" problem is driven by a desire for self-promotion is a bit of an oversimplification. Models are becoming more obtuse, as are the statisical methods to validate (or reject) them. Even for an expert reviewer, or the self-promoting researcher, difficulties abound in deciding if the data fit model which fits the theory, or if the theory fits the model which fits the data.

Academics, ask yourself this: Does your research follow these steps? 1) idea/research question, 2) preliminary/informal data collection, 3) confirmation of idea leads to formal lit review and hypotheses, 4) formal data collection and analysis, 5) publication, and 6) hopes that Dubner or Hammermesh reads your work.

Sam_L

Stephen,

Perhaps you have rewarded the the Chronicle of Higher Education - maybe they referenced your blog in an attempt to get exposure for their article (and better search engine placement) - for suggesting implausible motivation.

clayton

Judging by your headline, I guess there is something to the criticism that you Freakonomics guys are too quick to infer causality!

Andreas Moser

Did I already point out my theory that not reading my blog causes death: http://andreasmoser.wordpress.com/2011/04/05/if-you-dont-read-my-blog-you-will-die/ ?

Dwight

Someone on another blog shared this corollary to Ockam's Razor with me: no one is sufficiently honest to be trusted with more than five independent variables.’

Owen

By the way, your blog does indeed have a bias for sensationalized stories and often has, at best, questionable facts.

http://voices.washingtonpost.com/ezra-klein/2009/10/the_shoddy_statistics_of_super.html

jane

keep the fire burning.