Search the Site

Posts Tagged ‘Academic Research’

The Retraction Penalty

In a new working paper called “The Retraction Penalty: Catastrophe and Consequence in Scientific Teams” (gated), Ginger Zhe Jin, Benjamin Jones, Susan Feng Lu, and Brian Uzzi explore a fascinating research question:

What are the individual rewards to working in teams? This question extends across many production settings but is of long-standing interest in science and innovation, where the “Matthew Effect” [a.k.a. “the rich get richer and the poor get poorer” suggests that eminent team members garner credit for great works at the expense of less eminent team members. In this paper, we study this question in reverse, examining highly negative events – article retractions. Using the Web of Science, we investigate how retractions affect citations to the authors’ prior publications. We find that the Matthew Effect works in reverse – namely, scientific misconduct imposes little citation penalty on eminent coauthors. By contrast, less eminent coauthors face substantial citation declines to their prior work, and especially when they are teamed with an eminent author. A simple Bayesian model is used to interpret the results. These findings suggest that a good reputation can have protective properties, but at the expense of those with less established reputations.

To me, this finding is a bit surprising at first glance but, upon second glance, not really — but still fascinating.

If you are even a little bit interested in this topic and don’t know about the Retraction Watch website, you should. A few recent examples:



To Test or Not to Test

Many folks always ask me what the impact of randomized trials are on development. We at Innovations for Poverty Action and the M.I.T. Jameel Poverty Action Lab are dedicated to randomized trials to help push forward evidence-based policymaking. Yet what is the evidence that evidence shifts views? Not always so easy to do. I’ve done some work on the donor side, which I’ve reported on here before.  Here is a meta-study that uses two of my studies that found fairly different results. One found that access to credit in South Africa led to increased income, the other found that access to credit in the Philippines had no discernible impact on income.

The researchers sent off about 1,500 mailers to microfinance institutions around the world, telling them about the positive study, the negative (or non-positive, technically) study, or a placebo (no mention of a study), and asked them if they wanted to participate in a randomized trial to measure the impact of their organization.  They then saw which microfinance leaders responded, and whether they responded favorably or negatively.



The Retraction Epidemic

In the Washington Post, Peter Whoriskey writes about the rising incidence of fraud in research labs:

It may be impossible for anyone from outside to know the extent of the problems in the Nature paper. But the incident comes amid a phenomenon that some call a “retraction epidemic.”

Last year, research published in the Proceedings of the National Academy of Sciences found that the percentage of scientific articles retracted because of fraud had increased tenfold since 1975.

The same analysis reviewed more than 2,000 retracted biomedical papers and found that 67 percent of the retractions were attributable to misconduct, mainly fraud or suspected fraud.

One of the less-obvious downsides of academic fraud:

The trouble is that a delayed response — or none at all — leaves other scientists to build upon shaky work. [Ferric] Fang said he has talked to researchers who have lost months by relying on results that proved impossible to reproduce.

Moreover, as [Adam] Marcus and [Ivan] Oransky have noted, much of the research is funded by taxpayers. Yet when retractions are done, they are done quietly and “live in obscurity,” meaning taxpayers are unlikely to find out that their money may have been wasted.



Are All Research Participants Outliers?

A Pacific Standard profile of noted social psychologist Joe Henrich has some staggering information about how social scientists conduct their research:

Economists and psychologists, for their part, did an end run around the issue with the convenient assumption that their job was to study the human mind stripped of culture. The human brain is genetically comparable around the globe, it was agreed, so human hardwiring for much behavior, perception, and cognition should be similarly universal. No need, in that case, to look beyond the convenient population of undergraduates for test subjects. A 2008 survey of the top six psychology journals dramatically shows how common that assumption was: more than 96 percent of the subjects tested in psychological studies from 2003 to 2007 were Westerners—with nearly 70 percent from the United States alone. Put another way: 96 percent of human subjects in these studies came from countries that represent only 12 percent of the world’s population.



Are Fake Resumes Ethical for Academic Research?

“Audit studies” have been popular in labor economics research for 10 years.  The researcher sends resumés of artificial job applicants in response to job openings. Typically there is a crucial difference in some characteristic of the person that indicates a particular racial/gender/ethnic or other group to which one person within a pair of resumés belongs while the other does not.  The differential response of employers to the difference in the characteristic implied by the resumés is taken as a measure of discrimination in hiring.

Is this ethical? 



“AI: Adventures in Ideas”: Getting to “Yes (I’ll Participate in Your Survey)”

Freakonomics readers may know that I’m not the most qualified person to talk about using surveys. My first attempt — asking street gang members “How does it feel to be black and poor? Very bad, bad, good, …” — was met with laughter, disbelief and, scorn. (I suppose it was all uphill from that point!)

A basic question social scientists confront is: Why would you want to participate in our survey? Interviews can be long and boring; who wants to sit on the phone or stand on a streetcorner answering questions? A few bucks may not be worth the time. In fact, you have likely already perfected methods of avoiding telemarketers and sidewalk interviewers. From a data standpoint, your skilled avoidance is our problem: the views of respondents can differ from non-participants. From political races to consumer habits to opinion polls … we love numbers, and we need participation to get an accurate reading.



Should We Be Searching for Dinosaur Vomit?

Yes! That’s the argument in a new Historical Biology paper called “A Call to Search for Fossilized Gastric Pellets.” Here’s the abstract:

Numerous extant carnivorous, piscivorous and insectivorous species – including birds, pinnipeds, varanid lizards and crocodiles and mammals – routinely ingest food combined with a high proportion of indigestible material that can be neither absorbed through digestion nor eliminated as faecal matter. Their solution is to egest the indigestible portion through the mouth as a gastric pellet. The status of gastric pellets in extant species is reviewed. Arguments based on phylogeny, anatomy and biomechanics strongly suggest that many extinct species, including crocodilians and pterosaurs, may also have produced gastric pellets routinely.



Beware: This Blog Apparently Causes Academic Fraud

Way to scapegoat, Chronicle of Higher Education!

An article about a Dutch psychologist accused of faking his research data wonders if academic fraudsters are responding to the wrong incentives:

Is a desire to get picked up by the Freakonomics blog, or the dozens of similar outlets for funky findings, really driving work in psychology labs? Alternatively—though not really mutually exclusively—are there broader statistical problems with the field that let snazzy but questionable findings slip through?



Research Retractions Rising

There’s a new trend emerging in academic research: more retractions. According to a recent article in Nature by Richard Van Noorden, “[i]n the early 2000s, only about 30 retraction notices appeared annually. This year, the Web of Science is on track to index more than 400 (see ‘Rise of the retractions’) — even though the total number of papers published has risen by only 44% over the past decade.”

The article suggests that the increase is a result of ‘an increased awareness of research misconduct” and “the emergence of software for easily detecting plagiarism and image manipulation, combined with the greater number of readers that the Internet brings to research papers.”

While scientists and editors support the change, they point to various problems with the system: policy inconsistencies across journals, “opaque” explanations for retractions, ongoing citation of retraction papers and the stigma surrounding retraction. “[B]ecause almost all of the retractions that hit the headlines are dramatic examples of misconduct, many researchers assume that any retraction indicates that something shady has occurred,” writes Van Noorden.



The Latest from the Brookings Panel

I’m back from my favorite conference of the year—the Brookings Papers on Economic Activity. It was a terrific line-up of papers. And to call the discussion lively would be an understatement. (Full disclosure: David Romer and I are the co-editors.)
While a close reading of technical research papers is my idea of a good time, I’m told not everyone is wired this way. So I went into the studio to record a very simple summary of my thoughts on the papers. You won’t quite get the whole two days of economic policy wonk-ery, but this video is a start:



A Debate on University Tenure

With only 8 percent of private employees belonging to trade unions, job security outside government employment has become a sometime thing. One group of employees, however, does have nearly total job security: tenured university professors. Faculty tenure is under attack as never before in the past 50 years.
I like tenure, but why should my group of workers get special protections against the vicissitudes of demand for our “product?” Self-interested arguments about job protection are unsatisfactory. I recently “debated” a journalist on this issue, with the resulting short video from the Texas Tribune:



Technology and Tenure

Should publishing requirements for tenure go up for scholars in the humanities and social sciences?



Why Don't Reputations and Salaries Rise Together?

Our new study poses a conundrum: in a professional market (for economists), having more scholars pay attention to your research raises your reputation and your salary. Conditional on that attention, though, writing more papers lowers your reputation — but it raises your salary!



An Academic Does the Right Thing

A few years back Dubner and I wrote a piece on Slate heralding a remarkable young economist, Emily Oster. She has continued to do great work. She also has done something incredibly rare for an academic economist: she has admitted she was wrong. In places like India and China, there are many “missing women.” In other words, the sex ratios . . .



‘Put Your Money Where Your Butt Is’

That’s the clever title of the latest paper from Dean Karlan (one of the founders of StickK.com, who was featured in this New York Times Magazine article yesterday along with my colleague John List) and co-authors Xavier Giné and Jonathan Zinman. The researchers had surveyors approach people on the streets of the Philippines and offer them the opportunity to open . . .



Does College Football Cause Higher Crime? A Guest Post

A few days ago, Levitt blogged about an interesting study finding that violent movies reduce crime (at least in the short run). The reason, according to the study’s authors, Gordon Dahl and Stefano DellaVigna, is simply that more violent movies means fewer drunken louts on the streets. It is simply an incapacitation effect. One way of testing this hypothesis would . . .



Schwarzenegger Lowers Crime

California Governor Arnold Schwarzenegger is one politician who can credibly claim that he is truly responsible for reducing crime, at least if you believe a new study by economists Gordon Dahl and Stefano DellaVigna. It isn’t his policies as governor, however, that he can take credit for, but rather his acting roles. In their new paper entitled “Does Movie Violence . . .




Congratulations to Dean Karlan

Yale economist Dean Karlan recently received a Presidential Early Career Award for Scientists and Engineers — the highest honor given by the federal government to young researchers. I believe the prize comes with half a million dollars of research support to use over the next five years. Dean has been doing important and innovative work, often using field experiments to . . .



College Football Polls Aren’t What You Think

It may not be surprising to you that Trevon D. Logan, an economics professor at Ohio State University, is interested in college football. Ohio State is, after all, a football mecca (as we experienced first-hand some time ago). What may surprise you, however, is what Logan has concluded about college football polls. In a new working paper, Logan used 25 . . .



‘Acting White’ Is Old School

Economist Roland Fryer has done research on “acting white,” i.e. the phenomenon by which black children who excel academically are stigmatized by their peers. Recently, he was in a New York City school and asked some of the seventh graders he was talking to whether they had ever heard the phrase “acting white.” The kids laughed at him and said, . . .



Did Banning Lead Lower Crime?

The major news media (see for instance here and here) have been reporting recently on the hypothesis that banning lead from gasoline caused a reduction in crime. This follows a similar article in the Washington Post a few months ago, which I blogged about at the time. Since a lot of people have written to us lately asking for comment . . .