How to Streamline Drug Research?

We all know that information is valuable, and that more information is generally better than less.
But in the realm of pharmaceutical research (as in others, to be sure), there’s a troubling paradox: while successes are widely publicized, and while the results of clinical trials are usually published, the research from projects that fail before that stage is usually kept hidden. “A result,” writes Natasha Singer in The Times, “is that companies waste many millions going down experimental paths that their competitors have already found to be dead ends.”
You should read Singer’s article in full, but here are a few highlights:

Now big-picture thinkers, within the industry and outside it, are re-examining every stage of drug development — from molecule to market — in an effort to foster faster innovation. It’s a holistic approach, called “systems thinking,” that originated in methods that engineers used to streamline projects in the aeronautics and automotive industries.
The Massachusetts Institute of Technology, for one, started a pharmaceutical innovation program this year to help drug companies adapt some successful approaches now used in aeronautics, like lean management and information-sharing among rivals.

And:

The M.I.T. project, called New Drug Development Paradigms, has gathered a powerful consortium of interested parties — including major drug makers and federal health authorities. One short-term goal is to identify, and rectify, the root causes of bottlenecks in the existing system. Longer term, the ambition is to create new prediction models, new ways to share information about the biology of diseases, and a new inclusiveness involving earlier participation of regulators, health insurers, health care providers, and patients.

And:

In the absence of a public database on failed drug compounds, a small group from the Sloan School of Management at M.I.T. and the Harvard Business School has created Pharmer’s Market (not this one), an online prediction market that uses crowd-sourcing to forecast the likelihood of a drug’s success. Introduced last month, the market has invited biomedical researchers and other drug industry experts to place anonymous bets, using virtual money, on the likelihood that certain breast cancer drugs, currently in clinical trials, will succeed or fail.
After the clinical trials conclude, researchers can assess whether this kind of collective intelligence may be a useful predictive tool for drug companies, said Ragu Bharadwaj, who helped devise the project as an M.I.T. graduate student.

There is an obvious conflict between open-sourcing and a competitive marketplace, but it is exciting to see smart people pushing hard to change the shape of that marketplace for the public good.