John Ioannidis’s 2005 paper, “Why most published research findings are false”[1] set off a cascade of discussion, despite its impenetrability, not least because of its sensational title. Concerns about non-reproducibility have been voiced more recently by the pharmaceutical industry, which increasingly relies on academic research for basic therapeutic ideas. Printz et al. reported in 2011 that their firm, Bayer, was unable to reproduce 65% of 67 published studies they attempted to verify.[2] Glenn Begley (Amgen) and Lee Ellis (MD Anderson) had even worse results in 2012. They confirmed the findings of only six of 53 published studies[3]. Not only that, but Begley & Ellis found that in many cases, the original authors were unable to reproduce their own work. Clearly there are serious problems in the field of biomedical research. This is Part 1 of an eight-part exploration of this situation and a statement of the position of Comparative Biosciences.

To make matters worse, now comes a book for the lay public by a popular science journalist: Rigor Mortis: How sloppy science creates worthless cures, crushes hope, and wastes billions, by Richard Harris (of NPR). The title is word play, suggesting that scientific rigor is dead (it isn’t and Harris does not try to make that case). The subtitle is hyperbole of course, but Harris hammers away and makes a persuasive case that there are many things seriously wrong in the biomedical research enterprise.

We at Comparative Biosciences take these problems very seriously, although we are not directly involved in the academic rat race so vividly described by Harris. Rigor Mortis is required reading here. Privately funded research is largely free of the institutional barriers and perverse incentives that Harris blames for the problems, such as the paucity of grant funding and the need for publication. We are not driven to publish, for example—let alone driven to publish first and in a high-impact journal. Contract research is even better insulated from these forces than other kinds of privately funded research. For example, investigator bias is a common problem, even in privately funded research such as at a biotech company. Investigators typically have a strong desire for their projects to succeed, which may color their results, consciously or unconsciously. As a contract house, we want our clients to be happy and successful, but scientific rigor and integrity are our highest values—and we do not depend upon the success of any one product. In the long run, we believe that we are all best served by objectivity and truth. We have had academic scientists tell us that, in their view, the best research today is probably done by CROs.

In our next post, we will consider Harris’s general critique of animal research found in Chapter Four of Rigor Mortis.

[1] See http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124

[2] See http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html

[3] See http://www.nature.com/nature/journal/v483/n7391/full/483531a.html