Historically, this blog has focused on “news you can use,” but in the spirit of two-way communication, for this post I thought I would try something that might generate more discussion. I’m sharing my thoughts on an issue I’ve been contemplating a lot: the hazards of overly hypothesis-driven science.
When I was a member of one study section, I often saw grant applications that began, “The overarching hypothesis of this application is….” Frequently, these applications were from junior investigators who, I suspect, had been counseled that what study sections want is hypothesis-driven science. In fact, one can even find this advice in articles about grantsmanship.
Despite these beliefs about “what study sections want,” such applications often received unfavorable reviews because the panel felt that if the “overarching hypothesis” turned out to be wrong, the only thing that would be learned is that the hypothesis was wrong. Knowing how a biological system doesn’t work is certainly useful, but most basic research study sections expect that a grant will tell us more about how biological systems do work, regardless of the outcomes of the proposed experiments. Rather than praising these applications for being hypothesis-driven, the study section often criticized them for being overly hypothesis-driven.
Many people besides me have worried about an almost dogmatic emphasis on hypothesis-driven science as the gold standard for biomedical research (e.g., see Jewett, 2005; Beard and Kushmerick, 2009; Glass, 2014). But the issue here is even deeper than just grantsmanship, and I think it is also relevant to recent concerns over the reproducibility of scientific data and the correctness of conclusions drawn from those data. It is too easy for us to become enamored with our hypotheses, a phenomenon that has been called confirmation bias. Data that support an exciting, novel hypothesis will likely appear in a “high-impact” journal and lead to recognition in the field. This creates an incentive to show that the hypothesis is correct and a disincentive to proving it wrong. Focusing on a single hypothesis also produces tunnel vision, making it harder to see other possible explanations for the data and sometimes leading us to ignore anomalies that might actually be the key to a genuine breakthrough.
In a 1964 paper, John Platt codified an alternative approach to the standard conception of the scientific method, which he named strong inference. In strong inference, scientists always produce multiple hypotheses that will explain their data and then design experiments that will distinguish among these alternative hypotheses. The advantage, at least in principle, is that it forces us to consider different explanations for our results at every stage, minimizing confirmation bias and tunnel vision.
Another way of addressing the hazards of hypothesis-driven science is to shift toward a paradigm of question-driven science. In question-driven science, the focus is on answering questions: How does this system work? What does this protein do? Why does this mutation produce this phenotype? By putting questions ahead of hypotheses, getting the answer becomes the goal rather than “proving” a particular idea. A scientific approach that puts questions first and includes multiple models to explain our observations offers significant benefits for fundamental biomedical research.
In order to make progress, it may sometimes be necessary to start with experiments designed to give us information and leads—Who are the players? or What happens when we change this?—before we can develop any models or hypotheses at all. This kind of work is often maligned as “fishing expeditions” and criticized for not being hypothesis-driven, but history has shown us just how important it can be for producing clues that eventually lead to breakthroughs. For example, genetic screens for mutations affecting development in C. elegans set the stage for the discovery of microRNA-mediated regulation of gene expression.
Is it time to stop talking about hypothesis-driven science and to focus instead on question-driven science? Hypotheses and models are important intermediates in the scientific process, but should they be in the driver’s seat? Let me know what you think.
Registration is now open for the 2014 Postdoctoral Preparation Institute: Career Transitions, a workshop we’re funding for postdoctoral fellows who will soon be seeking positions in a variety of career sectors. The workshop, which is being run by FASEB, will take place near NIH on June 5-6. It follows two successful prior NIGMS postdoc workshops in 2010 and 2012.
