I previously told you about the development of an NIGMS clearinghouse site where members of the research community will be able to find grantee-produced training materials designed to teach rigorous experimental design and enhance data reproducibility. Since then, NIH has established two new related sites. The first is a Rigor and Reproducibility web portal that provides general information about NIH efforts and offers resources that include guidelines for how research results should be reported and links to publications written by NIH authors on rigor, reproducibility and transparency.
The second site is focused on grants and funding and includes a summary of NIH’s proposal to clarify its instructions to applicants to emphasize expectations that rigorous experimental design and reproducibility of results should be considered during the application and review process. You may have read about the changes in a recent Rock Talk blog post that announced the publication of two new NIH Guide notices: Enhancing Reproducibility through Rigor and Transparency and Consideration of Sex as a Biological Variable in NIH-funded Research. We anticipate that the new instructions will be released in the fall of 2015 and will take effect for all research grant applications submitted on or after January 25, 2016.
As always, if you have questions or concerns, contact your program director. We’re also interested in hearing how your lab validates key biological and chemical reagents, so tell us about your procedures!
Mike Rogers, who has directed the NIGMS Division of Pharmacology, Physiology, and Biological Chemistry for the past 22 years, retired today. Throughout his NIH career, Mike has been a champion for chemistry and its important role in biomedical research.
Before joining NIGMS 26 years ago, Mike worked for more than a decade in what is now the Center for Scientific Review, where he oversaw the Bioorganic and Natural Products study section.
Between these two positions, Mike completed a detail assignment on Capitol Hill working for Senator Ted Kennedy’s Health, Education, Labor and Pensions Committee, an experience that he says allowed him to see NIH from a different perspective.
Throughout his time at NIGMS, Mike has sought to build scientific bridges. He created the chemistry-biology interface predoctoral training program, which aims to cross-train students in both disciplines. He was instrumental in developing the large-scale collaborative project awards program that “glued” together scientists with diverse expertise to tackle big, unanswered questions in biology. More recently, he forged a link between two fields to help form the new field of quantitative and systems pharmacology. Along the way, he mentored and encouraged others to develop major NIGMS and trans-NIH initiatives, such as those in glycoscience, pharmacogenomics and synthetic organic chemistry.
As part of a series of NIH-wide initiatives to enhance rigor and reproducibility in research, we recently launched a Web page that will serve as a clearinghouse for NIH and NIH-funded training modules to enhance data reproducibility. Among other things, the site will house the products of grants we’ll be making over the next few months for training module development, piloting and dissemination.
Currently, the page hosts a series of four training modules developed by the NIH Office of the Director. These modules, which are being incorporated into NIH intramural program training activities, cover some of the important factors that contribute to rigor and reproducibility in the research endeavor, including blinding, selection of exclusion criteria and awareness of bias. The videos and accompanying discussion materials are not meant to provide specific instructions on how to conduct reproducible research, but rather to stimulate conversations among trainees as well as between trainees and their mentors. Graduate students, postdoctoral fellows and early stage investigators are the primary audiences for the training modules.
Also included on the page are links to previously recorded reproducibility workshops held here at NIH that detail the potentials and pitfalls of cutting-edge technologies in cell and structural biology.
Training is an important element of the NIGMS mission and a major focus of NIH’s overall efforts to enhance data reproducibility. In addition to the training modules we’ll be funding, we recently announced the availability of administrative supplements to our T32 training grants to support the development and implementation of curricular activities in this arena.
I hope you find the resources on this site useful, both now and as we add more in the future.
Although researchers have made major progress in achieving a balance between male and female subjects in human studies—women now account for roughly half of the participants in NIH-funded clinical trials—a similar pattern has not been seen in pre-clinical research involving animals and cells. To inform the development of policies that address this issue, NIH has issued a request for information (RFI) on the consideration of sex as a biological variable in biomedical research.
As NIH Deputy Director for Extramural Research Sally Rockey wrote in a recent blog post announcing the RFI, “Sex is a critical variable when trying to understand the biological and behavioral systems that fundamentally shape human health.” Appropriate representation of animals and cells is also relevant to NIGMS and NIH efforts to enhance scientific rigor and data reproducibility in research.
And in her own blog post about the RFI, NIH Associate Director for Research on Women’s Health Janine Clayton said that while many scientists have already expressed support for the policy change, she has also heard from many sources that it needs “to be carefully implemented, as a true benefit to science—and not become a trivial, bureaucratic box to check.” She noted that comments in response to the RFI will guide NIH in creating “meaningful change in a deliberate and thoughtful way.”
Since NIGMS supports a significant amount of basic biomedical science that utilizes animal models and cells, we encourage our grantees to submit their input on this topic by the October 13 deadline.
UPDATE: The deadline for submitting input has been extended to October 24.
In February, we asked for input on training activities relevant to enhancing data reproducibility, which has become a very serious issue for both basic and clinical research. The responses revealed that there is substantial variation in the training occurring at institutions. One reason is that “best practices” training in skills that influence data reproducibility appears to be largely passed down from generation to generation of scientists working in the laboratory.
To increase the likelihood that researchers generate reproducible, unbiased and properly validated results, NIGMS and nine additional NIH components have issued a funding opportunity announcement to develop, pilot and disseminate training modules to enhance data reproducibility. Appropriate areas for the modules include experimental design, laboratory practices, analysis and reporting of results, and/or the influence of cultural factors such as confirmation bias in hypothesis testing or the scientific rewards system. The modules should be creative, engaging, readily accessible online at no cost and easily incorporated into research training programs for the intended audience, which includes graduate students, postdoctoral fellows and beginning faculty.
The application deadline is November 20, 2014, with letters of intent due by October 20, 2014. Applicants may request up to $150,000 in total costs to cover the entire award period. For more details, read the FAQs.
Data reproducibility is getting a lot of attention in the scientific community, and NIH is among those seeking to address the issue . At NIGMS, one area we’re focusing on is the needs and opportunities for training in areas relevant to improving data reproducibility in biomedical research. We just issued a request for information to gather input on activities that already exist or are planned as well as on crucial needs that an NIGMS program should address.
I strongly encourage you and your colleagues to submit comments by the February 28 deadline (no longer available). The information will assist us in shaping a program of small grants to support the design and development of exportable training modules tailored for graduate students, postdoctoral students and beginning investigators.
UPDATE: NIGMS and additional NIH components have issued the Training Modules to Enhance Data Reproducibility (R25) funding opportunity announcement. The application deadline is November 20.