Reproducibility Update: New Resources and Expected Changes to the SF424 Application Guide

I previously told you about the development of an NIGMS clearinghouse site where members of the research community will be able to find grantee-produced training materials designed to teach rigorous experimental design and enhance data reproducibility. Since then, NIH has established two new related sites. The first is a Rigor and Reproducibility web portal that provides general information about NIH efforts and offers resources that include guidelines for how research results should be reported and links to publications written by NIH authors on rigor, reproducibility and transparency.

The second site is focused on grants and funding and includes a summary of NIH’s proposal to clarify its instructions to applicants to emphasize expectations that rigorous experimental design and reproducibility of results should be considered during the application and review process. You may have read about the changes in a recent Rock Talk blog post that announced the publication of two new NIH Guide notices: Enhancing Reproducibility through Rigor and Transparency and Consideration of Sex as a Biological Variable in NIH-funded Research. We anticipate that the new instructions will be released in the fall of 2015 and will take effect for all research grant applications submitted on or after January 25, 2016.

As always, if you have questions or concerns, contact your program director. We’re also interested in hearing how your lab validates key biological and chemical reagents, so tell us about your procedures!

Resource Spotlight: iBiology Videos

In this iBiology video, NIGMS Director Jon Lorsch gives an overview of the Institute and careers in scientific public service. He also answers questions from the scientific community during an iBiology Q&A session.
In this iBiology video, NIGMS Director Jon Lorsch gives an overview of the Institute and careers in scientific public service Exit icon. He also answers questions from the scientific community Exit icon during an iBiology Q&A session.

One of the research and training resources we help fund is iBiology Exit icon, a collection of high-quality, free online videos of scientists talking about their research, career paths and related topics. The project, which also receives support from the National Science Foundation and other organizations, produces material that is relevant to those at a range of educational and career levels, especially undergraduate students, graduate students and postdocs.

When Ron Vale started the iBiology project in 2006, his goal was to give people around the world broader access to research seminars. Since then, the scope has expanded. The collection now includes 350 videos that fall into three main categories:

Continue reading

Resource Spotlight: Knockout Mouse Phenotyping Program

If you use genetically modified mice or work on a gene in another model organism for which a homolog exists in mice, the Knockout Mouse Phenotyping Program (KOMP2) Exit icon may benefit your research. It’s a resource that generates mice carrying specific genetic mutations and systematically phenotypes them according to uniform, high quality-control standards.

KOMP2 targets a range of phenotypes in order to improve the chances of gaining preliminary insights into the function(s) of genes that influence multiple traits, including targeting genes for which no information is currently available. The resource also captures negative results and disseminates them broadly. It examines male and female mice and provides data down to the individual mouse level.

Continue reading

Clearinghouse for Training Modules to Enhance Data Reproducibility

NIH and NIH-funded Training Modules to Enhance Data ReproducibilityAs part of a series of NIH-wide initiatives to enhance rigor and reproducibility in research, we recently launched a Web page that will serve as a clearinghouse for NIH and NIH-funded training modules to enhance data reproducibility. Among other things, the site will house the products of grants we’ll be making over the next few months for training module development, piloting and dissemination.

Currently, the page hosts a series of four training modules developed by the NIH Office of the Director. These modules, which are being incorporated into NIH intramural program training activities, cover some of the important factors that contribute to rigor and reproducibility in the research endeavor, including blinding, selection of exclusion criteria and awareness of bias. The videos and accompanying discussion materials are not meant to provide specific instructions on how to conduct reproducible research, but rather to stimulate conversations among trainees as well as between trainees and their mentors. Graduate students, postdoctoral fellows and early stage investigators are the primary audiences for the training modules.

Also included on the page are links to previously recorded reproducibility workshops held here at NIH that detail the potentials and pitfalls of cutting-edge technologies in cell and structural biology.

Training is an important element of the NIGMS mission and a major focus of NIH’s overall efforts to enhance data reproducibility. In addition to the training modules we’ll be funding, we recently announced the availability of administrative supplements to our T32 training grants to support the development and implementation of curricular activities in this arena.

I hope you find the resources on this site useful, both now and as we add more in the future.

Sustaining a Community Resource for Cell Lines and DNA Samples

Cryogenic tanks filled with liquid nitrogen and millions of vials of frozen cells. Credit: Coriell Institute for Medical Research.

Cryogenic tanks filled with liquid nitrogen and millions of vials of frozen cells. Credit: Coriell Institute for Medical Research.

We have just funded a new, 5-year award to continue operation of the NIGMS Human Genetic Cell Repository, an important resource for the scientific community since 1972. The repository contains more than 11,300 human cell lines and 5,700 DNA samples derived from them. These high-quality, well-characterized and rigorously maintained resources, which you can order for a nominal fee, include:

  • Specimens from individuals with inherited diseases, apparently healthy individuals and those of diverse geographic origins that are divided equally between those from males and those from females.
  • A group of 39 induced pluripotent stem (iPS) cell lines that carry disease gene mutations or that are normal control iPS cell lines.
  • An inherited disease collection that represents almost 900 disorders.

Last year, 1,500 scientists received more than 5,000 cell lines and 40,000 DNA samples. I encourage you to peruse the catalog Exit icon and consider whether these specimens may be useful in your research program.

NIH Data Science Leader’s Vision of a Digital Enterprise for Biomedical Research

Phil BourneI recently had the opportunity to talk to Phil Bourne, NIH’s associate director for data science, about some of the current Big Data to Knowledge (BD2K) initiative activities. I asked him how they tie together his vision of a digital enterprise for biomedical research and how they might benefit NIGMS grantees.

Phil explained that the goal of his office, commonly referred to as ADDS, is to achieve efficiencies in biomedical research, such as by making it easier for researchers to locate and manipulate data and software. “If we could just achieve a 5 percent improvement in efficiency in research that would be, in NIH budget dollars, more than $150 million a year that could be spent on funding more people and doing more research,” he said.

An active area that we at NIGMS are engaged in with ADDS is sustaining biomedical data resources, of which we support a fair number. As someone who previously set up databases and who now oversees them, I’m very passionate about this topic. A key question is how to sustain support of data resources in the current research budget environment. Led by Phil’s team, NIH has issued a request for information on sustaining biomedical data repositories that seeks input on every aspect of maintaining these resources. I encourage you to share your ideas by the March 18 response date.

Training is important in Phil’s vision for a digital enterprise, too. He told me of a number of recent training activities at NIH, including a “software carpentry” workshop for experimental researchers to learn how to use a wide variety of analysis tools. In a blog post about this and another event Exit icon, the ADDS office asks for suggestions on other types of data science courses to offer. They want to provide workshops that train more experimentally versed scientists to work with big data and take those skills back to their labs. In addition, the ADDS office is planning to stand up a workforce development center to catalog classroom and online courses in the data sciences.

Another effort that’s in the works is creating a virtual space called the Commons where researchers can share, locate, utilize and cite datasets, software, standards definitions and documentation. Phil anticipates that the first components of the Commons will be available in 2016.

I’m really excited about Phil’s efforts and believe that they will help drive the “data quantum leap” I described in my first Feedback Loop blog post.

Report and Recommendations from the Future of Structural Biology Committees

NIGMS Advisory Council Meeting: Report of the NIGMS Future of Structural Biology Committees

Dr. Leemor Joshua-Tor’s presentation of the report begins at 00:36:22 on the archived videocast of the National Advisory General Medical Sciences Council meeting.

I previously told you about the formation of two committees focused on Protein Structure Initiative (PSI) transition planning. These committees were charged with identifying high priorities for future NIGMS investments in structural biology and determining what unique resources and capabilities developed during the PSI should be preserved to address the needs of the scientific community. Dr. Leemor Joshua-Tor, one of the committee co-chairs, presented the groups’ report at the National Advisory General Medical Sciences Council meeting on January 23.

The committees’ recommendations for preserving PSI resources that the committees felt will be important for the community in the future include:

  • Support for a modest number of protein expression resources to serve the needs of the community.
  • Continued support for a materials repository Exit icon similar to the one that has been supported through PSI.
  • Possible continued support for a structural biology knowledgebase Exit icon.

The committees identified these areas as high priorities for the future of structural biology:

  • Continued support for synchrotron beamlines for crystallography.
  • Support for modern cryo-EM resource centers.
  • Continued support for NMR resources for structural biology.
  • Support for the integration of structural biology methods.
  • Support for collaborative, multi-investigator efforts in membrane protein and large macromolecular assembly structure determination.

We’re now developing plans for implementing the report’s recommendations.

Tune in for Video Resources on Navigating Peer Review, eRA Commons

The NIH Extramural Nexus blog has published posts on video resources that you may find helpful:

New Webinars Connect Applicants to NIH Peer Review Experts: The Center for Scientific Review is hosting webinars in early November to give R01, R15, SBIR/STTR and fellowship grant applicants and others useful insights into the submission and review processes. Register by October 28.

New Video Tutorials Can Help You Navigate eRA Commons: A 10-part series of short video tutorials walks you through the steps for submitting just-in-time information, a no-cost extension, a relinquishing statement and more. Watch the tutorials on the NIH Grants playlist Exit icon on YouTube.

Ensuring Synchrotron Beamline Access for Biomedical Researchers

The National Synchrotron Light Source (NSLS) at Brookhaven National Laboratory (BNL) closed earlier this week as a newer, more advanced facility, NSLS-II Exit icon, began to come online.

Thousands of NIH researchers have used beamlines at NSLS over the last 30 years to collect data to characterize biological macromolecules including drug targets, ion pumps and enzymes. Because the beamlines for biological research Exit icon at NSLS-II will not be available until 2016, other synchrotron facilities are temporarily expanding their capacity to address the beamline reduction.

Here are some sources that will help you identify and access beamlines at other U.S. synchrotrons:

If you have questions about NIH-funded synchrotron resources, please contact me or Ward Smith.

Give Input on Structural Biology Resource and Infrastructure Needs

Earlier this year, I told you about the formation of two committees focused on Protein Structure Initiative (PSI) transition planning. These committees are charged with determining what unique resources and capabilities developed during the PSI should be preserved after the initiative ends and how this preservation should be done.

An important part of this process is getting input from the community, so we have just issued a request for information (RFI), NOT-GM-14-115, seeking comments about structural biology resources that have a high impact on the community, whether those resources have been supported through the PSI or by other means. We also want to hear what you think about the future of structural biology-related technology development, which has been an important feature of the PSI.

While the RFI invites comments on these specific topics, you should not feel limited to them—we welcome any comments that you feel are relevant.

To respond to the RFI, send an e-mail to nigmspsirfi@mail.nih.gov by May 23, 2014. When we compile the responses, we’ll remove any personal identifiers like names and e-mail addresses and only use de-identified comments.

If you have any questions about the RFI or the transition committees, please let me know.