Tune in for Video Resources on Navigating Peer Review, eRA Commons

The NIH Extramural Nexus blog has published posts on video resources that you may find helpful:

New Webinars Connect Applicants to NIH Peer Review Experts: The Center for Scientific Review is hosting webinars in early November to give R01, R15, SBIR/STTR and fellowship grant applicants and others useful insights into the submission and review processes. Register by October 28.

New Video Tutorials Can Help You Navigate eRA Commons: A 10-part series of short video tutorials walks you through the steps for submitting just-in-time information, a no-cost extension, a relinquishing statement and more. Watch the tutorials on the NIH Grants playlist Exit icon on YouTube.

CSR Launches Competition for Ideas to Detect Bias and Maximize Fairness in Peer Review

As part of NIH’s efforts to address racial disparities Exit icon in grant funding, the Center for Scientific Review (CSR) has just launched two America COMPETES Act Challenges. We hope that the ideas we receive will help us maximize the fairness and vitality of the peer review process, and we encourage you to enter.

One challenge, New Methods to Detect Bias in Peer Review, solicits ideas for strategies to detect possible bias in the NIH peer review process. Submissions can include approaches, strategies, methodologies and/or measures that would be sensitive to detecting bias among reviewers due to gender, race/ethnicity, institutional affiliation, area of science and amount of research experience. We’ll award first place ($10,000) and second place ($5,000) prizes in two categories: best empirically based idea and most creative idea.

The other challenge, Strategies to Strengthen Fairness and Impartiality in Peer Review, seeks ideas for reviewer training methods aimed at enhancing fairness and impartiality in NIH peer review. The submission does not require the full development of training materials. However, ideas should be presented with enough detail to allow assessment of their ability to address fairness and impartiality in review with regard to gender, race/ethnicity, institutional affiliation, area of science and amount of research experience. We’ll give first place ($10,000) and second place ($5,000) prizes for the best overall idea.

The challenges close on June 30, and winners will be announced on September 2. Details on the rules and submission procedures are on the CSR Challenge Web site and at http://www.challenge.gov Exit icon.

This contest is just one of many initiatives CSR is working on to evaluate the sources of racial disparities in grant funding in collaboration with the ACD Diversity Working Group Subcommittee on Peer Review.

Encouraging Model Organism Research

Image of a planerian from the lab of Dr.  Alejandro Sánchez  Alvarado. Nervous system in green and photoreceptors in red. Image credit: Sarah Elliot and Lisandro Maya-RamosA recent article in NIH’s Peer Review Notes illustrated how Alejandro Sánchez Alvarado developed the planarian as a model organism for studying regeneration.

When he applied for his first NIH grant in 1997, Sánchez Alvarado was a new investigator at the Carnegie Institution of Washington who had little preliminary data supporting his proposal. The peer reviewers astutely recognized the promise of his grant application, and they scored it favorably.

NIGMS funded the grant and continues to support Sánchez Alvarado’s studies, now at the Stowers Institute for Medical Research. Since receiving that first grant, Sánchez Alvarado has helped turn planaria into tiny workhorses that have opened new avenues for understanding tissue regeneration and other developmental processes. He has also trained and mentored graduate students and postdocs and served the scientific community as a study section and NIGMS advisory council member.

Sánchez Alvarado’s work is just one example of how NIGMS-funded research on model organisms, including those that are currently considered unconventional, has significantly advanced scientific progress. Think of the protozoan Tetrahymena thermophila and the discovery of telomeres, or the nematode Caenorhabditis elegans and the discovery of RNA interference. The fungus Neurospora crassa has been an important player in understanding the genetics of circadian rhythms, and the bacterium Caulobacter crescentus has led to important insights on the regulation of cell division.

We have long supported research using these and many more model organisms, including mammalian species and other vertebrates, invertebrates, plants, yeast, and additional bacteria. We know that findings made in such systems can translate rapidly—and often unexpectedly—into leaps in conceptual understanding of fundamental life processes, the development of better research tools and approaches, and new strategies for diagnosing and treating diseases. We also know that we can never predict where the next breakthrough will come from—or in which organism. So we continue to encourage applications for projects that use model organisms, including uncommon ones, when these systems offer distinct scientific advantages.

Identifying the Study Section for Your Application

One of my jobs as an NIGMS program director is helping investigators navigate the review process. This includes understanding where their applications will be reviewed and how they can make a recommendation about this assignment.

Applications can be grouped for review by research area or grant mechanism, or as a cohort submitted in response to a specific funding opportunity announcement (FOA). I would like to briefly walk through a few scenarios and share some advice along the way.

The “Review and Selection Process” (V.2) section of FOAs provides clues about where your application will be reviewed. An application can be reviewed at the Center for Scientific Review (CSR) or at an individual NIH institute or center (IC), depending upon the particular FOA. Information about who will review your application is posted in your eRA Commons account soon after it is determined, but you should contact your program director or CSR if you have questions or concerns.

The vast majority of applications received by NIH on topics relevant to NIGMS are in response to “parent” program announcements, such as PA-13-302, for unsolicited R01 applications. Section V.2 of PA-13-302 states, “Applications will be evaluated for scientific and technical merit by (an) appropriate Scientific Review Group(s) convened by CSR.” This means that the application will be reviewed by a regular NIH study section or a special emphasis panel (SEP) with expertise in the research area explored by the application. To identify a possible “review home” for your application, I suggest you peruse CSR’s list of study sections, find the ones that seem most suitable for your application and then use NIH RePORTER to search for funded applications that have been reviewed by those study sections. This will allow you to identify the group of scientists who have the appropriate research expertise to review your application.

Specific requests for applications (RFAs), such as RFA-GM-14-003 (Revisions for Macromolecular Interactions in Cells), are often reviewed together in the IC that issued the RFA. For example, section V.2 of GM-14-003 states, “Applications will be evaluated for scientific and technical merit by (an) appropriate Scientific Review Group(s) convened by the NIGMS.” This means that the NIGMS Office of Scientific Review will organize a review panel to review applications submitted in response to this RFA.

For applications that will be reviewed by groups convened by CSR, I encourage the investigators I speak with to write a brief cover letter for their applications that indicates which study section they think is appropriate and how they arrived at that conclusion. Sometimes, it is also helpful to indicate the type(s) of expertise you believe is needed to review your application, but you should not provide a list of reviewers, as that creates issues with potential conflicts of interest. The recommendations made in the cover letter are advisory, but the CSR Division of Receipt and Referral makes every effort to accommodate reasonable requests.

Resumption of NIH Extramural Activities

One of the biggest challenges facing NIH after the government shutdown is that it occurred during a peak review period and caused the cancellation of several hundred peer review meetings. On October 22, NIH announced that most of these meetings would be rescheduled so as to minimize the disruption of the submission/review/Council/award timeline.

While NIH may not be able to preserve the timeline for all applications, at this point, it looks as though most will still go to January 2014 council meetings. More details are available in a new NIH Guide notice.

Most of the study sections run by the NIGMS Office of Scientific Review were not affected by the shutdown and will proceed as planned over the next few weeks. While a few meetings will have to be rescheduled, we expect the results of the rescheduled meetings to be available in time for our January council meeting.

The other big challenge facing NIH and the extramural community is the disruption in the application process, since funding opportunity announcements (FOAs) could not be accessed and applications could not be submitted during the shutdown. NIH has rescheduled the submission dates that were lost and extended the dates for those FOAs that were adversely affected. More details are available in the same NIH Guide notice.

Sally Rockey’s blog provides more on NIH’s efforts to minimize the disruption of the shutdown’s effects on the extramural community.

Additional Reviewer Scoring Guidance for Overall Impact Scores on Research Applications

NIH has posted additional scoring guidance for reviewers to consider when determining overall impact scores for grant applications. Here are answers to some key questions about this new guidance.

Why provide more guidelines?

Scientific review officers and program directors noticed that reviewers have tended to arrive at overall impact scores by comparing the number of weaknesses and strengths in an application, rather than balancing the importance of the weaknesses and strengths. In addition, there was significant compression of overall impact scores around the perceived funding range. Both practices made it more difficult to gauge reviewer assessments.

What’s changed?

The additional guidance chart simplifies and clarifies the way in which reviewers should evaluate the overall impact of an application. In particular, it encourages reviewers to focus on the importance of the research problem and the likelihood that the project will succeed. The chart emphasizes a balanced assessment of the review criteria and the use of the entire scoring range. The general Scoring System and Procedure also includes a new, simplified scoring guidance chart for assigning individual criterion scores.

When will this change occur?

Some Center for Scientific Review (CSR) study sections used the new guidelines in application reviews for May 2013 advisory council meetings. So that these applications would not be disadvantaged by the deviation from the impact score distribution of previous review cycles used for percentiling, CSR recalculated the percentile base. The vast majority of study sections and special emphasis panels are using the new scoring guidance with the reviews for fall 2013 advisory council meetings.

Note: New scoring guidelines are also available for fellowships, career awards and institutional training grants.

CSR’s Percentiling Recalibration

The Center for Scientific Review (CSR) has recently adjusted the base that a number of permanent study sections and special emphasis panels (SEPs) use to calculate percentiles for certain application types. Applications that were reviewed in a SEP meeting for the May 2013 Council round will be assigned a percentile ranking calculated from the new base. In some cases, revised summary statements will be issued for those applications missing percentile scores or with an incorrect percentile. The percentile ranking will also appear in the eRA Commons account of the principal investigator(s).

We here at NIGMS are often asked to explain how percentile rankings factor into determining which awards to make. We do not rely solely on a percentile cutoff; we consider a number of additional factors, including an investigator’s career stage, a laboratory’s other research funding, NIGMS research portfolio balance, and other programmatic priorities. For more information, see http://www.nigms.nih.gov/Research/Application/Pages/SuccessRateFAQs.aspx.

How NIH Makes Grant Application Assignments

Here are answers to some of the questions we’re frequently asked about grant application assignments.

Who receives applications and makes assignments?

All applications are received by the Division of Receipt and Referral (DRR) in the Center for Scientific Review (CSR). The DRR gives each application two assignments, one for review and one for funding consideration.

How are review assignments made?

Referral officers in CSR assign an application to an integrated review group (IRG), a branch of CSR that manages the review of applications in a general scientific area. The chief of the IRG or his/her representative, in consultation with scientific review officers (SROs) in the IRG, makes the final assignment to a specific scientific review group (SRG), which can be a standing study section (SS) or a special emphasis panel (SEP). An SS reviews applications in a specific topic area within the purview of the IRG. A SEP reviews applications on special topics or when conflicts of interest exist. Find an IRG, SS or SEP.

Why isn’t the study section listed on my application?

Since the application is first assigned to an IRG, the IRG abbreviation initially appears as the review assignment. This abbreviation is replaced by the SRG label once that assignment has been made. A similar process occurs with applications assigned to an NIH institute or center (IC) for review. At NIGMS, the initial assignment is to our Office of Scientific Review, followed by assignment to a specific review group and SRO.

How are assignments for funding consideration made?

The DRR assigns the application to an IC for funding consideration. In some cases, the DRR may make a secondary assignment to another IC. The assignment is based on referral guidelines developed by each IC. These describe the IC’s focus and mission areas, interests shared with other ICs, and the funding opportunity announcements that the IC supports. Other considerations may include requests made by investigators or by ICs for secondary assignments.

Why wasn’t my application assigned to the person I thought was my program director?

It’s likely that your application received a “temporary” assignment. When an IC receives an application, it assigns a temporary program director. At NIGMS, this person is Ann Hagan, our associate director for extramural activities. Once the application has been matched with the most appropriate NIGMS division, branch and program director, it will receive a final assignment.

Why was my application assigned to an IC or SS that I didn’t request?

If you have traditionally been funded by one IC and request a change to another IC, reassignment will occur only if the application is deemed a better fit with the new IC. Another factor is the IC’s program interests, as described in its referral guidelines, program announcements (PAs) and requests for applications (RFAs). Many of these announcements are only supported by one or a subset of ICs. If an application is submitted in response to a funding opportunity announcement that is not supported by the requested IC, then it can’t be assigned to that IC.

Several factors influence the likelihood that a request for review assignment to a particular SS will be honored. The most important factor is whether the proposed research is a good fit for the scientific focus of the requested SS. Like science itself, the scientific focus of an SS evolves over time. Therefore, the SS that reviewed your application 4 years ago may no longer be suitable, and the IRG may make a different assignment. The funding opportunity announcement can also play a role in the review assignment. Applications for many RFAs and PAs with special receipt, referral and/or review considerations are reviewed by SEPs organized by CSR or by IC-specific scientific review offices. If the application is responding to one of these funding opportunity announcements, it can’t be assigned to a standing SS for review.

What should I do if I don’t get a requested study section or IC assignment?

You should contact the SRO or program director who was assigned the application. If, after discussion, a reassignment is warranted, that person will facilitate the change. If you still have concerns, you should contact the DRR.

For more details on this topic, read the CSR’s The Assignment Process.

Advisory Council Meeting

The next meeting of our Advisory Council is later this week. For more information on Council activities, see these previous Feedback Loop posts:

Discussing Your Application’s Review with Your Program Director

Study sections review applications three times a year, about halfway between the submission date and the second level of review by an advisory council. We are currently in the midst of the review cycle for the January 2013 Council meeting, which means that applicants will be getting their summary statements soon.

A recent post described a new NIH resource to explain the next steps after the review of your application. One of them is to contact your program director to discuss the critique, and I highly encourage this. Program directors read hundreds of summary statements each year, so we have a good idea about the comments that might have influenced your score, the likelihood that your application will be funded, and the types of revisions that might make your application more competitive.

If you’d like this input from your program director, the best first step is to send an e-mail to him or her after the summary statement is released (typically a few weeks after the study section has met to review your application) to arrange a time to talk. Program directors usually gain access to the summary statement around the same time you do, but giving us a chance to read and think about your review can facilitate a useful and productive conversation.

In addition to helping you determine next steps for your application, the conversation offers an opportunity to foster a relationship with the program director who manages grants in your area of research.