One of the key principles of the NIH Enhancing Peer Review efforts was a commitment to a continuous review of peer review. In that spirit, NIH conducted a broad survey of grant applicants, reviewers, advisory council members and NIH program and review officers to examine the perceived value of many of the changes that were made. The results of this survey are now available. The report analyzes responses from these groups on topics including the nine-point scoring scale, criterion scores, consistency, bulleted critiques, enhanced review criteria and the clustering of new investigator and clinical research applications.
Author: Jeremy Berg
As former NIGMS director, Jeremy oversaw the Institute’s programs to fund biomedical research and to train the next generation of scientists. He was a leader in many NIH-wide activities and also found time to study a variety of molecular recognition processes in his NIH lab.
Posts by Jeremy Berg
We just reached a milestone—our 100,000th grant application. Interestingly, it’s for a K99/R00 (Pathway to Independence) award. This program enables promising postdoctoral scientists to receive mentored—and later independent—research support. Given our strong commitment to research training, mentoring and workforce development, it’s somehow fitting that this application is for a program that addresses these needs.
I remember well when I submitted my first independent grant application, R29GM038230. It was for a FIRST award, an earlier program directed toward helping early stage investigators develop their independent careers. I submitted the application before there was a Grants.gov—and even before there were overnight delivery services. Since I was located in Baltimore, not far from NIH, I personally drove the application down to the old Westwood Building, where the predecessor to the NIH Center for Scientific Review (and NIGMS) was housed at the time. After I presented my carefully wrapped box, I watched as it was thrown on top of a pile of other applications that reminded me of the warehouse scene at the end of “Raiders of the Lost Ark” . Needless to say, I was relieved when I got the self-addressed card indicating that my application had been received and assigned a grant number.
While this is just one small example of how much things can change over time, it leads me to think about other changes and other milestones. One major milestone is coming in 2012, when NIGMS will mark its 50th anniversary. It will be a time for reflecting on the great scientific progress that has been made with NIGMS grant support and also on the opportunities and challenges that lie ahead. You can expect to hear more about the events associated with this anniversary as our plans develop.
An important step in the annual budget process is the release of the President’s budget request, which happened yesterday. The NIGMS slice is detailed in our FY 2012 budget justification, which includes a budget mechanism table, budget graphs, a Director’s overview and a justification narrative.
The proposed budget for NIH calls for an increase of 2.4% over Fiscal Year 2010 actual expenditures. The proposed budget for NIGMS represents a 2.5% increase over Fiscal Year 2010.
The reason that the comparisons are to Fiscal Year 2010 is that NIH is currently funded at that year’s level under a continuing resolution through March 4, 2011. As described in NIH Guide Notice NOT-OD-11-037, until a final Fiscal Year 2011 appropriation is enacted, NIH is issuing noncompeting research grant awards at a level below that indicated on the most recent Notice of Award (generally up to 90% of the previously committed level). This is consistent with our practice during the continuing resolutions of Fiscal Years 2006 to 2010. NIH will consider upward adjustments to these levels after a final appropriation is enacted.
The NIH Scientific Management Review Board will hold a teleconference on Wednesday, February 23, from 11:30 a.m. to 1:30 p.m. Eastern time. The discussion will focus on the NIH proposal to create a new National Center for Advancing Translational Sciences (NCATS) and its impact on other programs at NIH, including the National Center for Research Resources.
The teleconference is open to the public and time will be available for public comment, but those wishing to make comments need to sign up in advance. Details are in a Federal Register notice .
The toll-free number to participate in the teleconference is 1-800-779-1545. The participant passcode to give the meeting operator is “NIH.”
In March 2010, I announced that we were in the process of developing a strategic plan for research training, and I asked for your input.
We heard from more than 300 stakeholders, who included university faculty and administrators, graduate students and postdocs, representatives from professional societies and individuals from government and industry. We also received input from our Advisory Council at its meeting last week.
NIGMS staff and I took these comments into account in producing a draft strategic plan for training. Reflecting the Institute’s core values and vision, the plan encompasses several key themes:
- Research training is a responsibility shared by NIH, academic institutions, faculty and trainees.
- Research training focuses on student development, not simply selection of talent.
- Breadth and flexibility enable research training to keep pace with the opportunities and demands of contemporary science and provide the foundation for a variety of scientific career paths.
- Diversity is an indispensable component of research training excellence, and it must be advanced across the entire research enterprise.
We’ve posted the draft training strategic plan for public comment. I invite you to read the plan and give us your input. Between now and February 15, you may submit your comments anonymously through our online form.
I recently described the role that an advisory council plays as the second level of peer review for applications submitted to NIH. One thing that neither advisory councils nor study sections do, however, is make funding decisions. How, then, are these decisions made?
In this post, I’ll describe the process we use at NIGMS.
The Institute is organized into five units: four divisions (Cell Biology and Biophysics; Genetics and Developmental Biology; Minority Opportunities in Research; and Pharmacology, Physiology, and Biological Chemistry) and a center (Bioinformatics and Computational Biology). Once the National Advisory General Medical Sciences Council has met, each unit organizes meetings (referred to as “paylist meetings”) attended by most or all of the program directors within that unit.
During a paylist meeting, applications are discussed and prioritized, beginning with the top-scoring applications. These applications (typically up to about half of the number that are expected to be funded) are given highest priority for funding unless there are specific issues, such as those related to the NIGMS well-funded laboratory policy or other concerns that came up at the Council meeting.
The discussion then turns to applications in the “gray area,” typically extending to about 10 percentile points beyond where we would expect to be able to fund applications if they were awarded in straight percentile order. Each application is discussed, typically in percentile order, although sometimes early-stage investigators (ESIs) are discussed first.
For each application, the responsible program director presents the scientific topic as well as factors such as whether the applicant is an ESI or new investigator, how much other support the applicant has (particularly if the application represents the only support available to the investigator), whether the Council has given us specific advice on the application, whether the scientific area is perceived to be particularly exciting, and how much other research we already support in the general area of the application. The other members of the unit listen to these presentations, and the group then produces a prioritized list of applications.
The other key factor for final funding decisions is, of course, the availability of funds. Funds are provided through the appropriations process, either through a regular appropriations bill or, sometimes toward the beginning of a fiscal year, a continuing resolution that typically funds government programs at the previous year’s level. When it is reasonably clear what level of funds is available at a particular point in the fiscal year, the funds are allocated to different mechanisms and programs (research project grants, training grants, various programs within the Division of Minority Opportunities in Research, and so on) based on our previously established budget. Funds for unsolicited R01s are allocated among the four units within NIGMS that fund these applications (the Divisions of Cell Biology and Biophysics; Genetics and Developmental Biology; and Pharmacology, Physiology, and Biological Chemistry; and the Center for Bioinformatics and Computational Biology), based on the fractions of applications that have scored well enough to be considered for possible funding.
Paylists are then developed using the prioritized lists, with budget adjustments for each application based on NIH and NIGMS-wide policies as well as considerations specific to the application provided by the responsible program director. Applications are paid until the available funds are exhausted. Applications that are relatively high on the priority list but could not be funded with a given allocation are flagged for consideration later in the fiscal year, when more funds may become available. This process leads, over the course of a full fiscal year, to the funding curves we recently posted.
NIH recently launched a new site for communication with the scientific community, http://feedback.nih.gov/. The site has already been quite active, since it requests input on a proposed National Center for Advancing Translational Sciences (NCATS) and a proposed institute focused on substance use, abuse and addiction research.
Of particular interest may be a recent post on a “straw model” regarding where current National Center for Research Resources (NCRR) programs might end up if they are redistributed as a result of the formation of NCATS. In this model, some NCRR programs would be transferred to NIGMS.
An even more recent post provides information about open conference calls for grantees and others who are interested in NCRR programs to discuss the straw model. These calls will be held today through Friday.
If you have an interest in these rapidly moving activities, the NIH Feedback site is a good place to find updates and to submit your thoughts.
Later this month, the National Advisory General Medical Sciences Council will hold the first of its three meetings in 2011. While many applicants, grantees and reviewers are familiar with the roles and processes of study sections, fewer know how an advisory council works. In this post, I’ll provide an overview of its many critical roles.
Council members are leaders in the biological and medical sciences, education, health care and public affairs. Their areas of expertise cover the broad range of scientific fields supported by NIGMS. The Council performs the second level of peer review for research and research training grant applications assigned to NIGMS. Council members also offer advice and recommendations on policy and program development, program implementation, evaluation and other matters of significance to the mission and goals of the Institute.
A portion of each Council meeting is open to the public.
For the peer review function, which occurs during the part of the meeting that is closed to the public, Council members read summary statements, providing a general check on the quality of the first level of peer review. They advise us if they find cases where the comments and scores do not appear to be in good alignment. Their evaluation complements the initial peer review done by study sections, as it focuses primarily on summary statements rather than on applications (although Council members may have access to the applications).
The Council also provides input on cases where staff are considering exceptions to the well-funded laboratory policy, and it approves the potential funding of grants to investigators at foreign institutions. Another area of Council input relates to Method to Extend Research in Time (MERIT) awards. Finally, Council members point out applications that they feel are particularly interesting based on their scientific expertise and knowledge of trends in particular fields. They explain their perspective to NIGMS staff, who incorporate this input in subsequent steps of the funding decision process. I’ll describe these steps in an upcoming post.
The policy and program advisory function includes discussing “concept clearances,” or ideas for new initiatives being considered within the Institute. These can take the form of proposed requests for applications (RFAs) or program announcements (PAs). Council members provide critical analysis and feedback about the appropriateness of proposed initiatives and factors to consider should they be implemented. Approved concept clearances are posted soon after each Council meeting on the NIGMS Web site and often on the Feedback Loop. NIGMS staff can then receive input from the scientific community as they refine the funding opportunity announcements.
This month’s meeting will include one concept clearance presentation, on macromolecular complexes.
Council members also give input and feedback on assessments and formal evaluations of specific NIGMS programs, such as the Protein Structure Initiative. When the need arises, Council members form working groups focused on specific issues. To ensure an appropriate range of expertise and perspectives, these groups can include non-Council members, as well. Finally, the Council receives periodic reports about ongoing initiatives in order to monitor how they are proceeding and offer advice about possible changes.
Fiscal Year 2010 ended on September 30, 2010. We have now analyzed the overall results for R01 grants, shown in Figures 1-3.
The total NIGMS expenditures (including both direct and indirect costs) for R01 grants are shown in Figure 3 for Fiscal Year 1996 through Fiscal Year 2010.
What do we anticipate for the current fiscal year (Fiscal Year 2011)? At this point, no appropriation bill has passed and we are operating under a continuing resolution through March 4, 2011, that funds NIH at Fiscal Year 2010 levels. Because we do not know the final appropriation level, we are not able at this time to estimate reliably the number of competing grants that we will be able to support. We can, however, estimate the number of research project grant applications in the success rate base (correcting for applications that are reviewed twice in the same fiscal year). We predict that this number will be approximately 3,875, an increase of 17% over Fiscal Year 2010.
UPDATE: The original post accidentally included a histogram from a previous year. The post now includes the correct Fiscal Year 2010 figure.
This morning, I announced that I will step down as NIGMS Director at the end of June 2011. I had no intention of leaving NIGMS at this point, but am doing so in support of the career of my wife, Wendie, a leading breast imaging clinical researcher. After a change in her situation in May, we have been looking for a suitable position for her to continue her work on testing new methods for breast cancer screening. She has been actively recruited by a number of institutions around the country, and we have particularly explored options in the Baltimore-Washington area.
After considering all known options, we have decided to accept positions at the University of Pittsburgh. She will be starting in the Department of Radiology at Magee-Womens Hospital of UPMC in March 2011. I will be waiting until the end of June to move in order to complete some important projects at NIGMS and to allow our youngest child to finish her freshman year of high school. I will be serving as the University of Pittsburgh’s Associate Senior Vice Chancellor for Science Strategy and Planning in the Health Sciences and as a faculty member in the School of Medicine’s Department of Computational and Systems Biology.
My time at NIGMS has been one of the highlights of my career. When I joined the Institute more than 7 years ago, I was immediately impressed with the dedication and competence of the staff at all levels. During my tenure, we have been able to recruit a number of outstanding individuals to join this team. So while I am very sad to leave such an outstanding organization, I am confident that it will be in good hands, and I look forward to the new adventures that await me and my family.