Further Analysis of Renewal Rates for New and Established NIGMS Investigator Projects

In an earlier blog post, I presented data on the first competing renewal rates of R01 projects that NIGMS awarded to new and established investigators. The analysis showed that no renewal application was submitted for a substantial percentage of projects—30% of new projects from new investigators and 45% of new projects from established investigators. This raises questions, such as those suggested by Feedback Loop readers, including:

  • Do projects for which no renewal is submitted generally have less productivity or scientific impact?
  • Are new projects awarded to established investigators more likely to represent the second or third award to that investigator?

I’ve tried to explore these questions in a further analysis.

Continue reading

Improved Success Rate and Other Funding Trends in Fiscal Year 2014

The Consolidated and Further Continuing Appropriations Act, 2015 Exit icon, provides funding for the Federal Government through September 30. NIGMS has a Fiscal Year 2015 appropriation of $2.372 billion, which is $13 million, or 0.5%, higher than it was in Fiscal Year 2014.

As I explained in an earlier post, we made a number of adjustments to our portfolio and funding policies last fiscal year in order to bolster our support for investigator-initiated research. Partly because of these changes, the success rate for research project grants (RPGs)—which are primarily R01s—was 25 percent in Fiscal Year 2014. This is 5 percentage points higher than it was in Fiscal Year 2013. Had we not made the funding policy changes, we predicted that the success rate would have remained flat at 20 percent.

Figure 1 shows the number of RPG applications we received and funded, as well as the corresponding success rates, for Fiscal Years 2002-2014.

Figure 1. Number of competing RPG applications assigned to NIGMS (blue line with diamonds, left axis) and number funded (red line with squares, left axis) for Fiscal Years 2002-2014. The success rate (number of applications funded divided by the total number of applications) is shown in the green line with triangles, right axis. Data: Tony Moore.
Figure 1. Number of competing RPG applications assigned to NIGMS (blue line with diamonds, left axis) and number funded (red line with squares, left axis) for Fiscal Years 2002-2014. The success rate (number of applications funded divided by the total number of applications) is shown in the green line with triangles, right axis. Data: Tony Moore.

Moving forward, it will be important to employ strategies that will enable us to at least maintain this success rate. In keeping with this goal, we recently released a financial management plan that continues many of the funding policies we instituted last year. As funds from the retirement of the Protein Structure Initiative come back into the investigator-initiated RPG pool, we’ll be working to ensure that they support a sustained improvement in success rate rather than create a 1-year spike followed by a return to lower rates.

Figures 2 and 3 show data for funding versus the percentile scores of the R01 applications we received. People frequently ask me what NIGMS’ percentile cutoff or “payline” is, but it should be clear from these figures that we do not use a strict percentile score criterion for making funding decisions. Rather, we take a variety of factors into account in addition to the score, including the amount of other support already available to the researcher; the priority of the research area for the Institute’s mission; and the importance of maintaining a broad and diverse portfolio of research topics, approaches and investigators.

Figure 2. Percentage of competing R01 applications funded by NIGMS as a function of percentile scores for Fiscal Years 2010-2014. For Fiscal Year 2014, the success rate for R01 applications was 25.7 percent, and the midpoint of the funding curve was at approximately the 22nd percentile. Data: Jim Deatherage.
Figure 2. Percentage of competing R01 applications funded by NIGMS as a function of percentile scores for Fiscal Years 2010-2014. For Fiscal Year 2014, the success rate for R01 applications was 25.7 percent, and the midpoint of the funding curve was at approximately the 22nd percentile. See more details about the data analysis for Figure 2. Data: Jim Deatherage.
Figure 3. Number of competing R01 applications (solid black bars) assigned to NIGMS and number funded (striped red bars) in Fiscal Year 2014 as a function of percentile scores. Data: Jim Deatherage.
Figure 3. Number of competing R01 applications (solid black bars) assigned to NIGMS and number funded (striped red bars) in Fiscal Year 2014 as a function of percentile scores. See more details about the data analysis for Figure 3. Data: Jim Deatherage.

It’s too early to say what the success rate will be for Fiscal Year 2015 because it can be influenced by a number of factors, as I described last year. However, we’re hopeful that by continuing to adjust our priorities and policies to focus on supporting a broad and diverse portfolio of investigators, we can reverse the trend of falling success rates seen in recent years.

Examining the First Competing Renewal Rates of New NIGMS Investigators

The successful entry and retention of new investigators into biomedical research is a priority for us, and the renewal rate of this group’s first R01 research grants is an important indicator for this goal. Here are the results of an analysis I did of the first competing renewal rates for new and established investigators.

Figure 1 shows that the first competing renewal rate of new investigators’ first NIGMS R01 or R29 grants has declined over the past 10 years. This trend is similar to the one for overall NIGMS R01 application success rates.

Percent Funded by End of Project Period (approximate) 53% 2002, 53% 2003, 52% 2004, 45% 2005, 44% 2006, 43% 2007, 43% 2008, 38% 2009, 39% 2010, 35% 2011, 32% 2012
Figure 1. Percentage of new investigators’ R01 and R29 grants that were successfully renewed. The horizontal axis is the fiscal year in which the first project period ended. The vertical axis is the percentage of these projects that were successfully renewed at least once (regardless of whether the new or amended competing renewal application was funded) by the end of Fiscal Year 2014.

Figure 2 gives a more complete picture of the renewal history of new investigators’ NIGMS R01 and R29 projects. In addition to the renewal rate (also shown in Figure 1), it shows the percentage of projects for which at least one renewal application was submitted but was not successfully renewed as well as the percentage of projects for which no renewal application was submitted.

Renewal history at the end date of first project based on paid, not paid, and no application submitted (approximate). 2002 53% paid, 20% not paid, 27% no application. 2003 53% paid, 23% not paid, 24% no application. 2004 52% paid, 27% not paid, 21% no application. 2005 45% paid, 32% not paid, 23% no application. 2006 43% paid, 34% not paid, 23% no application. 2007 43% paid, 30% not paid, 27% no application. 2008 43% paid, 33% not paid, 24% no application. 2009 38% paid, 33% not paid, 29% no application. 2010 39% paid, 36% not paid, 25% no application. 2011 35% paid, 38% not paid, 27% no application. 2012 32% paid, 35% not paid, 33% no application.
Figure 2. Renewal history of new investigators’ R01 and R29 grants between Fiscal Years 2002-2012. The bottom section (green) shows successful renewals (paid), which are also shown in Figure 1; the middle section (red) shows grants for which renewal was attempted but was not successful (not paid); and the top section (blue) shows grants for which no renewal application was submitted (no app).

Figure 3 shows that success in renewing an NIGMS-funded R01 grant correlates positively with how long the grant has been active.

Percentage of Projects funded by grant years (approximate) 31% 1st renewals for grant years 4-6, 49% 2nd renewals for grant years 7-12, 54% for 3rd and greater renewals for grant years 12 or greater
Figure 3. Competing renewal rates for the first, second and third or more renewals of NIGMS R01 grants awarded between Fiscal Years 2004-2007 to new and established investigators.

Since first renewals have lower success rates than subsequent renewals, Figure 4 addresses whether new investigators seeking to renew their first R01 grants are competitive with established investigators who are renewing long-term and/or new projects. The figure shows that the renewal rate for all projects from established investigators, including new as well as long-term projects, is higher than the renewal rate of projects from new investigators (46 percent in the left column versus 36 percent in the right column). However, when focusing only on the first renewals of new projects (in the middle and right columns), new investigators are renewing at a higher rate than are established investigators (36 percent versus 30 percent).

Renewal history by percentage of projects FY2004-FY2007 by type of investigator (approximate). All renewals by established investigators 46% paid, 24% not paid, 30% no application. Renewals of new projects by established investigators 30% paid, 25% not paid, 45% no application. Renewals of new projects by new investigators, 36% paid, 35% not paid, 29% no application.
Figure 4. Renewal history of NIGMS R01 projects from new and established investigators that were initially funded by NIGMS between Fiscal Years 2004-2007.

Figure 5 shows the relative success of new and established investigators in renewing new projects as a function of the percentile score obtained on the initial award. As the “Paid” sections of the bars indicate, for each of the percentile groups, the overall renewal rate for new investigators’ new R01s was higher than that for established investigators’ new R01s.

Percentage of new and established investigators' projects renewed in relation to the percentile ranking of the original award (approximate). Percentile 0-9, new investigators 41% paid, 28% not paid, 31% no application. Percentile 0-9 established investigators, 34% paid, 22% not paid, 44% no application. Percentile 10-19, new investigators, 32% paid, 41% not paid, 27% no application. Percentile 10-19, established investigators 28% paid, 29% not paid, 43% no application. 20th percentile and greater, new investigators 41% funded, 36% not paid, 23% no application. 20th percentile and greater, established investigators 26% paid, 29% not paid, 45% no application.
Figure 5. Renewal history of NIGMS R01 projects from new and established investigators that were initially funded between Fiscal Years 2004-2007 in relation to the percentile ranking (0-9th, 10-19th, and 20th and higher percentiles) of the original award. The number of projects in each of the six categories analyzed from left (new investigators, 0-9th percentile) to right (established investigators, ≥20th percentile) are: 239, 328, 299, 347, 172 and 102.

Recognizing the importance of new investigators in sustaining the vitality of biomedical research, we give special consideration to applications from them, and in some cases, we fund these applications at percentiles beyond those for most established investigators. The data in Figure 5 supports this practice by showing that the renewal rates of new investigators whose original applications scored at or above the 20th percentile are about the same as, or higher than, those for new and established investigators whose original applications scored in the 0-9th percentile range.

More About This Analysis

This analysis includes Recovery Act projects and excludes withdrawn applications and multi-principal investigator grants.

Definitions
R01 projects: Research project grants.
R29 projects: First Independent Research Support and Transition (FIRST) awards, R01-type research project grants awarded to new investigators available from 1987 to 1998.
Renewal rate: Percentage of grants that were successfully renewed by the date of this analysis (end of Fiscal Year 2014), regardless of whether a new or amended competing renewal application was funded.
Grant year: Grant year in which the renewal R01 application was submitted.
New investigator: An individual who has not previously competed successfully as a program director/principal investigator for a substantial NIH independent research award (see http://grants.nih.gov/grants/glossary.htm#NewInvestigator).

More on My Shared Responsibility Post

Thanks for all of the comments and discussion on my last post. There were many good points and ideas brought up, and these will be very useful as we consider additional policy changes at NIGMS and NIH. I hope these conversations will continue outside of NIH as well.

Several people asked about the current distribution of funding among NIGMS principal investigators (PIs). Here are a few relevant statistics:

  • In terms of the NIH research funding of NIGMS grantees, in Fiscal Year 2013, 5 percent of the PIs had 25 percent of this group’s total NIH direct costs and 20 percent of the PIs had half of it. A similar pattern was recapitulated NIH-wide.
  • NIGMS PIs who had over $500,000 in total NIH direct costs held approximately $400 million in NIGMS funding.
  • The figure below shows the distribution of total NIH direct costs for NIGMS-supported investigators as well as the average number of NIH research grants held by PIs in each range.
Graph representing distribution of NIGMS investigartors' total NIH direct costs for research in FY2013
Figure 1. The distribution of NIGMS investigators’ total NIH direct costs for research in Fiscal Year 2013 (blue bars, left axis). The number below each bar represents the top of the direct cost range for that bin. The average number of NIH research grants held by PIs in each group is also shown (red line with squares, right axis). The direct costs bin ranges were chosen so that the first four bins each included 20 percent of NIGMS investigators.

With regard to changes NIH might make to help re-optimize the biomedical research ecosystem, NIH Director Francis Collins recently formed two NIH-wide working groups to develop possible new policies and programs related to some of the issues that I highlighted in my blog post and that were discussed in the subsequent comments. The first group, chaired by NIH Deputy Director for Extramural Research Sally Rockey, will explore ways to decrease the age at which investigators reach independence in research. The second, chaired by me, will look at developing more efficient and sustainable funding policies. Once these committees have made their recommendations, Sally plans to set up a group to consider the question of NIH support for faculty salaries.

As I mentioned in my post, we at NIGMS have been working for some time on these issues. We’ll be discussing additional changes and ideas with the community in the coming weeks and months on this blog and in other forums, including our upcoming Advisory Council meeting.

A Shared Responsibility

The doubling of the NIH budget between 1998 and 2003 affected nearly every part of the biomedical research enterprise. The strategies we use to support research, the manner in which scientists conduct research, the ways in which researchers are evaluated and rewarded, and the organization of research institutions were all influenced by the large, sustained increases in funding during the doubling period.

Despite the fact that the budget doubling ended more than a decade ago, the biomedical research enterprise has not re-equilibrated to function optimally under the current circumstances. As has been pointed out by others (e.g., Ioannidis, 2011; Vale, 2012; Bourne, 2013; Alberts et al., 2014), the old models for supporting, evaluating, rewarding and organizing research are not well suited to today’s realities. Talented and productive investigators at all levels are struggling to keep their labs open (see Figure 1 below, Figure 3 in my previous post on factors affecting success rates and Figure 3 in Sally Rockey’s 2012 post on application numbers). Trainees are apprehensive about pursuing careers in research (Polka and Krukenberg, 2014). Study sections are discouraged by the fact that most of the excellent applications they review won’t be funded and by the difficulty of trying to prioritize among them. And the nation’s academic institutions and funding agencies struggle to find new financial models to continue to support research and graduate education. If we do not retool the system to become more efficient and sustainable, we will be doing a disservice to the country by depriving it of scientific advances that would have led to improvements in health and prosperity.

Re-optimizing the biomedical research enterprise will require significant changes in every part of the system. For example, despite prescient, early warnings from Bruce Alberts (1985) about the dangers of confusing the number of grants and the size of one’s research group with success, large labs and big budgets have come to be viewed by many researchers and institutions as key indicators of scientific achievement. However, when basic research labs get too big it creates a number of inefficiencies. Much of the problem is one of bandwidth: One person can effectively supervise, mentor and train a limited number of people. Furthermore, the larger a lab gets, the more time the principal investigator must devote to writing grants and performing administrative tasks, further reducing the time available for actually doing science.

Although certain kinds of research projects—particularly those with an applied outcome, such as clinical trials—can require large teams, a 2010 analysis by NIGMS and a number of subsequent studies of other funding systems (Fortin and Currie, 2013; Gallo et al., 2014) have shown that, on average, large budgets do not give us the best returns on our investments in basic science. In addition, because it is impossible to know in advance where the next breakthroughs will arise, having a broad and diverse research portfolio should maximize the number of important discoveries that emerge from the science we support (Lauer, 2014).

These and other lines of evidence indicate that funding smaller, more efficient research groups will increase the net impact of fundamental biomedical research: valuable scientific output per taxpayer dollar invested. But to achieve this increase, we must all be willing to share the responsibility and focus on efficiency as much as we have always focused on efficacy. In the current zero-sum funding environment, the tradeoffs are stark: If one investigator gets a third R01, it means that another productive scientist loses his only grant or a promising new investigator can’t get her lab off the ground. Which outcome should we choose?

My main motivation for writing this post is to ask the biomedical research community to think carefully about these issues. Researchers should ask: Can I do my work more efficiently? What size does my lab need to be? How much funding do I really need? How do I define success? What can I do to help the research enterprise thrive?

Academic institutions should ask: How should we evaluate, reward and support researchers? What changes can we make to enhance the efficiency and sustainability of the research enterprise?

And journals, professional societies and private funding organizations should examine the roles they can play in helping to rewire the unproductive incentive systems that encourage researchers to focus on getting more funding than they actually need.

We at NIGMS are working hard to find ways to address the challenges currently facing fundamental biomedical research. As just one example, our MIRA program aims to create a more efficient, stable, flexible and productive research funding mechanism. If it is successful, the program could become the Institute’s primary means of funding individual investigators and could help transform how we support fundamental biomedical research. But reshaping the system will require everyone involved to share the responsibility. We owe it to the next generation of researchers and to the American public.

Graph representing NIGMS principal investigators (PIs) without NIH R01 funding between 200 and 2014.
Figure 1. The number of NIGMS principal investigators (PIs) without NIH R01 funding has increased over time. All NIGMS PIs are shown by the purple Xs (left axis). NIGMS PIs who were funded in each fiscal year are represented by the orange circles (left axis). PIs who had no NIH funding in a given fiscal year but had funding from NIGMS within the previous 8 years and were still actively applying for funding within the previous 4 years are shown by the green triangles (left axis); these unfunded PIs have made up an increasingly large percentage of all NIGMS PIs over the past decade (blue squares; right axis). Definitions: “PI” includes both contact PIs and PIs on multi-PI awards. This analysis includes only R01, R37 and R29 (“R01 equivalent”) grants and PIs. Other kinds of NIH grant support are not counted. An “NIGMS PI” is defined as a current or former NIGMS R01 PI who was either funded by NIGMS in the fiscal year shown or who was not NIH-funded in the fiscal year shown but was funded by NIGMS within the previous 8 years and applied for NIGMS funding within the previous 4 years. The latter criterion indicates that these PIs were still seeking funding for a substantial period of time after termination of their last NIH grant. Note that PIs who had lost NIGMS support but had active R01 support from another NIH institute or center are not counted as “NIGMS PIs” because they were still funded in that fiscal year. Also not counted as “NIGMS PIs” are inactive PIs, defined as PIs who were funded by NIGMS in the previous 8 years but who did not apply for NIGMS funding in the previous 4 years. Data analysis was performed by Lisa Dunbar and Jim Deatherage.

UPDATE: For additional details, read More on My Shared Responsibility Post.

A Look at Our AREA Grants

Academic Research Enhancement Award (AREA, R15) grants support small-scale research projects in the biomedical and behavioral sciences conducted by faculty and students at educational institutions that have not been major recipients of NIH research grant funds. Recently, a faculty member at an AREA grant-eligible institution wrote to NIGMS Director Jon Lorsch urging the Institute to support more AREA grants, arguing that these grants not only train students but are also cost-effective. This prompted us to take a close look at our portfolio of R15 grants. I’d like to share what we found. Thanks to Tony Moore and Ching-Yi Shieh for providing data in the figures.

NIGMS receives the largest number of R15 applications of any NIH institute. This is not surprising, since faculty and students at eligible institutions typically focus on basic research using model organisms and systems. Table 1 shows that the number of AREA grants awarded by NIGMS in each of the last 10 fiscal years has varied from a high of 63 in Fiscal Year 2007 to a low of 36 in Fiscal Year 2010 and that total funding for these grants has ranged from $8.9 million to $18.4 million. As shown in the first figure, NIGMS funds more R15s than any other institute, in recent years between 21% and 29% of the NIH total.

 Fiscal Year Number of Applications Number of
Awards
Total Funding
($ in millions)
 2004 128 48 $9,867
 2005 142 49 $10,382
 2006 171 50 $10,602
 2007 200 63 $13,387
 2008 167 53 $11,158
 2009 172 42 $8,903
 2010 199 36 $9,766
 2011 313 62 $18,441
 2012 306 56 $17,925
 2013 304 45 $16,035

Table 1. Number of R15 applications received and awarded by NIGMS and the total funding for R15s in Fiscal Years 2004-2013.

Figure 1. Percentage of NIH R15 dollars awarded by NIGMS. NIGMS (in yellow) has typically supported between 21% and 29% of NIH-funded R15s. The exception was in Fiscal Year 2010, the last year of Recovery Act funding, when the NIH Office of the Director (OD) co-funded a large number of R15s.

The NIGMS success rate for R15s tends to be higher than the overall NIH success rate, although both have been declining steadily over the past 10 years (Figure 2). This decline is due to several factors: an increase in the number of applications, a bump-up in the size of AREA grants in Fiscal Year 2010 from $150,000 to $300,000 in direct costs, and a flat NIH budget. Figure 2 also shows that success rates for R01 grants have been falling as well. While success rates for both NIGMS and NIH R15 grants had usually been higher than those for R01s, in the last several years they have been lower.

Figure 2. Success rates of NIGMS and NIH R15s and R01s in Fiscal Years 2004-2013. Although the NIGMS success rates for both R15s and R01s tend to be higher than the NIH R15 and R01 success rates, all have been declining for the past 10 years. The declines have been greater for R15s than for R01s. In Fiscal Year 2004, 38% of NIGMS R15 applicants were awarded R15s versus 15% in Fiscal Year 2013. The decline in success rate for R15s is due largely to the increase in the number of applications and, since Fiscal Year 2010, to the increased amount of money an applicant can request.

As you can see in Figure 3, over the 10-year period, the funds spent on R15 grants have fluctuated and have made up between about 0.7% and 1.3% of NIGMS’ budget for research project grants (which are largely R01s). Across NIH, AREA grants account for an even smaller amount, about 0.5% last year compared with 1.2% for NIGMS.

Figure 3. Dollars (in thousands) spent by NIGMS for R15s in Fiscal Years 2004-2013 (blue bars, right axis) and the percent of the NIGMS research project grant (RPG) budget that went to R15 grants (green line, left axis).

Does our investment in AREA grants pay off? There are a number of ways to estimate their impact, including quantitative measures such as the number of publications that result from the project, as well as outcomes that are more difficult to measure such as encouraging students to pursue careers in biomedical research and enhancing the educational environment.

While the number of publications per grant is far from a perfect indicator of research productivity, we found that the number of publications attributed to any AREA grant over its entire duration varies tremendously, as shown in Figure 4. Nearly three-fourths of all AREA grantees publish at least one paper, and some produce many publications over the lifetime of their awards. Considering that AREA grantees often have heavy teaching loads and employ undergraduates rather than graduate students and postdocs to assist with the research, these numbers are encouraging.

Figure 4. Number of publications over the duration of NIGMS R15 grants in Fiscal Years 2004-2013.

We hope you will share your AREA grant success stories with us.

Budget for Fiscal Year 2014 and Beyond

We recently posted our financial management plan for Fiscal Year 2014 (no longer available) and a budget table showing our Fiscal Year 2014 operating plan by mechanism. Consistent with our emphasis on bolstering our commitment to investigator-initiated research, we’re making every effort to move financial resources into research project grants (RPGs), which include R00s, R01s, R15s, R21s, R37s, P01s, DP1s, DP2s and U01s.

We estimate that these efforts will increase our RPG success rate from 19.9% in Fiscal Year 2013 to more than 22% in Fiscal Year 2014. This translates into funding about 100 more competing RPGs than we did in Fiscal Year 2013.

NIGMS’ plan aligns with NIH’s policies and includes these key elements:

  • We will fund noncompeting RPGs at the committed levels. We will restore those already funded at 90% to the committed levels.
  • The overall average cost of competing RPGs will be at approximately 2% above the Fiscal Year 2013 level.
  • We will fund noncompeting IDeA and AIDS research centers, which have required budget levels or a specific mandated policy, at the committed levels.
  • We will reduce other P41, P50 and U54 noncompeting awards by 10% from the committed levels. We will revise those already funded at levels below 90% accordingly. Future funding levels for these center mechanisms will depend on the Institute’s budget and efforts to increase support for our investigator-initiated RPG pool.
  • We will reduce by approximately 50% the funds set aside for new and competing research center awards in response to the targeted funding opportunity announcements we issued recently:
  • We will increase Ruth L. Kirschstein undergraduate and graduate student stipends by 2%. Entry-level postdoctoral stipends will increase to $42,000, with 4% increases between the years of experience levels. See NOT-OD-14-046 for the full range of Fiscal Year 2014 stipends.
  • We will continue to support new investigators on R01-equivalent awards at success rates comparable to or better than those of established investigators submitting new (Type 1) R01-equivalent applications.

Looking ahead to Fiscal Year 2015, you can find information about the President’s budget request for NIH and read the NIGMS Fiscal Year 2015 budget justification to learn more about the specifics of the proposed budget for our Institute.

Funding Trends and Factors Affecting Success Rate

On January 17, the President signed into law the Consolidated Appropriations Act of 2014 Exit icon, funding the Federal Government for the remainder of the fiscal year and providing increased support for NIH relative to the post-sequester levels of 2013. The NIGMS Fiscal Year 2014 budget is $2.359 billion, which is about $66 million, or 2.9%, higher than it was in Fiscal Year 2013.

It’s too early to know what this will mean for the NIGMS grant application success rate—the number of competing R01 applications we fund divided by the total number of competing R01 applications we receive. A number of other, largely independent factors in addition to the budget can impact the success rate, as well. We hope that our latest analysis of NIGMS funding trends illuminates the interplay among some of these factors. Thanks to Tony Moore, Jim Deatherage and Ching-Yi Shieh for help with the data collection and analysis.

Figure 1 shows the percentage of R01 applications funded by NIGMS as a function of percentile scores for Fiscal Years 2009-2013. Fewer grants scoring above about the 12th percentile were funded last year than in the previous 4 years. In fact, the funding curves shifted more to the left each year in this period, except in 2012, which had a slightly better success rate than 2011. This was due in part to a dip in noncompeting R01 grants that had to be funded along with no increase in the number of competing applications relative to 2011. (For a 4-year R01, the second, third and fourth years are all noncompeting grants because they are funded without review by a study section. The funds required to pay the noncompeting awards are often referred to as “out-year commitments.”) Funding levels fell again in Fiscal Year 2013 due to the sequester cuts and increases in the numbers of competing applications and noncompeting grants (see below).

Figure 1. Percentage of competing R01 applications funded by NIGMS as a function of percentile scores for Fiscal Years 2009-2013. For Fiscal Year 2013, the success rate for R01 applications was 21%, and the midpoint of the funding curve was at approximately the 17th percentile. See more details about Figure 1 analysis.

Figure 2 presents a more granular view of the data for Fiscal Year 2013. The solid black bars correspond to the number of NIGMS competing R01 applications that scored at each percentile. The striped red bars show the number of these applications that we funded.

Figure 2. Number of competing R01 applications (solid black bars) assigned to NIGMS and number funded (striped red bars) in Fiscal Year 2013 as a function of percentile scores. See more details about Figure 2 analysis.

Figure 3 shows the success rate for Fiscal Years 2000-2013 (green line with triangles; right axis), the total number of NIGMS R01 applications each year (blue line with diamonds) and the number of funded competing R01 grants (red line with squares, left axis). Between Fiscal Years 2000 and 2003, the last year of the NIH budget doubling, the success rate was 37-38%. After the budget doubling ended, the success rate declined, falling to 26% in 2006. In 2007, the success rate jumped to 33%, largely due to a combination of a budget increase for NIH and a dip in the number of noncompeting grants NIGMS had to fund that year. (See below for more about changes in the number of noncompeting grants.) Over the next 6 years, the success rate for R01s dropped to 21%, the lowest level in two decades. Note that the Fiscal Year 2013 success rate for all research project grants (RPGs), which include R00s, R01s, R15s, R21s, R37s, P01s, DP1s, DP2s and U01s, was 19.9%. This is lower than the success rate for R01s alone, which was 21%.

Figure 3. Number of competing R01 applications assigned to NIGMS (blue line with diamonds, left axis) and number funded (red line with squares, left axis) for Fiscal Years 2000-2013. The success rate is shown in the green line with triangles (right axis).

One reason the success rate has fallen is that the number of applications increased between Fiscal Years 2002-2005 and then again between Fiscal Years 2010-2013. Similar trends are seen NIH-wide and are the result of a 50% increase in the number of investigators applying for grants along with a smaller increase in the average number of applications submitted per investigator.

Another factor that influences the success rate is the number of noncompeting awards. The more noncompeting awards we need to make in a given year, the fewer competing grants we can fund. The number of noncompeting RPGs cycles with a 4-year period, as shown in Figure 4.

Figure 4. Number of noncompeting (blue line with diamonds, left axis) and competing (red line with squares, right axis) RPGs funded by NIGMS for Fiscal Years 2000-2013. Note that the Y axes do not start at 0.

This cycle was apparently set in the early 1990s when the Institute shifted the duration of most competing RPGs from 5 to 4 years. The result was that the last group of 5-year awards and the first group of 4-year awards ended in the same year, creating a significant dip in noncompeting grants and a corresponding jump in competing grants that the Institute could fund. The bolus of grants awarded that year kept noncompeting commitments high for the next 3 years, until they all came up for renewal again. As you can see in Figure 4, the cycle has persisted to this day.

The cycle may lead you to wonder if you should try to time your application to correspond to a trough in noncompeting grants. You might get lucky and have all of the factors that drive success rate work in your favor. However, if other factors act to decrease the success rate, you might end up worse off. For example, even though 2011 was a trough year for noncompeting grants (Figure 4, blue line with diamonds), a jump in the number of applications (Figure 3, blue line with diamonds) worked in the opposite direction, and the success rate actually fell (Figure 3, green line with triangles).

Finally, the Institute’s budget is also a factor in determining the success rate. Figure 5 shows the NIGMS budget (red line with squares, left axis), the budget committed to competing and noncompeting RPGs (blue line with diamonds, left axis) and the ratio of the RPG budget to the total NIGMS budget (green line with triangles, right axis).

Figure 5. Total NIGMS budget (red line with squares, left axis) and budget committed to competing and noncompeting RPGs (blue line with diamonds, left axis) for Fiscal Years 2000-2013. The green line with triangles shows the ratio of the RPG budget to the total NIGMS budget (right axis). The jump in the NIGMS budget and corresponding drop in the RPG/NIGMS budget ratio occurred when large, primarily non-RPG programs were transferred to NIGMS along with their associated funds from the former National Center for Research Resources.

The Institute’s budget grew during the NIH budget doubling (1998-2003). It also jumped by nearly $400 million when the Institutional Development Award (IDeA) and Biotechnology Research Resources programs moved from the former National Center for Research Resources to NIGMS. A variety of pressures were responsible for the ~2% decline in the ratio of RPG funds to the NIGMS budget that occurred during Fiscal Years 2005-2011, including commitments to targeted initiatives and increased funding for training programs.

Because we don’t yet know the number of applications we will receive or the number of noncompeting grants that will end early due to retirements or other events, we can’t predict what our success rate will be in Fiscal Year 2014. We hope, however, that the steps we are taking to bolster our commitment to investigator-initiated RPGs will have a positive impact on the success rate.

Bolstering Our Commitment to Investigator-Initiated Research

As part of an ongoing examination of our grant portfolio to ensure that we invest taxpayer money as effectively and efficiently as possible, we recently analyzed changes over time in the distribution of investigator-initiated research compared to research funded through targeted funding opportunity announcements (FOAs).

Changes over time in NIGMS investments in investigator-initiated research (research grant funds not associated with targeted FOAs) (right axis) and research funded through targeted FOAs (left axis). The analysis does not include fellowship, career development and training awards; programs transferred to NIGMS from the former National Center for Research Resources; and some other programs. For more details about the analysis, which was performed by Jim Deatherage, chief of our Cell Biology Branch, see the NIGMS Funding Trends Web page.

The figure shows that in the early 1990s, 99% of NIGMS’ grant budget supported investigator-initiated research, compared to 80% today. During the budget doubling in Fiscal Years 1998-2003, the Institute’s investment in research funded through targeted FOAs increased dramatically, then continued to increase at a slower rate during Fiscal Years 2004-2009.

As I discussed in a previous post about our large-scale research initiatives and centers, there were many good reasons for using FOAs to target specific areas of research with some of the funds made available by the budget doubling. For example, FOAs allowed the Institute to experiment with catalyzing the development of such new and emerging fields as structural genomics, pharmacogenomics and systems biology.

Since the budget doubling ended, however, maintaining steady support for our targeted research portfolio has made it difficult to maintain steady support for investigator-initiated research project grants (RPGs). Partly as a result, the success rate for RPGs (the number of funded RPGs divided by the number of RPG applications) fell below 20% in Fiscal Year 2013. Although a number of factors have contributed to the declining success rate, a significant one is that targeted and investigator-initiated research grants compete directly with each other. To bolster the success rate, we need to decrease our commitment to targeted FOAs. Furthermore, because none of us knows where the next major advances will arise, the soundest investment strategy is to have a distributed portfolio in which researchers investigate a wide range of scientific questions. History strongly suggests that letting scientists “follow their noses”—which involves a combination of curiosity, expertise, creativity and serendipity—is the most productive route to findings that will eventually translate into medical and technological breakthroughs.

To rebalance our portfolio in order to renew and reinvigorate our commitment to investigator-initiated research, we will be reducing our use of targeted FOAs, generally reserving them for cases in which they are likely to have a major impact on a large segment of the biomedical research enterprise. These cases could include promoting the rapid development of accessible, cost-effective new technologies that enable major advances in understanding biological systems; more efficiently organizing the Nation’s basic biomedical research resources to provide scientists throughout the country access to high-end instrumentation and technical expertise; and, in some instances, using targeted FOAs with defined lifetimes to catalyze the rapid development of emerging research areas.

It is important to note that we are making a distinction between investigator-initiated research and targeted research, not between investigator-initiated research and team science. We strongly support team science, which can certainly be investigator-initiated, and we expect such collaborative efforts to increase as research probes more deeply into the complexities of living systems. Currently, team-based, investigator-initiated research can be funded through multi-PI R01s and can also occur through groups of individually funded PIs working together. In special cases, program project grants (P01s) may be appropriate, particularly for long-term, interdisciplinary collaborations that require dedicated core facilities. As we move forward with our strategic planning process, we will be exploring additional ways to support investigator-initiated team science. I invite you to send us ideas you have for how best to do this.

NIH’s Sally Rockey on PubMed Comments, 2013 Success Rates, Lead Time for Inviting NIH Staff to Meetings

Within the last week, NIH’s Sally Rockey has published posts that may be of interest to you:

PubMed Gets Interactive: The broader public now can view opinions and information shared by authors on scientific publications in PubMed.

Application Success Rates Decline in 2013: An early analysis of 2013 competing research project grant applications and awards at NIH shows a downward trend for success rates. NOTE: We at NIGMS are currently working on our annual funding trends post.

Understanding Lead-time for NIH Staff Participation in Scientific Meetings: Invitations for NIH program, review or other staff to speak at or attend a meeting need to be made as early as possible due to approval requirements and budget constraints. NOTE: Sally Rockey’s post suggests at least 4 months advance notice, but at NIGMS, we recommend at least 6 months.