Application, Review, Funding, and Demographic Trends for Maximizing Investigators’ Research Awards (MIRA): FY 2016-2018

NIGMS has made MIRA awards to Established Investigators (EI) and Early-Stage Investigators (ESI) for three full Fiscal Years (FY). In this Feedback Loop post, we provide an analysis of application, review, funding, and demographic trends for the MIRA program.

For the first two rounds of EI MIRAs, eligibility was limited to well-funded NIGMS investigators: PIs with two or more NIGMS R01-equivalent awards or one NIGMS R01-equivalent award for >$400,000 in direct costs. For the FY 2018 EI competition and beyond, eligibility was expanded to include any investigator with a single PD/PI NIGMS R01-equivalent that is up for renewal. For the FY 2016 ESI MIRA competition, ESIs and New Investigators (NI) at the assistant professor or equivalent level were eligible, whereas eligibility was restricted to ESIs in subsequent rounds. As always, a PI can apply for an extension of ESI status for various life and career events, including childbirth.

Table 1 shows the number of awards made and associated success rates by fiscal year for EIs and ESIs.

Table 1. Number of MIRAs Funded by Fiscal Year and Cohort (Success Rate)

FY	Established Investigators	Early-Stage Investigators
2016	136 (76.0%)	93* (29.1%)
2017	95 (68.3%)	100 (28.8%)
2018	115 (50.2%)	112 (36.8%)
	*FY 2016 ESI MIRA includes 27 awards to Non-Early-Stage New Investigators.

Because the EI MIRA applications generally proposed continuations of research programs, we compared them to R01 renewal applications during the same time period. As shown in Table 2, the EI MIRA success rates are much higher than the success rates for R01 renewal applications.

Table 2. Number of NIGMS R01 Renewals and EI MIRAs (Success Rate)

FY	R01 Renewals	Established Investigator MIRA
2016	390 (40.0%)	136 (76.0%)
2017	332 (40.1%)	95 (68.3%)
2018	259 (44.1%)	115 (50.2%)

Overall, ESI MIRAs had a significantly higher success rate over the last 3 years (31.4%) compared to the ESI R01s (25.7%). The ESI MIRA success rate was also substantially higher than the success rate for all new (non-renewal) NIGMS R01s during the same time period.

Table 3. Number of NIGMS New R01s, ESI R01s, and ESI MIRAs (Success Rate)

FY	New R01s: ESI only	All New R01s	ESI MIRAs
2016	83 (23.1%)	443 (18.1%)	93* (29.1%)
2017	113 (30.2%)	538 (21.3%)	100 (28.8%)
2018	79 (23.6%)	508 (20.8%)	112 (36.8%)
	*FY 2016 ESI MIRA includes 27 awards to Non-Early-Stage New Investigators.

Over the last 3 fiscal years, 1,890 investigators were eligible for the EI MIRA program, and 516 (27%) of those eligible chose to apply. For well-funded investigators who had >$400,000 in NIGMS direct costs for their research prior to applying for MIRA, their MIRA budgets were on average 12% lower than their previous level of NIGMS research support. In exchange, MIRA provides a longer funding period (5 years instead of 4), increased funding stability and scientific flexibility, and reduced administrative burden. This trade-off made it possible for the Institute to fund a larger number of PIs and enabled us to increase the budgets for most ESI MIRAs relative to what the PIs would have received with an R01 (median annual direct costs of $250,000 vs. $210,000, respectively). In addition, we are working to generally give established PIs with only one modular R01 who convert to MIRAs budgets of at least $250,000 in order to mitigate some of the losses to inflation that have occurred over the past 15 years. In this manner, the MIRA program has contributed to two key strategic objectives of NIGMS [PDF 702KB]: Investing in and sustaining a broad and diverse portfolio of highly meritorious research, and promoting the ability of investigators to pursue new research directions, novel scientific insights, and innovative ideas. More information on MIRA budgets can be found in our recent Feedback Loop post on factors to consider when applying for an EI MIRA and in the FAQs for the ESI and EI MIRA programs.

In addition to scientific diversity, NIGMS monitors other factors contributing to the breadth of its portfolio, including demographic and geographic diversity. Over all three rounds of MIRA, men submitted 71% of ESI applications and 77% of EI applications (among applicants whose gender was reported). The skewed gender distributions in ESI and EI applicants is, unfortunately, consistent with NIH-wide trends and faculty demographics (e.g., Hechtman et al., 2018 ; Valantine et al., 2016 ). Across all six MIRA cohorts, no significant differences were observed between men and women in the fraction of applications reviewed (some applications are not reviewed if the proposal is outside of the NIGMS mission, the investigator is ineligible, or the application is incomplete), the mean impact scores, or the success rates (Table 4). The fraction of applications discussed during review was also not significantly different in five of the six cohorts, although a significantly greater percentage of applications from women than men were discussed in the FY 2018 EI cohort (87% vs. 69%, p = 0.01).

Table 4. Review Process by Gender, ESI and EI MIRA

	ESI MIRA			EI MIRA
	2016	2017	2018	2016	2017	2018
Percentage of Applications Received: Men	73%	70%	71%	80%	75%	77%
Percentage of Applications Received: Women	27%	30%	29%	20%	25%	23%
P-value	NA	NA	NA	NA	NA	NA
Percentage of Received Applications Reviewed: Men	79%	85%	89%	99%	93%	91%
Percentage of Received Applications Reviewed: Women	79%	90%	85%	94%	97%	88%
P-value	p = 1.0	p = 0.15	p = 0.46	p = 0.18	p = 0.45	p = 0.62
Percentage of Reviewed Applications Discussed: Men	54%	54%	54%	100%	100%	69%
Percentage of Reviewed Applications Discussed: Women	46%	54%	57%	100%	100%	87%
P-value	p = 0.31	p = 1.0	p = 0.60	p = 1.0	p = 1.0	p = 0.01
Mean Impact Scores: Men	40.0	39.2	33.6	27.1	27.4	36.9
Mean Impact Scores: Women	39.2	39.2	32.3	28.1	29.1	34.9
P-value	p = 0.73	p = 0.97	p = 0.56	p = 0.72	p = 0.49	p = 0.45
Percentage of Reviewed Applications Awarded: Men	30%	29%	38%	77%	66%	47%
Percentage of Reviewed Applications Awarded: Women	25%	30%	38%	73%	74%	62%
P-value	p = 0.48	p = 1.0	p = 1.0	p = 0.65	p = 0.40	p = 0.08

Analysis of the review process as a function of race/ethnicity groups, on the other hand, did reveal some significant differences in review rates, average scores, and success rates. We focus first on White and Asian investigators, who comprised over 85% of MIRA applicants. Applications from White investigators were reviewed significantly more often than those from Asian investigators in all three ESI cohorts, as well as the combined ESI cohort, but not in either the individual or combined EI cohorts (Table 5; statistically significant values are highlighted in red). Applications proposing research outside the NIGMS mission accounted for a strong majority of the non-reviewed ESI applications while most of the remaining applications in this category came from investigators who were not eligible at the time they applied or from investigators who received other funding after applying for MIRA, thus becoming ineligible before the MIRA application was reviewed. The main driver of the differing review rates between Whites and Asians was the higher rate at which applications from Asian investigators were determined to be outside of the NIGMS mission. Because of this observation, in FY 2017 NIGMS began using a blinded initial assessment of whether an application for MIRA was within the Institute’s mission. This assessment was based on the application title and abstract, excluding the investigator’s name and institution, and was made independently by two program directors. Any recommendations to decline an application were then reviewed by the relevant NIGMS division director and NIGMS leadership using the full application, if necessary. Because the difference in review rates between White and Asian applicants persisted after these new procedures were instituted, beginning with the FY 2018 ESI cohort, the likelihood of an application’s relevance to the NIGMS mission was assessed initially by machine learning tools using only the titles, abstracts, and specific aims of funded NIH R01s as a training set, followed by a blinded process similar to that used in 2017. The results of the machine learning tools largely corresponded with subsequent human assessment. Despite these steps to mitigate potential bias in determining if an application falls within the mission of the Institute, there continued to be a significant difference in review rates between Whites and Asians in the FY 2018 ESI MIRA cohort, suggesting that factors other than bias are responsible for the disparity. We will continue monitoring these outcomes and using a similar multi-tiered process as described above in evaluating applications for relevance to NIGMS’ mission.

Table 5. Review Rate, Whites and Asians

	Whites	Asians	Fisher p-value Individual Cohorts	Fisher p-value Combined Cohorts
ESI 2016	84.1	73	0.02	ESI combined: 0.00004
ESI 2017	89.9	80.2	0.02
ESI 2018	92.5	82.6	0.02
EI 2016	97.9	100	1.0	EI combined: 0.66
EI 2017	93.6	88.9	0.41
EI 2018	90.6	89.8	0.79

Some differences in average scores between White and Asian investigators were also observed. For the FY 2016 ESI MIRA cohort, Asians had a mean priority score 5.5 points higher (worse) than Whites, which was a statistically-significant difference (Table 6), although this disparity was not observed in the FY 2017 or 2018 cohorts nor in the combined FY 2016-2018 group. When the non-ESI NIs are excluded from the FY 2016 cohort, the mean score for Whites increases by half a point (38.3) whereas the mean score for Asians is unchanged (43.3), a difference that is no longer statistically significant (p = 0.11). Thus, some of the score

Table 6. Mean Scores, Whites and Asians

	Whites	Asians	Wilcoxon p-value Individual Cohorts	Wilcoxon p-value Combined Cohorts
ESI 2016*	37.8	43.3	0.028	0.23
ESI 2017	40.0	38.1	0.44
ESI 2018	32.2	33.1	0.79
EI 2016	26.2	31.9	0.09	0.01
EI 2017	26.4	32.4	0.07
EI 2018	35.6	37.6	0.44
	*FY 2016 ESI MIRA includes 27 awards to Non-Early-Stage New Investigators.

disparity between White and Asian applicants in the FY 2016 cohort could have been driven by more senior NIs who were eligible in that round, but not in subsequent rounds, and were more competitive on average than ESIs because of their more advanced career stage. In all three of the EI MIRA cohorts, the mean score for Asian applicants was higher (2-6 points) than for White applicants. Although these differences did not reach statistical significance in any individual cohort, there is a significant difference for the combined FY 2016-2018 group.

Success rates largely tracked these scores differences, with a significant difference in the ESI FY 2016 cohort but not for any other individual or combined ESI cohort (Table 7). By contrast, success rates were higher for Whites than Asians in each individual EI MIRA FY cohort and, although the difference in the combined cohort was only marginally statistically significant, if future cohorts follow the same trend it would likely become more significant. We are working to understand the sources for these observed disparities, which have also been noted in studies of NIH-wide R01 funding (Ginther et al., 2016 ).

Table 7. Percentage of Reviewed Applications Awarded, Whites and Asians

	Whites	Asians	Fisher p-value Individual Cohorts	Fisher p-value Combined Cohorts
ESI 2016*	36.1%	18.5%	0.003	0.06
ESI 2017	25.8%	25.8%	1.0
ESI 2018	37.2%	36.8%	1.0
EI 2016	78.3%	72.4%	0.48	0.08
EI 2017	71.8%	58.3%	0.22
EI 2018	53.4%	45.5%	0.40
	*FY 2016 ESI MIRA includes 27 awards to Non-Early-Stage New Investigators, who were eligible for ESI MIRA in FY 2016 but not in subsequent years.

The remaining MIRA applications were received from investigators whose race/ethnicity is Hispanic; Black or African-American; American Indian or Alaska Native; and from investigators with multiple, unknown, or withheld race/ethnicity. Because of the small number of investigators in these groups, it is difficult to make meaningful comparisons at this stage, and privacy considerations prohibit presenting the data for individual groups and cohorts. Therefore, we present the data for Hispanic, Black or African-American, and American Indian or Alaska Native investigators collectively (as “underrepresented”) in comparison to the combination of White and Asian investigators (“well-represented”).

Well-represented investigators were reviewed more frequently (86% vs. 71%) than underrepresented investigators in the ESI FY 2017 cohort (Table 8). No statistically significant differences were found in the other five individual cohorts nor in either combined cohort.

Table 8. Percentage of Applications Reviewed, Well-Represented vs Underrepresented

	Applications Received, Well-represented	Applications Received, Under- represented	Applications Reviewed, Well-represented	Applications Reviewed, Under- represented	Wilcoxon p-value Individual Cohorts	Wilcoxon p-value Combined Cohorts
ESI FY 2016	340	26	272 (80%)	22 (85%)	0.8	0.07
ESI FY 2017	314	34	271 (86%)	24 (71%)	0.02
ESI FY 2018	278	28	248 (89%)	22 (79%)	0.12
EI FY 2016	170	< 11	167 (98%)	< 11 (> 80%)	0.13	0.24
EI FY 2017	137	< 11	127 (93%)	< 11 (> 90%)	1
EI FY 2018	229	16	207 (90%)	14 (88%)	0.66

For ESI applicants, comparison of mean scores showed no significant differences between well-represented and underrepresented investigators in any individual FY cohort nor in the combined cohort (Table 9). For EI applicants, well-represented investigators scored significantly better than underrepresented investigators in the FY 2016, FY 2017, and combined cohorts (Table 9).

Table 9. Mean Scores, Well-Represented vs Underrepresented

	Mean Impact Score, Well-Represented	Mean Impact Score, Underrepresented	Wilcoxon p-value Individual Cohorts	Wilcoxon p-value Combined Cohorts
ESI FY 2016	39.5	36.3	0.44	0.63
ESI FY 2017	39.3	37.9	0.65
ESI FY 2018	32.5	33.5	0.77
EI FY 2016	27.2	43.2	0.02	0.004
EI FY 2017	27.5	21.3	0.34
EI FY 2018	36.0	50.3	0.008

Turning to success rate, no statistically significant differences were found in any individual or combined FY cohorts for ESIs (Table 10). Consistent with the scores shown in Table 9, the success rate for well-represented EIs was significantly higher than for underrepresented EIs for the FY 2018 cohort (Table 10) and for the combined EI cohort (65% vs. 32%)

Table 10. Percentage of Reviewed Applications Awarded, Well-Represented vs Underrepresented

	Well-Represented	Underrepresented	Fisher p-value Individual Cohorts	Fisher p-value Combined Cohorts
ESI FY 2016	30%	23%	0.63	0.59
ESI FY 2017	26%	38%	0.23
ESI FY 2018	37%	41%	0.82
EI FY 2016	77%	40%	0.09	0.003
EI FY 2017	69%	67%	1
EI FY 2018	52%	21%	0.05

A comparison of the mean ages of the ESI MIRA cohorts and the ESI R01 cohorts from the corresponding years is shown in Table 11. All three ESI MIRA cohorts have a lower average age than their matched ESI R01 cohorts. This difference is statistically significant for the FY 2017 cohort and for the combined cohort, but not for the FY 2016 and 2018 cohorts. This age difference is reflected in the mean ages of applicants for ESI MIRA and NIGMS ESI R01 grants in the FY 2017, FY 2018, and combined cohorts (Table 11) suggesting that the MIRA program is encouraging people to apply sooner than they would for an R01. In addition to encouraging investigators to apply earlier, the MIRA program also appears to be selecting people for earlier awards, as shown by the lower average age of ESI MIRA awardees than ESI MIRA applicants in the combined cohort. The overall decrease in average age observed for the ESI MIRA program relative to ESI R01 awardees represents a step toward an NIH-wide goal of reducing the average age at which investigators receive their first major award.

Table 11. Mean Age (Years) by Cohort

	ESI MIRA Applicants	ESI R01 Applicants	Fisher p-value Applicants	ESI MIRA Awardees	ESI R01 Awardees	Fisher p-value Awardees
FY 2016*	38.8	38.8	0.82	37.5	38.3	0.17
FY 2017	37.6	38.7	0.00003	36.7	38.3	0.0009
FY 2018	37.8	39.0	0.0008	37.7	38.7	0.25
Combined	38.1	38.8	0.00001	37.3	38.4	0.00008
	*FY 2016 ESI MIRA includes 27 awards to Non-Early-Stage New Investigators, who were eligible for ESI MIRA in FY 2016 but not in subsequent years.

Finally, we examined the geographic distribution of MIRAs. The Institutional Development Award (IDeA) program is intended to build research capacity in states that have had historically low NIH funding levels. Figure 1 shows the number of ESI and EI MIRAs (FY 2016-2018) in each state, with IDeA states highlighted in blue (note: the Institute has already made some EI MIRA awards in FY 2019, but these are not reflected in the map or the data above). Of the 23 states and one territory in the IDeA program, 16 have at least one MIRA. Investigators in IDeA states comprised a much larger fraction of applicants to the ESI MIRA program (11%) than to the EI MIRA program (4%). In both groups, the IDeA state investigators’ success rate was not significantly different from non-IDeA state investigators. A total of 14 EIs and 27 ESIs in IDeA states received MIRAs over FY 2016-2018. Thus, investigators in all but 8 states have MIRAs.

Figure 1: ESI vs. EI MIRA Awards by State or U.S. Territory

As MIRAs become an increasingly large fraction of our research portfolio (from 7.8% of R01-equivalents in FY 2016 to 16.0% in FY 2018), we will continue to monitor these trends and other outcomes measures to help evaluate the effectiveness of the program.

Share...