I recently had the opportunity to speak at the Model Organisms to Human Biology meeting sponsored by the Genetics Society of America. I shared some of my perspectives on the powerful interplay between studies of model organisms and studies of humans (both individuals and populations) enabled through genetics. I illustrated why results over many decades have shown that studying fundamental mechanisms in a wide range of organisms can elucidate important processes relevant to human health and disease.
I also discussed aspects of the NIH peer review system, particularly with regard to proposed studies of model organisms.
One of the key changes in the new peer review system is the use of individual scores for five specific criteria. During my talk, I focused on the significance criterion:
Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?
This definition is intended to cover the entire range of research supported by NIH, spanning basic studies of fundamental mechanisms through applied studies that have the potential for direct clinical impact.
Some applicants who use model organisms try to explain the significance of their project by making relatively tenuous links to specific clinical areas. As an alternative, they should consider highlighting the study’s importance to a basic field of biomedical or behavioral research and the reason for using a specific experimental system.
To examine how reviewers apply the significance criterion in determining overall impact scores, I analyzed 360 NIGMS R01 applications reviewed during the October 2009 Council round. A plot comparing the average significance scores with the overall impact scores for these applications is shown below.
As anticipated, the scores are reasonably strongly correlated, with a Pearson correlation coefficient of 0.63. Similar comparisons with the other peer review criteria revealed correlation coefficients of 0.74 for approach, 0.54 for innovation, 0.49 for investigator and 0.37 for environment.
This analysis indicates that approach and significance are the most important factors, on average, in determining the overall impact score, at least for this sample of NIGMS R01 grant applications.
UPDATE: Jeremy Berg has posted similar analyses of the approach and innovation criteria.
The Advisory Committee to the Director, NIH (ACD) is a group knowledgeable in the fields of research pertinent to the NIH mission. It includes individuals from the academic and private sector research communities as well as representatives of the general public.
In March, I announced the launch of a strategic planning process focused on training and career development. I encourage all stakeholders, including faculty members, postdocs, graduate students, university administrators and government and industry scientists to provide input on this important topic.
Since I first wrote about the death of former NIGMS director Dr. Ruth Kirschstein, plans have progressed for a daylong commemorative event on Monday, May 17, at NIH. The program will include several remembrances as well as scientific presentations and posters by recipients of the Ruth L. Kirschstein National Research Service Award.