Posts

Posted on

More on Criterion Scores

by
6 comments

In an earlier post, I presented an analysis of the relationship between the average significance criterion scores provided independently by individual reviewers and the overall impact scores determined at the end of the study section discussion for a sample of 360 NIGMS R01 grant applications reviewed during the October 2009 Council round. Based on the interest in this analysis reflected here and on other blogs, including DrugMonkey and Medical Writing, Editing & Grantsmanship Link to external web site, I want to provide some additional aspects of this analysis.

As I noted in the recent post, the criterion score most strongly correlated (0.74) with the overall impact score is approach. Here is a plot showing this correlation:

Plot of approach and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Plot of approach and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Similarly, here is a plot comparing the average innovation criterion score and the overall impact score:

Plot of innovation and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Plot of innovation and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Note that the overall impact score is NOT derived by combining the individual criterion scores. This policy is based on several considerations, including:

  • The effect of the individual criterion scores on the overall impact score is expected to depend on the nature of the project. For example, an application directed toward developing a community resource may not be highly innovative; indeed, a high level of innovation may be undesirable in this context. Nonetheless, such a project may receive a high overall impact score if the approach and significance are strong.
  • The overall impact score is refined over the course of a study section discussion, whereas the individual criterion scores are not.

That being said, it is still possible to derive the average behavior of the study sections involved in reviewing these applications from their scores. The correlation coefficient for the linear combination of individual criterion scores with weighting factors optimized (approximately related to the correlation coefficients between the individual criterion scores and the overall impact factor) is 0.78.

The availability of individual criterion scores provides useful data for analyzing study section behavior. In addition, these criterion scores are important parameters that can assist program staff in making funding recommendations.

Posted on

Model Organisms and the Significance of Significance

by
13 comments

I recently had the opportunity to speak at the Model Organisms to Human Biology meeting Link to external web site sponsored by the Genetics Society of America. I shared some of my perspectives on the powerful interplay between studies of model organisms and studies of humans (both individuals and populations) enabled through genetics. I illustrated why results over many decades have shown that studying fundamental mechanisms in a wide range of organisms can elucidate important processes relevant to human health and disease.

I also discussed aspects of the NIH peer review system, particularly with regard to proposed studies of model organisms.

One of the key changes in the new peer review system is the use of individual scores for five specific criteria. During my talk, I focused on the significance criterion:

Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?

This definition is intended to cover the entire range of research supported by NIH, spanning basic studies of fundamental mechanisms through applied studies that have the potential for direct clinical impact.

Some applicants who use model organisms try to explain the significance of their project by making relatively tenuous links to specific clinical areas. As an alternative, they should consider highlighting the study’s importance to a basic field of biomedical or behavioral research and the reason for using a specific experimental system.

To examine how reviewers apply the significance criterion in determining overall impact scores, I analyzed 360 NIGMS R01 applications reviewed during the October 2009 Council round. A plot comparing the average significance scores with the overall impact scores for these applications is shown below.

Plot of significance and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

Plot of significance and overall impact scores in a sample of 360 NIGMS R01 applications reviewed during the October 2009 Council round.

As anticipated, the scores are reasonably strongly correlated, with a Pearson correlation coefficient of 0.63. Similar comparisons with the other peer review criteria revealed correlation coefficients of 0.74 for approach, 0.54 for innovation, 0.49 for investigator and 0.37 for environment.

This analysis indicates that approach and significance are the most important factors, on average, in determining the overall impact score, at least for this sample of NIGMS R01 grant applications.

UPDATE: Jeremy Berg has posted similar analyses of the approach and innovation criteria.

Posted on

Electronic Awards Administration

by
1 comment

As with the transition to electronic grant applications, NIH is now transitioning to electronic award administration. Here are some important upcoming changes.

Progress Reports

Beginning August 1, 2010, you must use the eRA Commons eSNAP (Electronic Streamlined Non-competing Award Process) feature to submit all eligible progress reports. Also, SNAP progress reports will be due 45 days prior to the next budget start date (instead of 60 days), and IRB and IACUC approval dates will not be required as part of the progress report.

You can refer to the Notice of Award to determine whether these new requirements apply to your grant. For detailed instructions, see the eSNAP User Guide.

Appointments and Terminations

Beginning January 1, 2011, you must use the xTrain feature in eRA Commons to electronically submit appointment forms and termination notices for research training, fellowship, education and career development awards. After this date, paper documents will not be accepted.

Posted on

Earlier Submission Deadlines for 2011 Pioneer and New Innovator Awards

by
0 comments

NIH Director’s Pioneer AwardNIH has announced the 2011 competitions for the NIH Director’s Pioneer Awards and the NIH Director’s New Innovator Awards. These awards support exceptionally creative scientists who propose highly innovative—and often unconventional—approaches to major challenges in biomedical or behavioral research. Both programs are part of the NIH Common Fund and are managed by NIGMS.

The Pioneer Award provides $2.5 million in direct costs over 5 years and is open to scientists at U.S. institutions at any career level. The deadline for applying is September 13, 2010.

NIH Director’s New Innovator Award

The New Innovator Award provides $1.5 million in direct costs over 5 years and is designed for early stage investigators at U.S. institutions who have not yet obtained an NIH R01 or similar grant. Applications are due by September 20, 2010.

For more information about the programs and links to the requests for applications, see the Pioneer Award Web site and the New Innovator Award Web site.

A highlight of these programs is the annual symposium. This year’s symposium will take place near the NIH campus in Bethesda on September 30 and October 1, and it will include research talks by the second graduating class of Pioneer Award recipients. If you’re in the area, consider attending the symposium. It’s free, doesn’t require advance registration and also offers the opportunity to view poster presentations by many of the Pioneers and New Innovators. If you can’t attend, the symposium will be videocast live and archived at http://videocast.nih.gov.

Posted on

60th Lindau Nobel Laureate Meeting Promises to Be a ‘Dynamite’ Event

by
1 comment

This year, I’m the lucky NIGMS program director attending the 60th Annual Lindau Nobel Laureate Meeting in Lindau, Germany. Like John Schwab, who traveled with last year’s group, my main job is to help the 16 super-energetic, really smart graduate students we sponsored interact with each other, hundreds of their peers from all over the world and, of course, dozens of Nobel laureates.

60th Lindau Nobel Laureate Meeting

The Lindau meeting, which started in 1951, is designed to “educate, inspire and connect” generations of scientists by bringing together Nobel laureates with young researchers. Unlike last year’s meeting, which focused on chemistry, this year’s is more interdisciplinary. It showcases all three of the Nobel Prize natural science fields: chemistry, physics and physiology or medicine.

As you might imagine, the competition for student slots is fierce, with more than 40,000 applying for 650 slots this year. The U.S. contingent consists of 75 students supported by the Oak Ridge Associated Universities (ORAU), the National Science Foundation, the Department of Energy, Mars (the company, not the planet!) and NIGMS/NIH.

It’s quite appropriate that NIGMS is involved in this program again this year. Our portfolio of funded research is extremely diverse, including a lot in chemistry and physics. After all, an interdisciplinary approach can help us better address problems relevant to human health. And around NIH, NIGMS is also widely called the “Nobel Prize Institute”—we’ve funded the Nobel Prize-winning of research of 73 laureates, and some of them are here in Lindau.

So far, the trip is going pretty well. We’re surrounded by World Cup fever! We haven’t had much of a chance to meet the laureates yet, since the opening ceremony was just on Sunday. I know that much of the close laureate-student interactions will take place through small group discussion sessions set up for this purpose.

I truly feel fortunate to be here—for the interactions, the interdisciplinary science and what I anticipate will be really inspiring lectures by the laureates. But even if you can’t be here, you can still watch the scientific proceedings online and follow the meeting using a variety of social media.

Post submitted from Lindau, Germany, on Monday, Jun 28, 2010 6:25 PM CEST

Posted on

New Funding Opportunity for Biologists to Collaborate on Structure Studies

by
0 comments

PSI Logo

We have just issued a new “partnership” program announcement for researchers interested in a biological problem of significant scope to collaborate with structural biology researchers.

The announcement is a part of the PSI:Biology initiative, which will apply high-throughput structure determination to a broad range of biological problems. Successful applicants will partner with researchers within the PSI:Biology network.

Applicants should propose projects for which the determination of many protein structures will be important. The projects should also involve proteins amenable to high-throughput structure determination or targets that motivate new technology development for determining more difficult structures.

Posted on

Beyond Bio2010

by
0 comments

Beyond Bio2010Last month, I attended the “Beyond Bio2010” conference held at the National Academy of Sciences. The meeting highlighted the progress made in implementing the Bio2010 Link to external web site recommendations, chiefly to transform undergraduate biology education by using the quantitative sciences (mathematics, chemistry, physics, engineering and computer sciences) to study biology and vice versa.

The 2-day meeting, which was held in the same room that the Bio2010 report was publically released in 2002, primarily consisted of teams of biology, math and computer sciences faculty discussing strategies used by their institutions to reform science curricula. Among the efforts that really caught my attention were those that:

  • Assessed student learning outcomes and showed improvement as a result of integrating math, physics, chemistry and computer science into biology;
  • Established new interdisciplinary majors or minors in areas like bioinformatics;
  • Described efforts to break down “departmental silos” through collaborations among biology, math, chemistry and other faculty;
  • Trained faculty for an integrated pedagogical approach (for a resource, see The National Academies Summer  Institutes on Undergraduate Education in Biology); and
  • Discussed the need for the administration to formally recognize faculty who reformed the curricula.

In addition, I presented a summary of NIGMS-supported efforts, namely the MARC Curricular Improvement grant, a competitive mechanism for eligible institutions to implement Bio2010 recommendations. The inclusion of all undergraduate students, with a focus on historically unrepresented populations, was a significant theme of the conference. In fact, I was delighted to see one of our MARC Curricular Improvement grantees from the University of Puerto Rico talk about her institution’s efforts on this front.

A unique feature of the conference was that several undergraduate students also presented posters on their interdisciplinary biology and mathematics research. Their posters showed math majors collecting samples in a stream and biology majors performing mathematical modeling. A faculty attendee at the conference noted that when he walked around and listened to the student presenters, he really could not figure out who was a math major and who was a biology major—a desired outcome of Bio2010!

If you are interested in finding more details about “Beyond Bio2010,” a report of the conference is slated to be published in the Fall 2010 issue of CBE—Life Sciences Education Link to external web site.

Posted on

Job Opportunity in Enzymology/Biochemistry

by
0 comments

We’re looking for a program director (or “health scientist administrator”) to oversee grants and other activities related to enzymology and other aspects of biochemistry. The job is within the Division of Pharmacology, Physiology and Biological Chemistry. We hope to find someone who can work effectively in a team setting and who already has some independent research experience in this area or a closely related area of biochemistry.

For details, see the job announcement. Please forward this information to anyone who might be interested in the position and feel free to send me questions about the job.

The listing closes July 12, 2010.

Posted on

Presenting NIGMS to the NIH Director’s Advisory Committee

by
0 comments

Presentation to the Advisory Committee to the DirectorThe Advisory Committee to the Director, NIH (ACD) is a group knowledgeable in the fields of research pertinent to the NIH mission. It includes individuals from the academic and private sector research communities as well as representatives of the general public.

The ACD meets in person twice a year to provide advice on a range of NIH activities. At almost every meeting, the NIH Director invites an institute or center director to present information about his or her organization, including his or her vision of its key features.

I was delighted when Dr. Collins invited me to present at the most recent ACD meeting. In my talk, I highlighted the Institute’s focus on investigator-initiated research and aspects of our role in training and workforce development. I also described some advances in five areas across NIGMS, including structural biology and the Protein Structure Initiative, RNA biology, pharmacogenomics, the Institutional Research and Academic Career Development Award (IRACDA) postdoctoral program and the Models of Infectious Disease Agent Study (MIDAS) program.

A video of my talk and a question-and-answer session is now available. It begins at minute 138:45 with an introduction by Dr. Collins.

Posted on

My NCBI Tool to Replace eRA Commons for Bibliography Management

by
0 comments

My NCBI screenshotNIH has announced a significant upgrade to the citation management capability of investigators’ personal profiles in the eRA Commons.

With the integration of the National Center for Biotechnology Information (NCBI) “My Bibliography” portal, direct database queries will replace manual citation entry in the Commons. This will have many benefits, most immediately for your eSNAP progress reports due to more accurate data and automated evaluation of each citation’s Public Access Policy compliance status.

You must now enter citations via My Bibliography accounts. Users (investigators or delegates) will need to have a My NCBI account and link it to their Commons account. Instructions are available on the Commons Web site at https://era.nih.gov/.

Please note that beginning July 23, you will not be able to manually enter a citation directly into the Commons. You will still be able to manually enter citations of publications and other items not indexed in PubMed (book chapters, meeting abstracts, etc.), but this must be done using My Bibliography. Also note that beginning on October 22, all citations that had been manually entered into the Commons will no longer be displayed. Publications abstracted in PubMed will automatically appear in My Bibliography; other citations must be added to My Bibliography to appear.