Peer Review of NIH
Research Grant Applications
Victor A. Fung, Ph.D.
Scientific Review Administrator
Center for Scientific Review
National Institutes of Health
Tel: 301-435-3504
Fax: 301-480-0287
E-mail: fungv@csr.nih.gov
Typical Timeline for a New
Individual Research Project
Grant Application (R01)
There are three overlapping cycles per year:
–Submit in February (June, October)
–Review in June (October, February)
–Council in September (January, May)
–Earliest award in December (April, July)
Cycle 1---Cycle 2---Cycle 3----
Applications Submitted to NIH
• Approximately 50,000
grant applications are
submitted to NIH each
year, of which 25-30%
are funded
• Competing grant
applications are
received for three
review cycles per year
Types of Scientific Review Groups
Where are Applications Reviewed?
GROUPS
CSR IRGs
Study Sections
Special Emphasis Panels
INSTITUTES
Scientific Review Groups
Contract Review Committees
APPLICATIONS REVIEWED
Research Projects
Academic Research
Enhancement Awards
Postdoctoral Fellowships
Small Business Innovation
Research
Shared Instrumentation
Program Projects
Centers
Institutional Training Grants
Conference Grants
Career Awards
Small Grants
RFAs
Contracts
Dual Review System for Grant Applications
First Level of Review
Scientific Review Group (SRG)
• Provides Initial Scientific Merit
Review of Grant Applications
• Rates Applications and Makes
Recommendations for Appropriate
Level of Support and Duration of
Award
Second Level of Review
Council
• Assesses Quality of SRG
Review of Grant Applications
• Makes Recommendation to
Institute Staff on Funding
• Evaluates Program Priorities
and Relevance
• Advises on Policy
PRESUBMISSION
PREPARATION
•
•
•
•
CONTACT PROGRAM DIRECTOR
SPECIFIC RESEARCH AREAS
SPECIAL INITIATIVES, RFAs, PAs,
CONTACT SCIENTIFIC REVIEW
ADMINISTRATOR
• IDENTIFY STUDY SECTION
Program Announcement
• Invites grant applications in a given research area
• May describe new or expanded interest in a particular
extramural program
• May be a reminder of a continuing interest in a
particular extramural program
• Generally has no funds set aside
• Applications reviewed in CSR along with unsolicited
grant applications
Requests for Applications (RFA)
• Announcement describing an institute initiative in a
well-defined scientific area
• Invitation to the field to submit research grant
applications for a one-time competition
• Set-aside of funds for a certain number of awards
• Applications generally reviewed within the issuing
institute
WHEN PREPARING AN
APPLICATION
• READ INSTRUCTIONS
• ATTENTION TO ADMINISTRATIVE
DETAILS
• FONT SIZE
• ANIMAL STUDIES – IACUC
• HUMAN STUDIES
• APPENDICES – MANUSCRIPTS
FONT SIZE
When Preparing an Application
• Never assume that the reviewers “will know
what you mean
• Refer to the literature thoroughly
• State rationale of proposed investigation
• Present an organized, lucid write-up
• Include well-designed tables and figures
• Obtain pre-view from colleagues
Importance of Clarity
• This Reviewer was unable to discern the
connections between A and B. This
reviewer cannot recommend support for a
project that cannot be understood.
There is no grantsmanship that
will turn a bad idea into a good
one, but……..
There are many ways to disguise
a good one.
William Raub, Past Deputy Director,
NIH
REVIEW CRITERIA
•
•
•
•
•
SIGNIFICANCE
APPROACH
INNOVATION
INVESTIGATOR
ENVIRONMENT
SIGNIFICANCE
•
•
•
•
Important area of research
Target sites: breast, lung, prostate, skin…
Carcinogenesis mechanisms
Development of new reagents,
methodology, treatment
• Convince reviewers that proposed work is
important
EXAMPLES
• Oncogene activation during skin tumor
progression – role of ROS in squamous
carcinoma progression – new therapeutic
strategies
• Role of oxidants and angiogenesis in
Kaposi’s sarcomas
• Roles of critical MMR genes in response to
DNA damage, chemotherapeutic challenges
EXAMPLES
• New mass spectrometry methodology to
address the important question: Do
mutation hot spots result from enhanced
chemical deposition vs. site specific
deficiencies in repair
• Chemopreventive activity of green tea
APPROACH
•
•
•
•
•
•
Hypothesis and Mechanism Based
Preliminary Data
Specific Aims
Experimental Design - questions
Results, impact, future directions
Potential problems and alternatives
PRELIMINARY DATA
•
•
•
•
•
Quality – peer reviewed journals
Quantity – enough
Evaluation – valid interpretation
Adequate controls
Support hypothesis, feasibility of studies
SPECIFIC AIMS
•
•
•
•
•
Test hypothesis
Rational
Integrated
Prioritized
Logical chronological sequence
EXPERIMENTAL DESIGN
•
•
•
•
•
•
•
Logical and focused
Address important target questions
Appropriate methodology
Adequate experimental details
Justification for proposed reagents, methods
Statistical analysis
Potential pitfalls and alternatives
RESULTS
•
•
•
•
•
•
Adequate Discussion
Anticipated & Alternative
Use of results
Significance – potential to move field ahead
Correlative vs. mechanistic
Have an impact regardless of expected or
unexpected outcome
Potential Pitfalls and Alternatives
• The substantial weakness is a serious lack
of potential problems and alternative
possible results, and consequent alternative
approaches which will be utilized. This
imparts the impression that the success of
the proposal is absolutely dependent on
achieving the expected results.
INNOVATION
•
•
•
•
•
Novel concepts and approaches
Project challenges existing paradigms
Original and innovative aims
Innovative use of existing technologies
Development of new reagents, models,
methodologies and technologies
INVESTIGATOR
•
•
•
•
Appropriate training and expertise
Collaborators and consultants
Productivity – track record
Network
NEW INVESTIGATORS
• CHECK NEW INVESTIGATOR BOX ON
PHS 398 FACE PAGE
• INDEPENDENCE, RESOURCES
• REFERENCE LETTERS
• PRODUCTIVITY
• PERIOD OF AWARD
• PAYLINES
ENVIRONMENT
• Intellectual and physical environment
• Does the scientific environment contribute
to the probability of success?
• Unique features – facilities, interaction with
experts
Scientific Review Group or
Study Section Actions
• Scored, Scientific Merit Rating
(priority scores)
• Unscored
• Deferral
K185pp.40
PRIORITY SCORES
•
•
•
•
•
OUTSTANDING
EXCELLENT
VERY GOOD
GOOD
ACCEPTABLE
1.0 – 1.5
1.5 – 2.0
2.0 – 2.5
2.5 – 3.5
3.5 – 5.0
Common Problems
SIGNIFICANCE
• Area of research is not important
• Proposed work is not important
COMMON PROBLEMS
•
•
•
•
•
APPROACH
Questionable reasoning in experimental
approach
Lacks an acceptable scientific rationale
Diffuse, superficial, or unfocused research
plan
Lack of sufficient experimental details
Unrealistically large amount of work
COMMON PROBLEMS
•
•
•
•
Innovation, Investigator, Environment
Lack of new or original ideas
Lack of experience in the essential
methodology
New PI’s independence is in question
Inadequate facilities
OPTIONS IF NOT FUNDED
• REVISE AND RESUBMIT ( 2 CHANCES)
• FUNDING OUTSIDE THE PAYLINE
• AER, EXCEPTION, SHANNON,
SUPPLEMENTARY SUPPORT
(COMPETING RENEWAL)
• PROGRAM DIRECTOR’S ADVICE
RESEARCH GRANT
National Institutes of Health
Principal
Investigator
School or Other
Research Center
Center for Scientific Review
Assign to
Initiates
Research Idea
Submits
application
IC and IRG
Scientific Review Group
Review
for
Scientific
Merit
Institute
Evaluate for
Relevance
Advisory Council or Board
Recommend
Conducts Research
Allocates
Funds
Action
Institute Director
Divisions - Extramural
Division of Cancer Biology
Division of Cancer Control and Population Sciences
Division of Cancer Prevention
Division of Cancer Treatment and Diagnosis
Division of Extramural Activities
Divisions and Centers - Intramural
Center for Cancer Research
Division of Cancer Epidemiology and Genetics
The origin of the grant in the United States goes back about 150 years
when the Congress made a grant in the amount of $30,000 to Professor
Samuel F.B. Morse to test the feasibility for public use of the electromagnetic
telegraph system. The Morse grant is significant since it represents one of the first
technical proposals accepted by the federal government and the first instance
in which Congress full participated.
National Cancer Institute's (NCI) first cancer research grant application was
received from William T. Salter at Harvard University. It requested $4,350
for cancer research and was assigned grant number 1C1. It was reviewed
November 1937 but not funded. The first cancer research grant funded by NCI
was awarded to Louis F. Fieser, also at Harvard University, on November 27, 1937.
It was funded for $27,550 for investigating chemical structure and carcinogenic activity.
The grant identificationnumber was IC3.
1
Distribution of FY 2004 Budget Request ($5,986,000,000)
2
FY 2004 Increase Request for Advancing Discovery
3
4
5
6
7
SBIR/STTR Participating Agencies
TOTAL ~ $1.2 B FY 2001
• DOD
z HHS
• NASA
• DOE
• NSF
• USDA
• DOC
• EPA
• DOT
• ED
SBIR/STTR
SBIR/STTR
SBIR/STTR
SBIR/STTR
SBIR/STTR
SBIR
SBIR
SBIR
SBIR
SBIR
NIH:
$410.575M SBIR
$ 24.635M STTR
$435.21 M Total
CDC: ~$ 6.5 M SBIR
FDA:
$ 0.58 M SBIR
AHRQ: ~$ 2.1 M SBIR
Program Descriptions
• SBIR: Set-aside Program for Small Business
Concerns to engage in Federal R&D-with potential for commercialization.
• STTR: Set-aside Program to facilitate
cooperative R&D between Small
Business Concerns and U.S. Research
Institutions-- with potential for
commercialization.
8
SYSTEMS AND METHODS FOR SMALL ANIMAL IMAGING (SBIR/STTR) RELEASE DATE:
November 18, 2002 PA NUMBER: PA-03-031
TELEHEALTH TECHNOLOGIES DEVELOPMENT (SBIR/STTR) RELEASE DATE: November
18, 2002 PA NUMBER: PA-03-030
NOVEL TECHNOLOGIES FOR IN VIVO IMAGING (R21/R33) RELEASE DATE: May 19, 2003
PA NUMBER: PAR-03-124
INNOVATIONS IN BIOMEDICAL COMPUTATIONAL SCIENCE AND TECHNOLOGY
RELEASE DATE: April 17, 2003 (see NOT-OD-03-044) PA NUMBER: PAR-03-106
TECHNOLOGY DEVELOPMENT FOR BIOMEDICAL APPLICATIONS RELEASE DATE:
February 27, 2003 PA NUMBER: PAR-03-075
NANOSCIENCE AND NANOTECHNOLOGY IN BIOLOGY AND MEDICINE RELEASE
DATE: December 12, 2002 PA NUMBER: PAR-03-045
CONTINUED DEVELOPMENT AND MAINTENANCE OF BIOINFORMATICS AND
COMPUTATIONAL BIOLOGY SOFTWARE RELEASE DATE: July 26, 2002 (see NOT-CA-03020) PA NUMBER: PA-02-141
BIOENGINEERING NANOTECHNOLOGY INITIATIVE (supercedes PA-00-018)
RELEASE DATE: July 2, 2002 (see also PA-00-018) PA NUMBER: PA-02-125
9
Below you will find some web sites that should be helpful to scientists who want to apply
for funding at the NIH.
•Funding Opportunities: http://grants1.nih.gov/grants/funding/funding.htm
•Grant Policy Guidelines: http://grants1.nih.gov/grants/policy/policy.htm
•Research Training and Fellowships: http://grants1.nih.gov/training/extramural.htm
•Award Data: http://grants1.nih.gov/grants/award/award.htm
•Applications and Forms: http://grants1.nih.gov/grants/forms.htm
•Standard Receipt Dates: http://grants1.nih.gov/grants/dates.htm
•Referral and Review Process: http://www.csr.nih.gov/refrev.htm
•Guide for Grants and Contracts: http://grants.nih.gov/grants/guide/index.html
•Computer Retrieval of Information
on Scientific Projects (CRISP): http://crisp.cit.nih.gov/
10
National Science Foundation
(NSF)
& Medical Physics
Ron Rardin, Program Director for
Operations Research and
Service Enterprise Engineering
NSF Engineering Directorate
Who Is This Guy?
• Ron Rardin, Program Director at the National
Science Foundation (NSF) for Operations
Research and Service Enterprise Engineering
• On rotation in Washington for 3 years
• Permanent position at Purdue Industrial Engineering
• 20 years at Purdue, 9 before at GaTech
• Long interest in exact and heuristic optimization
NSF Overview
• Broad mission and
influence
• Total about $5B per year
• Small fraction of federal
R&D
• About 1200 employees
• Only about 10%
engineering (+10%
computer science &
engineering)
Othr NSF
DOE 7% 4%
10%
NASA
13%
HHS
21%
DOD
45%
Directorate Structure
Office of the
Inspector General
National Science Board
Director
Rita Colwell
Total $5B
Joe Bordogna
Directorate for
Biological
Sciences
Directorate for
Administration
Directorate for
Geosciences
$526M
Directorate for
Engineering
$488M
John Brighton
DMII
$691M
Directorate for
Computer and
Information
Science and
Engineering
$527M
Directorate for
Education and
Human
Resources
$908M
Directorate for
Mathematical
and Physical
Sciences
Directorate for
Social,
Behavioral,
and Economic
Sciences
$942M
Polar Programs $304M
$196M
Priority Areas
Nanoscale Science & Engr
Information Technology
Mathematical Sciences
Social, Behavioral & Economic
Biocomplexity in the Environment
21st Century Workforce
(Cyber Infrastructure)
$M FY03
221
286
60
10
79
185
coming
Engineering Directorate
Divisions
Engineering Directorate (ENG)
John Brighton, Assistant Director
Bert Marsh, Deputy Assistant Director
$488M
Bioengineering & Environmental Systems (BES)
Design, Manufacture & Industrial Innovation (DMII)
Bruce Hamilton, Division Director
$43.9M
Warren DeVries, Division Director
Academic Research $57.6M
SBIR/STTR $83.6M
Chemical & Transport Systems (CTS)
Civil & Mechanical Systems (CMS)
Esin Gulari, Division Director
$58.9M
Galip Ulsoy, Division Director
$57.8M
Electrical & Communications Systems (ECS)
Engineering Education & Centers (EEC)
Vasu Varadan, Division Director
$66.7M
Bruce Kramer, Division Director
$119.5M
Engineering Directorate
(AAPM Interests)
Engineering Directorate (ENG)
John Brighton, Assistant Director
Bert Marsh, Deputy Assistant Director
$488M
Bioengineering & Environmental Systems (BES)
Bruce Hamilton, Division Director
$43.9M
Design, Manufacture & Industrial Innovation (DMII)
Warren DeVries, Division Director
Academic Research $57.6M
SBIR/STTR $83.6M
ENG Interests in Imaging and Radiation Treatment
BES (Sept 1 thru Oct 15 annually) : Biomedical Engineering program
funds a wide variety of imaging and biosensing research
DMII (Sept 1 thru Oct 1 & Jan 1 thru Feb 1 annually): Service
Enterprise Engineering program has broad interest in health care
delivery optimization including radiation therapy (focus on math
modeling & analysis)
NSF Award Examples
• BES Example: High-Resolution Cortical Imaging of Brain
Electrical Activity. The focus of this proposal is to achieve high
spatial and temporal resolution in brain imaging by mathematically
combining MRI and EEG data.
• DMII Example 1: Mixed Integer Programming for Radiotherapy
Optimization.The research will develop and test MIP methodology
to provide solutions that satisfy prescribed constraints with a
tumor dose objective value that lies within a provable bound around
the optimum.
• DMII Example 2: A Prototype Radiation Therapy Treatment
Planning Research Toolkit.This Study Grant supports the research
of the NCI-cosponsored Optimization Collaborative Working Group
(CWG) developing standards for web-based tools that will enhance
the development and validation of dose computation and
optimization algorithms in the delivery of radiation treatments.
Unsolicited Awards
• Most NSF proposals are unsolicited
• no specific topic list
• Awards are grants
• no specific deliverables
• Typical grant size is $350K over 3 yrs
• PI summer pay, grad students, travel
• 10-20% of proposals funded
• Target audience = engineering academia (with
medical collaborators)
Career Grants
• CAREER = special NSF program for young
faculty
• Untenured assistant prof (3 time limit)
• Provide $80K per year for 5 years
• Advantages: prestige, compete with peers
• Still very competitive
• 10-20% funded
• Come out of same budget as unsolicited grants
SBIR/STTR Programs
• Opportunities also exist in SBIR/STTR
small business innovation programs
• near term commercializable research
• Phase I: proof of concept, $100K
• Electronics & Information Tech = June 12
• Bio Tech & Advanced Manufacturing = Jan 20
• Phase II: to commercialization, $500K
• All fields = July 29
Proposal Formats
• Proposal format
•
•
•
•
Summary (abstract): 1 page
Description: 15 pages
References, bios, current support
Budgets
• State what problem you wish to study, how you
propose to approach it, what is novel, why team
is qualified, and why results would be significant
(science & social)
• No hypothesis format
• Electronic submission thru web-based FastLane
NSF Peer Review
Panels
• Almost all proposals are peer reviewed by
panels of 8-10 researchers evaluating 1525 proposals (3-5 on each proposal)
• Merit criteria
• Intellectual Merit = the science proposed.
Novelty, technical strengths, team
qualifications, research plan
• Broader Impacts = impacts beyond the
immediate topic. Development of research &
education infrastructure, social/economic
importance
Differences vs NIH
• Panels are generated ad hoc to respond to each
submitted proposal pool of program – not
standing
• Panels are recruited and moderated by program
managers – not a separate organization
• Program managers need not follow panel
priorities
• Program managers have broad authority to
renegotiate budgets before award
• No revise and resubmit process (formal)
Proposal Writing
Hints
• Get to the point early
• Summary/abstract is most important, then
Intro
• Industrial/medical partners/endorsements
offer valuable context knowledge &
evidence that the work is worth doing
• Remember that you are writing to a panel
that spans the program to which
submitted, not just experts on your topic