The R01 Grant Application
Writing with Impact
January 10, 2013
ARC, Room123-ABC
Office of Faculty Development
Panelists
Janis K. Burkhardt, PhD. Associate Professor, Department of
Pathology and Laboratory Medicine
Douglas A. Coulter, PhD. Professor, Department of Pediatrics,
Division of Neurology
Andrea Kelly, MD, MSCE. Assistant Professor, Division of
Endocrinology and Diabetes
Sheela N. Magge, MD, MSCE. Assistant Professor, Division of
Endocrinology and Diabetes
Victoria A. Miller, PhD. Assistant Professor, Department of
Anesthesiology and Critical Care Medicine
Berenice Saxon, Sponsored Projects Office, CHOP Research
Institute
Virginia A. Stallings, MD. Professor, Department of Pediatrics,
Division of Gastroenterology, Hepatology and Nutrition
Office of Faculty Development
Overview
•
•
•
•
•
•
•
•
•
Review Process and Scoring System
Writing the Grant
Advice for Clinical Researchers
Submitting the Grant
Interpreting Summary Statements
Deciding Whether to Re-submit
Responding to Reviewers
Lessons Learned as New Investigator
New Investigators and Early Stage Investigators
January 2013
Office of Faculty Development
4
Scientific Review Group (SRG)
“Study Section”
• Chair
– Moderates discussion
– Serves as reviewer
• Scientific Review Officer (SRO)
– Assigns applications to reviewers
– Attends and oversees administrative and regulatory aspects of
peer review meetings
– Prepares Summary Statements for all applications
January 2013
Office of Faculty Development
5
Scientific Review Group (SRG)
“Study Section”
• Members (n ~ 20 to 30)
– Receive grant applications six weeks before meeting
– Prepare a written critique
– Assign a numerical score to each of five review criteria
(next slide)
– Make recommendations concerning the scientific merit
of applications
Note: R01s can be triaged and not discussed at all
Resource: Video of Mock Scientific Review Group
January 2013
Office of Faculty Development
6
Review Criteria
1.
2.
3.
4.
5.
Significance
Investigator(s)
Innovation
Approach
Environment
(detailed in handout)
January 2013
Office of Faculty Development
7
Impact v. Significance
Overall Impact: the synthesis/integration of the
five core review criteria of which Significance is
one.
“Reviewers will provide an overall impact score
to reflect their assessment of the likelihood for
the project to exert a sustained, powerful
influence on the research field(s) involved, in
consideration of the five core review criteria, and
additional review criteria (as applicable for the
project proposed).”
January 2013
Office of Faculty Development
8
Impact/Priority Score
• 9-point rating scale (1 = exceptional; 9 = poor)
• Score: mean of all members' impact/priority scores X 10.
10 (high impact) to 90 (low impact)
• Numerical impact/priority scores are not reported for
applications that are not discussed. Such applications
typically rank in the bottom half.
January 2013
Office of Faculty Development
9
Writing the Grant
• Lead the reviewer through your proposal.
– Organize the proposal in logical order.
– Keep it simple, but address the concerns.
• Be sure to address all the NIH review criteria.
• Don’t neglect the “extra” sections.
January 2013
Office of Faculty Development
10
Lead the reviewer through your proposal
Organize your argument logically
• Make sure your big ideas come across.
– Do: Capitalize or italicize the central hypotheses.
– Don’t: Lose the forest in the technical trees.
• Think carefully about order of aims.
– Does Aim 2 hinge on the success of Aim 1?
– What is the most efficient and interesting way to lay out your questions?
• Sometimes these are in opposition!
– Where possible, show interrelationships.
• Don’t be afraid to refer back and ahead.
• Try to balance complexity of aims.
– Don’t give them one to cut.
– Sometimes they just aren’t balanced – address this.
• Make clear what you expect to learn, and how it will advance the field.
– Sometimes this means describing long term plans.
January 2013
Office of Faculty Development
11
Lead the reviewer through your proposal
Organize your argument logically
• Picture the tired reviewer, with a stack of these things, who
knows that only 1-2 favorites will likely be funded.
• Keep writing clear and concise.
• Take care to lay out your argument logically, so that the
“bones” are readily apparent.
• Take care with page layout.
– Choose fonts of a reasonable size.
– Leave spaces.
– Intersperse figures.
January 2013
Office of Faculty Development
12
Page Layout Dos
• Use a systematic outline with consistent headings and
subheadings.
• Place figures near text they support, and use to break up the
text.
• Make the document do what you need to do. Create a
consistent style with repeating special sections.
– Consider dispensing with experimental systems up front.
– Consider an introductory paragraph at the beginning of each aim or
sub-aim to tell the reviewer what to expect later on.
– Consider ending each section with a paragraph like “Pitfalls and
alternative approaches”.
January 2013
Office of Faculty Development
13
Page Layout Don’ts
• Don’t jam the page full of text.
– Reviewers are mean when they’re tired!
• Don’t capitalize half the grant.
– Reviewers don’t like to be shouted at!
• Don’t try to put science where it doesn’t belong (e.g. in Vertebrate
animals).
– There are people whose whole job it is to send grants like that
back!
• Don’t try to get clever with size 10.5 font, etc.
– They really do check!
– Exception – figure legends.
January 2013
Office of Faculty Development
14
Use Figures Wisely
• Keep them simple.
• Make sure there is a figure legend and they are referred to
in the text.
• Don’t assume that if it’s published, they will see it.
– If your proposal hinges on a piece of data – show it.
• Don’t overlook the power of a diagram, table or flow chart.
– It can be a big space saver, and can really put your big ideas into
perspective.
January 2013
Office of Faculty Development
15
(C) APPROACH
Interaction of T cells with APCs induces dramatic remodeling of F-actin at the site of TCR
ligation, in a process that is required for productive T cell activation. Understanding how the
actin cytoskeleton facilitates T cell activation requires identifying the relevant regulatory
molecules, and determining how they work together to generate the appropriate actin
architecture at the appropriate place and time. Our overarching hypothesis is that actin
dynamics at the immune synapse are controlled by a regulatory complex comprised of
WASP, WIP and HS1. We propose that each of these proteins plays a distinct role in
regulating actin architecture, while at the same time modulating the function of the
others. To test this hypothesis, we ask whether and how complex formation is controlled (Aim
1); we will compare the contributions of individual complex components to actin responses and
other aspects of T cell activation (Aim 2); and we will conduct structure-function analysis of the
least well understood complex component, HS1 (Aim 3).
Fig. 25 Diagram of HS1-WIP-WASP
interactions. Note the modular nature of the three
proteins, and the presence in WIP and HS1 of the
WH2 and acidic motifs that WASP uses to drive
Arp2/3-dependent actin polymerization. Known
domain interactions are indicated. 1) WASP
catalyzes the formation of branched actin
filaments, an activity that may be regulated by WIP
binding. 2) Acting together, HS1 and WIP can also
catalyze new actin polymerization. 3) The HTH
region of HS1 can also stabilize actin filaments, as
can WIP.
January 2013
Office of Faculty Development
16
Address Each of the NIH
Review Criteria
•
•
•
•
•
January 2013
Significance
Investigators
Innovation
Approach
Environment
Office of Faculty Development
17
Significance
Without this, all is lost.
• In an era when funding is severely restricted, projects that are
the most clinically and/or biologically significant are likely to
be viewed most favorably.
• Spell it out up front.
–
–
–
–
What diseases are involved?
How many people would benefit?
What obstacles would be overcome?
Basic knowledge? OK – but why should people care?
• Make sure you hit the priorities of the chosen institute.
• Consider reiterating at the end of the Approach section.
January 2013
Office of Faculty Development
18
Investigators
• Pay careful attention to your biosketch.
– Most reviewers will not know who you are and
make use of your biosketch to assess your
credentials and accomplishments.
• Use the paragraph well. Tailor it to each application.
– Why are you uniquely well qualified to do this
work?
• Choose papers thoughtfully.
• Don’t neglect the investigative team.
January 2013
Office of Faculty Development
19
Investigator(s)
• Collaborators can really strengthen an application.
• Make sure that their role is commensurate with their
involvement.
•
•
•
•
Co-investigator?
Collaborator? Consultant? Other significant contributor?
Paid vs. unpaid?
Add a Biosketch if it strengthens your application.
• Lab members can also strengthen a team.
• Include Biosketches. Mention in budget justification.
January 2013
Office of Faculty Development
20
Environment
• Why are you at a great place to do this work?
– What equipment is available?
– What expertise is available?
• Goes back to Investigators, and beyond.
– What resources are available?
• Patients, specimens, etc.
• This mostly gets dealt with on Resources page.
– Start with a special paragraph analogous to the one on the Biosketch.
– Make sure your equipment list is appropriate to the grant.
– If collaborative, address contiguity or other logistics.
January 2013
Office of Faculty Development
21
Innovation
• Spell it out.
• Think of this as – “Why will I succeed where others have not?”
– New technology?
– New question?
– New way of asking an old question?
• Although reviewers are attracted to novel ideas and methods, be
careful. Propose good solid experimentation.
– Not technically innovative? – tell them why the old way is still the
best way.
January 2013
Office of Faculty Development
22
Approach
• Provide clear rationales for proposed experiments.
• Don’t be overly ambitious; propose what you think you can
reasonably accomplish over the tenure of the grant.
– provide a tentative timeline
• It is essential to provide potential alternative approaches -- not
every experiment works!
– Better yet – show them the result will be interesting either way.
– Provide essential experimental detail, but don’t write a lab manual.
•
Don’t neglect statistics – more on this…
January 2013
Office of Faculty Development
23
Approach: Preliminary Data
• Ideally, some preliminary data should be provided to
support each major line of investigation.
– Although less is expected from new investigators, it is
important to establish feasibility and some evidence
of experimental competence.
• Sprinkling preliminary data throughout the approach
section works well.
• Don’t assume reviewers will dig out your papers.
January 2013
Office of Faculty Development
24
Address Feasibility
• Don’t assume that they will trust that you can do a
technique.
• If its challenging, address that.
• Show an example
• Refer to your publications (not just a citation)
• Name a skilled collaborator
– Make sure that the collaborator’s role is commensurate with the
work involved.
• Are there alternative approaches?
• Even if you have no space to articulate everything, show that you
are conscious of the issue.
January 2013
Office of Faculty Development
25
Note!
The application does
not begin or end with
the Approach Section!
January 2013
Office of Faculty Development
26
Abstract and Specific Aims
are crucial
• It is likely that only 2-3 members of a study
section will read your proposal in detail, but
every member votes.
• What the other members know independently
about your proposal may be limited to these
sections.
• Polish these sections when you have finished
the application.
– Make sure they are intelligible on their own.
January 2013
Office of Faculty Development
27
Budget
• The charge of a study section is to evaluate and
rank the science of the proposals they review.
But they also review the budget after the science
has been scored.
• Be careful to justify each item in your budget.
• Propose a budget that you think is realistic for the
work you propose and say why.
January 2013
Office of Faculty Development
28
Don’t blow off the other parts
• Biosketches
• Letters of support
– Request these early.
– Be prepared to draft these yourself
•
•
•
•
•
Resources
Equipment
Vertebrate Animals*
Human Subjects*
Resource Sharing Plan
Some of these are opportunities to sell your team or environment.
Others are actually scorable and can hurt you if carelessly done.
January 2013
Office of Faculty Development
29
Cover letter
You can, and should, tell the NIH which
SRGs and Institutes you think would be
best able to judge your work.
January 2013
Office of Faculty Development
30
Location, Location, Location
• Center for Scientific Review (CSR) Study Sections and Study Section members
are listed on the Office of Extramural Review (OER) website.
– Competitors are often less dangerous than people who don’t value your
topic.
• Pick a Study Section that is likely to be familiar with the work you propose.
• Pick an Institute whose mission best matches your research
– If it’s a tie, consider paylines and other factors (e.g. mechanism).
– You can request cross-listing, but usually only one institute will consider
your grant.
• Ask a more senior colleague for advice about where to direct your proposal.
• When in doubt, call a Program Officer. They are there to help you.
– Build a relationship. It may save you one day.
• State your preferences in the cover letter with a sentence justifying your choice.
January 2013
Office of Faculty Development
31
Advice for Clinical Researchers
Preparation
• Involve biostatistician early
– Sample size will dictate budget
– Budget may dictate scope of work
• Other documents are not “after-thoughts”
– Protection of Human Subjects
– Women & Minorities
– Children
– Planned Enrollment
– Multiple Leadership Plan
– Letters of Support – allow people time to write as they
may have deadlines as well
– Sub-awards
January 2013
Office of Faculty Development
32
Advice for Clinical Researchers
Preparation
• Have a supportive “mentor” (even for an R01)
• Clarify the roles of different members of research
team – i.e. collaborator vs. consultant vs.
coinvestigator, etc. This has an impact if the
proposal is funded
• Look at scientific review committee members
– Can contact SRO if you feel that additional expertise is
needed in the form of ad hoc members
– Also, you can request particular committee in your cover
letter
January 2013
Office of Faculty Development
33
Budget Preparation in Clinical Research
• Start early!
• Must work with Clinical Trial Finance
Management (CTFM)
• Clinical Translational Research Center (CTRC)
– Contact Veronica Kain for pricing
• CTRC will work with CTFM on budget
– Biochemistry Core Lab Costs—David Stokes
• Expect budget cuts even if you are funded
January 2013
Office of Faculty Development
34
Submitting the Application:
Sponsored Projects Office
1.
Complete the SmartForm data and forward.
 Program Announcement (PA), personnel, dates, subaward/prime
institutions
2.
3.
Initiate certification requests.
Research Business Manager (RBM) assists with budget planning and
entry.
 Correct forms from subs
4.
5.
6.
7.
January 2013
Sponsored Projects Officer (SPO) ensures correct forms and reviews
PA.
Use Activity button to upload science.
Authorized Organizational Representative (AOR) generates 424 from
Smartform.
PI has 2 days to critically review assembled proposal and make
revisions; submit early to allow for this!
Office of Faculty Development
35
Interpreting the Summary Statement
• Look for commonalities amongst reviewers
– Both strengths and weaknesses
• Look at scoring pattern
– Low scores in one area or across the board?
• Unscored/Not Discussed ≠ Hopeless
January 2013
Office of Faculty Development
36
Common Problems
Cited by Peer Reviewers
• Significance
–
–
–
–
Problem not important enough
Study not likely to produce useful information
No rationale for the proposed experiments is provided
Clinical implications not clear
• Not ready for prime time
– Studies based on shaky hypothesis or data
– Alternative hypotheses not considered
– Problem more complex than applicant realizes
• Problems with focus
–
–
–
–
January 2013
Lack of focus in Hypotheses, Aims, or Research Plan
Research priorities not clearly defined
Project is a fishing expedition
One of the Aims is “underdeveloped”
Office of Faculty Development
37
Common Problems
Cited by Peer Reviewers
• Feasibility concerns
– Insufficient preliminary data or preliminary data do not support the
project’s feasibility
– Overambitious research plan
– Methods unsuited to objective
• Concerns about the investigator
– Investigator too inexperienced with proposed techniques
– Too little detail to convince the reviewers that the applicant knows
what he/she is doing
– Concerns about productivity of investigator
– Inappropriate or lack of team members with expertise
January 2013
Office of Faculty Development
38
Common Problems
Cited by Peer Reviewers
• Innovation gone wrong
– Proposal is technology-driven (method in search of a problem)
• Insufficient attention to statistics
– Important to include statistician
• Sloppiness, inconsistencies, typos
– Will cost you points
January 2013
Office of Faculty Development
39
To Resubmit or Not?
• Are there fatal flaws (vs. you simply didn’t make your case)?
– If design has to change significantly new opportunities for weakness
• Was there some enthusiasm for the project?
– Did reviewers think idea was significant?
– “The topic itself for children with chronic illness is innovative and of
importance.”
– Did reviewers think design/topic was innovative?
• “The use of an “accelerated” longitudinal design is clever and seems
appropriate.”
January 2013
Office of Faculty Development
40
40
Planning the Resubmission
• Summarize all key points of review, separated
by major and minor problems
• Decide how each major problem will be
addressed:
– Could mean adding prelim data, changing
methods, enhancing team, or providing more
clarity/detail/justification (e.g., making your case)
• Minor problems can be addressed in one
sentence of Introduction or not at all
January 2013
Office of Faculty Development
41
Responding to Reviews in a
Resubmission
• Start with reminder of strengths:
– “We are gratified that the reviewers found our original
application “innovative and of importance,” with
“detailed attention to theoretical and developmental
context,” and that “the study follows in a
programmatic fashion” from our earlier work. In
response to the reviews, we have made the following
changes:”
• Categorize major areas, provide brief response to
each, and refer to appropriate section of grant.
January 2013
Office of Faculty Development
42
42
Responding to Reviews in a
Resubmission
Examples of Unscored R01 Critiques, PI
Response, and New Critiques (Funded)
January 2013
Office of Faculty Development
43
43
Preliminary Data
• Unscored application weakness:
– “Not enough preliminary data to support the study as designed”
(although 1 reviewer thought prelim data was strong)
• PI response:
– “The reviewers were in disagreement regarding the sufficiency
of the preliminary data. Since the initial application, we reran
our analyses on a sample of 181 dyads. We added data related
to the Child Report version of the CDMI and test-retest
reliability. We also included analyses that support the validity of
the CDMI and provide the foundation for several hypotheses of
this proposed R01.”
• Resubmission application strength:
– “The CDM tool, while still undergoing further evaluation, has
been developed and tested by the study team and seems to be
a valuable tool for assessing several components of the decision
making process.”
January 2013
Office of Faculty Development
44
44
Clinical Implications
• Unscored application weakness:
– “The potential impact for the future clinical management
of these pediatric illnesses is not clearly laid out.”
• PI response:
– “A new section in the Research Strategy addresses clinical
implications. In addition, our team now includes nurse
practitioners and physicians from the Diabetes and CF
Centers, who will provide feedback regarding clinical
implications.”
• Resubmission application strength:
– “If successful, this research could potentially inform the
development of interventions and training for both
adolescents and for parents.”
January 2013
Office of Faculty Development
45
45
Funded Resubmission
• Strengths of resubmission:
– Same as before, plus new strengths were directly
related to changes made in response to first
review:
• “The applicant was very responsive to prior critiques.”
• Weaknesses of resubmission:
– Only minor weaknesses
• Overall design, justification, and significance were
unchanged from original application, so there were not
new opportunities for weakness
January 2013
Office of Faculty Development
46
46
Lessons Learned as a New Investigator
• Leave no question unanswered, especially regarding
design and methodology
– Clear and consistent writing is critical
– Include timeline to show feasibility
• Repetition and consistency are good
• Select the right team
January 2013
Office of Faculty Development
47
47
Lessons Learned as a New Investigator
•
Be explicit about significance and clinical
implications (and everything!)
– How is proposed research the logical next step based on
your preliminary data and field in general?
– Exactly how will data be used to inform clinical issues?
Don’t take for granted that these will be obvious to
the reviewer
January 2013
Office of Faculty Development
48
48
New Investigator
• PI is a New Investigator (NI) if he/she has not
been PD/PI for a substantial NIH independent
research award.
– Okay to have had R00, R03, R15, R21, F, K, etc.
Basically, okay to have had anything other than an
R01.
• Multiple PD/PIs: If application involves more than
one PI, all PIs must meet definition of NI in order
for application to have NI status.
– Consider carefully: Multiple PI role not ideal for new
PIs
January 2013
Office of Faculty Development
49
49
Early Stage Investigator
• Early Stage Investigators (ESI) are NI within 10
years of completing terminal research degree or
medical residency
– Developed because of increasing age at first R01 and
decreasing R01s from new investigators
– Goal is to accelerate transition to an independent
scientific career
– Implemented in 2009
January 2013
Office of Faculty Development
50
50
Early Stage Investigator
• Special consideration during peer review:
– Reviewed separately from other R01s
– Reviewers focus more on proposed approach than track
record and expect less preliminary data than would be
provided by an established investigator.
• Special consideration at time of funding:
– Varies by institute:
• E.g., NICHD 2010 funding strategy:
– R01 payline: 13th percentile
– R01 payline for NI and ESI: 15th percentile
January 2013
Office of Faculty Development
51
51
New Investigator and Early Stage Investigator
Status in eRA Commons
• To ensure that NIH recognizes your Early Stage
Investigator (ESI) status, you must update your eRA
Commons profile to reflect the date of completion of
your terminal research degree or the end of your
residency.
• NIAID New Investigator Guide to Funding:
http://www.niaid.nih.gov/researchfunding/grant/Pages
/newpiguide.aspx
January 2013
Office of Faculty Development
52
52
Grant Writing Resources
• The Grant Application Writer’s Workbook
Stephen W. Russell, DVM., PhD and David C. Morrison, PhD
– NIH, NSF, and Any Agency versions available
– Each version begins with refinement of idea, progresses through tips
and strategies for each section of the proposal, and concludes with
pre-submission review and writing of the accompanying cover letter.
– Purchase at link above or borrow from Office of Faculty Development
Lending Library, Mary Field fieldm@email.chop.edu
•
January 2013
NIH Links
– Sample R01 Applications and Summary Statements
– Scientific Review Group Rosters
– Video of Mock Scientific Review Group
Office of Faculty Development
53
Grant Opportunities
• Research Institute list of federal, foundation
and industry funding
• Grants.gov search function
Comprehensive listing of federal funding
opportunities, including but not limited to NIH
January 2013
Office of Faculty Development
54