NHMRC Rebuttals
Prof. Jane Pillow,
Centre for Neonatal Research & Education
UWA, 2012
What is the Assessment that
you will receive?
• Goal is for 3 reviews/grant
– 2 External Reviewers – comments are important, but
scores don’t count for much
– Primary Spokesperson (part of multidisciplinary panel, may
not be expert
• The assessment comments wont identify whether they
are from an external reviewer or a spokesperson
• Since 2011, 2nd Spokesperson does not submit an
assessment until after you receive your review, but still
speaks to/supports the application during panel meeting
• Your grant will ONLY be scored by the GRP panel to
which your application is assigned
Grant Assessment Process
• Based on initial scoring by external reviewers, and both the
primary and secondary spokesperson, the bottom 1/3 of
applications in each GRP will be ranked as NFFC (not for
further consideration).
– This list is provided to the GRP 1 week prior to panel meeting
– Panellists can “rescue” an NFFC application which they feel
merit full discussion by the panel
• GRP’s meet for 5 days and categorise/rank the remaining
60-80 grants allocated to their panel based on:
– Grant application
– Assessor’s reports
– Rebuttal
• Your spokespersons are there to push your grant forward
– Each spokesperson is likely to only get 2-3 grants up of the
total of ~ 16 grants for which they have responsibility
Scoring Criteria
• Scientific Quality (50 %)
• Significance and Innovation (25%)
– Nb: your grant may not be especially innovative, but it may
have major significance for health outcomes.
• Track Record (25%)
Change since 2011:
Each panel member will have to vote on each of the above
criteria for each grant, rather than providing a single overall score
- potential opportunity in rebuttal to improve a subscore
identified as weak!
[7] HIGHEST INTERNATIONAL QUALITY AND
RESEARCH PERFORMANCE (1-5%)
Funded
[6] HIGHLY COMPETITIVE (5-10%)
[5] EXCELLENT (15%)
Fundable
[4] GOOD (25%) Remaining 50%
[3] SATISFACTORY
[2] MARGINAL
[1] POOR
Uncompetitive
Funded
Fundable
Not Fundable
Number of Applications
3000
2500
25
19
2000
37
1500
1000
30
23
24
48
49
58
46
52
27
27
23
23
23
39
33
37
500
30
41
43
44
42
36
34
34
36
23
23
22
22
40
21
42
21
0
2000
2002
2004
2006
2008
2010
2012
When?
2 periods:
1. Monday 18 June to Thursday 28th June; and
2. Monday 2nd July to Thursday 12th July;
A list of which applications will be considered during each
period is available from the NHMRC website (and has been
emailed to the CIA)
There is NO SCOPE for NHMRC to provide alternative
assessment periods – so CIA’s need to work with their CI team
to ensure an applicant response is provided in this period.
You have assessors reports…..
So what do you do now?
Take it seriously – your rebuttal counts!
A measured/considered rebuttal will improve a grant
• May change a borderline application into fundable
• May clarify what is otherwise seen as a fundamental flaw
BUT
A poor rebuttal or failure to submit a rebuttal may be fatal!
Don’t Despair!
Remember the objective:
You want money to do your
research
Make sure YOUR grant is one that your primary
spokesperson wants to get up
• Give them the ammunition they need!
• Make your grant stand out above the crowd
• Be convincing about a successful outcome (supported with
facts!)
Reality Check
The way you see your
NHMRC application
The way the NHMRC panel
sees your grant
Can you upgrade it in time?
(slide from N Morrison & M Forwood, Griffith Uni:
http://www.nccarf.edu.au/userfiles/Forwood%20-%20NHMRC.pdf)
Be familiar with the rubric and make it
easy for the panel to score your grant!
Can you include elevate your score in each of the subcategories?
Can you internationalise your grant?
Reinforce uniqueness of resources and opportunities
Stay Cool and Remain Professional !
• Keep your spokespersons batting on your side:
– Don’t attack the assessor OR the system
– Avoid sarcasm and derogatory statements
• Remember that the assessors are your peers and/or
superiors and that you assess NH&MRC grants yourself.
2010 Grant 1:
•
•
•
•
Comments received 3rd week of August!
1 Reviewer only
Stunningly brief
 general comments only, no significant critique
A few seemingly trivial questions
Initial response – INDIGNATION and DESPAIR!
Action – Contact Rob Roche
•
•
•
Simply respond, in a professional manner, to any issues raised as best you can.
Use your response to emphasise what you feel are the strengths of the project and the
research team.
Consult with the co-CIs in the drafting of the response
Was our project doomed? – No!
• Start Early
• Successful response writing
requires feedback and constructive
support from collaborators and
uninvolved peers
• THIS TAKES TIME and your
COLLABORATORS/PEERS ARE
BUSY (possibly prioritising their
own CIA grants!)
• Identify and highlight issues for
Fellow CI’s to address – and set
deadlines for receipt of their
responses!
• Remember RGMS peak periods –
don’t leave it until the last minute
Examine the Assessment Content
• Take note of the descriptors used by the assessors and compare them
to the scoring rubric
– Does one of the 3 scored areas drag your grant down?
– Be aware to assessor rankings (often unrealistic)
• Identify the major issues raised and categorise them according to the
rubric and put in a grid to assist prioritisation
Assessor 1
Scientific Quality
Issue 1
Issue 2
Significance & Innovation
Issue 1
Issue 2
Track Record
Issue 1 etc
Budget
Assessor 2
Assessor 3
Formulate your response
• Summarise the positive comments and important
features of your grant in the first 2-3 sentences
• Combine issues from different reports using relevant
short headers
• Prioritise issues within headers and order accordingly
• Formulate response
– Stay focussed, answer factually and honestly
– Include new data that
• supports hypothesis (without showing you have done the complete
first year of the grant!)
• demonstrates feasibility/technical expertise with new methodology
(reduce technical risk)
• addresses any perceived weakness identified in reviews
– Apportion response to each issue relative to importance
– Don’t answer all the minor remarks unless space permits
Formulate your response (2)
• Reference where needed to support contentions
• Do not substantially redesign the study:
– you have been assessed on the submitted grant and cannot
be re-evaluated on a new study
• Use clear, logical and positive statements
– Positive language is important:
• We aim to, the aims are, we will use, it is planned, we
propose, we showed, has the potential; we discovered…..
– Negative language hinders:
• Hope, wish, might, probably (Hopes and wishes might get you
funded, but they probably won’t)
Formulate your Response (3)
• Assessors/GRP frequently make valid and useful points.
• Acknowledge the useful points and thank the assessor, but
keep your responses strong! Turn criticism into advantage.
• Reinforce/emphasise positive comments by assessors –
but ensure apparently good assessments are valid
• You cannot appear to have missed something important
entirely.
– If you actually have, you are doomed – unless you can
rescue it in your response!
• However, if the assessor is wrong, carefully point out that
the assessor have “misunderstood”.
– Carefully identify error in scientific fact in a hostile review
Track Record and Budget
• If there is space add any new publications/major grants
– Don’t include full citation details if clearly evident in
PubMed (eg CIA – 4 additional 2011 publications on
PubMed)
– If lots of publications – detail those MOST RELEVANT to
this application
• Bolster track record of a CI that might otherwise draw
team track record down if possible
– Eg International Keynote talks
• CIA is most important
Budget
• Some budget cut is almost inevitable – especially if you
have not justified all your costs!
• Respond to any major suggested cuts that may
compromise ability to complete grant
– Clarify justification and workload
Develop Your Document
• MS Word Template available on NHMRC Project Grant
website under “Documentation to apply for a Project
Grant
– Pay attention to the formatting requirements
– If insufficient space
• Eliminate “flowery” language
• Prioritise space allocated to responses
Then What?
• Circulate to your colleagues
– Co-investigators (CI’s and AI’s)
– Experienced but less emotionally involved colleagues
• Leave it for a few days
• Revise
• Submit your response WITHIN the ALLOCATED TIME PERIOD
• Date for submission is specified on response letter
• Submitted on RGMS
If you are stuck?
Rob Roche has been the UWA conduit to NH&MRC for a
long, long time.
If you are having difficulties with RGMS:
a) Contact RGMS help desk
b) Contact Robert
Robert Roche: robert.roche@uwa.edu.au
Tel: 6488 2033
c) Obtain advice from other experienced colleagues
What next?
DO NOT contact panel members directly!
3
2
1
4
Steps:
1. Results and Rebuttals
2. Select Round
3. Filter
4. Click the properties box of the
application to access and view
the rebuttal letter
If the letter is available, the folder
icon will be visible in the “Letter of
Result” column
Click on the folder icon to
download letter
To upload the rebuttal response
Click on the rebuttal tab.
1
2
3
1. Proceed to the Rebuttal link in the sub menu.
2. To upload the rebuttal response.
3. Click Save.
1
2
1. Select Yes, submit my response to
NHMRC when upload is complete.
2. Click Save or submit to lock the
response and to make it available
to NHMRC.