Berlin Institute of Health
(BIH)
Luisenstraße 56
10117 Berlin
Germany
Evaluation Guidelines and Evaluation Template
for
Collaborative Research Grant Applications
PART I: GENERAL OVERVIEW ...................................................................................................................... 3
INTRODUCTION ...................................................................................................................................................... 3
THE SCOPE OF FUNDING ........................................................................................................................................................ 3
DURATION OF FUNDING ........................................................................................................................................................ 4
ELIGIBILITY REQUIREMENTS ................................................................................................................................................ 4
IMPORTANT TIMELINES AND DEADLINES........................................................................................................................... 4
THE ASSESSMENT PROCEDURE ............................................................................................................................................ 4
CONFIDENTIALITY ................................................................................................................................................................. 5
DECLARATION OF INTEREST ................................................................................................................................................ 5
DIVERSITY AND EQUAL OPPORTUNITIES ............................................................................................................................ 5
EVALUATION GUIDELINES..................................................................................................................................................... 6
TABLE I: NIH-BASED SCORING CHART ............................................................................................................................... 6
PART II: SCORING AND WRITTEN EVALUATION OF THE RESEARCH PLAN ................................. 7
1.
2.
3.
4.
5.
6.
7.
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
8.
STRATEGIC RELEVANCE OF THE PROPOSAL FOR FUNDING BY THE BIH..................................................... 7
SCIENTIFIC QUALITY AND RELEVANCE OF THE RESEARCH PROPOSAL ........................................................ 7
ENVIRONMENT, ORGANIZATION AND MANAGEMENT OF THE CONSORTIUM .............................................. 8
QUALIFICATIONS AND TRACK RECORDS OF THE COORDINATORS .............................................................. 8
COLLABORATION BETWEEN INDIVIDUAL SUBPROJECTS ............................................................................. 9
PROJECT IMPLEMENTATION AND COSTS ...................................................................................................... 9
EVALUATION OF THE INDIVIDUAL SUBPROJECTS......................................................................................... 9
SUBPROJECT 1 ............................................................................................................................................................. 9
SUBPROJECT 2 ........................................................................................................................................................... 10
SUBPROJECT 3 ........................................................................................................................................................... 11
SUBPROJECT 4 ........................................................................................................................................................... 12
SUBPROJECT 5 ........................................................................................................................................................... 12
SUBPROJECT 6 ........................................................................................................................................................... 13
SUBPROJECT 7 ........................................................................................................................................................... 14
SUBPROJECT 8 ........................................................................................................................................................... 15
THE OVERALL SCORE .................................................................................................................................. 15
PART V: YOUR FEEDBACK ON THE EVALUATION PROCESS ........................................................... 17
Part I: General Overview
Introduction
In biomedical science, translational research combines the findings of basic researchers
and of clinician scientists to transform discoveries and observations into new diagnostic
and therapeutic procedures with the ultimate goal to improve health. Translational
research projects aim at bringing together a multidisciplinary team of researchers and
clinicians in order to address unmet clinical needs, and to achieve internationally visible
expertise beyond the scope of conventional scientific cooperation.
Collaborative Research Grants (CRGs) of the BIH aim to provide competitive funding for
large-scale research projects with high translational relevance. Following the paradigm
of systems medicine, they should aim to deliver substantial solutions for emerging
questions of pathogenic mechanisms and unmet clinical needs. Projects have to be
interdisciplinary, ideally including scientists from different research fields, and should
aim to contribute substantially to the understanding of basic mechanisms of disease
processes as well as to the exchange of knowledge between basic research and clinical
application (e.g. diagnostic and therapeutic development). The projects should employ
the approach of systems medicine, show their potential for translational research and
have a clear fit to the long-term objectives of the BIH.
The scope of funding
Large consortia of basic scientists and clinicians from both MDC and Charité will be
funded. The main scientific theme of the proposal should be developed within 6 to 8
more specific sub-projects. Each sub-project should have a clearly defined translational
perspective.
The priority areas of the 1st CRG Call are:
1. Exploring the immune system to control disease
2. Overcoming biological barriers to treatment
3. Proteostatic mechanisms in health and disease
4. Exploring the heritable dimensions of disease for improved clinical outcome
CRGs will cover costs of personnel and consumables. Justified requests for contributions
to running costs for central facilities may be supported. Acquisition of additional
competitive funding for the consortium’s work from other extramural agencies (e.g.
DFG, BMBF, EU, etc.) is encouraged. The extramural funding should clearly be of
complementary nature and non-redundant. Individual principal investigators can apply
within different consortia provided the proposed research is non-redundant.
Duration of funding
Funding is initially granted for up to four years. It is foreseen that an extension of the
project will be possible after a successful evaluation in the fourth year.
Eligibility requirements
Each consortium should include both basic scientists and clinicians from both MDC and
Charité, and be led by a consortium coordinator and a deputy coordinator. The
coordinator will represent the research consortium towards the BIH and third parties
and will have the overall responsibility for the management of the consortium. Ideally,
coordinator and deputy coordinator should not come from the same institution. Each
principal investigator of the consortium should have a proven scientific track record
either in biomedical or in clinical research. He or she should also have enough protected
time to contribute to the project. The coordinator should have a proven track record of
translational biomedical research, the ability to organize research consortia and be
willing to dedicate a significant amount of time to the project.
Important timelines and deadlines
1st Call:
Submission of proposals: November 5 – January 5, 2014
Return of the reviews: February 11, 2014
Interviews: February 28, 2014
Earliest start of project: April 1, 2014
The assessment procedure
Please evaluate the collaborative research grant application assigned to you using this
form. After completing all fields, please save the document, preferably as PDF file to
prevent any subsequent changes and submit your review to the BIH head office to
christian.hirsch@mdc-berlin.de by February 11, 2014. Your identity as reviewer will
remain anonymous to the authors of the proposal; however your anonymized comments
will be made available to the applicants. Based on your reports, as well as on structural
and strategic considerations of the BIH Directorate, selected applicants will be shortlisted for interviews. Based on the vote of the Scientific Advisory Board, the BIH
Directorate will decide which proposals will be funded. Resubmission of applications for
subsequent calls will be possible.
If you have any questions regarding the review procedure, please contact
Dr. Christian Hirsch at the BIH Head Office.
Dr. Christian Hirsch
Berlin Institute of Health (BIH)
Luisenstraße 56, 10117 Berlin
Phone: +49 30 9406 2070
Email: christian.hirsch@mdc-berlin.de
Confidentiality
By taking cognizance of the proposal, reviewers commit themselves to observe strict
confidence. Reviewers must keep all information pertaining to the grant and its
assessment confidential and agree to never retain, use or copy the information of the
proposal. Reviewers must not make use of the research design or findings or allow
others to do so.
Declaration of interest
To avoid conflicts of interest between reviewers and applicants during the assessment,
all potential conflicts must be declared before reviewing a proposal. Conflicts of interest
may include the following:
o Close scientific collaborations, e.g. joint research projects or joint publications
o Direct scientific competition
o Close professional proximity, e.g. members of the same institute or closely
connected institution during the past 5 years
o Personal economic interests in the funding decision
o Personal relationships, connections or conflicts
Diversity and equal opportunities
Please pay particular attention to avoid any discrimination regarding non-scientific
aspects of the application/applicants such as age, gender, nationality or disabilities.
Therefore, consider the applicant’s scientific career development rather than her or his
age. Unavoidable delays in the applicant’s career, for example childcare responsibilities
causing gaps in publication lists, less time spent abroad or longer periods of
qualifications, should be taken into consideration.
Evaluation guidelines
The evaluation form is divided into eight sections. The first six concentrate on the
proposal as a whole. Section 7 focuses on the individual subprojects and shall be
completed for each subproject (or at least those subprojects that you deem most worthy
of commenting). Finally, we ask you in section 8 for your overall score for the proposal.
Sections 1-7 consist of two units: A score questionnaire and a free-text field. In the score
questionnaire, please use the pull-down menus to answer each question. When you
select such an item, a menu appears just below it. In most instances, you can rate the
questions with an impact score ranging from 1 (exceptional) to 9 (poor) in accordance
with the NIH-based scoring scale in table I. In some cases, you can answer with yes or no
or choose a percentage. Please base your assessment on the highest international
standards in the relevant research field. We strongly encourage you to use the entire
range of scores, to facilitate a distinct ranking of the submitted proposals.
In the free-text field we ask you to assess the program using the corresponding
questionnaire as guide and to discuss your scorings. While we intend to use the score
charts to gain an overview, we consider the free-text fields as the main part of the
evaluation form.
Please also use this field to evaluate or comment on aspects of the section that are not
covered by the questionnaire. Where appropriate, justify comments with references to
key publications. Please avoid making any suggestions; your goal is to evaluate what has
been presented, not how it could have been done better.
When commenting on the proposal, please avoid any direct comparison with other
proposals. Please keep in mind that not everyone reading your comments is a specialist
in the field. Thus, please write comments that a generalist is able to understand.
As this is our first evaluation, we also ask you for your feedback at the end of the form to
improve future evaluations.
Table I: NIH-based scoring chart
Impact
high
moderate
low
Impact score
1
2
3
4
5
6
Descriptor
Exceptional
Outstanding
Excellent
Very Good
Good
Satisfactory
7
Fair
8
9
Marginal
Poor
Detailed explanation
Exceptionally strong with essentially no weaknesses
Extremely strong with negligible weaknesses
Very strong with only some minor weaknesses
Strong but with numerous minor weaknesses
Strong but with at least one moderate weakness
Some strengths but also some moderate weaknesses
Some strengths but with at least one major
weakness
A few strengths and a few major weaknesses
Very few strengths and numerous major weaknesses
Definitions:
Minor: addressable weakness that does not substantially lessen the impact of the project.
Moderate: weakness that lessens the impact of the project.
Major: weakness that severely limits the impact of the project.
Part II: Scoring and written evaluation of the research plan
1. Strategic relevance of the proposal for funding by the BIH
Questions:
Score
How well does the proposal fit into the BIH funding line of “Collaborative
Research Grants”?
-
Please rate the translational perspective of the proposed program.
-
How well does the proposal apply systems-oriented approaches to medicine?
-
How well does the program foster cooperation between basic researchers and
clinician scientists?
How do you rate the chances of success for this proposal,
if submitted to major national or international funding agencies?
-
In the following free-text field, please assess the strategic relevance of the program
for funding by the BIH using the above questionnaire as guide. Please evaluate or
comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 1 page)
2. Scientific quality and relevance of the research proposal
Questions:
Score
How do you rate the scientific quality of the research proposal?
-
Are the research plan, objectives and hypotheses clearly presented and realistic?
-
Overall feasibility and chances for success of the research proposal.
-
How original, innovative and coherent is the proposal?
-
How do you rate the availability of required research methods and materials for
the project?
Please rate the potential relevance of the proposal for health care and clinical
practice.
-
In the following free-text field, please assess the scientific quality and relevance of the
research program using the above questionnaire as guide. Please evaluate or comment
important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 1 page)
3. Environment, organization and management of the consortium
Questions:
Are the project management, resources and the division of labor and the
proposed schedule appropriate and well planned?
Does the described available research environment support the research
program, including appropriate research infrastructures (e.g. BIH core
facilities)?
What is the situation and outlook for female researchers in the proposal?
Are appropriate policies taken to ensure equal opportunities?
How do you rate the measures (inviting internationally recognized researchers
for presentations and collaboration, presenting at international meetings, etc.)
aimed at increasing international visibility of the consortium and the BIH?
Score
-
Please assess the environment, organization and management of the consortium
using the above questionnaire as guide. Please evaluate or comment important aspects
of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
4. Qualifications and track records of the coordinators
Questions:
Score
How do you assess the training and suitability of the coordinator to conduct the
research program?
-
How do you rate the track record of the coordinator?
-
How do you assess the training and suitability of the deputy to conduct the
research program?
-
How do you rate the track record of the deputy?
-
How appropriate are the leadership approach, governance and organizational
structure for the project (coordinator and deputy coordinator)?
-
How do you rate the investigators complementary and integrated expertise?
-
How do you rate the qualification and track record of the consortium members
in terms of basic research and clinical research, respectively?
-
Please assess the coordinators’ qualifications and track record using the above
questionnaire as guide. Please evaluate or comment important aspects of the section
that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
5. Collaboration between individual subprojects
Questions:
Score
Please rate the appropriateness of the division into individual subprojects in
order to achieve the overall goal.
-
Please assess the degree of cooperation between the individual subprojects.
-
Please rate the potential for synergies between individual projects.
-
Please rate the measures for the exchange of ideas and discussions on the
projects’ progress (i.e. in the form of joint seminars or meetings).
-
Please assess the collaboration between individual subprojects using the above
questionnaire as guide. Please evaluate or comment important aspects of the section
that are not covered by the questionnaire.
<Your text here> (approximately 0.5 page)
6. Project implementation and costs
Questions:
Score
Is the requested amount of funding appropriate?
-
Has the requested amount of funding been sufficiently justified?
-
Please rate the approach taken regarding intellectual property that may arise
from the project.
-
Is there evidence for double funding of projects?
-
Please assess the project implementation and costs using the above questionnaire as
guide. Please evaluate or comment important aspects of the section that are not covered
by the questionnaire.
<Your text here> (approximately 0.5 page)
7. Evaluation of the individual subprojects
7.1 Subproject 1
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
-
-
Please assess the subproject 1 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.2 Subproject 2
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
Please assess the subproject 2 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.3 Subproject 3
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
-
-
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
Please assess the subproject 3 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.4 Subproject 4
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
-
-
Please assess the subproject 4 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.5 Subproject 5
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
-
-
Please assess the subproject 5 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.6 Subproject 6
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
Please assess the subproject 6 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.7 Subproject 7
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
-
-
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
Please assess the subproject 7 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
7.8 Subproject 8
Questions:
Score
How do you rate the scientific quality of the subproject?
-
How realistic are the subproject’s objectives and hypotheses?
-
How do you judge the overall feasibility and chances for success for the
subproject
-
Are the experimental design and timeline of the subproject feasible?
-
Do the PIs sufficiently acknowledge potential problems and consider
alternatives?
Please rate the availability of required research methods and materials for the
project.
-
How original, innovative and coherent is the subproject?
-
Please rate the potential relevance of the subproject for health care and clinical
practice?
-
Please rate the qualification of the principal investigators of the subproject.
-
Please rate the relevance of this subproject for developing a joint research space
between Charité and MDC.
-
How well does the subproject fit with the overall goals of the collaborative
project?
If you were asked to cut 30 % of the overall budget, by what percentage would
you cut the budget of this subproject (please specify below)?
-
-
-
Please assess the subproject 8 using the above questionnaire as guide. Please evaluate
or comment important aspects of the section that are not covered by the questionnaire.
<Your text here> (approximately 0.5 - 1 page)
8. The overall score
Questions:
Score
Please indicate your overall grade for this application.
-
Part V: Your feedback on the evaluation process
As this is our first CRG evaluation and all documents were prepared in a rather short
time, the evaluation procedure is still far from perfect. To improve the forthcoming
evaluations, we would be very thankful for your suggestions on how to improve the
documents (i.e. important points that we missed in this form) and any other suggestions
that you deem relevant.
<Your text here> (no space limit)