Lisa 1
KINNITATUD
Sihtasutus Eesti Teadusagentuuri juhatuse
25.02.2014 käskkirjaga nr 1.1-4/14/27
Institutsionaalse uurimistoetuse taotluse hindamisjuhend
Guidelines for evaluating institutional research funding applications
Introduction: Legal background.
Institutional research funding is one of the most important instruments of research funding in Estonia. Its
aim is to support high quality research in R&D institutions, and thus to guarantee the sustainability of
institutional research. These dual goals – to support high quality research and to take into account the
institutional stability of the research system - are rather unique for a funding instrument. Therefore,
assessment procedures need a comprehensive guide. The aim of this document is to set and describe
the guidelines for evaluating institutional research funding applications. The evaluation criteria and their
content, as well as the rating scales are described. The guidelines should provide a uniform basis for
evaluators to assess the applications and also help applicants to understand the evaluation procedure
and fill in the application forms.
For additional information, details of the legal background are made available in the endnote to this
documenti.
II Relevant terms
1) institutional research funding – funding allocated for financing high level research and
development (R&D) and related activities (research topics) of an R&D institution, for ensuring the
consistency of research and development of an R&D institution, and for upgrading, supplementing,
and maintaining the infrastructure necessary for this purpose;
2) research topic – an R&D institution’s basic and applied research conducted within the framework
of a clearly defined and scientifically justified research and development activity plan;
3) research funding applicant – an R&D institution operating as a legal entity or state institution
whose research and development has received regular positive evaluation in at least one research
field as of the application date.
4) research team – a team consisting of research team leader, senior staff, and supporting staff.
5) research team leader – a researcher in charge of carrying out a research topic.
6) senior research staff – a researcher or researchers responsible for carrying out an essential part
of a research topic.
7) support staff – all other members of a research team besides the research team leader and the
senior research staff. The support staff may include scientific and technical staff, and Ph.D. and
Master students.
1
III Structure, content, and evaluation of an institutional research funding application
An institutional research funding application consists of five sections. Its structure and expected content
are outlined below for the attention of the reviewers. Sections 1 and 2 are to be evaluated by external
reviewers and will serve as input for the evaluation panels and the Evaluation Committee. Sections 3-5
are to be evaluated by the evaluation panels and by the Evaluation Committee.
1. Assessment of the research topic
For the attention of the reviewers: the applicant had to outline the contents of the proposed research
topic (up to 30,000 characters with spaces across all sections; the list of references, pictures and
schemes uploaded as separate files) using the following structure.
1) Introduction and motivation
The applicant had to present the general motivation for the proposed research topic, explain its role
and importance to science, present its general theoretical background, and main objectives and
their justification.
2) Details of the proposed research topic
The applicant had to present a detailed justification of the proposed research topic, working
hypotheses, methods, etc., and an outline of the work plan. If appropriate, this should include an
explanation of how ethics requirements will be adhered to and fulfilled (in the case of animal and
human experiments).
3) Expected results
The applicant had to outline the expected results and possible future developments of the proposed
research topic, and explain their importance for science, society, and culture.
Based on the above information, the reviewers should provide answers and comments to the following
questions (up to 2000 characters to each question):
1.1. Is the proposed research topic scientifically well motivated, and its theoretical background and
main objectives clearly outlined? Please provide comments to justify your answer.
1.2. Is the proposed research topic well justified? Please provide comments to justify your answer.
1.3. Are the working hypotheses, methods, and the work plan well described and appropriate for
achieving the goals? Please provide comments to justify your answer.
1.4. What is the expected impact of the proposed research on the specific field of research, society, and
culture? Please provide comments to justify your answer.
1.5. If applicable, have the ethical requirements for human and animal studies been met? Please
provide comments to justify your answer.
Other comments on Section 1.
Overall score for the research topic (1-5 on a 5-point scale, see part IV)
2. Assessment of the research team
For the attention of the reviewers: The applicant had to explain the quality of the research team (up to
2000 characters) using the following structure.
1) Qualifications of the research team leader and the senior research staff
2
The applicant had to present information on the academic qualifications of the team leader and the
senior research staff based on their work over the past 10 years, review their leadership qualities,
involvement in the supervision of Ph.D. students and postdoctoral researchers, participation in
international projects, collaboration, national research programs, etc. If applicable for the particular
research field, the applicant should provide additional information about each senior staff member,
e.g., their number of publications, number of citations, Hirsch index, etc.
2) Previous research of the research team leader and senior research staff, and their collaboration on
the proposed topic
The applicant had to present information on how the previous research of the research team leader
and senior research staff is related to the proposed research topic, including their previous
collaboration, what the roles of the research team leader and senior research staff are in forming
the research team, and what (complementary) expertise they bring to the team.
3) Description of support staff and their roles
The applicant had to provide information (up to 1000 characters) about the support staff: their
number, expertize, and justification for the positions applied for.
Based on the above information, the reviewers should provide answers and comments to the following
questions (up to 2000 characters to each question):
2.1. Is the research team leader an internationally recognized researcher whose previous research
results have been widely acknowledged over the last 10 years? Please compare his/her scientific
achievements with top researches in this field worldwide. Please provide comments to justify your
answer.
2.2. Does the level of the senior research staff correspond to that of an international-level independent
researcher over the last 10 years? Please comment on the academic level and on strengths and
weaknesses of each team member and provide comments to justify your answer.
2.3. Do the team leader and the senior staff have sufficient experience in the management of both
international and national research projects? Please evaluate the leadership qualities of the senior
research staff. Please provide comments to justify your answer.
2.4. Have the research team leader and senior research staff been involved in supervising Ph.D.
students and postdoctoral researchers? Please provide comments to justify your answer.
2.5. Are the size and age composition/balance of the research team leader and senior research staff
justified and optimal to achieve the research objectives, the sustainability of the research topic, and the
education of a new generation of researchers? Please provide comments to justify your answer.
Other comments on Section 2.
Overall score for the research team (1-5 on a 5-point scale, see part IV)
3. Research infrastructure
This section will be evaluated only by local experts.
The applicant had to describe (up to 3000 characters) any existing research infrastrucure and, if
relevant, which national/international research infrastructures the team has access to in order to carry
out the presented research topic. Where appropriate, the applicant had to explain how the research
infrastructure is related to national/international research programs, and research infrastructure
roadmaps, etc.
3
Based on this information, local experts should provide answers and comments to the following question
(up to 2000 characters):
3.1. Are the infrastructure and research environment appropriate for the proposed research topic?
Please provide comments to justify your answer.
Overall score for the research infrastructure (1-3 on a 3-point scale, see part IV)
4. Institutional and national relevance
This section will be evaluated only by the Evaluation Committee.
The applicant had to provide information on the importance of the research topic for the institution and
on its national relevance using the following structure. The total sum of the earmarked strategically
important topics could be up to 30% of the sum of the targeted and institutional research funding of the
R&D institution ending this year.
1) Institutional relevance.
The applicant had to describe (up to 2000 characters) the importance of the research topic to the
strategic interests of the institution, e.g., for Ph.D. and Master’s programs and state research
institutes for carrying out their statutory goals.
2) Position of the research topic in Estonia.
The applicant had to describe (up to 2000 characters) the role of the research topic in maintaining
the level of specific scientific knowledge in Estonia, the uniqueness of the research topic, the extent
of cooperation with other institutions, etc.
3) Institutional commitment
The applicant had to describe (up to 2000 characters) its commitment to the sustainable
development of the proposed research topic, e.g., how many postdoctoral fellows and junior and
senior researchers the applicant R&D institution has employed during last 10 years to work on the
research topic, and which (additional) resources the applicant R&D institution has allocated to the
research topic.
4) Connection to national strategic plans, research infrastructure roadmap, national research programs
If relevant to the research topic, the applicant had to desribe its connection to national strategic
plans, national research programs, research infrasructures, etc.
Based on the information provided to the above, the Evaluation Committee should provide answers and
comments to the following questions (up to 2000 characters to each question):
4.1. Is the proposed topic among the strategic topics of the R&D institution and based on its strategic
objectives? Please provide comments to justify your answer.
4.2. Does the research topic have a crucial role in maintaining the level of specific scientific knowledge
in Estonia? Please provide comments to justify your answer.
4.3. Are there similar/close research topics in other Estonian R&D institutions? If yes, what makes the
proposed research topic unique and strategically important compared to others? Please provide
comments to justify your answer.
4.4. Describe the previous institutional commitment to building up expertize in the research topic, e.g.,
how many new postdocs and junior and senior researchers has the applicant R&D institution
employed during last 10 years to work on the research topic? Which resources has the applicant
R&D institution allocated to the research topic?
4
4.5. Is the application connected to national strategic plans, a research infrastructure roadmap, national
research programs, etc.? Please provide comments to justify your answer.
Other comments to Section 4.
Overall score for the institutional and national relevance (1-3 points). The strategically important
research topics earmarked by the R&D institution will be awarded 2 points on top of the score.
5. Budget
This section will be evaluated only by the Evaluation Committee, but not scored.
The applicant had to outline (up to 1000 characters) the justification for the number and structure of
research positions necessary for carrying out the research topic, and present the budget of the research
topic according to the guidelines set by the Estonian Research Council.
The evaluators should answer and comment (up to 2000 characters) on the following:
5.1. Are the expenses related to personnel and research justified for carrying out the planned research?
Please provide comments to justify your answer.
5.2. Is the accommodation of other monetary resources which will become available due to the merger,
expiry or termination of other projects into this research justified? Please provide comments to
justify your answer.
Overall comments on the budget.
Overall assessment
This section will be filled in by the Evaluation Committee.
Overall score of the application (to be summarised by the Evaluation Committee).
Overall score for an application earmarked as a strategically important research topic can be up
to 18 points and for an application not earmarked - up to 16 points.
IV Evaluation criteria and rating scales to be used in the review
In evaluating the applications, please take into account the following guidelines.
A 5-point rating scale is used in evaluating sections 1 and 2 of the application (outstanding, very good,
good, satisfactory or unsatisfactory). A 3-step rating scale (unsatisfactory, good or outstanding) is used
in evaluating sections 3 and 4 of an application.
The values for criteria in the drop-down menu for sections 1 and 2 are as follows:

Outstanding (5);

Very good (4);

Good (3);

Satisfactory (2);

Unsatisfactory (1).
5
The values for criteria in the drop-down menu for sections 3 and 4 are as follows:

Outstanding (3);

Good (2);

Unsatisfactory (1).
Research topics which receive less than three points for sections 1 and 2 or less than 2 point for
sections 3 and 4 do not qualify for funding.
The rating scales correspond to the following assessments.
1. Assessment of the research topic
1.1. Is the proposed research topic scientifically well motivated and are its theoretical background and main
objectives clearly outlined?
Unsatisfactory. The proposed research topic is scientifically poorly defined, its theoretical background and main
objectives are poorly outlined.
Satisfactory. The proposed research topic is scientifically poorly motivated, its theoretical background and main
objectives are somewhat outlined.
Good. The proposed research topic is scientifically motivated, its theoretical background and main objectives are
outlined.
Very good. The proposed research topic is scientifically well motivated, its theoretical background and main
objectives are clearly outlined.
Outstanding. The proposed research topic is scientifically highly motivated, its theoretical background and main
objectives are clearly outlined.
1.2. Is the proposed research topic well justified?
Unsatisfactory. The proposed research topic is weak.
Satisfactory. The proposed research topic is somewhat original and innovative research at the national level; a
methodologically sound study but some areas require revision.
Good. The proposed research topic is well justified, this is original and innovative research at the national level; a
methodologically sound study.
Very good. The proposed research topic is well justified; this is original and innovative research at the
international level, and includes novel methodology and design.
Outstanding. The proposed research topic is highly justified; this is highly original and innovative research at
international level, and it includes novel methodology and design.
1.3. Are the working hypotheses, methods and the work plan well described and relevant for achieving the goals?
Unsatisfactory. The working hypotheses and methods are weak, research plan needs profound revision; the
division of subtasks/subtopics is poor and hardly supports the achievement of the overall goal.
Satisfactory. The working hypotheses and methods are somewhat articulated and justified, research plan needs
major improvements, the division of subtasks/subtopics is satisfactory but additional clarifications and
adjustments are inevitable. It is not clear whether the proposed approach supports the achievement of the overall
goal.
Good. The working hypotheses and methods are articulated and justified, research plan needs some clarification,
the division of subtasks/subtopics is presented well and the overall goals can be achieved, but certain
improvements and adjustments are still necessary.
6
Very good. The working hypotheses, methods and the work plan are clearly described and relevant for achieving
the goals, they are clearly articulated and justified; the division of subtasks/subtopics is clear and justified and
supports the achievement of the overall goal.
Outstanding. The working hypotheses, methods and the work plan are very clearly described and relevant for
achieving the goals, they are very clearly articulated and justified; the division of subtasks/subtopics is very clear
and justified and supports the achievement of the overall goal.
1.4. What is the expected impact of the proposed research on the specific research field, society, and culture?
Unsatisfactory. The expected impact of the proposed research on the specific research field, society, and culture
are vaguely outlined. The expected results and possible future developments of the proposed research topic are
also vaguely outlined. The expected results do not ensure national and international acceptability,
competitiveness or high quality of the research topic
Satisfactory. The expected impact of the proposed research on the specific research field, society, and culture
are vaguely outlined. The expected results and possible future developments of the proposed research topic are
also vaguely outlined. The expected results somewhat ensure national and international acceptability,
competitiveness, and high quality of the research topic
Good. The expected impact of the proposed research on the specific research field, society, and culture are not
very clearly outlined. The expected results and possible future developments of the proposed research topic are
not very clearly outlined. The expected results somewhat increase the knowledge base in the specific research
field, and if appropriate, are of relevance for societal problems and culture.
Very good. The expected impact of the proposed research on the specific research field, society, and culture is
high. The expected results and possible future developments of the proposed research topic are clearly outlined.
The expected results definitely increase the knowledge base in the specific research field, and if appropriate, are
of high relevance for societal problems and culture.
Outstanding. The expected impact of the proposed research on the specific research field, society, and culture is
high. The expected results and possible future developments of the proposed research topic are very clearly
outlined. The expected results substantially increase the knowledge base in the specific research field, and if
appropriate, are of high relevance for societal problems and culture.
1.5. If applicable, have the ethical requirements for human and animal studies been met?
(This criterion will be assessed but not numerically scored by reviewers)
The ethical issues for human and animal studies are considered insufficiently.
The ethical issues for human and animal studies are partially considered.
The ethical issues for human and animal studies are adequately considered.
Not applicable.
2. Assessment of the research team
2.1. Is the research team leader an internationally recognized researcher whose previous research results have
been widely acknowledged over the last 10 years? Please compare his/her scientific achievements with top
researches in this field worldwide.
Unsatisfactory. The research team leader is not an internationally recognized researcher, but her/his previous
research results have been somewhat acknowledged over the last 10 years.
Satisfactory. The research team leader is an internationally recognized researcher whose previous research
results have been somewhat acknowledged over the last 10 years.
Good. The research team leader is an internationally recognized researcher whose previous research results
have been acknowledged over the last 10 years.
7
Very good. The research team leader is an internationally recognized researcher whose previous research results
have been well acknowledged over the last 10 years.
Outstanding. The research team leader is an internationally recognized researcher whose previous research
results have been widely acknowledged over the last 10 years.
2.2. Does the level of the senior research staff correspond to the international level of an independent researcher
over the last 10 years? Please provide comments on the scientific level, on strengths and weaknesses of
each team member.
Unsatisfactory. None of the senior research staff correspond to the international level in their respective field over
the last 10 years.
Satisfactory. Some of the senior research staff corresponds to the international level in their respective field over
the last 10 years.
Good. Most of the senior research staff correspond to the international level in their respective field over the last
10 years.
Very good. The senior research staff correspond to the international level in their respective field over the last 10
years.
Outstanding. The senior research staff correspond well to the international level in their respective field over the
last 10 years.
2.3. Do the team leader and the senior research staff have sufficient experience in the management of both
international and national research projects? Please evaluate the leadership qualities of the senior research
staff.
Unsatisfactory. The team leader and the senior research staff have no experience in management of either
international or national research projects.
Satisfactory. The team leader and the senior research staff have little experience in management of both
international and national research projects.
Good. The team leader and the senior research staff have some experience in management of both international
and national research projects.
Very good. The team leader and the senior research staff have solid experience in management of both
international and national research projects.
Outstanding. The team leader and the senior research staff have wide experience in management of both
international and national research projects.
2.4. Have the research team leader and senior research staff been involved in supervising Ph.D. students and
postdoctoral researchers?
Unsatisfactory. The research team leader and senior research staff have not been involved in
supervising Ph.D. students and postdoctoral researchers and there is still need for an increase of Ph.D.
and postdoctoral research supervision to meet the challenges of the program.
Satisfactory. The research team leader and senior research staff have been somewhat involved in supervising
Ph.D.students and postdoctoral researchers but there is still need for an increase of Ph.D. and postdoctoral
research supervision to meet the challenges of the program.
Good. The research team leader and senior research staff have been to a certain extent involved in supervising
Ph.D. students and postdoctoral researchers but there is still need for an increase of Ph.D. and postdoctoral
research supervision to meet the challenges of the program.
Very good. The research team leader and senior research staff have been well involved in supervising Ph.D.
students and postdoctoral research.
Outstanding. The research team leader and senior research staff have been very well involved in supervising
Ph.D. students and postdoctoral research.
8
2.5. Is the size and age composition/balance of the research team leader and senior research staff justified and
optimal to achieve the research objectives, the sustainability of the research topic, and the education of a
new generation of researchers?
Unsatisfactory. The size and age composition/balance of the research team leader and senior research staff are
at an inadequate level; the research team is not sustainable. The research team lacks balance between the
senior and junior research staff. There is insufficient senior research staff for the successful running of the
established research plan. The number of junior research staff is limited to guarantee the sustainability of
proposal. The overall size as well as competence of the research staff do not support the achievement of
established objectives.
Satisfactory. The size and age composition/balance of the research team leader and senior research staff are on
satisfactory level, but the research team may not be sustainable.
Good. The size and age composition/balance of the research team leader and senior research staff are justified
and clear in order to achieve the research objectives; the research team is sustainable.
Very good. The size and age composition/balance of the research team leader and senior research staff are
justified and clear in order to achieve the research objectives; the sustainability of the research topic and
education of a new generation of researchers is high.
Outstanding. The size and age composition/balance of the research team leader and senior research staff are
justified and optimal to achieve the research objectives; the sustainability of the research topic and education of a
new generation of researchers is very high.
3. Research infrastructure
3.1. Are the infrastructure and research environment appropriate for the proposed research?
Unsatisfactory. The infrastructure and research environment are inappropriate for the proposed research.
Good. The infrastructure and research environment fully support the needs of the proposed research.
Outstanding. The infrastructure and research environment fully support the needs of the proposed research and
are related to Estonian or international research infrastructure roadmaps.
4. Institutional and national relevance
4.1. Is the proposed topic among the strategic topics of the R&D institution and based on its strategic objectives?
Unsatisfactory. The research topic is not among the strategic topics of the R&D institution or based on its
strategic objectives.
Good. The research topic is among the strategic topics of the R&D institution and based on its strategic
objectives.
Outstanding. The proposed topic is definitely among the strategic topics of the R&D institution and based on its
strategic objectives.
4.2. Does the research topic have a crucial role in maintaining specific scientific competencies in Estonia?
Unsatisfactory. The research topic has no role in maintaining the level of specific scientific knowledge in Estonia.
Good. The research topic has a certain role in maintaining the level of specific scientific knowledge in Estonia.
Outstanding. The research topic has a crucial role in maintaining the level of specific scientific knowledge in
Estonia or is unique and strategically important.
4.3. Are there similar/close research topics in the other Estonian R&D institutions? If yes, what makes the
proposed research topic unique and strategically important among others?
9
Unsatisfactory. There are a number of similar/close research topics in other Estonian R&D institutions.
Good. There are similar/close research topics in other Estonian R&D institutions.
Outstanding. There are no similar/close research topics in other Estonian R&D institutions; the topic is unique.
4.4. Describe the previous institutional commitment into building up expertize in the research topic, e.g., how
many new postdocs, junior and senior researchers have the applicant R&D institution employed during last
10 years to work on the research topic? Which resources has the applicant R&D institution allocated to the
research topic?
Unsatisfactory. There are no previous institutional commitments into building up expertize in the research topic;
the number of new postdocs, junior and senior researchers that the applicant R&D institution has employed
during last 10 years does not guarantee the sustainability of the research topic. The applicant R&D institution has
allocated no resources to the research topic.
Good. There are some previous institutional commitments into building up expertize in the research topic, the
number of new postdocs, junior and senior researchers that the applicant R&D institution has employed during
last 10 years guarantees the sustainability of the research topic. The applicant R&D institution has allocated
some resources to the research topic.
Outstanding. The previous institutional commitment into building up expertize in the research topic has been
significant; the number of new postdocs, junior and senior researchers that the applicant R&D institution has
employed during last 10 years fully guarantees the sustainability of the research topic. The applicant R&D
institution has allocated considerable resources to the research topic.
4.5. Is the application connected to national strategic plans, research infrastructure roadmap, national research
programs, etc.?
Unsatisfactory. The application is not connected to national strategic plans, the Estonian research infrastructure
roadmap, or national research programs.
Good. The application is connected to national strategic plans, the Estonian research infrastructure roadmap, or
national research programs.
Outstanding. The application is very well connected to national strategic plans, the Estonian research
infrastructure roadmap, or national research programs.
Andres Koppel
juhatuse esimees
SA Eesti Teadusagentuur
iAwarding
of institutional research funding is based on the provisions of the Organization of Research and Development Act
which entered into force in 2012, and regulation number 73 ”Conditions and procedure for applying for, awarding and
amending the amount of institutional research funding” adopted on 27 December 2011 (amended 13 February 2013 and 13
February 2014) (hereinafter regulation) of the Ministry of Education and Research established on the basis of Subsection
152 (4) of the Organization of Research and Development Act.
Regulation § 8 section (5):
The evaluation committee of the Estonian Research Council shall evaluate applications for research funding based on the
opinions of expert panels and considering:
1) an overview of the R&D institution’s role, present tasks and goals as well as its sustainability in the research domains
covered by the present application (e.g., for carrying out Ph.D. and Master’s education in the case of universities, or in the
case of state research institutes for carrying out their statutory goals
2) The association of the proposed research topic to national strategic development plans and its importance in carrying out
these plans
3) The general theoretical background of the planned research topic and its place in research, main objectives, hypotheses,
methods, and work plan;
10
4) If applicable, an explanation of how ethical requirements have been met in the case of animal and human studies;
5) The expected results and possible future developments of the proposed research topic, its importance for science,
society, and culture;
6) The research team leader and senior research staff, their track record, the number of research positions necessary for
carrying out the research and their structure;
7) The existence and state-of-art of the infrastructure necessary for carrying out the research topic;
8) The justified budget of the research topic;
9) The justification for accommodating into the research topic of other monetary resources by the R&D institution which
become available due to the merger, expiry, or termination of other targeted research themes or institutional research
funding topics.
11