The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
Assessment in the Service of Learning
Professor Michal Beller
Director General, RAMA
The National Authority for Measurement and Evaluation in Education
March 2008
Introduction
Three of the main goals of the Israeli education system are: a) the fulfillment of each and
every pupil’s intellectual and creative potential; b) the narrowing of achievement gaps in
education, and c) value education. Assuming there is a consensus regarding these goals,
the question is how would we know whether these goals have been achieved?
How would a parent know whether the education system has provided his or her child
with the tools necessary to function in modern society? How would all those involved in
the education process (teachers, principals, the different echelons of the education
system) know whether he or she has executed his or her role satisfactorily? How would
the public know whether the future generation of children has been equipped with the
skills needed to meet the challenges of the twenty first century? How can we reassure the
public that the considerable resources spent on education are being used wisely? And
how can we convince decision-makers that the increasing investments in education are
worthwhile even at the expense of other important areas of needs?
The narrowing of achievement gaps is one of the most crucial goals to be achieved in
order to ensure the resilience of Israeli society. How can the gap be detected, and how
would we know whether it has narrowed? The detection of gaps should be done by topics
(such as violence and the learning environment) and in the different subject areas. It is
also important to detect gaps between sectors, cities, settlements, socioeconomic strata,
etc.
Valid and professionally constructed measurement tools are needed in order to monitor
the achievement of the aims of education. It is known that measurement is a complex
1
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
issue in the business sector and even more so in the public sector and in the field of
education, in particular. The process of learning is unmatched in its complexity due to the
differences between students and the variety of pedagogical approaches for achieving
educational goals. Similarly, these approaches require appropriate types of measurement
and assessment.
Often we tend to look for short, simple, straightforward answers to questions related to
educational measurement and assessment. However, a complex educational process
should learn to benefit also from complex answers that would provide educators, mainly
teachers and principals, with indications of positive and negative directions, and with
suggestions as to what is more appropriate and what is less appropriate for specific
pupils.
There is no uniform “one size fits all” solution that answers all the needs, and there is no
magic solution to the means of measurement. However, the professional, ethical and
public dilemmas that accompany the different measurement instruments do not exempt us
from facing the challenge and providing an adequate answer.
The National Authority for Measurement and Evaluation in Education (RAMA) was
recently founded in order to provide a professional response to the issues of measurement
and evaluation in the education system. RAMA, from the first day of its founding, has
adopted an ideology that is based on two tenets: a) assessment in the service of learning;
and b) constructing an assortment of professional solutions that effectively incorporate
different components of measurement and assessment (about RAMA – in the box).
About the negative phenomena that accompany high stakes testing
High stakes tests, those tests which are perceived as determining the future of the pupil,
the teacher, or the principal, to some extent, are naturally accompanied by negative
reactions. Among these are test anxiety, cheating, attempts to exclude weak pupils,
teachers helping pupils during the test, narrowing of the curriculum and focusing on what
is tested, diverting resources (assigning "good" teachers to the classes and subjects that
are tested, diverting instructional hours from other subjects, preparing for the test), and
abuse of results, etc.
The education system needs to act forcefully in order to minimize these negative
phenomena, first and foremost by a cultural change by which measurement and
assessment are intended to serve the learning process and not inhibit it. However, these
negative phenomena are not, in themselves, a reason to avoid the thoughtful use of tests.
2
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
About centralization versus decentralization of assessments
A constant dilemma facing the education system is setting the balance between local
assessment and centralized assessment, in other words, between decentralization and
centralization of measurement and assessment. The utopian solution is, of course, to
allow maximum freedom to each school to decide on the assessment procedures that best
suit it. According to this approach one could also consider giving this freedom to each
teacher in the school. However, considerations of accountability, transparency,
professionalism, cost benefit, and mainly the capacity to make valid comparisons
necessitates that at least a part of the overall assessment be centralized and allow for valid
comparisons and systemic monitoring over time.
About "ratings" and "comparisons" – Methodology in a nutshell
Two of the more common uses of national test scores are ratings ("league tables") and
comparisons ("we've gone up", "we've gone down", "we've improved", being a few of the
more common expressions). However, these types of ratings, when they are made at the
level of the school in an irresponsible manner, may cause great damage. Not every
comparison is valid and valuable. For example, is there any meaning in comparing the
grades of pupils in Savyon (a prestigious, high SES neighborhood in the center of the
country) with those of pupils in Rahat (a Bedouin village in the far south of the country)?
Does a grade of 90 on an easy test in a given year imply an improvement over a grade of
75 on a difficult test in a previous year?
RAMA is working to enhance the public's understanding that ratings only have meaning,
if at all, among relevant populations, and comparisons over years have value only when
the rating scales are calibrated.
Comparisons across years that are often carried out between scores on the Meitzav (this is
the Hebrew acronym, and in English it is known as GEMS – Growth and Efficiency
Measures of Schools) national tests administered to all fifth and eighth grade pupils, and
those on the high school matriculation exams (Bagrut) are not valid, as the scoring scales
on these tests are not completely calibrated. RAMA is working now to professionally
calibrate the Meitzav scores so that the year 2007 will serve as a base year for meaningful
comparisons of future Meitzav tests. When RAMA becomes responsible for developing
and grading Bagrut exams, it will begin dealing with this subject for these exams also.
3
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
Updated assessment and evaluation structure
The format for national assessment was updated in 2007 (not including the matriculation
exams – Bagrut1). The new format was designed and consolidated by RAMA, in
conjunction with various factors at the Ministry of Education and in consultation with
school principals and many teachers. In the field of education, this format is intended to
provide a suitable, professional answer to all stake holders in the education system:
schools and other external bodies.
The new format is intended to maximize the advantages of the current assessment system
while providing a solution for its disadvantages and is based on the following principles:
 Implementing a culture of "assessment for the benefit of learning" – assessment
that is intended to support continuous improvement of learning, to align with learning
goals and the school vision, and based on the understanding that tests are not a goal in
themselves but rather a tool in the service of learning.
 Meaningful integration between internal and external assessment and between
formative and summative assessment.
 Maximum decentralization of assessment by supplying professional assessment
instruments provided to the schools by RAMA.
 Empowerment of school principals and teachers.
 Reduction of pressure and the amount of external tests.
The new format includes four components (some are already implemented and some are
not): internal school-based assessment, national sample assessment, school-based
assessment with external tools (Meitzav) only a small number of which will be
administered and checked by RAMA ("external Meitzav ") and most will only serve the
school ("internal Meitzav "). The following are details of the components:
School-based assessment
RAMA will help in strengthening the internal assessment that is being carried out in the
schools, by providing banks of tasks and items, "off the shelf" tests, diagnostic tests,
questionnaires and other tools that will be accompanied by in-service training of school
staff.
National sample assessment
National longitudinal trends will be monitored based on a representative, national sample
encompassing a small number of classes. This monitoring will give feedback from
questionnaires on a variety of educational topics as well as tests in different subjects that
will include a wide and in-depth coverage of the curricula. This feedback will serve as an
1
For more on the matriculation exams see: http://www.bibl.u-szeged.hu/oseas/bagrut.html
4
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
indicator for policymakers interested in examining the extent of success of various
instructional methods, changes in the size of educational gaps, etc.
School-based assessment with external tools
The Meitzav is a set of school measures that serve the school principal and his or her staff
as a central tool for the monitoring of curricular planning and for utilization of resources.
The new format of the Meitzav integrates external and internal school assessment.
The development of the Meitzav is based on the understanding that the school is a
complex entity that is comprised of many interrelated parts. It is about an organization
that includes a learning environment, curricula, achievement, and professional
development programs for staff, many relationships (pupils-pupils, teachers-teachers,
teachers-pupils, teachers-parents) and more.
The Meitzav in its new format will go back to focusing on its original task, providing
school feedback, as listed below:
 The Meitzav will be carried out towards the end of the school year and the tests will
cover the material learned up until that particular point (material could be from the
same year and also previous years, specifications for each subject are published on
the RAMA internet site).
 The Meitzav will be given in four subjects – Language Arts (Hebrew/Arabic),
Mathematics, English and Science and Technology – and will also include
questionnaires related to school climate, pedagogical setting, and more (below
referred to as "climate questionnaires").
 The tests will be administered to pupils in fifth and eighth grades. In Language Arts
there will also be a test administered in second grade.
 There will be a calibration of the grading scales of the Meitzav in a way that will
enable valid, future comparisons of the grades across the years. The year 2007 will
serve as the base year.
The External Meitzav –As of 2007, the external tests will be administered to a school
every other year and they will be composed of a pair of subjects (Two out of the four
subjects that are included in the Meitzav). In this way, once every four years a school
will be tested externally in a specific subject. Climate questionnaires will be administered
on each administration of the external Meitzav. As a consequence, each school will
receive a report every other year that will include the achievement scores in the two
subjects as well as a report on the climate and pedagogical setting of the school.
The Internal Meitzav – As of 2007, each school will administer an internal Meitzav in
each of the subjects, in those years in which there is no external Meitzav. Schools will
5
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
not be required to report the results to any external body and they will remain
internal to the school. The internal Meitzav will be based on the integration of three
components: a) an objective, external, national test having psychometric properties of
reliability and validity developed by RAMA in conjunction with a professional
pedagogical committee and with the chief inspectors who ensure that the tests are in
alignment with the curricula and the standards of knowledge and understanding that
pupils are expected to reach in each subject; b) internal grading by the school staff (with
the assistance of a rubric attached to the test) which provides the staff with timely
feedback about the mastery of students (or groups of students) in each subject; and c)
comparison of the students' achievement in the school with external norms (national,
regional, district and sector) that are calculated from the external Meitzav tests
administered at the beginning of the following year.
International Assessments
RAMA is also responsible for the international tests, in which many countries participate.
The studies based on these international tests allow for a comparison of the achievement
of pupils from different countries in a number of subjects, and with regard to other
educational issues. In addition, the results of the studies allow for a comparison between
sectors and different subgroups of the population within each participating country. The
international organizations that develop the tests are among the leaders in the field of
assessment and measurement. The tests are translated into different languages and
because they are professionally constructed they have a high level of reliability and
validity. The tests and questionnaires are administered to a representative sample of the
population in each country and the scores are reported only at the national level and not at
the level of the class or school. Over the last few years Israel has participated in a series
of international tests including TIMSS, PISA, PIRLS and SITES.
The results of the studies provide true and reliable information about the education
system in Israel from an international perspective. This information may have great
importance for policymakers who want to focus on the strong and weak points of the
education system in Israel with an objective comparison to other education systems
around the world. Furthermore, the participation in international studies allows Israel to
learn about new and innovative approaches in the subjects studied and to examine its own
curricula in light of those implemented in different countries around the world. Thus, for
example, the need for strengthening literacy studies in Science, Mathematics and
Language Arts was identified. The studies also provide information about successful
models in education from different countries. This can be achieved by comparing
countries with high achievement to countries with low achievement levels, and by
examining the relationship between student achievement and various background
6
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
variables (such as education of parents, socio-economic status, student attitudes, etc.).
Thus, for example, the world was able to acknowledge the successful education system in
Finland where pupils reached the highest level of achievement in the PISA study mainly
due to the cultural and social norms there and due to a reform that empowered schools.
The media tends to emphasize the ranking of each country in these studies, and to
compare these rankings to previous studies by presenting the ranking of the countries as
the main findings of these international tests. However, the main value of these studies
lies in the possibility of each country to compare and check its own achievement of
the goals that it set for itself. For example, in Israel, the results of the study allow a
comparison between the achievement of pupils who are speakers of Arabic and the
achievement of their Hebrew speaking peers and between other sub-groups (for example,
boys and girls). In particular, these studies allow us to monitor achievement in Israel
across time. This is possible thanks to the cyclical nature of these studies which are
carried out in a similar format once every three to five years, maintaining calibration of
the scoring scale from one study to the next. More details regarding each of the
international tests are available on the tests’ websites of the tests and on the RAMA
website.
The "bottom line"
RAMA would like to pass on the message that there is no simple "bottom line" in the
field of measurement and assessment; there is not and will never be any "magic number"
or single "general conclusion" that can sum up complex educational processes, neither on
the level of the individual nor on the system level. RAMA sees its main mission as
helping the education system improve through the thoughtful use of assessment and
measurement. The concept of "assessment in the service of learning" is not just a motto –
RAMA sees it as the raison d'etre.
7
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
The National Authority for Measurement and Evaluation in Education –
RAMA
The National Authority for Measurement and Evaluation in Education (RAMA) is a leading,
professional organization of the education system, specializing in educational measurement and
assessment. The authority acts as an independent, objective, professional body serving all stakeholders
within the education system and without. In order to assure that RAMA can fulfill its mission it was
founded (in 2005) as an independent, governmental authority, reporting directly to the Minister of
Education, and its activities are anchored in the RAMA article of law.
The vision
RAMA believes that effective processes of measurement and assessment can help in the improvement
of education: in identifying gaps between what is and what should be, in aspiring for continuous
improvement, in promoting excellence, in attending to students with special needs, and in supporting
weak populations.
Measurement and evaluation are a means and not an end, and they are supposed to be part of a
ongoing teaching and learning process. RAMA will act to imbed the principle of "assessment in the
service of learning": emphasizing the alignment and the feedback that are required between content,
standards, instructional methods and assessment.
"Assessment in the service of learning" is a comprehensive approach that supports the integration of
internal assessment of schools with external assessment, and also the integration of summative
assessment of pupil achievement with formative assessment that supports and promotes instruction.
RAMA is a part of the education system, and as such attaches importance to the constant and direct
contact with principals, teachers and parents. RAMA is in daily interaction with the headquarters of
the Ministry of Education and with the districts and municipalities, in order to be attentive to their
needs and be involved in processes, and in order to contribute to the implementation of the findings.
RAMA acts to strengthen itself as a reliable, relevant, professional and objective body. It will carry
out and report the assessment and evaluation results fairly and precisely, without fear or bias.
The mission
RAMA's mission is to contribute to the improvement of the education system through:
 setting up an effective assessment system that is an inseparable part of the learning process;
 developing a variety of assessment instruments for the continuous monitoring of the
8
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
development of the education system and for assessing the progress of educational
programs;
 professional training of all those involved in assessment and measurement in the education
system;
 providing findings that are supported by data and tools for scientific analysis to
policymakers in the education system, to researchers and interested parties in academia, and
to the general public;
 helping shape national educational policy that relies on findings and facts.
RAMA's activities
The following topics will be included in stages within the responsibilities of RAMA, and will be dealt
with in conjunction with the various departments within the Ministry of Education (see table below).
9
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
The leading topic
Examples of sub-topics
Large scale assessments,
questionnaires and surveys
Development of tests of literacy and basic skills;
Meitzav tests, five year plan tests for the Arab
sector, tests for the religious sector; international
tests; Bagrut tests; tests for special populations;
tests and questionnaires that will be defined from
time to time.
Construction of task banks; construction of
instruments for formative assessment in the
service of teachers.
Development of methodologies and statistical
instruments for improving the construction of tests
and grading; development of a computerized bank
of diverse assessment instruments.
Publication of annual reports; provision of data
banks and national information to the public and
to the community of researchers.
Planning of training programs on topics dealing
with measurement and assessment for school
assessment coordinators, teachers, principals and
district role holders.
Evaluation of the effectiveness of large-scale
national projects and interventions such as the five
year plan in the Arab sector, language instruction
for new immigrants, remedial education,
educational reforms, etc.
Formative assessments in the
service of learning
Research and development
Reporting to the public and
providing data banks and
information
Training for in-service courses
in the field of measurement
and assessment
Project evaluation and
educational interventions
RAMA sees schools, teachers and principals as the main consumers of its services, and is in constant
contact with the "field" through internet forums and activating a "field team" which provides RAMA
with immediate and continuous feedback about its activities.
Further details appear on the RAMA site: http://rama.education.gov.il
10
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il
The State of Israel
Ministry of Education, Culture and Sports
NATIONAL AUTHORITY FOR EDUCATIONAL MEASUREMENT AND EVALUATION (RAMA)
11
Kiryat Hamemshala, 125 Menachem Begin Blvd, Tel Aviv 67012, ISRAEL
Tel: +972-3-7632880 ▪ Fax: +972-3-7632889 ▪ e-mail: rama@education.gov.il