From Exams to Schoolbased Assessment:
Opportunities and
Challenges
Gordon Stanley, PhD
Honorary Professor of Education
The University of Sydney
Emeritus Professor of Psychology
The University of Melbourne
Global reform trend
 How we collect data about student learning and
achievement is at the centre of education reform.
 Are our practices fit for the national purpose of human
development?
 Traditional assessment focused on sorting function.
 Over-reliance on external system wide tests and
exams.
Traditional school-based practice
 Teachers give quiz or test to chart progress of students.
 Teachers mark and grade performance using implicit
standards based on experience.
 Often marks given based on relative performance
rather than with alignment to common standards.
Standards-based assessment
 Reporting required to be based on credible evidence
about developmental progress and achievement.
 Reported with reference to a series of standards or
verbal descriptions of what students ‘know and can do.’
 Not sufficient to say: ‘You are an ‘A’ student”.
Modern History ‘A’
 Comprehensively analyses the key features of
specific periods of twentieth century history and
evaluates the role of key individuals, groups,
events and ideas.
 Evaluates the relative significance of factors
contributing to change and continuity in the
modern
world.
Displays
a
sophisticated
understanding of historical terms and concepts.
 Utilises a variety of relevant historical sources and
evaluates their reliability. Assesses different
historical interpretations and perspectives.
 Communicates high level argument through wellstructured and detailed texts.
Giving meaning to the standard
 The verbal description helps to create an ‘image’ of
student capability.
 A clearer image emerges when examples of student
work typical of a student who has achieved an ‘A’ grade
are available.
 A standards-based curriculum and reporting along a
clear developmental (growth) pathway allows new
thinking about how best to collect evidence.
 The assessment for learning movement has highlighted
the way in which assessment and pedagogy are
interconnected in classroom activities.
 Recognition of the opportunities in school-based
assessment also raises challenges about how it is to
be managed
The process of assessment
 Assessment involves professional judgment based
upon an image formed by the collection of information
about student performance.
 This definition draws attention to two important facts.
Fact one
 All assessment involves professional judgment i.e.
judgment based on deep understanding of the
curriculum and reporting standards and the images of
student performance associated with progress along
the developmental pathway described by these
standards.
 External exams and so-called ‘objective tests’ require a
considerable amount of professional judgment in their
design, construction and interpretation.
Fact two
 Assessment is based on collection of information, which can
range from unstructured classroom observations and short
quizzes to standardized tests and exams.
 This information provides the judge, be it the teacher or
examiner, with an on-balance picture (the ‘image’) of where
the student is with respect to the developmental pathway.
 Importantly all this information should be scrutinized for any
inconsistency. All sources of information provide a window
on development of the individual.
Two questions
 How can we ensure that professional judgment with
respect to standards is consistent across an education
system?
 How much information do we need to collect?
No easy answers
 Assessment cannot be a precise activity. What we must
ensure is that our processes are as reliable and fair as
we can make them given the constraints of time and
effort available.
 We need to avoid being over-zealous in the collection
of information.
 Information needs to be robust enough for the task at
hand.
Design requirement
 A major design requirement for quality assessment,
whether involving external tests or school-based
assessment, is the need to ensure alignment to
curriculum and syllabus goals and expected outcomes.
 Alignment helps define the standards.
 We need assessment to report developmental progress
towards and/or attainment of the curriculum outcomes
External exams
Role of external exams
 External exams are common in Commonwealth and
European countries, especially at the point of
graduation from school and for entry to college and
university.
 They are used for certifying student achievement and
provide the basis for competitive selection.
 Students and their teachers, as well as tutors and
coaching colleges, all take an intense interest in the
process and outcomes
Traditional external exam scene
Strength of external exams
 Developed by exam authorities who have subject
specialists produce the papers and ensure their
security until students sit the same exam at the same
time and under common conditions of supervision.
 As all students respond to a common set of questions
at the same time it is possible for a direct comparison
of results.
 Student responses are marked by trained markers in a
process independent from the school in which the
student has been taught.
 Commonly exam authorities are subject to quality
audits and generally have high public confidence.
Weaknesses of external exams
 The time available as well as the length of external
exams is limited by cost factors.
 Samples of domain knowledge and skills need to be
made, thereby limiting coverage of the curriculum in the
exam.
 Additionally there are often significant parts of the
curriculum, which cannot be assessed by traditional
exams
 Pressure is placed on examination authorities to avoid
‘surprise’ questions based on fine detail and to ensure
that questions are ‘accessible’ to all students.
 As a result papers are often fairly predictable. With high
stakes being associated with public exams teachers
and tutors put a lot of effort into preparing students for
particular types of exam questions.
 ‘Though fairly reliable tests of narrow textbook content,
Indian school board exams are rarely valid tests of
desired competencies and broader curricular
objectives, even within the cognitive domain.’
(From National Council of Educational Research and Training (2006)
Position Paper: National Focus Group on Examination Reforms, p 7.)
 Similar comments can be made about exams in a
number of Commonwealth countries.
Weakness of only external exams.
 High stakes leads to tutoring or coaching industry.
 Focus on ‘teaching to the test’.
 One-shot approach can lead to considerable stress.
 Sometimes performance can be affected by chance
factors associated
preparation.
with
lucky
choice
of
exam
 Thus the external exam may be measuring a personal
best due to last minute coaching and lucky breaks
rather than representing a stable achievement.
 In other cases test anxiety or tiredness may make the
student performance less than expected.
Stress reducers
Cheating
 The high stakes with external exams have often lead to
students seeking ways to get an advantage by having
unauthorised material available to them.
 Most authorities have had to develop procedures to
minimise such behaviour but there are always new
attempts being made.
Need for reliable information
 If we wish to be confident about the assessment of
knowledge and skills attained we need to have reliable
information.
 More information available should increase the
reliability of our assessment as well as the validity of
our judgment.
 Hence assessment can be enhanced considerably
through incorporating school-based assessment into
the process.
School-based assessment
 A renewed interest in school-based assessment
recognises its everyday
teaching/learning process.
role
as
part
of
the
 Teachers use a variety of ways to get an understanding
of their students and to provide feedback and guidance
to improve learning.
 As a critical source of information, school-based
assessment provides many opportunities, but also
poses many challenges to education systems as they
manage change.
 Today in most public policy goals for modern education
there is concern about evidence of depth of
understanding, and evidence of 21st century and
employability skills.
 The traditional external exams using pencil and paper
tests relying on essays and multiple-choice questions
are thought to be inadequate by themselves for these
current demands.
 Laboratory skills can be assessed effectively by
observing students conducting practical work in the
laboratory.
 Research skills may be best assessed by having the
student carry out a project and defending the outcome
in the classroom.
 Working collaboratively is best assessed through
observation of the students at work in groups.
Observing, questioning and interviewing are not typically
incorporated in current external exams, but are part of
everyday classroom activity.
School-based assessment in high
stakes exams
 Many countries have responded to the criticisms
levelled at the traditional one-shot external exams by
incorporating school-based assessment into their high
stakes exam systems.
 Some have given equal weighting to school-based and
external exam results in determining the level of
achievement.
 Other systems have not found the journey of
implementation
completely
successful
and
consequently have given a lower weighting to such
assessment.
 It cannot be assumed that school-based assessment
necessarily avoids problems associated with external
testing without some assurance of consistent
professional standards for effective assessment by
classroom teachers.
 Teachers may teach a narrow curriculum by selecting
their favourite bits for emphasis and may employ
classroom assessments, which create similar
pressures to external exams.
Problems with high stakes
assessment
 It may be in the nature of national and system level
assessment that such corruption of the intent of
curriculum delivery occurs, especially when student
results are used for performance management of
teachers and schools.
 To
counteract such trends requires a strong
professional ethic and system level investment in
developing and maintaining high quality teaching and
assessment regimes.
Need for teacher assessment skills
 School systems need to support processes to develop
assessment competence in teachers, a task that is
often not maintained effectively beyond an initial
implementation cycle.
 The professional status of teachers is enhanced when
assessment occurs at the level of the classroom.
 Teacher involvement in assessment moderation and
standard setting are invaluable in helping them to
assign performance levels correctly according to
national standards.
Requirements for school-based
assessment
 For effective school-based assessment in an education
system there are a number of issues that need to be
addressed, ranging from workload to ensuring
consistent and quality engagement by teachers and
students.
 Comparing education systems in different countries
one can see common challenges emerging.
 There is agreement that professional development and
moderation of practice are vital to gain acceptance and
fairness.
Manageability and evidence records
 Tensions often arise between the activities performed
by teachers in embedding assessment into their daily
teaching and the requirements for quality assessment.
 Different classroom teaching and learning situations
provide varying opportunities to observe and record
information to inform judgments about student
achievement.
 The amount of evidence collected and how it is
recorded is a significant issue.
 Practices differ in the extent of data collection and
recording ranging from detailed assessment protocols
to ‘on-balance’ judgments of attainment of assessment
criteria.
 Tradeoffs need to be made with respect to the degree
of task standardization, the number of tasks on which
assessment is based, the degree of recording and the
storage of information to justify the teacher judgment
and the impact on teachers’ workloads.
 Since a prime focus is to provide an opportunity to
make judgments about student progress and to guide
the next step in learning it is important that the
assessment process is manageable in terms of
workload and provides timely feedback
Effectiveness
 The effectiveness of school-based assessment models
needs to be evaluated in the policy context in which
they are operating so that effectiveness can be
measured against the purpose/s required by the
education system.
 Much of the literature on the reliability or validity of
assessment by teachers has been based on relatively
small scale studies embedded in contexts which may
not fit well into a system wide reporting and
accountability framework.
 Where teacher training in assessment has been
strongly embedded school-based assessments have
found to be more reliable than when they are not.
 It is reasonable to expect student’s levels of skill to
develop over time as a function of instruction and
practice and the rate of such development may vary
significantly across individuals.
 Usually exam authorities advise schools to weight
school assessment components closer to the
summative external exam higher than earlier assessed
components.
Alignment to standards
 MacCann and Stanley (2010) compared school
assessment grades in Yr 10 science with grades on an
external test for science.
 The school assessment programme was reported with
respect to performance descriptors for five levels as
was the external test.
 The test performance descriptors are similar to those
for school reporting though somewhat more narrowly
focused on scientific literacy.
 The alignment of scores on school assessment to
grade descriptors was made at school level and
submitted to the examination board whereas the
external test score alignment to grades was made
centrally by a small team of experienced judges.
 With an annual candidature of around 80,000 one
might expect reasonably stable results.
% at Grade ‘A’
% at Grade ‘C’
 Natural tendency for the classroom teacher to see their
students in the best possible light, despite the training
received in aligning portfolios to the system-wide
standards.
 Clearly there is a need for external moderation of some
sort, depending on the degree to which the
assessments based on classroom teacher judgments
are high stakes.
Moderation of school-based
assessment
 To ensure comparable outcomes across schools with
school-based assessment a moderation process is
usually invoked.
 The accuracy required of a given moderation process
bears a practical relationship to the degree to which the
measures are high stakes.
 Programmes which determine tertiary entrance are
based most commonly on both public examinations
and moderated school assessments.
 Having external examinations reassures the public
concern about fair comparison across schools.
 The combination of external plus moderated internal
results improves both the reliability and validity of the
scores or grades.
 Moderation of the internal results involves making sure
that the school assessments are consistent with
system-wide reporting in relation to standards.
 Moderation can involve a statistical process or a social
(consensus) process.
Statistical moderation
 In statistical moderation the criterion for moderation is
either an external test or public exam that is attempted
by the students in addition to being assessed within the
school.
 When the distribution of criterion marks obtained by the
school group on the external exam has a similar
distribution shape to that of the school assessments, it
is common to use a linear conversion line.
 The main advantage of statistical moderation is its
transparency and reproducibility.
Social (consensus) moderation
 Social moderation involves a number of procedures for
aligning school assessments based on professional
judgment.
 School assessments are compared with work samples
and compared across different schools by referring
them to a standards-referenced scale.
 Work samples are commonly in portfolios of student
work that are assigned to a certain performance level
by the school.
 These are then referred to external moderators who
determine whether the judgments are accurate or too
lenient or too harsh.
Comments on social moderation
 For high stakes competitive assessments most
systems rely on external testing to moderate or monitor
the assessments rather than social moderation.
 Using external exams to moderate has public
confidence because of external marking and the
identity of the students not being known to the markers.
 It is cost effective because it does not require extensive
deployment of moderators and can be accomplished in
a shorter time-frame
Conclusion
 School-based assessment to be respected in a
standards world has to demonstrate that it is based on
evidence that is consistent with appropriate alignment
to the curriculum standards expected by end-users.
 School-based assessment ensures that evidence about
student performance is well-grounded and supported
by the teaching and learning programme at school
level.
 For quality assessment there needs to be real
investment in developing assessment competency and
ensuring common standards are being implemented.
 In India the Continuous and Comprehensive Evaluation
(CCE) programme implemented by CBSE provides an
excellent example of a system level approach to
implementing school-based reform.
 The programme of assessment training developed by
the Centre for Assessment, Evaluation, and Research
will provide support for schools and teachers to build
confidence that their assessment will assist student
learning and progression towards each student
achieving their potential.
Thank you for listening
Email: gordon.stanley@sydney.edu.au