Quality Assurance in the School Based
Assessment Component of a Public
Examination
by
Stafford A. Griffith (Professor)
Institute of Education
University of the West Indies, Mona Campus
Introduction and Overview
SBA in public examinations is an issue of continuing concern.
Despite this, the secondary and post-secondary examination systems of
many countries are expanding the use of SBA.
Queensland in Australia, for example, abolished external examinations more
than three decades ago and replaced them with a system of SBA (Maxwell
2004, Third ACEAB Conference).
The UK Working Group on 14-19 Reform, suggested that for three of the four
levels of certification proposed, “teacher-led assessment should be the
predominant mode of assessment” while for the fourth “assessment should
remain a balance between external examinations and in-course
assessment” (Working Group on 14-19 Reform, 2004).
Nevertheless, the quality control issues that are often associated with the
use of SBA cannot be ignored.
A number of boards, including the Caribbean Examinations Council (CXC),
have put in place certain quality assurance measures to address these
issues.
This paper examines the benefits of SBA in a public examination, with
specific reference to CXC, and the quality assurance measures taken to
respond to some of the critical concerns of stakeholders.
It gives special attention to limitations that the teacher variable imposes on
quality assurance and suggests ways of addressing these.
Benefits of SBA
SBA is central to CXC examinations, both CSEC and CAPE.
The SBA component does not mirror the external assessment measures.
It seeks to explore the unique opportunity provided by this component to
assess competencies that cannot be adequately or easily assessed using
MC or essay tests.
While SBA in CXC examinations may take a variety of forms, most tasks
are in the form of projects.
Projects require students to produce something and often include
preparation of reports or the making of a presentation.
The advantages of SBA are, perhaps, well crystallised in its capacity to
facilitate and promote authentic assessment.
Authentic assessment focuses on the application of knowledge to real-life
situations.
It requires students to apply what they know and to demonstrate the
competencies needed for success in the real world.
The SBA component for many CXC subjects requires authentic assessment
tasks.
CSEC History syllabus provides an option that requires students to
undertake a History research project where they behave like a historian
identify a suitable area of research,
collect historical data from a number of sources, and
prepare a report using writing conventions of historians.
SBA also facilitates the individualizing of a part of the curriculum.
The teacher assesses the achievement of each student on tasks that are
specially selected or defined to respond to the interest and the environment
of the student.
The extent to which the opportunity is provided, varies among subjects and
public examination boards.
CXC, in most of its subjects, provides for SBA to contribute 20 percent of
the overall marks for the examination, thus providing the opportunity to
individualise approximately 20 percent of the curriculum.
Exceptions to this proportion are made for subjects that are more practical
or performance oriented, where the percentage for SBA may account for 30
to 60 percent, depending on the nature of the subject.
The Issue of Fairness
The limitations in SBA are generally related to issues of fairness that may
affect the validity and reliability of the scores awarded.
Fairness involves a lack of bias, equitable treatment in the testing
process, and equal scores for students who have equal standing on the
tested constructs (AERA, APA, NCME, 1999).
Fairness is a central aspect of public examinations and any inherent
unfairness in any aspect would be incongruous with the concept.
While public examinations have gained acceptance, generally, as valid,
reliable and fair measures of student achievement, there is still much
concern about the fairness or unfairness of the SBA component.
Yip and Cheung (2005) in a HKEAA study found that teachers were
concerned about fairness.
They were of the view that SBA “could not provide a valid and reliable
assessment of student work because of the great variation among
different schools in the conditions under which assessment is made”
The variations cited by teachers included:
 the degree of guidance or assistance given to students,
 the types of practical work being assessed and
 the marking standards of teachers.
CXC, like several other public examination boards, has instituted a
number of quality assurance measures to:
 reduce these potential sources of invalidity and unreliability and
 counter possible unfairness.
However, much more needs to be done to treat with these issues.
Quality Assurance Measures
Quality assurance in SBA may be conceptualized as all the quality control
mechanisms put in place by an examining board that provide confidence that
the grades awarded for the SBA are in keeping with the required standards.
Quality assurance measures are important to treat with issues of fairness
that can affect the reliability and validity of the marks or grades awarded to
students for the SBA component of a public examination.
Like many other public examination boards, CXC uses moderation as the
primary tool of quality control.
Moderation may be described as “a process of aligning the standards of
different examinations or different components of an examination,
conducted in different places at different times and marked by different
examiners” (Broomes, 1997).
The moderation procedures used by CXC involves procedures prior to,
during, and after the assessment of students.
Prior to initiating the SBA tasks, teachers and students are provided with
guidance about what is expected.
This guidance is contained in the subject syllabus and represents an initial
effort to achieve a common understanding of what is required.
Further guidance is provided through mark schemes or scoring rubrics for
the award of marks for SBA tasks.
The CXC syllabus guidance and scoring rubrics are developed by Panels of
subject specialists.
The use of pre-defined mark schemes or scoring rubrics,
provides a more objective approach, leading to increased scorerreliability (Moskal, 2002).
In addition to guidance in the subject syllabus and the scoring
rubrics, CXC undertakes SBA workshops that seek to standardize
or calibrate teachers who use the rubrics.
Workshops are conducted, periodically, using the expertise of both
contracted subject matter specialists and CXC staff.
Moderation measures are also instituted during SBA activities.
Teachers are expected to provide guidance to students in keeping with
guidelines provided in syllabuses and at workshops.
Further guidance is provided through a SBA Manual for Principals.
However, the teacher is permitted considerable independence of action in
working with students to undertake the required tasks and
assigning scores for the completed tasks.
The degree of independence has caused concerns about
the validity of the assignments undertaken by some students and
the reliability of the scores awarded by some teachers.
This is an area that requires greater attention for improved quality control.
CXC may benefit from the more detailed guidance given by boards such as
the IEB for moderation of SBA by teachers both within and between schools.
The moderation guidance addresses
measures for standardization,
fairness and equity,
validity of tasks for the subject,
accuracy and
standards of marking.
The IEB provides extensive guidance on how various issues should be
addressed, including the frequency, timing, venue and dates for meetings
within schools and between schools.
The adoption of similar measure by CXC will contribute, significantly, to the
improvement of procedures for quality assurance.
The Council should also consider the recommendations made by the UK
Working Group on 14-19 Reform for:
“robust internal quality assurance systems including a potentially pivotal
role for Chartered Assessors in institutions, responsible for co-ordinating
assessment across the institution, and ensuring that standards are applied
consistently and codes of fair practice are adhered to”.
Both the recommendation of the UK Working Group and the practice of the
IEB emphasize cross validation among teachers and schools during the
conduct of SBA as an important measure for quality assurance.
The adoption of such procedures will assist CXC with a more comprehensive
response to the quality assurance challenges in SBA.
Apart from adopting quality control measures, through moderation, prior to,
and during SBA, CXC undertakes a review of the scores or grades, and
where necessary, makes adjustments.
Broomes (1997) gives two important reasons for this review.
First, despite the fact that teachers may be scrupulously honest in making
assessments of the work of their students, “the ‘halo’ phenomenon may cause a
teacher to assess a student too generously”.
Secondly, “as in any profession, there are a few teachers whose standards are not
of the highest [quality]” .
The review of scores or grades awarded by teachers assists in detecting
any unfairness or malpractice that may have taken place.
CXC undertakes the review of teachers’ scores by using either of two
procedures.
For some subjects, particularly those with a practical component, the moderator
visits the school, participates in the assessment of some or all students and makes
adjustments, where necessary, to the scores awarded by teachers.
For other subjects, assignments of five students, selected as requested by CXC, are
remarked by CXC moderators.
The scores of the moderators are compared with those of the class teacher
and a relationship between the two sets of scores statistically established
and used to generate moderated scores for all students in the class.
This form of moderation, or variants of it, is widely used by several other
public examination boards.
Quality Control and the Teacher
Much of the concerns about fairness in SBA relate to the role of the
teacher in the assessment process.
The support for teachers’ involvement in the assessment of their
students is widespread, particularly because of the value in improving
instruction and, ultimately, student learning.
However, differential degrees of guidance or assistance given to students
in SBA is a source of continuing uneasiness among teachers, parents and
other stakeholders.
Much of the uneasiness is caused by misunderstandings about the dual
role of the teacher as instructor and assessor.
The misinformed teacher or stakeholder may incorrectly regard
teacher input into SBA as unfair.
But teacher input in guiding students on a continuous basis is
necessary and legitimate.
This input is part of formative assessment that should be
distinguished from summative assessment, which is also involved in
SBA.
In summative assessment, the aim is to report how far the student
has reached in the learning journey.
This distinction between formative assessment and summative
assessment is often unclear, leading to:
different practices among different teachers and schools and
different perceptions among different stakeholders about what is fair and
acceptable.
Differential inputs from teachers that may unfairly reward or penalize
students are often the result of differential understanding of their role.
At one end of the continuum are teachers who may treat the
assignment purely as an external examination while at the other are
teachers who end up doing much of the assignment for the student.
Moderation of the marks awarded by teachers is not an appropriate
quality assurance measure for this type of improper practice among
teachers.
Moderation at that stage is concerned with the work
submitted/completed.
Once the marks awarded by the teacher for the assignment are in
keeping with the CXC standards
no adjustment would be required and
the Council is not likely, without cause, to investigate the classroom practice
that led to the quality of an assignment in the sample/assignments
moderated.
Under the circumstances, the Council depends on the teacher to
follow fair and equitable practice.
Measures such as those that have been instituted by the IEB and
recommended by the UK Working Group on 14-19 Reform, for
quality control during the process of SBA, are helpful in alleviating
the differential practices within and between schools.
However, more effective training of teachers is required to ensure
that they understand and can perform the dual role expected of
them.
The training of teachers to understand and perform the dual role
as instructor and assessor in SBA should not be left to the
workshops conducted by the examination board.
It should form an integral part of the pre-service training provided
by teacher training institutions.
Although stakeholder perception about the legitimacy of teachers’
guidance and assistance to students in SBA is not, in itself, a quality
assurance measure, it is an important consideration for:
treating with any incorrect perception about the role of the teacher and
obtaining support for the quality control measures in relation to teacher
involvement.
As a complement to teacher training, therefore, more effective
public education about the dual role of the teacher is necessary.
In particular, such public education should seek to develop, among
all stakeholders, a proper understanding of the importance of
formative assessment in SBA.
Summary and Conclusions
SBA is likely to remain a part of public examinations in the foreseeable
future.
It has significant educational benefits in developing and assessing
important educational skills and in individualizing a part of the subject
curriculum.
However, concerns of fairness need to be addressed.
Examination boards such as CXC have instituted a number of quality
assurance measures to treat with this issue.
In treating with quality assurance issues in SBA, the variable of teacher
input requires the greatest attention.
The quality control measures undertaken by CXC may be augmented by
adopting measures such as those
practiced by the IEB and
recommended by the UK Working Group on 14-19 Reform
that emphasize cross validation among teachers and schools during the conduct
of SBA.
However, teachers need to be trained more extensively to understand
and to practice, more effectively, their dual role as instructor and
assessor in SBA.
Such training should not be left merely to workshops but should be an
integral part of the teacher preparation undertaken by teacher training
institutions.
THANK YOU