Introduction
1. The design of Assessment Instruments must be based on the following
assessment principles: validity, reliability, impact, practicability and
fairness
2. Validity is the extent to which test is assessing what it intended to assess.
3. Validity is the mainstay in the modern conception of assessment and
evidence of validity should be visible in all processes, setting of papers,
marking process and grading of learners performance and resulting
4. Reliability is the ability of the test to yield consistent results and consistent
demonstration of a particular skill by learners confirms the validity of the
skill possess by learners.
Focus for 2013 and Beyond
a) Finalise Policy on the Quality Assurance of SBA.
b) Systems Audit focussing on support of SBA implementation.
c) Moderation of assessment tasks
d) Moderation of learner evidence in selected districts, across PEDs.
e) Development of Assessment exemplars.
f) Moderation of common tests/exams set for the Preparatory
examinations.
g) Common Assessment Task for Life orientation.
Quality Assurance Approach
Systems Audit
a) The system audit was conducted in which the team of moderators
audited the provincial and district system to ascertain its potential to
support school based assessment
b) DBE moderators met the curriculum planners and subject advisors of the
sampled districts.
c) The audit focussed on all policy documents, reports, guidelines,
curriculum materials, instruments of moderation and quality assurance of
assessment tasks
d) The audit was followed by the moderation of learner evidence and
assessment tasks
e) 20 schools were sampled from two districts
Quality Assurance Approach
Methodology
a) A stratified purposive sampling technique was followed and schools
performing within the range of underperforming (0% – 60%) and good
performing (80% - 100%) were sampled
b) Two instruments were used:
Systems Audit instrument:
-
Location of the SBA,
the relationship between the exams and curriculum,
training programme for subject advisors and teachers,
feedback,
common tests
Moderation Instrument
- Content coverage,
-
cognitive levels,
language and technical aspects
Quality Assurance Approach
c)
The following subjects were sampled namely: Accounting,
Mathematics, Physical Sciences, Life Sciences, Geography, English
First Additional Languages, History and Life Orientation.
d) 68 Moderators were appointed, rigorously trained and deployed
across provinces.
e) Judgement is based on the set criteria as exemplified by the system audit
document and moderation instrument
f) The judgement is generally based on a group (subject experts)
observation, discussion and judgement based on consensus
g) Findings can therefore be considered to be valid, reliable, dependable
and credible
Moderation Instrument (criteria)
a)
Questions are clear, concise, precise, intelligible, and worked using language
appropriate to the range of candidates for whom the test or Task is intended.
b)
Repetition of questions from previous examinations is avoided.
c)
Appropriate curriculum content which is adequately covered within the test/task.
d)
Appropriate distribution of learning outcomes and assessment standards
e)
Correlation between item difficulty, time allocations and mark allocations
f)
Compliance with content and construct validity
g)
Appropriate distribution of cognitive levels
h)
Language level is in keeping with the vocabulary and comprehension levels of Grade 12
learners
Moderation Instrument (criteria)
i)
Questions or tasks are free from subject error from an academic point of view, e.g.
historic or scientific facts.
j)
Bearing in mind the different national, ethnic and religious backgrounds of candidates,
questions and/or source material will not cause offence due to inappropriate subject
matter and/or Language
k)
Instructions on the assessment tasks and tests were clear and precise.
Quality Assurance Findings
Support systems provided by the district and provinces
1. In some provinces the evidence points to the fact that there is little
interaction between subject advisors and teachers
2. The quality assurance of SBA moderation in some provinces has revealed
that there is a lack of a functional, collaborative partnership between the
directorates of Curriculum and Examinations and Assessment.
3. Some provinces are more advantaged than others regarding the number
of subject advisors
4. In some provinces subject advisors are overloaded with both GET and
FET workload and are not able to provide effective support.
5. There is generally very limited opportunities for the training of teachers
across provinces
6. There is limited training of subject advisors in provinces especially in
relation to assessment issues
Quality Assurance Findings
Compliance
1. There is general compliance to the requirements of SAG and subject
policy guideline documents.
2. The learner evidence and teacher files are mostly neatly organised
Moderation – pre-moderation of learner evidence
a) Provincial moderation where it did take place was meaningful and
constructive.
b) There is evidence of moderation and pre-moderation across provinces
c) Prevalence of inaccurate and inconsistent marking of learner tasks.
d) Poor marking process resulting in the inflation of marks -inaccurate
marking guideline, and misapplication of rubrics
e) Poor moderation of learner evidence which manifests into the so-called
shadow marking
f) Lack of quality pre-moderation or quality pre-moderation of tasks
Quality Assurance Findings
Quality of Assessment Tasks
1. In terms of tests and exams the format, layout and presentation of the
question papers are generally neat in few instances
2. Evidence of post moderation in some form is present in most provinces. In
most cases this reflected school, district and provincial levels of
moderation.
3. The content coverage in most cases is within the scope of the grade 12
syllabus.
4. The quality of assessment tasks and tests generated at school level are of
a poor standard.
Quality Assurance Findings
Quality of Assessment Tasks
5. The technical quality of papers is very poor: cut and paste from previous
papers, no sequencing of the number of questions, no mark allocations,
poor formatting noted
6. Over dependence on past year papers was becoming an unhealthy trend.
This dependence stifles creatively and originality that is expected from
teachers.
7. Investigation tasks in Maths and Physical Sciences was poorly done.
Some provinces deviated from the required norm.
Quality Assurance Findings
Feedback
a) There were no examiners’ reports that accompanied assessment tasks
b) The history of the paper was only provided by examiners that set the
common tests and for tests and tasks set at school level only few schools
managed to submit the history of the paper.
c) The quality of the comments on the papers moderated were not
appropriate and led to the production of poor papers and also lulled
teachers into a false sense of security
d) Inaccurate feedback to learners, due to inconsistent marking, flawed
marking tools and shadow marking
e) Reports emanating from centralised moderation are not accurate and
serve as barriers in the development of teachers
Proposed Intervention - 2014
Focus
(a) Urgent need for the national training of subject advisors on SBA .
(b) Development of Training Programme to be implemented by Subject Advisors.
(c) Development of Assessment Exemplars (Alternative forms of assessment)
Proposed Intervention - 2014
Suggested Approach
1. Three subject advisors per province, per subject to be nominated as SBA champions in
the province.
2. Provincial SBA Champions to be exposed to a two week intense training programme
nationally, coordinated by the DBE.
3. Focus on seven key subjects in 2013 (Mathematics, Physical Science, Life Sciences,
Accounting, History, Geography and Economics) and incrementally phased in to include
other subjects and GET.
4. Training Programme will focus on:
(a)
Design of high quality SBA assessment tasks.
(b)
Develop a Training Manual for use in the training of subject advisors.
(c)
Development of assessment exemplars
Assessment for Learning
1. Assessment for learning needs to be driven by the curriculum and teacher
development sections and there is a need to keep teachers within the
classroom and support them within the school rather outside the school.
2. Authentic assessment for learning considers assessment as a service of
learning, active engagement of learners and learning as a process and not
a product.
3. These central tenets of assessment for learning cannot be achieved
through centralised moderation of assessment tasks and learner evidence
4. Therefore curriculum section needs to be strengthened and need to ensure
that subject advisors are capacitated with the requisite knowledge of their
subject to assist teachers with teaching methodologies, assessment
strategies and the teaching of higher order skills such as critical thinking
and problem solving.
5. Classroom assessment activities that are driving teaching and learning
needs to be the main focus of curriculum and teacher development
6.
Conclusion
The focus on FET teacher training is highlighted in Action Plan 2014 (Goal 16:
Improve the professionalism, teaching skills, subject knowledge and computer
literacy of teachers throughout their entire careers), and is still largely focused
on ensuring compliance with Curriculum Policy and Assessment
requirements.
The systems audit indicates there is rich opportunity for the teacher training
to build the skills of teachers in the design and development of School Based
Assessment Tasks, tests and the overall management of assessment