Key Stages 2/3 Cluster Group External Moderation, Pilot 2010
English Chief Moderator Report December 2010
The evidence base:
Following a moderator training day on the 13th of November, 79 learner profiles from
20 clusters were moderated by a team of 10 moderators over the 18th-20th of
November. Four moderator pairs succeeded in moderating 4 clusters over 2 ½-3
days; one pair moderated 3 clusters in 3 days. Moderator feedback from this training
is extremely positive.
CODES
A
A(i)
A(i)* [2
steps]
Ua [disagree
with level]
Ua*
[insufficient
evidence]
N
KS2
7
8
0
KS3
2
9
0
5 profiles from 5 different clusters – 4
at KS2 and one at KS3.
18 [+2]
26 [+2]
1
0
40
40
TOTAL
%
9
17
0
11.25%
21.25%
-
5
6.25%
44+ 4 profiles where
no actual AT best-fit
was indicated to
agree/disagree with =
50
1 [+ No evidence sent
from Monmouth]
80 profiles
60%
1.25%
-
Section 1:
Main messages for school clusters/local authorities
Range of evidence received by moderators against expectations (i.e. based on
WJEC’s guidance):
In the guidance document for Clusters we asked that:
‘Each learner profile should contain evidence for all 3 attainment targets, (Oracy,
Reading and Writing) taken from the Programme of Study for English (Skills and
Range) to demonstrate the level characteristics of Curriculum 2008.
Further guidance:
 Oracy evidence can be provided in audio/audio-visual format or as secondary
evidence through teacher commentary on the task and learner performance.
This evidence should come from normal classroom practice.
 In order to demonstrate performance across the Range, reading and writing
evidence should include literary and non-literary evidence (see Programme
of Study for further guidance).
 Where the same evidence is used to demonstrate performance in more than
one attainment target, e.g. oral assessment of reading evidence used to
support both reading and oracy attainment, commentary should refer to
both ATs.
 It is important for moderation that relevant resources/stimulus materials
and/or frameworks for work in each attainment target are included with the
sample. In relation to reading, copies of texts should be included, e.g. copy of
a poem, leaflet, extract from prose text etc.
Each learner profile must include teacher/cluster commentary:
 To demonstrate the cluster’s understanding of the level descriptions
1

In summary form for each profile to explain the cluster’s best-fit judgement of
lower end of top end of the relevant level.’
Whilst many clusters provided evidence from across the three ATs, 60% of the
profiles moderated included insufficient evidence for moderators to agree with the
cluster’s best-fit judgement. Whilst this was sometimes due to a lack of cluster
commentary and/or stimulus/resource materials, it was usually due to a lack of
evidence for one of more attainment target e.g. only one task for Oracy, Reading
and/or Writing; or insufficient range, particularly literary and non-literary evidence for
Reading and/or Writing. One piece of work for an attainment target evidenced
standardisation and was therefore insufficient to demonstrate moderation. In addition
limited evidence from across the range within an attainment target did not
demonstrate sufficient characteristics from the level description(s) in question to
provide robust evidence of cluster moderation.
In addition there was an over-reliance, particularly at KS3, for clusters to use past
SATs material as evidence for reading. Whilst some had successfully made use of
these to reflect more open tasks using the SAT stimulus material, some were still
evident of test/examination conditions and therefore could not be considered part of
‘normal classroom practice’. Use of more open and creative activities often within the
same profiles were actually a better indicator of learner attainment. This aspect was
reported on as part of the previous KS3 external moderation exercise and should
have been acted on since then. As expected there were few examples of
standardised test results being used as part of the evidence for Reading. Naturally
where these arose cluster commentary struggled to make any relevant links between
the standard score and characteristics of the level in question.
Successful learner profiles demonstrated a range of evidence, usually two or three
tasks for each attainment target with evidence of group and individual work for Oracy
and both literary and non-literary evidence for Reading and Writing. These were
supported by resource/stimulus materials such as copies of texts and or oracy/writing
stimulus. Judgements relating to the work were summarised in concise cluster
commentary which made clear the level characteristics identified by the cluster,
including reference to adjacent levels to demonstrate performance at the lower/top
end of the level as requested. Such profiles were, however, few in number [11.25%].
Although there were other profiles where moderators were also able to agree with the
best-fit judgment [21.25%], these would be stronger cluster profiles if further
evidence such as resource/stimulus materials had been consistently included or if
cluster commentary had made more explicit reference to adjacent levels.
Only one cluster failed to present one of the four learner profiles requested although
a number of clusters had clearly not identified whether their profiles were lower/top
end of the level as requested. The need to present top or lower end profiles was
included in the information meeting presentation and in the guidance. In all cases
these were moderated in order to provide clusters with the feedback needed to build
on their current practice and ensure reliable and consistent learner profiles for
school, department and cluster moderation at KS2 and KS3.
.
Main findings from the pilot in terms of cluster groups’ understanding of
national standards.
It is pleasing to note that whilst moderators were unable to agree with the best-fit
level of a majority of learner profiles due to insufficient evidence, moderators only
disagreed with the best-fit level for 5 out of the 80 profiles [6.25%]. In most cases
this was due to features of a lower level being more evident in the profile than the
cluster had indicated. In addition, there were a few profiles where although
2
moderators agreed with the cluster’s overall best-fit judgement they were unable to
agree that they were top of the level as indicated by the cluster. In these instances
moderators suggested that the profiles would be better suited to demonstrate a
profile securely within the level.
This demonstrates a consistency in the identification and application of level
characteristics to learners’ work at both key stages. The issues linked to
insufficiency of evidence are likely to be linked to clusters still moving from
standardising to moderating. This demonstrates a secure movement forward from
the last KS3 external moderation report, particularly in relation to the accuracy of
judgements at KS3.
Identification of cluster group good practice/areas for development
Moderators noted the following in identifying good practice:
 Detailed commentary related to level descriptions
 Learner profile overview
 Contextual information included
 Resources included
 Good range of evidence covered
 Reference to adjacent levels in cluster commentary
 Interesting range of tasks that reflect NC 2008
 Poetry included as evidence of Writing
 Good range of texts used for Reading tasks
 Common cluster commentary framework used by both key stages
 Summary paragraph/overview at the end of each attainment target which
makes clear the best-fit level for that attainment target
 Clear indication and rationale for the best-fit level for the profile as a whole.
Moderators noted the following in relation to areas for development:
 Wrong levels submitted i.e. Levels 3 and 6
 Focus on evidence from years other than years 6 and 9 (i.e. not end of key
stage)
 Commentary didn’t reflect cluster understanding of level description
 Cover sheet not included
 Lack of non-literary evidence (Reading and/or Writing)
 Over emphasis on literary tasks, especially at KS3
 Use of ‘tests’ e.g. SATs papers (under timed conditions)
 Lack of stimulus materials for Reading to assist moderation
 Tasks submitted too similar, not reflecting range
 Lack of evidence resulting in moderators being unable to agree with cluster
judgment
 Quality of evidence making photocopies difficult to read / audio evidence
inaudible
 Difficult to assess over-scaffolded tasks
 Group writing tasks difficult to use to assess individual contribution
 Tasks included not referenced to a specific attainment target
 Lack of clear commentary referring to 2 levels (to exemplify best-fit at
lower/top end of level).
Summary of key messages for clusters on identifying future learner profiles
and commentary.
3
To build on current best practice at KS2 and KS3 clusters should ensure that:







they demonstrate evidence of moderating learner profiles rather than
standardising individual pieces of work;
learner profiles include evidence for each attainment target. As noted in the
guidance and exemplified during the pilot’s information meetings, secondary
evidence through teacher commentary for Oracy is appropriate;
learner profiles for school and cluster moderation include a range of evidence
for each of the attainment targets. The evidence should sufficiently
demonstrate the level characteristics relevant to the learner profile (lower
end/top end or securely within);
cluster commentary reflects the level characteristics demonstrated in the
learner profile and demonstrates the cluster’s overall best-fit judgement;
task setting reflects the Skills and Range of the 2008 English programme of
study e.g. a balance of non-literary and literary evidence for both Reading and
Writing and evidence of a range of Oracy tasks. This is essential to ensure
that assessment reflects the normal classroom practice relevant to the
programme of study including the 2008 level descriptions.
cluster commentary includes contextual information for each task;
evidence includes resource/stimulus material where relevant.
Summary of possible implications for teaching (i.e. task setting) and
assessment:
As noted in the Guidance, tasks/activities should come from normal classroom
practice relevant to the programme of study for English within the 2008 National
Curriculum. Whilst moderators saw good examples of this across the three ATs
across both key stages, there was more evidence of Range and normal classroom
practice at KS2. Often the issues that arose at KS3 were as a result of insufficient
non-literary evidence. This should have been addressed following the previous
external moderation exercise. Schools also continue to make use of past SATs
materials. In some cases these were adapted to reflect aspects of normal classroom
practice e.g. through the nature of the task set and the nature of the feedback given.
In the worst examples these had been undertaken as an examination/test and are
therefore not evidence of normal classroom practice.
Inclusion of non-literary reading evidence would have helped to exemplify the
characteristics of Levels 4 and 5 that refer to pupils being able to ‘locate and use
ideas and information on a specific topic from more than one printed source and use
them effectively’ [Level 4] and to ‘retrieve and collate information from a range of
sources’ [Level 5]. It is interesting to note that these characteristics reflect some of
the reading skills exemplified in the PISA reading proficiency levels where there are
the greatest differences between OECD scores and Wales (PISA reading proficiency
levels 3 and 4). The PISA report also notes that ‘the text types for reading in PISA
2009 consisted mainly of non-fiction texts, including non-continuous texts, such as
charts, graphs, tables, maps and forms’ and confirms that due to the 2008
programme of study for reading in English, ‘learners should now be well equipped to
deal effectively with the texts encountered in PISA’. However, feedback from the
pilot would indicate that there is currently insufficient evidence of non-literary reading
across the learner profiles and that this is a key aspect to address through national
roll out. However, whilst we recognise this as a priority, teacher assessment needs
to be based on more than just evidence of reading to demonstrate a clear
understanding of the three attainment targets and their weighting in coming to a bestfit judgement for a learner profile.
4
Some clusters had included responses to moving image – this is appropriate under
the programme of study for English within Curriculum 2008. Where this evidenced
response to literature, however, and no other non-literary evidence was included a
narrow range of evidence was presented resulting in insufficient evidence to fully
demonstrate the features of Level 4/5.
There was also evidence of some creative task setting to demonstrate both Oracy
and Reading and/or Reading and Writing through the same task. As noted in the
guidance materials this is appropriate provided commentary refers specifically to
performance against both attainment targets. There were examples where
insufficient contextual information such as task setting and/or stimulus materials
made it difficult for moderators to agree with such evidence however, e.g. how far an
empathy piece based on a novel demonstrated a learner’s reading and writing
attainment; or evidence for both attainment targets not clearly distinguished within
the cluster’s commentary
5