Standardising and Annotation
Regional Support Network Meetings
Autumn 2013
1 of x
Version 3.0
Copyright © AQA and its licensors. All rights reserved.
PLEASE NOTE
In order to meet regulatory requirements, this meeting will be recorded.
2 of x
Copyright © AQA and its licensors. All rights reserved.
Internal Standardising
In this session we will:
• consider how TOLS might be used
• discuss the marking of candidates’ work
• explore the importance of good annotation
• consider models of internal standardising
3
Copyright © AQA and its licensors. All rights reserved.
TOLS
• Three candidates in each of the four units. One set of standardising
materials does not ‘replace’ another; there is a library of materials.
• Has to be in the secure area so username / password necessary.
• Each person completes the appropriate units on-line allowing others to see
their marks by a deadline specified by the Head of Department.
• One person completes the appropriate units, gains access to the
commentaries and marks and arranges department standardising when
appropriate.
• Departments focus on particular sections close to when they are marking
that task.
4
Copyright © AQA and its licensors. All rights reserved.
The Importance of Annotation
Work with at least one other person:
• read the list of paragraphs from candidates’ work which are accompanied by
Centre annotation
• discuss their appropriateness
• choose one of the complete annotated responses
• decide your own mark
• consider the appropriateness of the annotation
5
Copyright © AQA and its licensors. All rights reserved.
The Importance of Rigorous Internal Standardising
The challenges:
• finding time
• lack of centre support
• ‘experienced’ staff
• ‘inexperienced’ staff
• pressure to produce results
Why it is important
• how Moderators operate: ‘samples’, ‘tolerance’
• what ‘regression’ is
6
Copyright © AQA and its licensors. All rights reserved.
Moderation
On a sample of 20:
Sub-sample 10
Stage 2
15
Stage 3
20
• English has 6 separate marks – tolerance for total is 3
• If four of the six marks are one mark out, accumulated
errancy = 4, so whole submission becomes out of
tolerance and the moderator has to move to stage 2.
•
If one piece is four out, the same applies.
7
Copyright © AQA and its licensors. All rights reserved.
Moderation
• We train moderators by standardising them with additional materials designed to
show them issues they will encounter
• We monitor moderators’ work by checking samples of what they do
• We ensure that problems in moderation are passed to a senior moderator for
second opinion
• We check speaking and listening records
• Moderators check S&L records for coverage of tasks and evidence of applied
criteria
• Moderators select separate samples for each component and report back
separately
8
Copyright © AQA and its licensors. All rights reserved.
Models of Internal Standardising
•
little, and as often as is reasonable
• focus on key borderlines
• task-based rather than folder-based
• use centre materials as well as AQA materials
• teacher pairing
• make sure the demands are very clear to school leadership – up to three
different units; six separate marks in English, five in English Language;
uniquely tight tolerances; uniquely pressurised subject as far as CA is
concerned; dealing with bands when the centre wants grades • tackle over and under-rewarding as it is found
• tackle ineffective task-setting as it is found
9
Copyright © AQA and its licensors. All rights reserved.
Band
5
‘Sophisticated
impressive’
25 - 30
4
‘Assured’
19-24
3
‘Clear, consistent’
13-18
2
‘Some’
7-12
English Language Criteria
• sophisticated interpretations of texts
• sophisticated engagement with writers’ ideas and attitudes; sophisticated
interpretations using imaginatively selected supporting textual detail
• sophisticated analysis of aspects of language and structure
• confident and developed interpretations of the text
• engagement with writers’ ideas and attitudes and provide perceptive
interpretations using precisely selected supporting textual detail
• analysis of aspects of language and structure in convincing detail
• clear evidence of understanding of significant meanings of the text
• ability to explain writers’ ideas clearly offering relevant and appropriate
supporting textual detail
• ability to display understanding of features of language and structure
supported by relevant and appropriate quotation
• some awareness of more obvious meanings in the text
• some awareness of writers’ ideas supported by relevant textual detail
• some awareness of obvious features of language and structure supported
by some relevant textual detail
Band
5.
‘Sophisticated,
impressive’
13-15
4.
‘Assured’
10-12
3.
‘Clear, consistent’
7-9
English Reading Criteria
• sustained, sophisticated interpretation of the text
• sustained and sophisticated engagement with writers’ ideas and attitudes. Sophisticated
interpretations using imaginatively selected supporting textual detail
• sophisticated analysis of aspects of language and structure
• perceptive and mature comment on the significance of the contexts
• sustained and developed interpretations of the text
• engagement with writers’ ideas and attitudes and provide perceptive interpretations using
precisely selected supporting textual detail
• analysis of aspects of language and structure in convincing detail.
• convincing connections between the text and its context
•
•
•
•
2
‘some’
4-6
•
•
•
•
clear evidence of understanding of significant meanings of the text
ability to explain writers’ ideas clearly offering relevant and appropriate supporting textual
detail
ability to display understanding of features of language and structure supported by
relevant and appropriate quotation
ability to explore the significance of aspects of the contexts matched to textual details
some awareness of more obvious meanings in the text
some awareness of writers’ ideas supported by relevant textual detail
some awareness of obvious features of language and structure supported by some
relevant textual detail
some relevant comments about the significance of the contexts
Band 5 33-40 ‘sophisticated’ ‘impressive’
•
•
•
•
•
sophisticated engagement with writers’ ideas and attitudes
sophisticated interpretations using imaginatively selected supporting textual detail
sophisticated analysis of aspects of language and structure
perceptive and imaginative exploration of points of links and comparisons
perceptive and imaginative comment on the significance of the contexts
Band 4 25-32 ‘confident’ ‘assured’
•
•
•
•
•
sustained and developed appreciation of writers’ ideas & attitudes
convincing interpretations using precisely selected supporting textual detail
analysis of aspects of language and structure in convincing detail.
thoughtful consideration of links & comparisons between the texts
thoughtful consideration of the significance of the contexts of the texts
Band 3 17-24 ‘clear’ ‘consistent’
•
•
•
•
clear/consistent understanding of writers’ ideas with relevant appropriate supporting textual detail
clear/consistent understanding of features of language and structure supported by relevant and appropriate quotation
clear/consistent understanding of links and comparisons between texts
clear/consistent grasp of the significance of some aspects of the contexts
Band 2 9-16 ‘some’
•
•
•
•
some familiarity with writers’ ideas supported by a range of textual detail
some familiarity with obvious features of language and structure supported by some relevant textual detail
some relevant comments about links between texts
some relevant comments about the significance of the contexts
Band 1 1-8 ‘limited’
•
•
•
•
limited awareness of writers’ ideas using a limited range of textual detail
limited awareness of obvious features of language and structure
limited awareness of links between texts
limited awareness of contexts and very limited comment on their significance