e-asTTle Writing
Seminar
The revised writing tool
e-asTTle Writing Seminar
The revised writing tool
Key changes?
Scope
Prompts & Purpose
Assessment
What
How
Rubric
Exemplars
Scale
Reporting
Creating an
assessment
Purpose
Considerations
Making own
prompt?
Prompts
Other prompts
Prompt pack
Administration
The scoring
rubric
What are R’s
By category
The layout
The exemplars
Overview
Generic exemplars
Notes
Struc & Lang. notes
R3 is not L3!
Have a read
Your thoughts
Marking a script
Scoring sheets
What you need
Splashing in the Waves
Considerations
Depressing Dogs
Suggested process
Third time lucky
Your thoughts
Entering the
data
Viewing reports
Interpreting the
Reflections
Overall Score
Imprecision
Individual Elements
Have a go
R3 does not equal R3
Teaching and
Learning
An assessment tool
Suggestion for analysis
Progress
Reflections and
Evaluation
Other prompts
The table
The ILP
E-asTTle Writing – why more changes?
•
•
•
•
Both Mathematics and reading have been updated and
recalibrated in the last two years. The writing was ‘out
of sync’ with the others.
The original scoring rubric uses curriculum levels.
Progressions in writing do not necessarily fit neatly into
curriculum levels.
The previous rubrics did go down to Level 1, but these
could not be recorded in e-asTTle and Year 1-3 student
scores could not be inputted into e-asTTle.
The scoring rubrics were not well enough supported
with exemplars and were difficult to use accurately even
with moderation processes in place.
Home
What are the key changes? – Scope of the Tool
• The tool can now be used to assess writing from
students in Year 1 through to Year 10. The previous
version assessed from year 4 to Year 10.
• In order to be assessed accurately by the new tool,
students should be able to write at least one or two
simple ideas.
(Students who score in the lowest category for every element
assessed are not well targeted by the assessment, as it may not
accurately indicate the students’ skills and next learning steps.)
Home
What are the key changes? - Prompts and purposes
• The tool emphasises ‘prompts’ rather than ‘tasks’.
• 20 open-ended prompts encourage students to
choose subject matter relevant to their experience.
• There is a range of prompts for younger or older
students.
• There are now five writing purposes, important to
writing in general.
Home
What are the key changes? - Rubric
• There is now one rubric for all writing purposes.
• There are seven elements of writing, similar to those
in the previous version but with changes that require
a shift in thinking by teachers.
• There is a range of achievement categories on a
continuum from R1 to R6/R7 for each element.
• The tool is aligned to the NZ curriculum:
- it reflects the intentions of the literacy learning
progressions and national standards
- it is not specific to a particular learning area
• However mainly developed through an analysis of
student responses to writing prompts.
Home
What are the key changes? - Rubric
Original
Audience Awareness and Purpose, Content/Ideas,
Structure/Organisation, Language Resources,
Grammar, Punctuation, Spelling
Revised
Ideas, Structure and Language, Organisation,
Vocabulary, Sentence Structure, Punctuation, Spelling
Home
What are the key changes? - Annotated Exemplars
• There 76 annotated exemplars include a set of prompt
specific exemplars and generic exemplars to
accompany each writing prompt.
• Prompt specific exemplars are examples of writing for
the prompt being used. They show a range of abilities.
• The generic exemplars are not necessarily for the
same purpose or prompt.
• The exemplars are actual samples of student writing,
and are representative rather than ideal examples.
• The annotated exemplars are essential tools for
ensuring consistent scoring decisions.
Home
What are the key changes? – Measurement scale
• The scale provides a measure of writing proficiency
and is linked to curriculum performance levels,
allowing students’ scores to be reported within
curriculum bands.
• The new scale is not directly comparable to the
previous e-asTTle Writing scale.
Note: A conversion table is being developed to provide
some comparison between scores in the new and
previous versions.
Home
What are the key changes? – Report formats
• There are some minor changes to the report formats
to accommodate the new rubric.
• The introduction of + to clearly show error of
measurement in scores.
Home
Assessment – What
E-asTTle writing assesses a student’s ability to
independently write continuous texts across a variety
of communicative purposes.
These are:
• Describe
• Explain
• Recount
• Narrate
• Persuade
Home
Assessment – How
• Up to 40 minutes
• Scored using rubric and annotated exemplars (both
generic exemplars and prompt specific exemplars).
• Scoring rubric linked to literacy learning progressions
but developed through analysis of student samples
• Curriculum level is not part of the marking
• When analysed a curriculum level for each element
and overall score will be available
Home
Creating a writing assessment - Purpose
• Create Test – Choose Writing/Tuhituhi prompt
• Choose the purpose first
•
•
•
•
•
Describe
Explain
Recount
Narrate
Persuade
• Consider the students being assessed, the type of
response that the purpose will deliver and the
information you are wanting to acquire.
Home
Creating a writing assessment – Can you make
your own prompt?
Yes and No
• Yes the rubric can be used to assess teacher’s own
prompts.
• However, DO NOT put the scored results for your
own prompts into e-asTTle as the difficulty of the
‘made up’ prompt is unknown.
• ‘Teacher made’ prompts should assess continuous
texts using the five purposes as this is what the rubric
was developed for. A conversion table is available so
that you do not put this data into e-asTTle.
e-asTTle training area
Home
Creating a writing assessment - Considerations
• When choosing a prompt consider possible sources
of difficulty
• Level of abstraction
• Complexity of the structure eg. Narrative harder than
description or recount
• Topics better suited to some age levels – 5 in slightly simpler
language as most likely to be used by younger students (the
recounts and 3 describes)
• Can use more than one prompt to cater for and engage
students, but probably wise to do so at different times
Home
Creating a writing assessment - Prompts
See other prompts here
Home
Creating a writing assessment - Prompts
Home
Creating a writing assessment – Test creation
Once a prompt has been chosen the prompt pack is
created. It includes:
• Admin instructions
• Student booklet
• Glossary
• Rubric
• Annotated exemplars
Home
Creating a writing assessment – Test
administration
• Don’t forget to assign tests to students – cannot put
results and generate results until this is done.
• Your turn – please spend time reading the
administration guidelines
Home
The scoring rubric – What are R’s
• The progressions for each element within the scoring
rubric is by category - not curriculum level
• R means Rubric
• Do not use the rubric in isolation – use it in
conjunction with the specific exemplars, the generic
exemplars and the Structure and language notes.
Home
The scoring rubric – by category
• Using student samples and the literacy learning
progressions the rubric has been able to capture
stages of skill development in writing.
• A little like children’s art?
Home
The scoring rubric – like categorising children’s art
Random scribble
Symbolism
Controlled scribble
Realism
Pre symbolism
3 dimensional space
Home
The scoring rubric – the layout
The focus
Definition
Category
Descriptor
Notes
Generic
exemplars –
for this
element only
Home
The scoring rubric – R3 is not Curriculum Level 3!
Curriculum
Level
1
2
Ideas
R1
Structure &
Language
Organisation
R2
R1
R1
Vocabulary
R2
R2
R1
R3
R1 R2 R3
Punctuation
R1 R2
R3
Spelling
R1
R3
R2
***
4
R4
R3
R6
R5
R6
R6
R4
R4
6
R5
R5
R3
R7
R5
R5
R4
R4
5
R4
R4
R2
Sentence
Structure
3
R3
R5
R6
R6
R5
R6
R7
R6
Representative diagram only
Home
The scoring rubric – Notes
• THEY ARE NOT REQUIREMENTS – for example, in
Punctuation there are notes for ‘Other Punctuation” at
R2 and R3. Other punctuation is not required in the
descriptor until R4.
• The intention of the notes is to describe ‘what else’
you might see in scripts.
Home
The scoring rubric – Structure and Language
notes
• These notes are prompt specific i.e. The notes vary
depending on which purpose is being written
• Check carefully that these notes relate to the
prompt/s being used.
• For example the prompts ‘Adult and child’ ‘Girl’ ‘The
Market’ ‘Stick Insect’ and ‘Dogs at the beach’ use the
same structure and language notes
Home
The exemplars
• 76 exemplars in total. There is a set of generic
exemplars as well as exemplars specific to each
prompt
• The exemplars are not exemplary items. They
represent actual samples of student writing
• The annotated exemplars provide excellent
comparative reference points for ensuring consistent
marking.
Home
The specific exemplars
• These are generated when creating a prompt pack in
e-asTTle.
• They directly relate to the prompt you have chosen.
• The show a range of scores for the prompt you are
using.
• DIRECT comparisons can be made
• Use these before going to the generic exemplars.
Home
The generic exemplars
• They are referenced at the bottom of the marking
rubrics.
• They do not necessarily directly relate to the prompt
you are using.
• Use these for further reference if the specific
exemplars have not enabled a definitive decision.
• There are only one set of generic exemplars.
Home
The scoring rubric – Have a read
• Your turn – please spend time becoming familiar with
the scoring rubric using the glossary and definitions
as reference material.
• Make sure you look at the Structure and Language
notes for ‘Dogs at the Beach’ which is ‘Describe a
moment in time’.
• Highlight bits which are important or you will need to
remember.
Home
What we found out about the scoring rubrics
- Key points from the group
Home
Marking a script – What you need
Home
Marking a script – Considerations
• Age is irrelevant! Mark it for what it is.
• Mark the script in front of you, not what you know about
the student who wrote it.
• You can only use the evidence in front of you. e.g. If
technical words are not there, there is a limit to how far you
can score in spelling.
• There is no scoring rubric for neatness of handwriting!
• Beware of ‘proximal marking’. i.e. Mark each element
independent and separate from the other elements.
• It is possible to be at R6 for spelling and R2 for
punctuation.
Home
Marking a script – Suggested process
1. Read the text right through
2. Go to one of the seven elements and read the
descriptors and notes. Get a feel for which category
it may fit.
3. Check against the prompt specific exemplars for
comparative qualities.
4. Still not sure? – use the relevant generic exemplars
as another check.
Home
Marking a script – Have a go
• The script being given to you is an annotated
exemplar.
• Have a go at marking it and then we will discuss the
difficult decisions
• We will then compare to the annotated scores.
Home
Splashing in the Waves – How close were we?
Element
R1
R2
R3
R4
R5
R6
R7
Ideas
Structure
and Lang.
Organisation
Vocabulary
Sentence
Structure
Punctuation
Spelling
Home
Have another go – same prompt, different
sample!
Depressing Dogs
• Have a go at marking and try and come to agreement
over scores for each element with a partner.
• Remember to use the rubric, structure and language
notes, generic exemplars and specific exemplars to
help with your decisions
Home
Depressing Dogs - How did we go this time?
Element
R1
R2
R3
R4
R5
R6
R7
Ideas
Structure
and Lang.
Organisation
Vocabulary
Sentence
Structure
Punctuation
Spelling
Home
Third Script – Your choice
• Choose one of the recount or narrative scripts and
mark by yourself or with a partner. Once complete we
will get into ‘script groups’ and analyse and scores
and debate.
• Don’t forget to use the group scoring grid.
Home
Third time lucky – your choice
Element
R1
R2
R3
R4
R5
R6
R7
Ideas
Structure
and Lang.
Organisation
Vocabulary
Sentence
Structure
Punctuation
Spelling
Home
How was it? – Your thoughts and reflections
Home
Entering the data
• Checkpoint No.1 – have you assigned it?
• Change the date to represent when the actual sample
was taken rather than the date the data was entered
• Put the attitude and seven element scores onto
e-asTTle
• Click save after each student
• Reports are ready!
Home
Viewing reports
Home
Viewing reports
Home
Viewing reports – Interpreting the Overall Score
• The curriculum levels reported for e-asTTle Writing are based
on a standard-setting exercise undertaken to link
performance on an e-asTTle assessment with the
descriptions of writing competence provided in the Literacy
Learning Progressions.
• A curriculum level of 4A for example means that
“given 40 minutes to write to a particular prompt under
test conditions, the student has produced a text of
sufficient quality to indicate they have the writing skills
and competencies described as appropriate for
students working at an advanced stage for Level 4 of
the curriculum.”
Home
Viewing reports – Interpreting the Overall Score
• e-asTTle curriculum level attempts to identify the student’s performance in
the context of an e-asTTle writing assessment.
• A student who has been assessed by e-asTTle Writing at a particular
curriculum level will not necessarily have produced a piece of writing that
is of the same standard as a National Standards illustration.
• National Standards illustrations show examples completed by students in
class “largely by themselves”.
• An e-asTTle assessment is completed in 40 minutes under test conditions
without any teacher or peer feedback, or access to writing aides such as
dictionaries.
Home
Viewing reports – Taking imprecision into account
• No educational assessment is perfectly precise. e-asTTle
Writing provides a margin of error by presenting scale scores
within a “plus or minus” range, for instance 1250 ± 40.
• If the student was to repeat the assessment we could
expect him/her to score somewhere between 1210 and 1290
scale units about 70% of the time.
• Imprecision also needs to be taken into account when
considering the curriculum level descriptor. A student who
scores 4A is probably somewhere in the range 4P to 5B.
Home
Viewing reports
Home
Viewing reports – R3 does not equal R3 or Level 3!
Element
R
score
Overall score
Explain - Caring for
Planet Earth
Narrate - I heard
a whisper
Recount – Time
with friends
2A
3B
2A
Ideas
R3
2A
3B
2A
Structure and
Lang.
R3
3B
3P
3B
Organisation
R3
2A
2A
2P
Vocabulary
R3
2A
3B
2A
Sentence
Structure
R3
3B
3P
3B
Punctuation
R3
2P
2A
2P
Spelling
R3
2P
2A
2P
Home
Viewing reports – Individual Elements
• In the revised e-asTTle writing each element has only 6 or 7
scoring categories. Moving from say a R2 to a R3 will involve
a large shift in the associated curriculum level.
• It is unwise to treat the element curriculum scores as precise
measures and comparisons with the old element scores will
be very tenuous.
• It is best for teachers to point to the rubric scores given for
each element when discussing a report with students.
.
Home
Viewing reports – Individual Elements
• The reporting system in the e-asTTle tool is built on a system
that uses many more bands than this and which are linked to
curriculum levels.
• These curriculum bands can impress a sense of precision
that isn't there and mean we have to be very aware of the
margin of error in an e-asTTle score when interpreting or
comparing them.
R1
R2
R3
R4
R5
R6
1B 1P 1A 2B 2P 2A 3B 3P 3A 4B 4P 4A 5B 5P 5A 6B 6P 6A
Home
Use to improve teaching and learning
• This is an assessment tool.
• “The primary purpose of assessment is to improve
students’ learning and teachers’ teaching as both
student and teacher respond to the information that it
provides.”
• Therefore we take the information from these
samples to identify areas for improvement and
deliberate teaching to support the student in his/her
learning.
Home
Use to improve teaching and learning
• Suggestion – use the curriculum level scores to
indicate needs and areas for future teaching because
R scores by themselves will be misleading.
i.e. R3 ≠ R3
• Then go back to the rubric to identify the progression
needed. Use documents such as the literacy learning
progressions to support you in the teaching of this
element.
Example: Sentence structure R3 – 2A
Use the rubric to see the demands of R4 to
identify next steps for this element.
Home
Viewing reports – showing progress between samples
Home
What now for you and your school?
• Is this a tool that can add to the improvement in
teaching and assessment of writing in your school?
• Is it useable and manageable?
• How might you use it?
• When?
• How?
Home
Evaluation
• Please spend a minute or two reflecting on this
seminar and how it has helped you come to terms
with the revised writing tool.
• Your feedback helps us to evaluate our own
performance and provides us with information on how
to improve the experience for others in the future.
• Thank you
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Other prompts
Home
Home