e-asTTle for Teaching and Learning
2015
The Essentials of e-asTTle
Seminar Overview
• Welcomes, Introductions
• Background to e-asTTle
• How e-asTTle works - rules of engagement
• Thinking differently about testing – Item Response Theory
• Misconceptions
• Creating a test
• Customised and Adaptive Tests
• Copy tests and Similar tests
• Writing test overview
• Reports & next steps
• Interpreting the reports
• e-asTTle updates
• Administration of e-asTTle
•
Reflection and Evaluation
Burning issues….
Electronic
assessment
tool for
teaching and
learning
Basic Rules of Engagement
• The data will be used to inform teaching and learning
• Students will share in the analysis of the data
• Testing will be by current achievement levels, not age
• The results will be interpreted in conjunction with other
forms of assessment
• The reports are only the beginning of analysis – not the
analysis itself
• The student’s test score in NOT an Overall Teacher
Judgment
Think differently about testing
– changing our hardwired thinking
Think differently about testing
• Students A, B and C all sit the same test
• The test has ten items
• All items are dichotomous (score 0 or 1)
• All three students score 6 out of 10
What can we say about the ability
of these three students?
Think differently about testing
Item
1
2
3
4
5
6









7
8
9
10
Score e-asTTle
Difficulty
Bob
Emma
Harry



6




6


6
Think differently about testing
Item
1
2
3
4
5
6









7
8
9
10
Score e-asTTle
Difficulty
Bob
Emma
Harry









6
1342
6
1392
6
1435
Think differently about testing
Item
5
6
7
8
9
10
Difficulty -3 -2 -1 -1 0
0
1
1
2
3
Bob
Emma
Harry
1


2

3
4















Score e-asTTle
Sublevel
6
1342
2A
6
1392
3B
6
1435
3P
Think differently about testing
• Two students get 6/10 correct.
• However, this does not mean they will receive the same
e-asTTle scale score.
• Item Response Theory is based on the difficulty of items a
student gets correct - not how many.
Implication
• Raw scores mean very little in e-asTTle.
Misconception #1
“Everyone needs to do the same test for
a fair comparison”
• If students of wide ability take the same test it is really only
the ‘average’ sector where any useful information will be
gained.
• Getting too many right or wrong provides us with very little
information.
Implication
• Test students on current achievement levels, not age.
• It may be necessary to create up to four different tests for
some cohorts. Each student is then assigned to a test which is
most suitable for him/her.
Misconception #2
“100% is best?”
• If we want to use asTTle to inform us, the students need to
get some answers wrong.
• A student that gets 100% correct gives little information on
where his/her next steps in learning are.
Implication
• We must make the tests challenging, even difficult for the
students.
Misconception #3
“Everyone must take the test at the
same time in the same place”
• E-asTTle provides the opportunity/challenge to test
students at different times and places, especially when
taking an online test.
• Students, if allowed in the assignment of the test, can even
sit a test at home.
• The MAIN PURPOSE of the test will dictate when, where and
how testing can take place.
Misconception #4
“e-asTTle means you take the test
on the computer”
• In many cases computer based testing is the best way
forward.
• However, paper based testing should also be considered
depending on your situation.
• Paper based testing is the recommended method for
areas such as Geometry or Measurement where
interaction with graphics is required.
CHECKPOINT 1
What were the
key messages
for me?
What action do I
need to take?
Creating Tests
The following tests can be created:
 A Customised Test
 An Onscreen Adaptive Test
 A Writing Prompt
And the following actions can also occur:
 Copy an Existing Test
 Create a Similar test
 Add Numeracy Data
Customised & Online Adaptive Tests
Customised Test
• This test is customized by the user and the process allows the person
creating the test to select from a range of curriculum strands,
different processes or time and control features, such as on-line
versus paper and pen.
Online Adaptive Test
• The main difference between an adaptive and a customized test is
that e-asTTle adjusts the difficulty of the test questions while the
student is sitting the test based upon the responses they are
providing.
Adaptive test
Creating a Writing Prompt
• School / teacher chooses a purpose and a prompt
• Student write for up to 40 minutes, maximum
• Writing is scored using…
• rubric
• annotated exemplars
• structure and language notes
• Curriculum level is not part of the marking.
• When rubric scores are entered a curriculum level for each
element and an overall score will be available
Creating a similar, or copying a test.
Why would you?
• Creating a similar test means that you can create a new test using
the same settings as a previous test but it will result in a new set of
questions
• Copying a test allows you to use the same test with a different group
of students, or, with the same group of students but at a different
time. It will have the same questions. The benefit is that it allows you
to keep the two reports separated.
CHECKPOINT 2
What were the
key messages
for me?
What action do I
need to take?
Time to have a go…
e-asTTle on TKI
http://e-asttle.tki.org.nz/
e-asTTle Training site – Teacher login
https://training.e-asttle.education.govt.nz/SCWeb/login.faces
e-asTTle Training site – Student login
https://training.e-asttle.education.govt.nz/StudentWeb/login.faces
CHECKPOINT 3
What were the
key messages
for me?
What action do I
need to take?
Online tests – Marking open response questions
• Closed questions (i.e. multi choice or true/false type questions) do not
require any data input or marking. The programme marks them
automatically
• Open questions are marked by question with student responses clustered.
Entering Paper Tests data
Entering e-asTTle Writing scores
e-asTTle Reports
The Console Reports
The Console Reports
The Comparisons Console Report (Schools like mine)
The Multi-test Console Report
The Individual Learning Pathway Report
The Individual Learning Pathway Report
The Individual Learning Pathway Report
• Standard error of measurement around the
students score (± )
• This is like the 'margin of error' reported in
political polls
• Two out of three times the student’s true
score will lie somewhere between the top
and the bottom of the red circle
• A difference in scores needs to be >22 to
be ‘statistically significant’
Error of Measurement
Error of Measurement is always
higher at the extremes of a scale
than in the middle of a scale…
no matter which tool you use!
Individual error of measurement
in writing, maths and reading.
e-asTTle standard error of
measurement in reading and
maths is 22 points and in writing
it is 80 points .
Error of Measurement – use cut scores
2A
1374
3B
1380
+6
2P
1295
2P
1342
+ 47
The Individual Learning Pathway Report
- Criterion information
When a dash “ – “appears in the e-asTTle ILP it means that the student has
failed to get more than 3 questions correct in that strand.
If a dash appears in the overall score it doesn’t mean the student
only got 2 correct answers in the test!
The Individual Learning Pathway Report
- Interpreting the quadrants
Strengths
• Wrong as expected
• Harder than student’s overall ability
Correct
Incorrect
• Unexpectedly correct
• Harder than student’s overall ability
To Be Achieved
Achieved
• Correct as expected
• Easier than or equal to the student’s
overall ability
Gaps
• Unexpectedly wrong
• Easier than or equal to the student’s overall
ability
e-asTTle reports – checking understanding…
Freddie Fangdangle scores 3B in reading or maths
Questions harder than 3B
which Freddie got correct
Questions harder than 3B
that Freddie got wrong
Questions equal or easier
than 3B that Freddie got
correct
Questions equal or easier
than 3B that Freddie got
wrong
e-asTTle reports – checking understanding…
Freddie Fangdangle scores 3B in writing
Element scores more
than a sublevel above 3B
e.g. 3A or higher
Element scores within a
sublevel of 3B
e.g. 2A, 3B or 3P
Element scores more
than a sublevel below 3B
e.g. 2P or lower
The Individual Learning Pathway Report
- Interpreting the quadrants
The Individual Question Analysis report
Individual Question Analysis – Student Speak
The Group Learning Pathway Report
The Group Learning Pathway Report
The Group Learning Pathway Report
The Curriculum Level Report (aka Skyline)
The Curriculum Level Report (aka Skyline)
The Curriculum Level Report (aka Skyline)
The Curriculum Level Report (aka Skyline)
2A
3P
3A
4A
The What Next Profile
The What Next Profile – in e-asTTle online
The What Next Profile – on the TKI site
http://assessment.tki.org.nz/Assessment-tools-resources/What-Next
The What Next Profile
The Tabular Report
The Tabular Report – Excel file (.csv)
The Progress Reports
The Progress Report – for 1 student
The Progress Report – for a group
Manage Students Area
Target setting
Student Result Summary
Have a look at some of your own students…
Manage students – Student details
Manage students – Student details
Download Student Logins
• by PDF or csv (excel)
• As a PDF you can print out as a card, hand it to the student and receive back once
the password has changed.
• To reset password go into “Manage existing students”, choose students and then
click “reset password”.
Manage students – Using groups
Manage students – Using groups
• Some groups will automatically be created when SMS data is imported.
Most often Class, Year etc.
• However you can create custom groups that enable the viewing of
reports etc. for that particular group
• By doing this you DO NOT need to test a certain group again. You can use
existing data but just view reports for the selected students in your group
• Examples of custom groups could be:
• Your top reading group
• An extension or remedial group
• Maori boys or similar groups that need to be monitored
• Target students
e-asTTle updates
CHECKPOINT 4
What were the
key messages
for me?
What action do I
need to take?
External Coordinators
External Coordinators are
people who may not be staff
members, but wish to access
school data for various
reasons.
Can only be set up by
someone with an
e-asTTle School Leader role.
Future seminars…
Dealing with data seminar - An introduction to using Excel in schools
• This practical seminar is for school leaders who need support in learning to use
Excel when working with student achievement data.
http://www.evaluate.co.nz/dealing-with-data-workshop/