Computer-Based Assessments
CBA Pilot for NILA & NINA
Autumn 2013
Assessment Coordinator Training
September / October 2013
1
Learning Intentions
At the end of the session you will be aware of:
• the context for computer-based assessment (CBA) in autumn 2013;
• developments to NILA and NINA;
• next steps for the pilot schools; and
• guidance and support available from CCEA and C2k.
Will have an opportunity to :
• access the CBA online training course;
• explore NILA and NINA; and
• contribute to discussion about using the NILA and NINA outcomes.
2
Overview of the Seminar
9.30
Registration
10.00
Welcome
Context of computer-based assessment in NI
10.45
Tea/Coffee
11.00
CBA context (contd.)
Developments - NILA and NINA
13.00
Lunch
13.30
Reporting Developments
Practical Activities
Next Steps for CBA Pilot
Guidance and support available from CCEA
14.45
Tea/Coffee
16.00
Questions, Evaluation & Close
3
Part 1
Setting the Context for the CBAs in 2013
4
Legislative Context
In his letter to schools in May 2013 the Minister for Education outlined
that:
• CBA assessment will not be specified as mandatory in 2013/14;
• Use of CBA assessments (NILA and NINA) in 2013/14 will be
optional for schools.
“Although CBA will not be specified in 2013/14, the Department expects
schools to carry out diagnostic assessment using an assessment tool
of its own choice and that the results of this assessment should be
shared with the pupil’s parent/guardian at a mutually convenient time.”
“I would like to encourage as many schools as possible to participate in
this pilot.”
5
Moving Forward
with the CBA Pilot
Online CBA Training live
as of 9 September
Initial Pre-pilot
11 – 13 September
Principal Information Seminars
11 – 25 September
CBA Pre-pilot (10 schools)
16 – 20 September
Assessment Coordinator Training
27 September - 11 October
CBA Pilot:
- Phase 1
- Phase 2
- Phase 3
- Phase 4*
7 October – 18 October
21 October – 8 November
11 November – 22 November
25 November – 13 December
Questionnaires issued to schools
November
6
Computer-Based Assessments
NILA - Northern Ireland Literacy Assessment
NINA - Northern Ireland Numeracy Assessment
• Assessments are available in English & Irish.
• Assessments are adaptive and not time-bound.
• Assessments are web-based and pupil responses to questions are
saved automatically online.
• Assessments may be completed by Year 4 to Year 7 pupils from the
pilot schools during autumn 2013.
• Assessments are processed immediately following completion and
schools can access results after 48 hours.
The purpose of the computer-based assessments is to help teachers
plan the learning and teaching during the school year to meet the
needs of pupils in their classrooms.
7
Computer-Based Assessments
The question items in the Literacy / Numeracy Assessments have
been developed using the following criteria:
• Definition of Literacy / Numeracy as set out by CCEA;
• The expectations of the NI Language & Literacy / Mathematics &
Numeracy Curricula;
• CCEA’s revised Lines of Development;
• Advice from ELBs regarding expectations at the end of each
academic year; and
• Levels of Progression.
8
Focus on Raising Standards
“The use of data from computer-based assessments remains a
key aspect of the Department’s school improvement policy”
Paragraph 3 – DE Letter to Principals Nov 2011
9
Computer-Based Assessments
10
NILA Overview
11
The Literacy Assessment
• Assesses some crucial aspects of Listening, Reading and Writing.
• Draws on topics from across the Northern Ireland Curriculum.
• Is varied in form, ranging from everyday texts, such as
advertisements and signs, to high quality literary passages.
• Uses 8 different answer techniques (cloze, drag and match, drag
into order, drag punctuation, drag to hot spot, pick multiple, pick one
and select word).
• Uses the potential of the on-screen presentation to provide
interactive and engaging experiences in different ways.
12
The Literacy Assessment
• There are two age related bands in the assessment, one for Years 4
and 5 and one for Years 6 and 7.
• The questions cover a range of difficulty within each band, designed
to allow pupils show what they can do whatever their ability.
• The assessment adapts to easier or more challenging questions
based on pupils’ responses.
• The assessment should take the majority of pupils around 35
minutes to complete.
• Provides teachers with diagnostic information about the skills and
understanding of their pupils.
13
Focus on Skills
Aspect of Literacy Skills Focus
Pupils’ ability to …
Listening
Information
Retrieval
Listen to and understand information
and ideas.
Reading
Sequence
Understand information and ideas
Information
Find and use information
Features of
Texts
Understand different types of texts
Opinions
Respond to different layers of
meaning
Writing
Punctuation Punctuate text
Spelling
Recognise accurate spellings
Grammar
Use grammar accurately
14
Adaptive Routing in NILA
15
NINA Overview
16
The Numeracy Assessment
• Samples the key concepts of Numeracy through two assessments
/ strands:
 Strand 1 Understanding & Using Number
 Strand 2: Shape & Space, Measures; Handling Data
• Each strand will be assessed using an adaptive approach to
assessment.
• Assessment for each strand will take approximately 20 minutes.
• Only mental arithmetic questions are timed.
• English Medium database is multi-instance.
17
Focus on Skills
Strand 1
Skills Focus
Pupils
Pupils’ability
abilityto
to…
…
Number
Facts
Counting & Understanding Number;
Facts and Mental Arithmetic
Operations
Operations with Whole Numbers,
Integers and Decimals
Fractions, Decimals, Percentages,
Ratio & Proportion
Problems – including Money,
Function Machines & Algebra
Shape and Space
Fractions
Problems
Strand 2
Shape and
Space
Measures
Handling Data
Measures
Handling Data
18
NINA Assessment Process
What is being assessed?
How difficult are the
questions?
Column
Information Provided
Skills Focus
Specific Area of the curriculum
Objective
Specific aspect of numeracy
Year
Curriculum year of the question
Band
Difficult, medium or easy
How the pupil performed? Outcome
Ticks and crosses
19
Adaptive Assessment
• All pupils start in their target year and are presented with questions
which are one year below their current school designated year e.g.
Year 3 questions for a Year 4 pupil.
• On the basis of their responses, pupils are promoted, demoted or
stay in their curricular year.
• A promotion may be reversed by a subsequent demotion and vice
versa.
• The overall result is then computed as a weighted average of the
two strand scores using the number of threads as the weighting.
• Outcomes from the assessment give both numerical and qualitative
diagnostic information about individual pupils. These will become
available 48 hours after completion of Strand 2.
.
20
Adaptive Assessment
21
Part 2
Developments to NILA and NINA
22
2012 CCEA Review
Evaluation of:
• online CBA training (users);
• CBA implementation during autumn 2012 (principal, teacher and
pupil);
• AROs and SSs (principal and teachers); and
• parent perspective.
Feedback from:
• helpdesk calls;
• ELB support teachers seminars;
• Principals’ focus groups; and
• written and verbal feedback from special schools
23
Developments
•
•
•
•
•
•
•
•
•
•
Revised Pupil Demos and associated resources;
NINA Practice Area;
Item Analysis and Question Banks;
SEN Improvements;
Monitoring Assessment Status;
Improved Reports (simpler, sortable, printable and saveable);
Charts and Tables showing progress over time;
Improved IM Language selection;
Modified Pupil Checklist; and
Streamlined Helpdesk.
24
Pupil Demos
Pupil Demos for autumn 2013
• Pace of the voice will be slowed
• Optional sub-titles available
• Scripts will be available.
NILA Practice Assessment Demo / NINA Practice Area Demo:
Introduces pupils to NILA / NINA and the techniques they need to be aware of
when completing their assessment.
NILA / NINA Real Assessment Demo:
Reinforces this knowledge and prepares the pupils for logging in to complete
their actual assessment.
25
Revised Practice Area
26
Revised Practice Area
27
Question Banks and
Item Analysis
NILA
• Additional questions added to NILA at the lower and upper end.
• Drag punctuation questions:
– Improved ‘grab’ of the box; and
– More accurate placement of the punctuation mark.
NINA
• Reviewed small number of questions with high proportion of
incorrect answers.
28
General SEN improvements
• Specific guidance included in the NILA / NINA Guidance
Documents;
• Setting an SEN entry point for individual pupils;
• Setting an HI entry point (removes Listening questions from NILA);
• Improved ‘drag punctuation’ questions in NILA;
• Voice in NILA and NINA Demos slowed down;
• Improved pupil demos (for the practice areas and the real
assessment);
• Streamlined NINA Practice area; and
• Simplified pupil checklists.
29
Introduction of
Special Entry Points
What qualifies a pupil to use the SEN entry point?
• Teacher and SMT must:
– reasonably expect that the pupil is performing 2 or more years below their
chronological age; and
– be in agreement that an SEN entry point is appropriate for the pupil.
What qualifies a pupil to use the HI entry point?
• Teacher and SMT must:
– have evidence that the pupil is receiving special support for HI; and
– be in agreement that an HI entry point is appropriate for a pupil.
N.B: Only the administrator can select the SEN and HI entry point.
30
NILA SEN and HI Entry Point
SEN Entry Point
Pupils will:
• have access to the same number of questions as their peers;
• allows pupils quicker access to easier and shorter questions; and
• aims to deliver a more positive and potentially shorter assessment.
Hearing Impaired Entry Point
Pupils will:
• only be assessed in Reading and Writing; and
• use an appropriate checklist.
31
Setting an SEN Entry Point
for a Pupil (NILA)
32
Setting an SEN Entry Point
for a Pupil (NINA)
33
NINA SEN Entry Point
Pupil’s
Curriculum Year
First Non-SEN Entry
Point questions will be
from:
First SEN Entry Point
questions will be from:
Year 4
Year 3
Year 1
Year 5
Year 4
Year 2
Year 6
Year 5
Year 3
Year 7
Year 6
Year 4
34
NILA: Monitoring Assessment
Progress for the School
Shows that the pupil has
started NILA in English
Shows that the pupil has
completed NILA in English
Shows that the pupil has
started NILA in Irish
Shows that the pupil has
completed NILA in Irish
35
NINA: Monitoring Assessment
Progress for the School
Shows that the pupil has started NINA
Shows that the pupil not started NINA
Shows that the pupil has completed NINA
36
CBA Roles and Responsibilities
• NILA / NINA Administrator - Administrators manage
NILA / NINA* at school level.
• Teacher - Teachers manage the delivery of NILA /
NINA* for their class.
*Note for NINA: there is an Admin Zone and a Teacher Zone
37
Administrator Role
The Administrator can:
• view the details for all teachers, pupils and classes registered in the
school;
• print / create teacher usernames and passwords for the school;
• print pupil usernames and passwords for each class;
• set an SEN / HI Entry Point for a pupil;
• monitor assessment progress for all classes in the school; and
• view and print tables and reports for individual pupils and classes
across the school.
38
Logging in as
the NILA Administrator
39
Logging in as
the NINA Administrator
40
Demonstrating NILA
• Demonstration of NILA administration
41
Part 3
Developments in Reporting
42
What NILA / NINA Tables and
Reports are available?
• NILA / NINA Results Tables provide schools with Age-Related
Outcome (ARO) Ranges and Standardised Scores (SS). These are
available for individual pupils, classes, year groups and at whole
school level. These build over time to show additional data as it
becomes available for new academic years.
• NILA / NINA Parent Reports contain the results table for an
individual pupil, showing only the Age-Related Outcome (ARO)
Range.
• NILA / NINA Objectives Reports provide specific information about
the questions asked and whether the pupil answered correctly or
incorrectly.
43
Printing
Teachers can view and print tables and reports for:
• their own class; or
• any of their own pupils.
Administrators can view and print tables and reports for:
•
•
•
•
the school;
any year group;
any class; or
any pupil.
44
Results Table (Individual Pupil)
45
Results Table (School,
Year Group and Class)
46
Parent Report
47
NILA Objectives Report
• NILA Objectives Report focuses on the assessment performance of
an individual pupil.
• It is designed to answer three key questions:
– what skills have been assessed?
– what was the difficulty of the questions?
– how did the pupil perform during the assessment?
48
NILA Objectives Report
What skills
have been
assessed?
What was the
difficulty of the
questions?
CY is now Age
> above what is expected in
the pupil’s curriculum year
= within what is expected in
the pupil’s curriculum year
< below what is expected in
the pupil’s curriculum year
How did the
pupil
perform?
Columns reordered so that
the Outcomes
are on the right.
49
NINA Objectives Report
• NINA Objectives Report focuses on the assessment performance of
an individual pupil.
• Three different views of the Objectives Report designed to answer
three key questions:
- what skills have been assessed?
- what was the difficulty of the questions?
- how did the pupil perform during the assessment?
50
NINA Objectives Report
Question Order
Question Order
Band
Single report
which includes
both Strand 1
and Strand 2.
E – Easy
M – Medium
D - Difficult
Printable
individually or in
batches within 48
hours for current or
previous years.
51
NINA Objectives Report
Skills Focus
Skills
Focus
Single report
which includes
both Strand 1
and Strand 2.
Printable
individually or in
batches within 48
hours for current or
previous years.
Difficulty of
Questions
answered is clear.
Number of
Questions will be
included with the
pupil details at the
top of the page.
Teacher will know
the pupil’s target
year.
Clear where
pupil has
answered
multiple
questions from
an objective.
52
NINA Objectives Report
Curriculum Year
Year
Single report
which includes
both Strand 1
and Strand 2.
Difficulty of
Questions
answered is clear.
Clear where
pupil has
answered
multiple
questions from
an objective.
Printable
individually or in
batches within 48
hours for current or
previous years.
53
Demonstrating NINA
• Demonstration of NINA administration
54
Activity: Using the ‘Outcomes’
In your pack you have a selection of NINA and
NILA reports.
How could you and your colleagues use this
information within your school?
55
Using Outcomes
For example,
• To monitor progress and for benchmarking performance within the
school and across Northern Ireland.
• To support clear and effective communication with parents in line
with feedback from schools in 2012.
• To aid communication with parents as to how they can help to
support learning at home.
• To support the planning for learning and teaching for individual
pupils.
• To inform individual education plans for pupils receiving targeted
support for special educational needs.
56
Reporting CBA Outcomes
to Parents
In his letter to schools in May 2013 the Minister for Education outlined
that:
“Although CBA will not be specified in 2013/14, the Department expects
schools to carry out diagnostic assessment using an assessment tool
of its own choice and that the results of this assessment should be
shared with the pupil’s parent/guardian at a mutually convenient
time.”
57
Practical Activities
• CBA online training course (10 mins)
• Exploring NILA (10-15 mins)
• Exploring NINA (10-15 mins)
58
Part 4
Next Steps
59
Key Dates for CBA Pilot
Prepare pupils for CBAs (practice
area/assessment)
September 2013 onwards
Online teacher training available
9 September 2013 onwards
Principal training
11 - 25 September 2013
Assessment Coordinator training
27 September – 11 October
2013
Upload pupil data from SIMS to
WebXchange
Not later than 30 September
2013
Phased Approach : Pupils complete live
assessments in non-transformed
schools.
Pupils complete live assessments in
transformed schools.
7 October – 22 November
2013
Questionnaires issued to schools
November 2013
25 November – 13 December
60
CBA Pilot
• ‘Pre-Flight Checks’.
• Both NINA and NILA assessments should be administered to as
many pupils as possible across years 4, 5, 6 and 7; and
• CCEA, C2k/Capita, and both CBA providers will be monitoring
assessment progress.
• CCEA and C2k/Capita will fully support you throughout the CBA
pilot.
You can contact the CBA helpdesk at any stage on: (028) 9026 1274 or
email: cba@ccea.org.uk.
For technical support contact C2k on: 0870 6011666.
NB – It is essential that only C2k PCs and laptops are used for the
assessments.
61
Assessment Preparation
Teachers should:
• use the online course CBA section of www.nicurriculum.org.uk;
• refer to the NILA / NINA Information for Schools Guidance
Documents;
• be familiar with the NILA / NINA pupil demos;
• explore the NILA practice assessment and the NINA practice area;
• check pupil usernames and passwords;
• ensure that pupils can have a break during the assessment/s;
• ensure that pupils know that NINA allows them to have the
question/s read aloud if required;
• consider resources required; and
• check the equipment.
62
Assessment Preparation
Teachers should ensure that pupils:
• view the pupil demos as part of a class discussion;
• know how to pause or exit an assessment by clicking on the red x
at the top right hand side of the screen and closing the window;
• know how to resume an assessment by clicking “Start Assessment
button”. Pupil should log in again with the same username and
password and see “Continue (first) assessment”; and
• understand the assessments they will be completing and have had
sufficient opportunity to use the NILA Practice Assessment / NINA
Practice Area.
NB - Pupils should be provided with pencils and paper for the NINA
assessment.
63
The Practice Assessment / Practice Area
Be sure to:
• take your pupils through the various question/response types;
• draw attention to the instructions for each question type;
• demonstrate how to answer using a data projector or smart board;
• give your pupils plenty of opportunity to use the practice assessment
(NILA) / practice area (NINA);
• show pupils how to undo/change an answer;
• evaluate how pupils cope with question types and advise; and
• show that an answer can be deleted or changed any number of times
before the ‘Next’ (NILA) / ‘Submit’ (NINA) button is clicked.
NB - Pupils can access the Practice Assessment / Practice Area
without usernames/password. No log-ins required.
64
Part 5
Guidance & Support for CBA
65
CCEA Support and Resources
for Autumn 2013
• Principals’ Information Seminars and Assessment Coordinator
Training
• Online Training Course for all new Year 4-7 Teachers
• C2k Instructions re: SIMS data export file
• CCEA email re: accessing NILA/NINA Admin passwords
• NILA / NINA Information for Schools Guidance Documents,
including guidance for pupils with SEN
• Website, online support & guidance materials at
www.nicurriculum.org.uk
• CCEA - CBA Helpdesk: 028 9026 1274
66
CBA On-line Training
www.nicurriculum.org.uk/cbaonlinetraining
67
68
Website URLs
• NILA
www.niliteracyassessment.org.uk
• NINA
http://www.ninumeracyassessment.org.uk
• Online Course
www.nicurriculum.org.uk/cbaonlinetraining
• NIC Website
www.nicurriculum.org.uk
• NILA Training
https://training.niliteracyassessment.org.uk
69
On line Training site
• www.training.ccea.org.uk
70