CBA Autumn 2014
Principal Information Seminar
September 2014
Diane Owens and Anne Cromie
1
Learning Intentions
At the end of the session you will be aware of the:
• context for CBA in Northern Ireland for autumn 2014;
• findings of the recent independent reviews of computer-based
assessments (CBA) commissioned by the Minister of Education;
• developments made to CBA in response to feedback from schools;
• reports and outcomes produced and how they can be used to inform
planning; and
• guidance and support available from CCEA and C2k.
2
Overview
9.00 or 13.00
Registration
9.30 or 13.30
Welcome
CBA Autumn 2014
Recent independent reviews
10.30 or 2.30
Tea/Coffee
10.45 or 14.45
11.45 or 15.45
Developments to CBA
Questions, evaluation & close
3
Focus on Raising Standards
“The introduction of the CBAs is
consistent with the national
strategy to improve pupil
performance within literacy
and numeracy.” p61
“…fit well with an official
assessment strategy that
focuses on assessment for
learning and a wider national
strategy to reduce inequity in
student outcomes.” p61
OECD Review of Evaluation and Assessment in
Northern Ireland Education 2013
4
Recent Evaluation and Review
5
Evaluation and Review
of CBA Since 2012
Standard
Evaluation
Commissioned Review
Additional
Annual and
ongoing
evaluation by
CCEA
1. CCEA review the operation of the
new assessments.
2. ETI review to determine how
effectively schools make use of
assessment information.
OECD Review
3. Independent review of the technical
issues - why they occurred and
what lessons can be learnt for the
future.
4. Independent review of policy and
practice.
5. Department review the policy of
statutory CBA.
6
Evaluation and Review
of CBA Since 2012
• Covered all aspects from procurement to operation
– Phased approach in autumn 2013
– 2014 mini trial
– Phased approach in autumn 2014
• Overall findings
– Consistent
– Reinforce key messages
• Since 2012:
– 184 schools chose to use CBA in 2013
– 194 in 2014
– 105 2012-2014
7
Moving Forward
Procurement Cycle
• Five year procurement cycle
Technical Support/ Transformed Primary Network
•
•
•
•
•
Continued partnership – CCEA/C2k/Capita/suppliers
Enhance CCEA/C2k helpdesk
Monitoring network stability
Ensure ICT equipment set up for CBA
Pre-flight check
– Support CBA on transformed network
8
OECD Review
9
OECD:
Particular Strengths of CBA
OECD has identified the following as particular strengths:
• Standardised to NI school population
• Aligned to revised curriculum
• Adaptive (difficulty of the questions adapts to the level of the pupil)
– reduces the risk of demotivation
– provides nuanced, fine-grained profile of the pupil’s performance.
• Consistent with the national strategy to:
– improve pupil performance in literacy and numeracy
– focus on assessment for learning
10
OECD:
Main Benefits of CBA
OECD has highlighted CBA as a reliable basis for…
•
•
•
•
•
assessing progress and learning needs
planning teaching to meet learner needs
supporting self-assessment and individualised target setting
reducing the workload of teachers as no marking is involved
providing teachers with clear and reliable information for parents on
their children’s strengths and weaknesses
11
OECD:
Learning for CBA
OECD identified the following regarding CBA:
• Technical challenges
– procurement process
– screen freezes/sound
– physical dexterity (keyboard and mouse)
• Feedback of results
– adaptivity
– delay in standardised scores
– parent reports
• Lack of continuity
–
–
–
–
procurement process
InCAs
longitudinal data
two providers
• Timeframe
12
Ministerial Statement
• OECD Review positive
‘The practice of having this sort of universal, formative
assessment in primary school, mapped to our own curriculum
and delivered at the start of the academic year, is noted with
approval by the OECD’
John O’Dowd, Oral Statement (11 March, 2014)
13
Legislative Context
In an email to schools on April 10, 2014, the Assessment and
Qualifications Team (DENI) noted that:
• NILA and NINA will not be specified for mandatory use in 2014/15;
• the Department expects schools to continue to carry out diagnostic
assessment using an assessment, or assessments, of their choice;
and
• the results of this assessment should be used as the basis for
engagement with the pupil’s parent/guardian.
14
NILA and NINA
15
Roles and Responsibilites
Role
Responsibility
Principal
Overview of school CBA
• accessing the school username and password
• arranging the SIMS upload to WebXchange
• allocating the CBA administrator role
Administrator
Manages at school level
• teacher passwords
• SEN/HI Entry Points
Teacher
Manages at classroom
level
•
•
•
•
Classroom assistant
Supports pupils
• continuation of general classroom good
practice
pupil usernames and passwords for class
preparing classroom
practising with pupils
administering assessments
16
SIMS Upload to WebXchange
Role to be assigned by Principal
• Quick to complete
• Complete well in advance of using assessments
• Need to have:
– Instructions sent to principals (approx. 26 August)
– School username and password – Principal can access this from ‘Principal
Documents with MY-SCHOOL area of C2k exchange (school username DENI
number)
• Does not affect use of practice assessment/area by pupils
• Last date to complete is 26 September 2014
If you experience any difficulty with SIMS/Webxchange or
accessing Principal Documents, please contact the C2k Service
Desk on 0870 601666.
17
Moving Forward
with the CBA 2014
Admin passwords available to schools
11 August
Preparation instructions provided to schools
26 August
Manuals and posters delivered to schools
5 September
Phase 1 assessments completed
8-12 September
Phase 2 assessments completed
15-18 September
Final date for SIMS upload
26 September
Phase 3 assessments completed
29 September-12 December
Evaluations live
8 September – 12 December
18
NILA and NINA
What do they have in common?
• May be completed only by Year 4 to Year 7 pupils
• Web-based and pupil responses to questions are saved
automatically online
• Adaptive and not time-bound
• Longer a pupil takes to answer the longer the assessment will take
• Can be paused and re-started from that point
• Available in English & Irish
• Special Entry Points available
• Pupils cannot repeat assessments
• Similar tables and reports
• Reports and outcomes available within 48 hours of completion
19
20
Focus on Skills
Aspect of Literacy Skills Focus
Pupils’ ability to …
Listening
Information
Retrieval
Listen to and understand information
and ideas.
Reading
Sequence
Understand information and ideas
Information
Find and use information
Features of
Texts
Understand different types of texts
Opinions
Respond to different layers of
meaning
Writing
Punctuation Punctuate text
Spelling
Recognise accurate spellings
Grammar
Use grammar accurately
21
The Literacy Assessment
•
•
•
•
•
Single assessment
No time limit
Most pupils complete 46 questions
Average time of 40 minutes
Four difficulties of questions
– age appropriate for the pupil (=), for a younger pupil (<), an older pupil (>) or an
even older pupils (>+)
• SEN Entry Point starts with (<)
• Question number is displayed at the bottom of the screen
• 8 different answer techniques
– cloze, drag and match, drag into order, drag punctuation, drag to hot spot, pick
multiple, pick one and select word
• Practice assessment
22
Adaptive Routing in NILA
23
24
Focus on Skills
Strand 1
Skills Focus
Pupils
Pupils’ability
abilityto
to…
…
Number
Facts
Counting & Understanding Number;
Facts and Mental Arithmetic
Operations
Operations with Whole Numbers,
Integers and Decimals
Fractions, Decimals, Percentages,
Ratio & Proportion
Problems – including Money,
Function Machines & Algebra
Shape and Space
Fractions
Problems
Strand 2
Shape and
Space
Measures
Handling Data
Measures
Handling Data
25
The Numeracy Assessment
• 2 strands:
– Strand 1 Understanding & Using Number
– Strand 2: Shape & Space, Measures and Handling Data
•
•
•
•
•
Neither has a fixed time for completion
Each strand assessed using an adaptive approach
Pupils take an average of 20 minutes per strand
Only mental arithmetic questions are timed
Irish Medium version only available to IM schools or those with IM
units
• Pencil and paper should be available to pupils
• Practice area
• Voiceover available for every question
26
Adaptive Assessment
• Start in their target year - one year below their current school
designated year e.g. Year 3 questions for a Year 4 pupil
• SEN Entry Point is 2 years below target year
• 3 difficulties of questions
– Easy, medium and difficult
•
•
•
•
•
First question is a ‘medium’ one
First set of 6-8 questions assesses general numeracy
Move up, down or stay in the year
Move up or down can be reversed
Process repeated until assessment determines they are answering
questions appropriate to their ability
27
Adaptive Assessment
28
Enjoy!
See you at 10.30/14.30
29
CBA Developments
30
Developments
For Autumn 2013
SEN Entry Point
Both
HI Entry Point
NILA
Skills Focus Report
NINA
Year Group Report
NINA
Improved Parent Report
Both
SS and AROs available within 48 hours
Both
Improved IM selection
Both
Modified pupil checklist
Both
Improved assessment monitoring
NINA
31
Parent Report
32
Special Entry Points
SEN (Special Educational Needs) Entry Point:
• for pupils experiencing difficulties with literacy/numeracy
– quicker access to easier questions
– more positive and potentially shorter assessment experience
– offers same access to the full range of questions as other pupils
HI (Hearing Impaired) Entry Point:
• for pupils who are receiving support for a hearing impairment
– removes the Listening aspect of the assessment
– allows pupils to be assessed in Reading and Writing
Use of the Special Entry Points will not be obvious to pupils.
33
Qualifying for
Special Entry Points
The teacher and the Senior Management Team must:
For the SEN Entry Point
• reasonably expect that the pupil is performing two or more years
below their chronological age; and
• be in agreement that an SEN Entry Point is appropriate for the pupil.
For the HI Entry Point
• have evidence that the pupil is receiving support for a Hearing
Impairment; and
• be in agreement that an HI Entry Point is appropriate for the pupil.
Only the Administrator can set a Special Entry Point
34
Developments
For Autumn 2014
Software adapted to operate on the C2k transformed network
Both
Improved SEN Entry Point selection
NILA
Improved HI Entry Point selection
NILA
Skills Focus Report
NILA
Question number added to the bottom of the screen
NILA
Improved filtering of reports
NILA
Reduction in question techniques
NINA
Improved practice area
NINA
35
Transformation
Transformation of C2k network
• Impacts all primary and special schools
• Close liaison between CCEA, C2k, Capita and suppliers
• Thorough piloting:
– December 2013
– June 2014
– Phased introduction Autumn 2014
36
Computers
• Only use pupil workstations
• Teacher laptops cannot be used
• Connect to the C2k network at least 30 minutes
before CBA use
• Computers not been connected to the C2k network
recently may take longer to update
• Pupils must first log into the C2k network with their
individual C2k username and password.
• Multiple pupils logged on with the same username
and password will not be able to access NILA and
NINA.
37
Check the Sound
NINA
• Go to the practice area
• Select any practice question and
• Click the sound icon
• You should be able to hear the sound play
NILA
• Go to the practice assessment
• Click the sound icon
• You should be able to hear the sound play
38
Accessing CBA
Teacher ‘MY-SCHOOL’ homepage
• ‘View All’
• ‘Curriculum Links’
• ‘NILA’ or ‘NINA’
39
Accessing CBA
Pupil ‘MY-SCHOOL’ homepage
• ‘My Links’
• ‘NILA’ or ‘NINA’
40
Pre Flight Check (PFC)
Key Points
• Simple and user-friendly resource for schools
• Automatically runs when pupils click on NILA or NINA link via their
‘MY-SCHOOL’ homepage
• Completed prior to commencing assessments
• Checking correct versions of software are installed
• In case of incorrect set-up - provide guidance and advice to schools
• When successful takes you straight to the assessment homepage
NINA
http://preflightcheck.ccea.org.uk/nina.html
NILA
http://preflightcheck.ccea.org.uk/nila.aspx
41
PFC Help
Identify Issues
• Every time the PFC runs CCEA is notified
• Gives us frequency of information
• Completing the pop up gives details of:
– school,
– workstation
– teachers comment
• Help identify patterns
• Need to include both DENI number and workstation number – little
sticker on computer
42
Improved Entry Point Selection
43
Individual Entry Point Selection
for NILA
44
NILA Skills Focus Report
45
Monitor Assessments
NILA EM completed
NINA completed
NILA EM started
NINA started
NILA IM completed
NINA not started
NILA IM started
NILA – to help monitor progress during an assessment the question
number is displayed at the bottom of the screen (most pupils answer 46
questions).
NINA – each strand takes approximately 20 minutes.
46
Reduction of
Question Techniques
Pupils are
allowed to
use pencil
and paper
47
48
During an Assessment
Question not fully loaded (images/buttons are misplaced/missing)
Step 1: If this issue continues for more than 10 seconds click on ‘F5’
once.
Step 2: If the problem persists for this user, close Internet explorer (IE)
and get the pupil to log back into their assessment.
Step 3: If the problem is still not resolved, please remove this device
from the assessments and use another workstation if available.
Step 4: If the problem remains, contact the C2k Service Desk urgently
on 0870 6011666.
49
Phoning C2k
Please make a note of the following information before calling the C2k
Helpdesk:
•
•
•
•
•
•
date and time of issue
assessment being attempted (NINA or NILA)
description of issue
workstation ID
pupil ‘C2k / NINA / NILA’ account user name
if possible please take a screenshot.
50
Outcomes and reports
51
What NILA / NINA Tables and
Reports are available?
• NILA / NINA Results Tables provide schools with Age-Related
Outcome (ARO) Ranges and Standardised Scores (SS). These are
available for individual pupils, classes, year groups and at whole
school level. These build over time to show additional data as it
becomes available for new academic years.
• NILA / NINA Parent Reports contain the results table for an
individual pupil, showing only the Age-Related Outcome (ARO)
Range.
• NILA / NINA Objectives Reports provide specific information about
the questions asked and whether the pupil answered correctly or
incorrectly.
52
Results Table (School,
Year Group and Class)
53
Results Table (Individual Pupil)
54
Skills Focus Reports
55
Using Outcomes
CBA “… are intended for diagnostic purposes and the assessment data
is primarily used for:
• Assessing pupil progress and identifying their learning needs
• Supporting self-evaluation and target-setting
• Helping teachers shape their teaching to address the learning needs
of their pupils
• Providing information for parents on their children’s strengths and
weaknesses within literacy and numeracy.”
OECD Review of Evaluation and Assessment in Education (2013)
p.54
56
CCEA Support and Resources
for Autumn 2014
• Face-to-face training (10 September - 1 October)
• Online Training Course (From 25 August)
• Preparation instructions – SIMS upload, school username/password
• NILA / NINA Guidance Documents, including guidance for pupils
with SEN
• ‘How to change an answer before you submit it in NINA’ poster
• Pupils demos and scripts
• NINA practice area
• NILA practice assessment
57
Websites and Helpdesks
Resources and Information for Schools
• Latest information about CBA and online resources for schools on
www.ccea.org.uk
Computer-Based Assessment Helpdesk
Monday – Friday (except for public holidays)
Email cba@ccea.org.uk
Hardware/ IT Difficulties or help with computers/laptops
• C2k helpdesk 0870 6011666
• C2K Exchange (desktop icon)
58
Questions, Evaluation and Close
59