NATIONAL ASSESSMENT
POLICY AND
STANDARDISED TESTING
IN SCHOOLS: SOME
ISSUES TO CONSIDER
Gerry Mac Ruairc
School of Education
University College Dublin
gerry.macruairc@ucd.ie
Overview
• Four core themes
– The broad context and the accountability agenda
– Some issues to be considered in the
implementation of current policy – some limitations
– Bias in testing
– Exploring alternatives
The broad context: Some key themes
• Increased testing at higher states = improved
attainment
• Impact of an outcome driven neo- liberal agenda
• Globalisation and the increasing economisation of
education policy (Lingard 2010)
• Education as a valid part of a comparative and
competitive market driven area of growth
PISA, PIRLS, TIMS, Sunday Times …
• Policy makers know best – and base their
knowledge on undeclared sources
•
A what works approach – transferrable
epistemologies and policy borrowings
Some discourses often silenced
• Standardised testing – a contested
terrain
• Patterns of results - we know this
already!
• Language ideologies underpinning
curriculum, textbooks and
pedagogy
• Language, culture and subtractive
schooling
Rising stakes – imperatives for testing in
Ireland
• Historically practice varied in schools
• Circular 8/99- link between standardized test results
and resources
• Part of the evidence base for tuairisci scoile and WSE
• Increased emphasis on results in evaluations
• Policy development on mandatory testing and
informing parents
• Collation of test results – National Literacy Strategy
• What is next?
National Literacy Strategy / Circular
The strategy and the circular
• Circular 0138/2006 required all schools to
implement standardised testing in English reading
and mathematics at two points in the primary
cycle.
• Now: English-medium schools the period
May/June for all students in 2nd, 4th and 6th
classes with effect from 2012 onwards
• You are requested to ensure that standardised
testing is implemented in your school on an
annual basis in the relevant classes beginning in
May/June 2012 (to facilitate valid comparisons
between schools and types of schools)
• Principal teachers in primary schools are required
to report annually aggregated assessment data
from standardised tests to the board of
management of their schools
• Primary schools will be required to report
aggregate standardised test results to the
Department of Education
• The template for reporting to boards of
management (included in the appendix) will be
used to collect the data – format not yet decided
The tests in context
• Tests should be used like dangerous drugs – handle with
care (Spolsky, 1981)
• Tests still retain a dominant position in terms of assumed
objectivity- enjoys enormous support and trust on the part of
the public and public/state institutions – firmly embedded in
systems
• Evidence reveals that what results is a more intensive
stratification on the basis of class and race rather than what
the rhetoric suggests
• Currently more questions are asked about ‘test takers’ than
about the construction of tests and ‘test makers’ (Willie
2001).
TESTING: Some issues to consider
• High stakes - Impact on practice: international
evidence
– Teach to the test – (Anagnostopoulos, 2005; McNeil
2000; Zigo, 2001; Lam and Bordignon 2001;
Alexander 2009 Darling Hammond 2010).
– Impoverished curriculum (UK Primary Curriculum
Review, 2009) - Wash back effect – narrows the
curriculum
– Avoidance of risk taking and innovative practice
(Williams & Ryan, 2000, Stobart, 2008).
– Negative impact particularly notable in schools
succeeding in connecting curricula and teaching to
the realities of students’ cultures, backgrounds and
economic conditions (McNeil, 2000, Hursh 2008).
IMPACT
Rising
stakes
Narrowing curriculum and pedagogy
Testing and views about good teaching
• Teachers begin to view themselves and their
effectiveness on the basis of student of outcomes on
tests – impacts practice related to testing – 43%
already speak of high correlation between test results
and perceptions of effectiveness (Rogerson 2011)
• DES funded UCD study on effective models of practice
– very strong messages related to what makes a good
teacher within the profession
• Main findings:
• Reflective Practitioner,
• Effective Planning and Management of Learning, Social
and Moral Dimension,
• Love for children/young people,
• Passion for Teaching and Learning
• Important that these are not negated in what is
considered it be a good teacher in the future.
Communication results to parents?
Not too good
just 1 in
reading
You have a 2
in reading
Not sure what it
means but she
said you have a
STEN score of 3 in
reading – that’s
not that good
She said our
Amy only
has a 2 /10
in reading
She got a
6/10 that’s
the top of
the class –
it’s not that
high
She got a
2 in
reading
I thought she
was great at
reading – but
she only got a
1/10
BIAS
• Despite considerable scholarship on bias in
testing there is a significant level of a
‘denial of bias’ among many policy makers
with respect to standardised tests.
• Broadly speaking test bias can occur in
– the design of test items and the language register of the
test
– the use that is made of the test results
– in the manner in which the test is administered.
AND
– The manner in which the test is completed by
children
Administering the test
• Test score pollution practices (Haladyna, Nolen
and Haas 1991; Rogerson 2011)
• One hundred and sixty primary school teachers
completed the questionnaire; 28 were male
(17.5%) and 132 were female (82.5%)
• “I am aware of other teachers engaging in nonstandard practices to raise the scores their
classes achieve 40% agreement
• “Some teachers’ practices around standardised
test administration and correction are a cause of
concern for you” 30% agreement
Table 1: Actions carried out by yourself or that you are aware other teachers have carried out in relation to standardised testing.
80.0%
70.0%
60.0%
50.0%
40.0%
30.0%
Self
Other Teacher
20.0%
10.0%
0.0%
Gave an answer
to a student
Allowed extra
time for test
completion
Encouraged
students to
redo/check an
answer
In the time
Explained
Noted common
leading up to vocabulary from errors made by
testing taught the English test
teachers to
topics knowing
paper
inform future
that such would
teaching
be examined in
the test
Taught items Followed all time Left materials on
from the test
and
view knowing
administration
that such
instructions materials would
exactly
aid students in
the test
16
BIAS: Some strategies used by students
in test completion
1. Means the same as /is the correct word
2. Straightforward Guess:
–Quick Pick
–Eenie, meanie, miney, mo
3. Makes sense: attempts to locate context for
the word in existing linguistic
repertoire/ experience
It makes sense or sounds right
• Stark differences between socio-economic
groups.
• For working class children - little by way of
familiar context for the language.
• Fundamental misunderstandings about the
meaning of word as a direct result of the
particular context in which the child had
previously experienced the word or part of the
word.
It makes sense- working class group
Are dogs domestic animals?
foreign/ friendly/ tame/ sensible
Shelly:
I picked friendly because domestic
means clean from Domestos the bleach
and clean …that’s good and so is friendly
They were unable to subdue him.
Quieten /rescue/assist/replace
Jack:
I put rescue ‘cause I thought subdue
meant to like get him back.
Peter:
I thought he was stuck down the hole
and you get him back.
Jack: To subdue him back to your gang- yeah
that’s it
The Middle Class sense
When the middle class child went to their
linguistic repertoire they often found an
appropriate context for the word
Jill:
Courteous kinda means nice… you know polite
Emma:
I don’t know why I know the meaning… I’ve heard
it like loads of times before.
Eire:
I don’t know where… you read it in books.
Kate:
You hear it around the place.
Rachel:
I picked on the outskirts because I’ve heard the
word before and I kinnda know what it means …
well I do know what it means.
Rachel:
I put permission because they say … in some
books they say like… he took off without his leave,
another word for permission.
it’s
What are we testing?
• The level of resonance between the language used in test
items and the middle-class linguistic repertoire- the
middle-class student in a notably privileged position in
terms of the potential for higher levels of attainment on
the test.
• This high level of linguistic resonance permeates the
whole test experience for the middle-class child, with the
result that each of the strategies used, including simple
guessing, have a great likelihood of success.
• The testing experience for the working-class child
on the other hand, is characterised by varying
degrees of struggle with the linguistic challenges
that the tests presents.
• Only one dimension of the linguistic
heterogeneity is tested.
• Validating the dominant linguistic register and
rewarding those who are immersed in this
register for their success in acquiring literacy of
their own linguistic world.
Significant limitations as a mode of
assessment
• Specific issues for disadvantaged schools in terms
of the broader asymmetrical patterns of
distribution* (following slide)
• In all cases the focus is too narrow and does not
adequately capture the current practice in
relation to literacy
• However, their simplicity, presumed ‘objectivity’,
the ‘validity’ of comparisons they give rise, render
these a formidable force to reckon with
So what do we do…
• A systematic, reliable approach to capturing the
work of the school across all dimensions of
literacy practice.
• Anecdotal is not enough; the legitimacy of the
standard is too strong and too embedded
• School wide development work needed here
• System wide leadership from within the
profession required
• The NCCA document and beyond
• Release the potential within schools to release the
potential within the system in order to
– celebrate the successes
– identify and address the challenges within all school
types schools in an authentic way
Otherwise
• Increasingly become more imprisoned by the
regimes of testing
• Continue to reproduce patterns of failure that can
obscure real success stories
• Impoverish the curriculum, particularly in
disadvantaged schools
• Narrow and reduce our expectations for the
outcomes of schooling, for pedagogy within
schools and for the education system more
broadly