Does the Index of Community
Socio-Educational Advantage have
Systematic Bias?
Barbara Preston
Barbara Preston Research
21 Boobialla Street
O’Connor ACT 2602
barbara.preston@netspeed.com.au
Ph. 02 6247 8919
AEU, AGPPA & ASPA NATIONAL SYMPOSIUM
Advice for Ministers and ACARA on NAPLAN, the use of student data,
My School and league tables
Friday 23 July 2010
Aerial Function Centre, University of Technology, Broadway, Sydney
Concern only with whether ICSEA has
systematic bias against public schools
Not dealing with other apparent biases:
– in favour of selective schools of all sorts and
against the schools ‘selected’ students would
otherwise attend
- in favour of high fee private schools and
against low fee schools
- against schools that actively seek to serve the
disadvantaged and ‘unacademic’.
Not dealing with the many other issues
around My School and NAPLAN.
The My School website and the
presentation of schools’ NAPLAN data
ICSEA and the presentation of
statistically similar schools
The high stakes involved
My School website
A central element of the My School website is the comparison between
‘statistically similar’ schools regarding their results on the National
Assessment Program – Literacy and Numeracy (NAPLAN).
On each school’s web page is a chart setting out the school’s average
NAPLAN results. Immediately below are the average scores for
‘statistically similar schools’ (SIM) and for all Australian schools (ALL).
There is a bar above each of the SIM and ALL scores that indicates by
its colour whether the particular school is ‘substantially above’ (green),
‘above’ (pale green), ‘close to’ (white), ‘below’ (pink), and ‘substantially
below’ (red) ‘statistically similar’ schools and all schools respectively. In
addition, a page is linked to each school’s main page that lists up to 60
‘statistically similar’ schools and provides similar data and colour
coding.
Thus, a scan of the dominant colours can quickly indicate to the viewer
whether or not the school with which they are concerned is
substantially above or substantially below ‘statistically similar’ schools,
or somewhere between.
Index of Socio-Educational Advantage (ICSEA)
Schools are classified as ‘statistically similar’ if they
have a similar ICSEA score.
ICSEA has several components, the most important of
which is an index based on 14 variables, covering
various income, occupation, education level and other
matters for all individuals/households in ABS Census
collection districts (a CD has around 250 households),
according to 2006 Census data. An index score is
calculated for each CD, and the value for a school is
derived from the weighted average of the scores of the
CDs of students’ home addresses.
The final ICSEA value for a school is adjusted for
remoteness and the proportion of Indigenous students,
and a small proportion of schools have additional
adjustments if considered warranted.
High stakes
... if some walk with their feet that’s
exactly what the system is designed to
do; that is to make sure that school
communities are being responsive to
the legitimate high expectations of
parents and kids ...
(Kevin Rudd 2009)
Transparency is critical. To improve
schools that are failing their students
we need information. And we want
parents to drive change ...
(Julia Gillard 2009)
The nature of Australia’s schooling
system:
•
•
Large private sector
Private sector much higher SES
than public sector
Share of total enrolments
From
1890 until
1980 the
public
sector’s
share
was
within 4%
of 80%.
It’s now
14%
below.
Year
Government
Nongovernment
1890
83%
17%
1900
80%
20%
1940
79%
21%
1954
78%
22%
1964
76%
24%
1971
78%
22%
1976
79%
21%
1981
78%
22%
1986
74%
26%
1991
72%
28%
1996
71%
29%
2001
69%
31%
2006
67%
33%
2009
66%
34%
Post
WWII
peak
Dynamics of selectivity
Allowing selectivity sets in train a vicious circle of increasing
social segregation and the residualisation of the
comprehensive and inclusive. (Selectivity includes
excluding those not wanted – those who are disruptive,
costly to educate, or simply low achieving.)
The realities of our schooling structures are complex and
the politics are difficult.
It is relationships that matter:
• between the private and the private
• between the selective/specialist & the comprehensive
• between the high fee and low fee.
Dynamics of selectivity
Low SES students achieve less in low SES schools than
they do in high SES schools – social segregation in
schooling exacerbates differences in educational outcomes.
Being a ‘positional good’ is inherent to schooling (in a way
that it is not to health).
Those already in or aspiring to high SES schools have a
vested interest in increasing social segregation (though this
may be countered to some extent by their commitment to
broader social justice and quality education for all). They
tend to be the articulate and politically powerful.
The private sector is inherently ‘selective’
(some parts more selective than others)
• Schools can be located where desired (with
some constraints and pressures from the
particular sector’s clientele community).
• Enrolment numbers for the school as a whole
and at grade levels can be planned and assured.
• Individual students can be selected on an
individual basis according to criteria of the
school’s choosing (with some constraints).
As the OECD noted in School: A Matter of
Choice (1994), it is often schools that choose
their pupils rather than the reverse.
Type of school attended by primary school students living in
localities from most disadvantaged to most advantaged
- ABS Census Collection Districts, deciles of SEIFA Index of Education & Occupation, 2006
90%
80%
70%
60%
50%
Govt
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
Disadvantaged
2
3
4
5
6
7
8
9
10
Advantaged
Type of school attended by secondary school students living in
localities from most disadvantaged to most advantaged
- ABS Census Collection Districts, deciles of SEIFA Index of Education & Occupation, 2006
90%
80%
70%
60%
Govt
50%
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
Disadvantaged
2
3
4
5
6
7
8
9
10
Advantaged
Percentage of students in government, Catholic and other
nongovernment primary schools with low, medium or
high family incomes, Australia, 2006
Low (< $1000/wk)
Medium ($1000-$1699/wk)
High (>$1700/wk)
P rim a ry s c h o o ls
60%
50%
40%
30%
20%
10%
0%
G o ve rn m e n t
C a t h o lic
O t h e r N o n g o ve rn m e n t
A ll s c h o o ls
Source: Preston 2007
Percentage of students in government, Catholic and other
nongovernment secondary schools with low, medium or
high family incomes, Australia, 2006
Low (< $1000/wk)
Medium ($1000-$1699/wk)
High (>$1700/wk)
Secondary schools
60%
50%
40%
30%
20%
10%
0%
Government
Catholic
Other Nongovernment
All schools
Source: Preston 2007
Indigenous secondary students
• 84% of all Indigenous secondary students attend
government schools.
• While 89% of LOW income Indigenous secondary students
attend government schools, only 69% of HIGH income
Indigenous secondary students attend government schools.
• In contrast, while only 10% of all Indigenous secondary
students attend Catholic schools, 20% of HIGH income
Indigenous secondary students attend Catholic schools.
• Similarly, while only 6% of all Indigenous secondary
students attend other nongovernment schools, 10% of
HIGH income Indigenous secondary students attend other
nongovernment schools.
(Pattern is similar at the primary level.
This data does not include students not living at home.)
Source: Preston 2007
Area based measures of SES & the
‘ecological fallacy’ – where is the debate?
Systematic bias in ICSEA?
Any systematic bias can only be investigated
indirectly:
• Census data is only available for students by
type of school attended (and level), not actual
schools.
• ACARA has not made available the area-based
index component of ICSEA.
• Students can be classified by CD in which they
live, by family income, whether they have an
internet connection at home, etc.
• CDs can be classified according to various
indexes of disadvantage / advantage such as
ABS SEIFA indexes.
Findings on family income:
1. In even the most disadvantaged CD (of
around 250 households) there are likely to
be some HIGH income families, and the
children in those families are more likely to
attend Catholic or other nongovernment
schools than their neighbours in LOW
income families.
2. In even the most advantaged CD there are
likely to be some LOW income families, and
the children in those families are more likely
to attend government schools than their
neighbours in HIGH income families.
Percentage of all government, Catholic and other
nongovernment primary students living in each decile
of disadvantage* who have HIGH family incomes
90%
80%
70%
60%
Government
50%
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
Disadvantage
2
3
4
5
6
7
8
9
10
Advantage
* Index of Education and Occupation, all Australian Census Collection Districts.
Source: ABS Census 2006
Percentage of all government, Catholic and other
nongovernment secondary students living in each decile of
disadvantage* who have HIGH family incomes
90%
80%
70%
60%
Government
50%
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
2
Disadvantaged
3
4
5
6
7
8
9
10
Advantaged
* Index of Education and Occupation, all Australian Census Collection Districts.
Source: ABS Census 2006
Percentage of all government, Catholic and other
nongovernment primary students living in each decile
of disadvantage* who have LOW family incomes
70%
60%
50%
Government
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
2
Disadvantaged
3
4
5
6
7
8
9
10
Advantaged
* Index of Education and Occupation, all Australian Census Collection Districts. Source: ABS Census 2006
Percentage of all government, Catholic and other
nongovernment secondary students living in each decile
of disadvantage* who have LOW family incomes
70%
60%
Government
50%
40%
Catholic
30%
Other NonGovernment
20%
10%
0%
1
2
3
4
5
6
7
8
9
10
* Index of Education and Occupation, all Australian Census Collection Districts. Source: ABS Census 2006
Findings on home internet connection:
In CDs from the most disadvantaged to the
most advantaged, students attending
government schools (primary or secondary)
are less likely to have a home internet
connection than their neighbours attending
Catholic or other nongovernment schools.
Percentage of all government, Catholic and other
nongovernment primary students living in each decile of
disadvantage* who have HOME INTERNET CONNECTION
100%
95%
90%
85%
80%
Government
75%
Catholic
70%
Other nongovernment
65%
60%
55%
50%
1
Disadvantaged
2
3
4
5
6
7
8
9
10
Advantaged
* Index of Education and Occupation, all Australian Census Collection Districts.
Source: ABS Census 2006
Percentage of all government, Catholic and other
nongovernment secondary students living in each decile of
disadvantage* who have HOME INTERNET CONNECTION
100%
95%
90%
85%
80%
Government
75%
Catholic
70%
Other nongovernment
65%
60%
1
2
Disadvantaged
3
4
5
6
7
8
9
10
Advantaged
* Index of Education and Occupation, all Australian Census Collection Districts.
Source: ABS Census 2006
Issues with judging schools according to:
1. Improvements over time at a given year
level:
– a school that is performing very badly in the base
year can show great improvement, but still not be
performing optimally (of course the improvement
should be recognised)
– a school that put much effort into improvement
before the base year and was performing very
highly may then have little room for improvement.
Issues with judging schools according to:
2. Improvements of particular students from
one test to the next several years later
– a school that draws from excellent feeder schools
and/or has excellent transition and orientation
programs will have students performing very well
in the first tests, with little room for improvement
before the subsequent tests
– a school that has poor feeder schools and/or poor
orientation will have students that have much
room for improvement between tests.
Campbell’s law
The more any quantitative indicator is
used for social decision-making, the more
subject it will be to corruption pressures
and the more apt it will be to distort and
corrupt the social processes it is intended
to monitor.
(D. T. Campbell, Assessing the Impact of Planned Social Change, 1976, p. 49)
Thank you.
barbara.preston@netspeed.com.au
barbara.preston59@gmail.com