Use of administrative
databases in the compilation
of academic value-added of
schools in Hong Kong
LEUNG Chun-wai, Chris
Census and Statistics Department, Hong Kong,
China
政府統計處
Census and Statistics Department
Introduction
• Public examination results are important
indicators of absolute standards.
• They may not be good indicators of school
effectiveness as they are often influenced by
the characteristics of the students, most
notably their prior attainment.
• The Education Bureau (EDB) has developed
a value-added system to measure the
extent to which schools have contributed to
the levels of attainment of their students.
政府統計處
Census and Statistics Department
Value-added System
• OECD Definition:
– “the contribution of a school to students’
progress towards stated or prescribed
education objectives (e.g. cognitive
achievement). The contribution is net of other
factors that contribute to students’ educational
progress.”
政府統計處
Census and Statistics Department
Concept of Value-added
Student actual attainment = Student predicted attainment +
政府統計處
(school residual + student residual)
Census and Statistics Department
Mathematical Form of the Model
• yij = β0 + β1x1ij + β2x2ij + … + βnxnij + (uj +
eij)
• yij : Normalised HKDSE Examination score
• x1ij: Academic Ability Index (AAI) score,
which measures the prior attainment.
• x2ij … xnij : Other control variables:
– School average AAI
– Gender
– All girls school
– All boys school
政府統計處
Census and Statistics Department
5
Mathematical Form of the Model
• yij = β0 + β1x1ij + β2x2ij + … + βnxnij + (uj + eij)
• uj : A residual term for school j, indicating the
school effect on performance (i.e. school
value added)
uj~N(0, σu2)
• eij : A residual term for student i in school j,
indicating that part of the student's
performance that could not be explained by
factors already included in the model
eij~N(0, σe2)
政府統計處
Census and Statistics Department
6
Mathematical Form of the Model
• Given that the examination scores have
been adjusted statistically for the factors
known to influence student performance,
the differences among schools can
reasonably be assumed to reflect
differences in their effectiveness.
政府統計處
Census and Statistics Department
7
Interpretation of value-added
• Value-added is also
presented in stanines
(groups from 1 to 9)
so that schools can
quickly compare their
value-added with that
of others.
政府統計處
Census and Statistics Department
Stanine
%
9
4
8
7
7
12
6
17
5
20
4
17
3
12
2
7
1
4
High
Performance
8
Use of administrative databases
• Hong Kong Diploma of Secondary
Education (HKDSE) Examination
Database
– Taken by most Hong Kong students at the end of
Secondary 6.
– Standards-referenced reporting: Written examinations
and school-based assessment (SBA)
– HKDSE qualification is recognised by the Universities
and Colleges Admissions Service (UCAS) in the UK
政府統計處
Census and Statistics Department
9
Use of administrative databases
• Hong Kong Diploma of Secondary Education
(HKDSE) Examination Database
– Some 65 000 day school students took part in the
HKDSE Examination in 2014.
– Database contains the scores and grades of
students in every subject entered.
– Value-added scores can be compiled for
individual subjects as well as for subject groups
such as the “Core 4” subjects (viz. Chinese,
English, Mathematics and Liberal Studies) and
the “Best 5” subjects.
政府統計處
Census and Statistics Department
10
Use of administrative databases
• Student Database
– contains students’ Academic Ability Index
(AAI) at Secondary 1 as their measure of prior
attainment.
– AAI is compiled based on the internal
assessments of students in their primary
schools, moderated by the academic
performance of the primary schools in the
territory.
政府統計處
Census and Statistics Department
11
Use of administrative databases
• Student Database
– To capture the effect of peers on HKDSE
performance, the average AAI of all students
in a secondary school is computed and
entered into the value-added model.
– Other information present includes
demographic variables, such as gender, date
of birth, nationality and language spoken, as
well as the records of student changing
schools.
政府統計處
Census and Statistics Department
12
Use of administrative databases
• School Database
– Information includes the mode of finance,
geographical location and whether the school
is a single-sex or a mixed-sex school.
政府統計處
Census and Statistics Department
13
Potential source of control
variables
• Socio-economic status (SES) is a commonly
adopted control variable in value-added
models worldwide.
• In Hong Kong, financially disadvantaged
families can apply for the Comprehensive
Social Security Assistance (CSSA) Scheme
and the School Textbook Assistance (STA)
Scheme. These databases serve as
potential sources of control variables for the
value-added model.
政府統計處
Census and Statistics Department
14
Potential source of control
variables
• Financial aid data are sensitive, and
consent from the financial aid recipients
needs to be obtained.
• Financial aid data can only identify
students with low SES, but are unable to
differentiate students at the high end of the
SES spectrum.
政府統計處
Census and Statistics Department
15
Potential source of control
variables
• Alternatively, the median monthly household
income of the District Council Constituency
Area (with an average population of around
17 000 each) that a student lives in can serve
as a proxy of SES.
• Drawbacks:
– In areas with diverse household income, the
median monthly household income is a poor
proxy of SES.
– With the construction of new housing estates, the
profile of household income in an area may
experience great changes in the 5-year intercensus / by-census period.
政府統計處
Census and Statistics Department
16
Conclusion
• Value-added provides an alternative
indicator of school effectiveness in
addition to the conventional public
examination results.
• To further enhance the quality of the
value-added model, it would be desirable
to review the value-added model, including
its control variables from time to time.
政府統計處
Census and Statistics Department
17
References
• Education Bureau (2013). “Schools Value-added Information
System: Technical Manual”
• OECD (2008). “Measuring Improvements in Learning Outcomes
Best Practices to Assess the Value-Added of Schools: Best
Practices to Assess the Value-Added of Schools”
• Hong Kong Examinations and Assessment Authority (2011).
“Grading Procedures and Standards-referenced Reporting in the
HKDSE Examination”
• Hong Kong Examinations and Assessment Authority (2013).
“Information on School-based Assessment”
• Value-Added Research Center, Wisconsin Center for Education
Research (2010). “Academic Growth over Time: Technical Report
on the LAUSD School-Level Model 2009-2010”
政府統計處
Census and Statistics Department
18
Thank you
政府統計處
Census and Statistics Department