Does the Department of Education’s audit refute the claim that Northern
Ireland’s education system is world-class?
The Belfast Telegraph (no doubt reflecting the views of the Department of
Education) claims that “The myth that Northern Ireland has a world-class education system has been exploded.” No such conclusion can be drawn from the data published in the March 7 th 2012 edition of the paper, because the Belfast
Telegraph has entirely misconstrued the nature of the data they’ve published.
The Department of Education are currently rank-ordering schools for GCSE pass rates and then drawing a line at 25% in respect of secondary schools and 85% in respect of grammar schools. Measurement error makes such precise distinctions
impossible. Measurement in education is not objective (as Belfast Telegraph readers are invited to think) but irreducibly uncertain. For example, one cannot use numerical data to infer any difference in the education on offer in a grammar school with a 100% GCSE rate (seven GCSEs at A * to C including English and mathematics) and a grammar school with 50% pass rate. It is instructive to point out some of the reasons (some conceptual, some statistical) which render the identification of “educationally unviable” schools an entirely meaningless activity.
(i) The performance of a school cannot be captured in a number. To illustrate the point suppose one picked up the tiniest fragment of soil from the floor of one of the school’s classrooms. Can the qualities of this dead microentity be captured in a number? If it can’t, how much confidence can we have in capturing the academic attainment of living, intentional beings (the pupils) using single numbers? Soil physics
(Philip, 1974) tells us that such a characterisation is impossible. The
Nobel Laureate Sir Peter Medawar (1977, p. 13) uses this argument to dismiss any attempt to capture complex constructs in a number: “The physical properties and field behaviour of soil depend on particle size

and shape, porosity, hydrogen iron concentration, material flora, and water content and hygroscopy. No single figure can embody itself in a constellation of values of all these variables in any single real instance.”
(ii) Goldstein & Speigelhalter’s famous 1996 paper in the Journal of the
Royal Statistical Society A – “League tables and their limitations:
Statistical issues in comparisons of institutional performance” - provides powerful mathematical support for the conclusion that
“definitive judgements on individual institutions” cannot be made
from rank orders of GCSE performance when pairwise 95% overlap intervals based on adjusted residuals are computed. Even for adjusted value-added measures, “finely graded comparisons between
institutions are impossible” (1996, p. 397).
(iii) Neither the Department of Education nor the Belfast Telegraph seems to realise the irreducible uncertainty inherent in the
procedures which generate the data published in the paper.
Cresswell (1986) showed that even the partitioning of a set of marks of reliability 0.90 (the maximum reliability is 1.0) into seven grades results in a quarter of candidates being awarded an incorrect
grade. It is important to note that such grade misclassification is completely unavoidable; the errors cannot be adjusted for by the relevant Awarding Body. In addition, the GCSE grade is computed by incorrectly adding together two entirely different measures: the examination score (which measures what the child can do alone) and the coursework score (which measures what the child can do with
others).

It is important to be clear about what is being claimed here. It could well be true that School A is educationally inferior to School B. It is just that the numbers reported in the Belfast Telegraph, because of their limitations,
simply cannot be used to support such a claim. What stands in the way of inferring “educational unviability” from these numbers is nothing less than the laws of mathematics.
In what sense is the Northern Ireland education system world class?
The Northern Ireland education system is world-class because its approach to teaching has been proven to be superior to all others in one of the most sophisticated studies ever conducted. This method of teaching is particularly effective in respect of children from “difficult” backgrounds. Unlike schools elsewhere in the world, Northern Ireland schools (grammar and secondary) cling steadfastly to a traditional model of teaching which is best described as “Direct
Instruction” (Adams and Engelmann, 1996). In particular, this approach to the classroom has been demonstrated to produce the best possible outcomes for children who live with disadvantage. The project which established the dominance of the approach to teaching which best characterises Northern Ireland schooling is “Project Follow Through.” The progress of 79,000 disadvantaged American children from 180 low-income families was tracked for two decades at a cost (in
1967) of a fraction of a billion dollars. The aim was simple: to identify, once and for all, the method of teaching which could raise the academic performance, particularly that of disadvantaged children to middle class levels.
Direct Instruction won hands down. ... When the testing was over, students in DI classrooms had been placed first in reading, first in math, first in spelling, and first in language. No other model came close. And DI’s dominance wasn’t just in basic skill acquisition. ... DI even did better in promoting students’ self-esteem than several child-centred approaches.
This is particularly striking because a central purpose of child-centred

teaching is to promote self-esteem by engaging children and making them the authors of their own education. ... More recent studies by both the
American Foundation of Teaches and the American Institutes for Research reviewed data on two dozen “whole school” reforms and found once again that the Direct Instruction model had the strongest empirical support. ...
The study concluded that when DI is properly implemented, the “results are stunning,” with DI students outperforming control students along every academic measure. (Ayres, 2007, p. 161)
Project Follow Through identified so-called “constructivist” approaches to teaching as being particularly ineffective, and Hattie’s (2009) synthesis of over 800 metaanalyses relating to academic achievement comes to precisely the same conclusion.
Incredibly, this same constructivist model underpins the Revised Curriculum and is the lens through which the Department of Education and Northern Ireland’s
General Teaching Council view the classroom. In short, in Northern Ireland, the
Revised Curriculum is founded on an approach to the classroom which has been proven to be ineffective with disadvantaged children.
Happily, Northern Ireland teachers, in their everyday professional lives, eschew education theory and take a common-sense view of instruction. It is this ability to ignore theory and stick with their gut instincts that makes Northern Ireland teaching world-class. Countries in which teachers slavishly follow the latest educational theory have very different educational outcomes to Northern Ireland.
The thinking of “learning theorists” has informed policy making in both CCEA and the Department of Education. It has been argued that learning theorists seem to have little regard for “what works” (Lagemann, 2000) and one will search in vain in
CCEA publications for any reference to Project Follow Through. The “Enriched
Curriculum” research carried out in Northern Ireland, which sought to identify the most effective way of teaching under-privileged children – the very same question posed by Project Follow Through four decades earlier - makes no mention of the

American research. Berube & Berube (2007, p. 5) highlight the non-cumulative and often self-contradictory nature of much education research:
[T]he education constituency “followed fad after fad, many of which were implausible and expensive, and none of which had the evidentiary basis.” In the search for innovation, Walberg contended, education theorists ignored
“much of the older research” because it pointed to “traditional, commonsense educational methods.”
While teachers in other countries have adjusted their teaching models to account for developments in education research, in Northern Ireland the dominant model of teaching departs little from the definition of Direct Instruction presented on pages 205 and 206 of Hattie (2009). In summary, it is not a myth that Northern
Ireland has a world class education system. Our education system is world –class because the method of instruction which informs everyday life in the classroom has itself been pronounced world-class. Moreover, the commitment of Northern
Ireland schools (grammar and secondary) to Direct Instruction reveals that it is a myth to suggest that post-primary teachers are not concerned to reduce the tail of under-achievement; given Northern Ireland’s world-class pedagogical approach, it could be argued there are few schooling systems better designed for the task.
References
Adams, G.L., & Engelmann, S. (1996). Research on direct instruction: 20 years
beyond DISTAR. Seattle, WA: Educational Achievement Systems.
Ayres, I. (2007). Supercrunchers. New York: Bantam Books.
Berube, M.R, & Berube, C.T. (2007). The end of school reform. New York: Rowman
& Littlefield Publishers.
Cresswell, M.J. (1986). How many examination grades should there be? British
Educational Research Journal, 12(1), 37-54.

Goldstein, H., & Spiegelhalter, D.J. (1996). League tables and their limitations:
Statistical issues in comparisons of institutional performances. Journal of the
Royal Statistical Society A, 159(3), 385-443.
Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating
to achievement. London: Routledge.
Lagemann, E.C. (2000). An elusive science. Chicago, IL: The University of Chicago
Press.
Medawar, P.B. (1977). Unnatural science. New York Review of Books, 13-18.
Philip, J.R. (1974). Fifty years of progress in soil physics. Geoderma, 12, 265-280.