THE LEANING TOWER OF PISA
Our Numeracy Tsar, Sir Robert Salisbury, has cited Northern Ireland’s PISA rankings to support his
claim that those who regard our education system as the best in Europe are promulgating a “myth.”
In the past few days Sir Robert has returned to this claim, invoking PISA data to argue that our postprimary schools are failing and, it would seem, that all our difficulties can be traced to academic
selection. At a time when our primary schools are – pace Sir Robert - being hailed as the finest in
Europe, these assertions must surely dismay those charged with securing inward investment in
Northern Ireland. What should add to their dismay is that such claims are entirely erroneous. There
is a profound conceptual error at the heart of the PISA scaling model and the PISA rankings simply
cannot be used to refute the claim that the post-primary element of our education system may also
be world class.
The paragraph which follows is quite technical and I beg the reader’s indulgence in advance , but it
is important that the record be put straight. Very damaging claims are being made about our postprimary schools and I have to make my case precisely in order that the Minister can have its
mathematical accuracy checked.
Underpinning the PISA tables is a probabilistic scaling model developed by Georg Rasch. Alas, there
is a serious mathematical error at its heart; Rasch confuses objective and subjective probability. It is
hard to imagine a more profound conceptual error (the error first appears on pages 10 and 11 of his
1960 book: “Probabilistic models for some intelligence and attainment tests”) and it renders the
PISA ranking all but valueless. It is simply not possibly to treat PISA scores as non-relational
properties of the pupil who took the test, let alone to draw inferences about the quality of the
education on offer in the country in which his or her school is located. The Rasch probability model
itself is entirely at odds with Rasch’s fundamental requirement that proficiency scores are itemindependent.
Hugh Morrison