What can social science contribute to
educational policy?
Evidence-Based Policies and
Indicator Systems
July 2006 London
Judy Sebba University of Sussex
j.c.sebba@sussex.ac.uk
We assume that:
• we understand policy-making sufficiently to
influence the process
• policy-makers are receptive to using evidence
• robust, reliable and valid evidence is available
• the highest quality research has greatest
influence
• research reports are accessible and draw out
implications for policy
Key influences in whether research
gets into policy
•
•
•
•
•
ministers’ own experiences
charismatic individuals/ studies
timing
role of the press
think tanks - creating a culture of research in
tension with culture of ideas (Rich, 2005)
• quality of research
• quality of reporting
The case of MMR: how much further damage?
• Wakefield published Lancet paper (1997) claiming
link between MMR vaccine & autism/ bowel disease
following tests in 12 children
• Many subsequently decided not to have MMR
• Wakefield paid (by lawyers) to find evidence to
support claim of damage by vaccine
• Wakefield also filed patent applications for new,
alternative single measles vaccine & potential
treatments for bowel disease
• Horton (editor Lancet) later issued a statement
acknowledging potential conflict of interest & study
had "fatal flaws" - should never have been published
• SR (Demicheli et al, 2005) concluded causal link
between vaccine & autistic colitis not proven
What can we learn from this?
• Fallibility of systems of Quality Assurance peer review & editing are isolated, untrained,
unrecognised (RAE, etc), unpaid roles
• Power of one guru (with media) or one study
needs to be challenged - e.g. reviews
• Role of press (& other mediators) spicing up,
manipulation, accuracy, etc
• Pressures to be fraudulent, exaggerate,
publication bias - (Cooper et al, 1997)
• Vested interests - e.g. MMR - SR identifies
‘What’ works is not enough: the example of
higher education
• Meta-analysis of 58 studies of the
relationship between research and
teaching in universities demonstrated
no relationship
(Hattie, J. & Marsh, H.W. 1996 Review of Educational Research)
• Students perceived clear benefits from
staff research including staff
enthusiasm and institutional reputation
(Jenkins, A. et al, 1998 Studies in higher education)
Formative assessment: a case of
research-informed policy & practice
• Evidence is robust - large effect sizes
replicated
• Practical examples in different contexts - see
OECD report
• Research has been successfully disseminated
- mediators were teachers?, academics, LEAs
• Policy-timing became right - fits ‘personalised
learning’ agenda
Lords Hansard text for 20 Feb 2001 (210220-45):
Lord Morris of Manchester moved Amendment
Lord Morris stated: In 1997, a review of research
on inclusion conducted by Judy Sebba for
Barnardo's found:
Pupils with identified difficulties or disabilities
appear to benefit educationally from schools
developing inclusive education by making
significant gains in reading, language, study skills
and living skills. Pupils who do not have identified
difficulties or disabilities appear to attain as good
or better standards ........
‘Too much testing is torture for girls’
(Daily Mail November 28, 2002)
The Eppi systematic review from which this
finding was extracted says:
• relationship between self esteem &
achievement increased after introduction of
the tests
• teaching style affected by tests
• repeated practice tests reinforce low self
image of lower achieving students
• students (particularly girls) show high level of
test anxiety
Improving the utility of research through the
research process
By being transparent/subject to public scrutiny,
systematic reviewing is giving users greater
access to research:
• identifying the relevant questions
• exposing ethics in carrying out research
• debating interpretations of findings
• providing mediation to increase access
Hence, both the generation & use of knowledge
are being considered
Role of think tanks in research mediation in US
(adapted from Rich, 2005)
Think Tanks: independent, nonprofit research
organizations.
• Fourfold increase in their numbers since 1970
• Majority of new think tanks are ideological
• Staff more likely to have media or political
experience than policy or research degree
• Prioritise publicity & marketing
What do mediators do?
(Fielding et al, 2005, p.103)
For transferring practice successfully, Brown
and Duguit (2002) recommend appointing an
‘organisational translator’ - someone ‘who
can frame the interests of one community in
terms of another community’s perspective’.
Such people are, unsurprisingly, rare. ...A
significant feature of ....research such as that
of Rudduck et al (2000) and ....Rudd et al
(2001) points to the importance of both
internal and external brokers or coordinators.
Maximising use of research
• Build use of research into infrastructure of
policy-making
• ‘educate’ policy-makers to develop realistic
expectations & ensure on-going involvement
of research specialists
• bring researchers, users and policy makers
together to develop dialogue throughout
• set up direct contact between ministers and
researchers but researchers need briefing
• seek to establish teamwork between
researchers, policy-makers and press
Receptivity to research: the public,
the professionals & the policy-makers
As a society we have been challenged by
issues such as BSE and genetically modified
food which highlight the need for a
scientifically literate population
Pilkington, M. & Stuart, M. (2001) Science for active
citizenship: the challenge for lifelong learning Journal of
Access and Credit Studies
Paradigm wars will not bring about social
justice or justify public expenditure
The goal of an emancipatory (social) science
calls for us to abandon sterile word-games
and concentrate on the business in hand,
which is how to develop the most reliable and
democratic ways of knowing, both in order to
bridge the gap between ourselves and others,
and to ensure that those who intervene
in other people’s lives do so with the
most benefit and the least harm.
(Oakley, A. 2000, Experiments in Knowing, p.3)
References
• Brown, J. S. & Duguid, P., (2002) ‘Organizing Knowledge’ in
Little, S., Quintas, P. and Ray, T. (eds) Managing Knowledge.
London, Sage
• Cooper, H., DeNeve, K.& Charlton, K. (1997) Finding the missing
science: The fate of studies submitted for review by a human
subjects committee. Psychological Methods, 2, 447-452
• Fielding et al (2005) Factors influencing the Transfer of Good
Practice. DfES
• Rich, A. (2005) Think tanks, public policy and the politics of
expertise. Cambridge: Cambridge University Press
• Rudd, P et al (2001) Beacon Schools: Further External
Evaluation of the Initiative Slough: NFER
• Rudduck, J. et al (2000) Schools Learning From Other Schools:
Co-operation in a Climate of Competition Research Papers in
Education 15 (3) pp 259-274