______________________________________________________
Evidence Use in Education Policy:
A New Perspective on an Old Problem
M. Stephen Weatherford
Lorraine M. McDonnell
University of California – Santa Barbara
W.T. Grant Foundation Conference on the Use of
Research Evidence
Santa Monica, California; January 24-25, 2013
______________________________________________________
______________________________________________________
Introduction / Overview
The legacy of early research on evidence use: two challenges.
Get research to policymakers in a form they can understand and
connect to their practical concerns – “transmission and translation.”
•
•
Problem has been widely recognized and engaged.
Practical innovations on both sides of the research-policy continuum
have gone far toward resolving it.
Persuade policymakers to take up research and make new knowledge
part of their own thinking about the issue at hand – “uptake.”
•
•
Distinct from getting the information.
Addressing it entails thinking about research use in a different way.
______________________________________________________
______________________________________________________
The Problem of Transmission and Translation
Two metaphors are prominent in the literature on research use:
“Speaking truth to power” and “Supply and demand.” They…
•
Portray the challenge as matching a fixed product with an unambiguous
problem.
•
Capture conventional wisdom about the limited use of research in
policymaking.
•
•
•
•
•
Research too pure and focused on questions of abstract theory.
Researchers fail to develop practical implications.
Research packaged in jargon that confuses or alienates potential users.
The politically powerful seek to use research findings for their own
purposes.
Identify the roots of the problem and the solution:
•
Communication is difficult between two separate communities defined by
distinct vocations.
•  Need to strengthen traits that make research more useful for policy and
practice.
•  Availability
•  Relevance
______________________________________________________
______________________________________________________
The CCSS process needed to go beyond showing the
connection between abstract research and local context.
The challenge: design standards that could be implemented in
the classroom and adopted by state boards subject to political
oversight involved…
•
•
•
Melding researchers’ findings with local knowledge of
educational practitioners.
Combining research knowledge across subfields.
Where validated research findings are incomplete, combining
research with other types of empirical information to bridge
the gap between what science knows and practitioners need.
Two defining aspects of the CCSS process addressed these
challenges.
•
•
Reject the dichotomy between research and other evidence.
Set up a process centered around inclusive, reciprocal
communication among researchers, practitioners, and
policymakers.
______________________________________________________
______________________________________________________
From “Research Versus the Rest” to Multiple Types of
Evidence.
Earlier approaches centered on sources with different types of
“home knowledge” – contextual, local, practitioner.
Our approach focuses on the way different types of evidence are
entitled to different sorts of claims on legitimacy or validity,
taking peer-reviewed research as the ideal type.
•
Defining properties of validated research findings: (1) explicit
description of methods and logic of inference; (2) wide sharing of
findings; (3) criticism focused on evidence and logic; (4) critics must
offer evidence and explicit argument; (5) alternative interpretations
must be empirically falsifiable.
Other types of evidence
•
•
•
•
Statistical data
Judgment by experts, about topics within their expertise
Practitioners’ experience
Value-based argumentation
______________________________________________________
______________________________________________________
From “Speaking truth to power” to “Persuasion.”
Builds on recent approaches to the study of implementation that reveal
the role of cognitive and social processes.
•
•
These approaches reject top-down image of “command and control.”
More nuanced model of process – “incorporation,” “sense-making,”
“uptake” – pointing toward the mechanism through which new
evidence would impact current practice.
CCSS process goes beyond “implementation” in four ways that allow us
to observe the structure and influence of a richer set of interactions.
•
•
•
•
Stage of the policy process – defining the problem and formulating standards.
More players and more disparate considerations.
Short time frame.
Process shielded from public criticism, at the most vulnerable early stage.
______________________________________________________
How did this structure influence the use of research and other
evidence in the formulation and adoption of the CCSS?
Elements
•  Typology of evidence types
•  Relaxes the hierarchy of “research versus the rest,” without abandoning
scientific criteria for validity.
•  Structured to foster sustained, reciprocal, critical interaction among
researchers, administrators, and practitioners.
Process – the social psychology literature on persuasion depicts key mechanisms.
•  Early literature emphasized receiving and accepting new information – the
psychological complement to transmission and translation.
•  More recent research shifts from static traits to how people interact with new
information – emphasis on the importance of active, reflective processing.
•  Different levels of information processing elicited by message content and
communication context.
•  Level of information processing is associated with distinctive outcomes:
•  Lower-level effort with implicit attitudes.
E.g., stereotypes, triggered via associative evaluations.
•  Higher-level effort with explicit attitudes.
Propositional reasoning – process connects evidence, premises, and
conclusions via syllogistic form disciplined by consistency checks
with other propositions relevant to the judgment at hand.
______________________________________________________
The CCSS Process Was Structured to Elicit High-level
Processing Effort.
•  Selection of participants – previous achievement, high interest.
•  Shared commitment to common, high-salience project.
•  “Research and evidence-based” – focus criticism on substance, not
ideology.
•  Complex task.
•  Transparent channels for communicating among participants; norms
highlight reciprocity and inclusiveness.
•  Social interaction / feedback – multiple rounds fostered evolution of
organizational culture.
•  “Pragmatism,” “problem-focused.”
•  “Learning from past mistakes.”
•  “Integrate state participants with researchers.”
______________________________________________________
______________________________________________________
Conclusions – Three Implications
•  Although transmission and translation remain a challenge, now effectively
addressed by a variety of organizations.
•  Development of CCSS never involved problems of transmission and
translation.
•  More demanding challenge is uptake and persuasion – creating processes
and incentives for policymakers to use evidence once it has been made
accessible and understandable.
•  Evidence derived from non-research sources can function in the policy
process much like research-based information.
•  Practitioner experience, professional judgment serve a critical role
independent of their potential as sources of political legitimacy.
•  Multiple types of evidence are best concatenated via on-going, problemfocused interaction among participants.
•  Conditions for effective uptake do not emerge naturally; institutional
arrangements that facilitate the use of science in policy play a
crucial role (cf. NRC 2012).
______________________________________________________