Why aren’t research results better
used by policy-makers?
How Policy-Makers View
Evidence: Lessons from the
RWJF Synthesis Project
Claudia Williams
„
Policy-makers are in information overload
„
Research does not pass the “so what” test
„
Research results are not translated for
policy decisions
AZA Consulting
June 25, 2006
„
1
•
Too technical and indirect
•
Isolated findings that have not been synthesized
Results are not fed into best channels
2
Mismatch between policy and research
Policy-maker
Researcher
Information
Channels
Opinion leaders, lobbyists
and press
Journals and research
conferences
Information
habits
Use people not paper
Peer-reviewed journals
Level of
Information
Often need basic
information
Information is technical
and sophisticated
Format
Won’t read long reports
Reports often 50+ pages
Skim for “story line” and
conclusions
Lead with methodology
and background. Results
last.
Joseph Nye expressed it well
“It's a totally different world, government, from academics. In academic life, there's
no premium on time, the premium is on getting it just right. In government, if you
haven't got the right answer by four o'clock this afternoon when the president
meets with the prime minister, that perfect paper you get in a little bit late is an "F.”
The idea that I would have time to read a thirty page paper -- ! I used to write them,
and I found that I couldn't read them (when I worked in government)…
So this problem of how do you take chaotic reality and try to shape the right
questions, even before you get answers, is very different in the government setting
than in the academic setting. Because in the academic setting there's a luxury,
there's no time limit. You can sort your way through it, figure it out; if you don't
have the answer you go back to the library and look up more data and so forth. In
government you either solve the problem or get the right answer quickly or it
doesn't happen at all. And it's quite a different set of skills. The premium we put on
time makes a huge difference.”
Joseph Nye, Dean of the Kennedy School of Government
1997 University of California at Berkeley Interview
4
Synthesis development process
Synthesis strategies
„
Choose topics carefully
„
Create meaning not noise
„
Synthesize bodies of information
„
Pair research with policy expertise
„
Use active dissemination
6
1
Choose topics carefully
„
Select perennial “thorn in the side” issues
„
Solicit topic ideas from policy audience
•
•
Start with “what is on their plates”
Make sure topics pass “so what” test
„
Certain policy decisions—often big ideological
questions—are not answered or addressed
through better information
„
Work with advisory group to narrow and select
best topics
7
Create meaning not noise
8
Synthesize bodies of information
„
Help organize and manage information,
addressing not aggravating information overload
„
Synthesize bodies of evidence instead of
producing isolated findings
„
Make synthesis visual and skimmable, leading
with conclusions and telling a story
„
„
Policy questions drive the information, not the
intricacies of the research
Organize, structure and make sense of
information, putting it into understandable
frameworks
„
Bring diverse findings onto the same footing
„
•
Use multiple layers so users can read at
different levels
•
Compare and weigh findings
Reach conclusions based on best evidence
9
Pair research with policy expertise
„
This approach does not come easily to
many researchers
„
Best known researcher often not best
synthesizer
„
Requires a team approach
„
The process is iterative and intense
10
Use active dissemination
11
„
Goals of dissemination: share findings and
foster discussion and dialogue
„
Researchers need a way to communicate
more directly with policy-makers
„
Partner with knowledge broker and
convening organizations
12
2
Synthesis framework
Methodological issues
„
Started with notion of a clear bar…but weaker
literature sometimes needs to be discussed
„
Policy staff want to reach their own conclusions
about research findings
„
s to
ult
Res ve
Use mpro ch
I
ear
Res
•
•
• Too long and complicated
• Fragmented
• Unclear and confusing
ENGAGE AND DISSEMINATE
The Problem
The Synthesis
Solution
TRANSLATE AND ORGANIZE
• Foster dialogue
• Actively engage
• Get press attention
Synthesis should not mask details on the
approach, methodologies and bias of underlying
evidence
•
RESEARCH FINDINGS ARE:
• Build conceptual frameworks
• Construct bridges
• Organize around policy issues
Partnerships?
Audience?
BUILD INFRASTRUCTURE
CREATE FORMAT AND DESIGN
• Tracking function
• Credible review
• Advisory group
Compare results
Evaluate methods
Technical information separated out
Topics?
•
•
•
•
Short and skimmable
Objective and balanced
Provocative
Links to other resources
13
Rules to live by
„
Government “think tank” staff are ideal audience
„
Syntheses should lay out areas where the
evidence points to a clear conclusion as well as
areas where there is debate
„
Syntheses need policy as well as research input
throughout
„
Syntheses should be candid, objective and
credible
Contact information
Project
email
web site
synthesisproject@rwjf.org
www.policysynthesis.org
David Colby, RWJF
Claudia Williams, AZA Consulting
phone
571.641.3030
email
cwilliams@azaconsult.com
15
16
3