ACPE Annual Meeting, April 26 30, 2013, New York
Evidence-Based Management
A New Approach to Teaching
the Practice of Management
Tony Kovner, Michiel Bosman, Eric Barends
EBMgt: My Experiences
 EBMgt as a Process
 Why Don’t Managers use it?
 Learning from Medicine
 Contributions of Organizational Behavior
 Experience in Teaching
 The Politics of EBMgt
EBMgmt: Definition
 “…the systematic, evidence-informed
practice of management, incorporating
scientific knowledge in the content and
process of making decisions.”
(Rousseau 2012)
EBMgt as a process
 Framing the question behind the decision
 Finding the sources of information
 Assessing the accuracy of information
 Assessing the applicability of information
 Assessing the actionability of information
 Determining if the information is adequate
(Hsu and others 2009)
Why don’t Managers use EBMgt?
 Never heard of it. They are using it.
 Cost now more probable than benefits in the
future
 Managers have to be persuaded and trained
 The process doesn’t clearly lead to one-best
way to intervene
 Politics impacts interests
Learning from EB-Medicine
 Interventions that achieve positive
predictable results
 Hindrances that blocked implementation of
EBM same now for EBMgt
 Financial as well as political impacts
Contributions of Organizational Behavior
 Hiring Talent-relying on structured interviews
(Rousseau)
 Challenging Performance goals impacting on
performance (Latham and Locke)
 Managers who set a vision outperform other
managers (Kirkpattrick)
Teaching EBMgt
 NYU / Wagner, Capstone course
 Projects
 Hourly nurse rounding process
 Redesigning the nurses’ clinical ladder
 Improving the supply distribution process
 Identifying causes of emergency department waiting
The Politics of EBMgmt
 Evidence is not sufficient to change people’s
behavior
 The quality of the argument and story-telling by
persons presenting the evidence is what persuades
stakeholders
 How persuade the manager that this is in his own
interest?
Michiel Bosman MD MMM FACPE
MD, University of Amsterdam
MMM, CMU
Exec PhD (2015), OSU Spears School
Collaborator, Center for EBMgmt
Serial Entrepreneur
What field is this?
 “there is a large research-user gap”
 “practitioners do not read academic journals”
 “the findings of research into what is an effective intervention
are not being translated into actual practice”
 “the relevance, quality and applicability of research is
questionable”
 “practice is being driven more by fads and fashions than
research”
 “many practices are doing more harm than good”
Medicine: Founding fathers
David Sackett
Gordon Guyatt
McMaster University Medical School, Canada
How it all started
Problem: too much ‘evidence’
 More than 1 million articles in 40,000 medical journals per
year (= 1995; now probably more than 2 million). For a
specialist to keep up this means reading 25 articles every
day (for a primary care physician more than 100!)
Problem: too much evidence
 HRM: 1,350 articles in 2010 (ABI/INFORM). For an HR
manager to keep up this means reading 3 to 4 articles
every day (for a ‘general’ manager more than 50!)
Problem: too many half truths
BTW: most of the research is seriously flawed or irrelevant
for practice. And some claim so is most management advice.
The 5 steps of Evidence-Based Practice
1. Formulate a focused question (Ask)
2. Search for the best available evidence (Acquire)
3. Critically appraise the evidence (Appraise)
4. Integrate the evidence with your professional
expertise and apply (Apply)
5. Monitor the outcome (Assess)
True or false?
1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
How evidence-based are managers?
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)
 959 (US) + 626 (Dutch) HR professionals
 35 statements, based on an extensive body of evidence
 true / false / uncertain
On average: 35% - 57% correct
Evidence-Based Practice
1991 Medicine
1998 Education
1999 Social care, public policy
2000 Nursing
2005 Criminal justice
2010 Marketing
2013 Healthcare Management?
Evidence-Based Consulting
 Close the research/practice gap

Professor + Consultant
 EB Mgmt Workshops

Teach the 5 step EB Practice process
 Answer business questions

CAT: Critically Appraised Topic
Closing the research/practice gap
 Executive Doctorate in Management
 University of Maryland College
 Oklahoma State University
 Development of tools to measure EB
Mgmt Attitudes (EBMAS) and
skills/competencies (Fresno)
Teaching Evidence Based Practice
to managers
Evidence?
intuition, expertise, personal
experience, collective experience,
organizational facts & data,
best practices, benchmarking,
outcome of scientific research
All managers base their
decisions on ‘evidence’
BUT ...
Managers give little
or no consideration to the
quality of the evidence
they base their decisions on
SO ...
Teach managers how to find
and and critically appraise
the evidence before they
make their decision
The 5 steps of EBP
1. Formulate a focused question (Ask)
2. Search for the best available evidence (Acquire)
3. Critically appraise the evidence (Appraise)
4. Integrate the evidence with your professional
expertise and apply (Apply)
5. Monitor the outcome (Assess)
0. Creating awareness
Why do we need it?
Trust me, 20 years of
management experience
Errors and Biases of Human Judgment
Errors and Biases of Human Judgment










Seeing order in randomness
Mental corner cutting
Misinterpretation of incomplete data
Halo effect
False consensus effect
Attribution error

Group think

Self serving bias

Sunk cost fallacy

Cognitive dissonance reduction 
Confirmation bias
Outcome bias
Authority bias
Small numbers fallacy
Recall bias
 Anchoring bias
 Inaccurate covariation detection
1. Formulate a focused question
Asking the right question?
 Does team-building work?
 Does the introduction of self-steering teams work?
 Does lean management improve the performance of
our hospital?
 Is 360 degree feedback for doctors effective?
Focused question?
 Does team-building work?
 What is a ‘team’?
 What kind of team?
 In what contexts/ settings?
 What counts as ‘team-building’?
 What does ‘work’ mean?
Answerable question: PICOC
P = Population
I = Intervention or success factor
C = Comparison
O = Outcome
C = Context
Focused question: PICOC
P = Population
I = Intervention or successfactor
C = Comparison
O = Outcome
 Employee productivity?
 Patient satisfaction?
 Return on investment?
C = Context
 Market share?
 Organizational commitment?
2. Finding the best available evidence
Evidence-based decision
Best available
experiential evidence
Best available
external evidence
Evidence-based
decision
Best available
internal evidence
Organizational values
and stakeholders’
concerns
Scientific databases: management
 ABI/INFORM
 Business Source Elite
 PubMed
 PsycINFO
 Web of Knowledge
 ERIC
3. Critical appraisal of studies
Making sense of evidence
Critical appraisal
 Construct validity
(lean six sigma = value stream mapping, root cause
analysis, goal setting, participative decision making)
 Internal validity
(does it work?)
 External validity
(will it also work for my employees / organization?)
Levels of internal validity
Best available
Best research design?
Critical appraisal
Step 4: Turning evidence into
practice
Applicable?
 organizational characteristics
 cultural & political aspects
 financial aspects /cost-effectiveness / ROI
 priorities
 change readiness / resistance to change
 implementation capacity
 timing
TEST?
CAT: Critically Appraised Topic
CAT-walk
Evidence-based practice:
If doctors can do it
managers can do it!