Postgraduate Course
7. Evidence-based management:
Research designs
Empirical circle
Postgraduate Course
Research designs: 4 elements
Postgraduate Course
 Randomization
 Comparison
 Before vs after
 Prospective vs retrospective
Research designs: terms
Postgraduate Course
 Randomized controlled study
 Controlled before-after study
 Cohort / panel study
 Experiment
 Before-after study
 Quasi-experiment
 Post-test only study
 Observational study
 Survey
 Longitudinal study
 Case study
 Cross-sectional study
 Qualitative study
CBA: controlled before-after study
Postgraduate Course
In a controlled before-after study two or more
groups are compared with each other, usually
comprising one group in which an intervention is
carried out (experimental group) and one group
where no or an alternative intervention is
conducted (control group).
CBA: controlled before-after study
Postgraduate Course
=
Postgraduate Course
Better (but not always feasible): randomization
In the case of randomization, the groups compared with each
other are selected entirely randomly, for example by drawing lots.
This means that each participant (or other unit such as a team,
department or company) has an equal chance of being in the
intervention or control group. In this way, the influence of any
distorting factors is spread over both groups so that these groups
are as comparable as possible with each other with the
exception of the intervention.
Randomized controlled study (RCT)
Postgraduate Course
=
Cohort / panel study
Postgraduate Course
Longitudinal study where large groups of people or
companies (cohort / panel) are followed over a long
period to see (prospective) whether differences occur
among the groups.
Starting point: intervention/exposure (independent variable)
=
Case-control study
Postgraduate Course
Longitudinal study in which one group of people or companies
with a particular outcome is compared retrospectively with a
group that does not have this outcome.
Starting point: outcome (dependent variable)
✔
VS
=
✗
VS
Examples of (flawed) case-control studies
Postgraduate Course
Before-after study
Postgraduate Course
Study in which data are obtained or particular
characteristics are measured of a population before
versus after an intervention / exposure / event, to
measure the effect or correlation.
versus
Posttest-only
Postgraduate Course
=
Cross-sectional study
Postgraduate Course
Study in which data of a statistically significant sample of
a population (managers, CEO’s, employees) is gathered
at one point in time. It provides a snapshot of the current
condition but does not explain cause and effect.
Cross-sectional studies
include surveys
Cross-sectional study
Postgraduate Course
?
?
?
Case study
Postgraduate Course
Study in which a large number (qualitative or quantitative)
of aspects of a single case (organization or team) was
investigated in depth over a long period within the case’s
own context.
Experimental research
Postgraduate Course
Experimental research refers to studies where the
researcher manipulates one (or more) variable and
controls the other variable(s) to determine whether there
is a causal relation between the manipulated variable and
the outcome.
 ‘True’ experiment: RCT
 ‘Quasi’ experiment: CBA
Observational research
Postgraduate Course
Observational research refers to studies where the
researcher merely observes but does not intervene,
with the intention of finding correlations among the
observed data
 Natural experiment
 Cohort / panel study
 Case-control study
Qualitative research
Postgraduate Course
Qualitative research explores and tries to understand
people's beliefs, experiences, attitudes, behaviour and
interactions. It generates non-numerical data. The bestknown qualitative research-methods include in-depth
interviews, focus groups, documentary analysis and
participant observation.
Qualitative research includes
 Case studies
 Ethnographic studies
 Field research
 Grounded theory approach
Longitudinal research
Postgraduate Course
A longitudinal study is an observational study that
involves repeated observations (measurements)
of the same variable(s) over long periods of time
(sometimes years or even decades).
 Cohort / panel study
 Repeated measures
 Times series
Mixed methods
Postgraduate Course
But ….
Postgraduate Course
Better than a single study:
a replication study
Better than a replication study:
a systematic review / meta analysis
If there were 100 studies, 99 of which gave a ‘negative’
result (where, say, the new intervention appeared to be
not effective), while one had a ‘positive’ result (were the
intervention appeared effective), it would obviously be a
mistake to consider only the single positive study.
Systematic review / meta-analysis
Postgraduate Course
The intention behind a systematic review is to identify as fully
as possible all the scientific studies of relevance to a particular
subject and to assess the validity and authority of the evidence
of each study separately. As the name indicates, a systematic
review takes a systematic approach to identifying studies and
has the methodological quality critically appraised by multiple
researchers independently of each other, as a consequence of
which the review is transparent and reproducible and can be
monitored. The use of statistical analysis techniques in a
systematic review to pool the results of the individual studies
numerically in order to achieve a more accurate estimate of
the effect is termed a “meta-analysis”.
Systematic review / meta-analysis
Postgraduate Course
?
Which terms belong together?
Postgraduate Course
1. Randomized controlled study
2. Controlled before-after study
3. Cohort / panel study
A. Experiment
4. Before-after study
B. Quasi-experiment
5. Post-test only study
C. Observational study
6. Survey
D. Longitudinal study
7. Case study
E. Cross-sectional study
F. Qualitative study