LITERATURE
EVALUATION PHCL 491
Courtesy of Lab Times online:
http://www.labtimes.org/editorial/e_024.lasso
Definitions
Background
Terminology
Definitions of Some Study Types
Introduction to Critical Appraisal Skill
 Programme (CASP)
Definitions
“Critical appraisal is the process of carefully
and systematically examining research to
judge its trustworthiness, and its value and
relevance in a particular context”. (Burls,
2003).
 “the assessment of evidence by
systematically reviewing its relevance,
validity and results to specific situations”
(Chambers, 1998)

Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Definitions

“application of rules of evidence to a
study to assess the validity of the data,
completeness of reporting, methods and
procedures, conclusions, compliance
with ethical standards, etc. The rules of
evidence vary with circumstances” (Last,
2001)
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Background

Every day we are confronted with
advertisements, news, brochures…etc that
try to persuade us that a certain
pharmaceutical or cosmetic products are
effective and safe. How make sure that the
studies behind these information sources
were not conducted in a way that was likely
to produce the results stated by them
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Background

Example from marketing (fictional):
Company named Starman co. claims in its
roll-ups, TV adds and brochures that 95% of
male costumers prefer to by its deodorant
“Supra-man” than buying similar products
from competing companies. How do you
know this is true?
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Background


If the male participants in the survey were
selected randomly and there is no bias in
the way they collected the data. Then this
result could be valid
If the male participant in the survey were
costumers who were already trying to buy to
product at the time and place of the survey,
or certain “gifts” were given to participants
who favored the product, then the results
are considered invalid.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Background

Literature evaluation skills are especially
needed by clinicians in:
• Making a clinical decision about a
subject which have conflicting
information in the literature
• Developing hospital drug formulary or
clinical practice guidelines.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Background

1.
2.
3.
Some reasons for why low-quality
researches may be published:
University teaching staff feel the pressure to
publish within a limited period in order to
advance in their jobs.
Lack of knowledge by either the researchers
of the peer-reviewers in study design and
statistical analysis.
Scientific fraud.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

Definition obtained from bandolier
website1
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Bias
 It is an a systematic error or deviation in
results or inferences. In studies of the
effects of healthcare bias can arise from
systematic differences in the groups that are
compared (selection bias), the care that is
provided, or exposure to other factors apart
from the interest (performance bias),
withdrawals or exclusions of people entered
into the study (attrition bias) or how
outcomes are assessed (detection bias)
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology


Bias does not necessarily carry an
imputation of prejudice, such as the
investigators’ desire for particular results.
This differs from conventional use of the
word in which bias refers to a partisan point
of view.
Selection bias refers to systematic
differences between comparison groups in
prognosis or responsiveness to treatment.
Random allocation with adequate
concealment protects against selection bias
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Confounding

Confounding refers to a situation in which a
measure of the effect of an intervention or
exposure is distorted because of the
association of exposure with other factor(s)
that influence the outcome under
investigation. This can lead to erroneous
conclusions being drawn, particularly in
observational studies
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

Confounding by indication can arise in
observational studies when patients with
the worst prognosis are allocated
preferentially to a particular treatment.
These patients are likely to be
systematically different from those not
treated, or treated with something else
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Validity

This term is a difficult concept in clinical
trials, it refers to a trial being able to
measure what it sets out to measure. A trial
that set out to measure the analgesic effect
of a procedure might be in trouble if
patients had no pain. Or in a condition
where treatment is life-long, evaluating an
intervention for 10 minutes might be seen
as silly.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

In diagnostic tests validity is the extent
to which a test measures what it was
intended to measure. High reliability is
necessary but does not guarantee high
validity
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Peer

review
Review of a study, service or
recommendations by those with similar
interests and expertise to the people who
produced the study findings or
recommendations. Peer reviewers can
include professional, patient and carer
representatives
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

The trouble is that peer review does not
always (or even frequently) work very
well. Many poor papers are published,
and even papers published in top
medical journals can have major flaws.
Just because something is found in a
peer-reviewed journal does not mean it
is right, or good, or sensible.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Blinding
 The process used in epidemiological studies
and clinical trials in which the participants,
investigators and/or assessors remain ignorant
concerning the treatments which participants
are receiving.
 The aim is to minimize observer bias, in which
the assessor, the person making a
measurement, have a prior interest or belief that
one treatment is better than another, and
therefore scores one better than another just
because of that
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

Blinding may be:
1. Single Blinding: either the participants are
blind to their allocation or the assessors
2. Double blinding: both the participants and
the assessors are blind to participants
allocation.
3. Triple blinding: the participants, the
assessors and the other group involved
with interpretation of data is also unaware
of subject assignment.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Placebo

It is a fake or inactive intervention, received
by the participants allocated to the control
group in a clinical trial, which is
indistinguishable from the active intervention
received by patients I the experimental
group
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Randomisation

Method analogous to tossing a coin to assign
patients to treatment groups (the
experimental treatment is assigned if the
coin lands heads and a conventional, control
or placebo treatment is given if the coin
lands tails). Usually done using a computer
that generates a list of random numbers,
which can then be used to generate a
treatment allocation list.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Stratified

randomisation
In any randomised trial it is desirable that the
comparison groups should be as similar as
possible as regards those characteristics that
might influence the response to the
intervention. Stratified randomisation is used
to ensure that equal numbers of participants
with a characteristic thought to affect
prognosis or response to the intervention will
be allocated to each comparison group
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

For example, in a trial of women with breast
cancer, it may be important to have similar
numbers of pre-menopausal and postmenopausal women in each comparison
group. Stratified randomisation could be used
to allocate equal numbers of pre- and postmenopausal women in each comparison
group. Stratified randomisation is performed
either by performing separate randomisation
(often using random permuted blocks) for
each strata, or by using minimization.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Quasi-random

allocation
A method of allocating participants to
different forms of care that is not truly
random; for example, allocation by date of
birth, day of the week, medical record
number, month of the year, or the order in
which participants are included in the study
(alternation).
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Open-label

study
A study in which both the assessors and the
participants are aware of the assignment of
the treatment groups (no blinding).
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Double-dummy

study
A technique used when treatments studied
are radically different , for example, when
comparing a treatment in the form of a tablet
with another in the form of an injection then
each group receives both a tablet and an
injection (i.e. one group receives a placebo
tablet and the other group receives a
placebo injection).
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Crossover

study design
The administration of two or more
experimental therapies one after the other in
a specified or random order to the same
group of patients.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology

Two main issues about cross-over designs:
1. Order effects: the order in which treatments
are administered may affect the outcome.
2. Carry-over between treatments: dealt with
by using wash-out period between
treatments or by making observations
sufficiently later after the start of a
treatment period that any carry-over effect
is minimized.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Parallel


study
A study in which the two groups receive two
distinct treatment, i.e. one group receives
treatment A while the other group receives
treatment B
Unlike a cross-over study in which one
group receives treatment A followed by
treatment B while the other group receives
treatment B followed by treatment A.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Control
 General definition: is something against
which we make a comparison.
 In clinical trials comparing two or more
interventions, a control is a person in the
comparison group that receives a placebo,
not intervention, usual care or another form
of care.
 In case control studies a control is a person
in the comparison group without the
disease or the outcome of interest
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology


In statistics control means to adjust for or
take into account extraneous influences or
observations.
Control can also mean programs aimed at
reducing or eliminating the disease when
applied to communicable (infectious)
disease.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Terminology
 Placebo-controlled

study
A clinical trial in the which the control group
receive a placebo that is identical to the
study drug in terms of appearance, odor,
taste but dose not contain the active drug.
Courtesy of THE BROSIUS: http://thebrosius.com/tips-guides/bro-basics/brocabulary-bro-terminology
Definitions of Some Study Types

The following diagram shows the
epidemiologically classification of
different study designs
Experimental
Studies
Randomized
controlled trials
Observational
Studies
Quasi-experimental
studies
Cohort
Analytical
studies
Case-control
Descriptive
studies
Cross-sectional
study
Definitions of Some Study Types


Different types of question require different
study designs (definitions from bandolier
website)1:
Qualitative study
Used to explore and understand people’s
beliefs, experiences, attitudes, behavior and
interactions. It generate non-numerical data,
e.g. patient’s description of their pain rather
than a measure of pain.
Definitions of Some Study Types
In health care, qualitative techniques have
been
commonly
used
in
research
documenting the experience of chronic
illness and in studies about the functioning of
organizations.
Qualitative
research
techniques such as focus groups and indepth interviews have been used in one-off
projects
commissioned
by
guideline
development groups to find out more about
the views and experiences of patients and
carers1.
Definitions of Some Study Types

When testing a particular treatment,
subjective anecdotal reports of benefit can
be misleading and qualitative studies are
therefore not appropriate.
An extreme example was the fashion for
drinking Radithor® a century ago. The
death of one keen proponent, Eben Byers,
led to the 1932 Wall Street Journal
headline, ‘The Radium water Worked Fine
until His Jaw Came Off’ 2
Definitions of Some Study Types

Randomised Controlled Trial (RCT)
A group of patients is randomised into an
experimental group and a control group.
These groups are followed up for the
variables/outcomes of interest. The point
about using randomisation is that it avoids
any possibility of selection bias in a trial1.
Definitions of Some Study Types
The test that randomisation has been
successful is that different treatment groups
have same characteristics at baseline. For
instance, there should be the same number of
men and women, or older or younger people, or
different degrees of disease severity.
Definitions of Some Study Types

The following is a flow diagram of the
progress through the phases of a
parallel randomised trial of two groups
(that is, enrolment, intervention
allocation, follow-up, and data
analysis)3.
Enrollment
1
Excluded (n=)
oNot meeting inclusion criteria (n=)
oDeclined to participate (n=)
oOther reasons (n=)
Assessed for eligibility
Randomized (n=)
Allocation
2
Allocated to intervention (n=)
oReceived allocated intervention (n=)
oDid not receive allocated
intervention (give reasons) (n=)
Follow-up
3
Lost to follow-up (give reasons) (n=)
Discontinued intervention (give
reasons) (n=)
4
Allocated to intervention (n=)
oReceived allocated intervention (n=)
oDid not receive allocated
intervention (give reasons) (n=)
Lost to follow-up (give reasons) (n=)
Discontinued intervention (give
reasons) (n=)
Analysis
Analyzed (n=)
Excluded from analysis (give reasons)
(n=)
Analyzed (n=)
Excluded from analysis (give reasons)
(n=)
Definitions of Some Study Types

Cross-Sectional Survey
It is a useful design to determine how
frequent a particular condition is. However,
when determining an accurate prognosis for
someone diagnosed with, say, cancer, a
cross-sectional survey (that observes people
who have the disease and describes their
condition) can give a biased result1.
Definitions of Some Study Types

Cohort Study
Involves identification of two groups
(cohorts) of patients one which received the
exposure of interest, and one which did not,
and following these cohorts forward for the
outcome of interest1.
Definitions of Some Study Types

Case-control study
A study which involves identifying patients
who have the outcome of interest (cases)
and control patients who do not have that
same outcome, and looking back to see if
they had the exposure of interest. The
exposure could be some environmental
factor, a behavioral factor, or exposure to a
drug or other therapeutic intervention1.
Definitions of Some Study Types

Systematic Review
A summary of the medical literature that
uses explicit methods to perform a thorough
literature search and critical appraisal of
individual studies and that uses appropriate
statistical techniques to combine these valid
studies1.
Definitions of Some Study Types


1.
2.
Health Economic Evaluation
“Ultimately, health economics is about
maximising social benefits obtained from
constrained health producing resources”4. It
has two roles:
Using statistical methods to aid healthcare
decision.
Introducing new way of thinking about
health resources and recognizing scarcity.
Introduction to CASP Programme

Critical Appraisal Skills Programme (CASP)
is a non-profit entity established in Oxford,
UK in 1993. It provides learning material and
workshops in critical appraisal for evidence
based practice in health and social care.
Now it has international network with branch
organizations in many country outside the
UK.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Introduction to CASP Programme

CASP’s learning resources include:

A set of eight critical appraisal skill tools to
use when reading research, these include tools
for Systematic Reviews. Randomised Controlled
Trials, Cohort Studies, Case Control Studies,
Economic Evaluations, Diagnostic Studies,
Qualitative Studies and Clinical Prediction Rule.
These are free to download and can be used by
anyone
under the Creative Commons License.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Introduction to CASP Programme

Critical Appraisal Skills Workshops
are offered for Systematic Reviews,
Randomised Controlled Trials and
Qualitative Studies, these are provided
on a competitively priced tariff.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Introduction to CASP Programme

Training the Trainer courses are offered
once a year for an International audience
and more frequently for UK based
participants. These combine experiential
learning in a typical CASP workshop
environment with a finding the evidence
module, and a two day facilitated peer to
peer CASP teaching and learning
experience.
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso

We are going to classify biomedical primary
literature in our study in to four major
categories:
Analytical (experimental)
Observational
Overview
Health outcome
Courtesy of Lab Times online: http://www.labtimes.org/editorial/e_024.lasso
Introduction
Clinical Trial
No of 1 Trial
Stability Study
Survey Research
Bioequivalence study
Introduction

True experiments
(experimental) provide the
highest level of evidence
for establishing cause-andeffect relationships.

However, experimental
studies are prone to
various sources of bias
and to poor execution.
Courtesy of EXPLORABLE:
http://explorable.com/experimentalresearch
Introduction

Observational studies are
used instead when it is
impossible or unethical to
perform an experimental
study (toxic agent and of no
therapeutic value), it would
be unethical to ask subjects
to voluntarily expose
themselves to the agent and
impossible to recruit patients
for such a study.
Courtesy of EXPLORABLE:
http://explorable.com/experimentalresearch
Introduction

For example: evaluation
of risk factors for cancer,
or the toxicity of
environmental or
industrial hazards, or the
teratogenicity of drugs
administered during
pregnancy. Under such
conditions we have to
employ follow-up (cohort)
or case-control studies.
Courtesy of EXPLORABLE:
http://explorable.com/experimentalresearch
Clinical Trial
 Clinical

Trials
The epitome of the
experimental study is the
randomized controlled trial in
which the investigator
demonstrates ``control'' over
the factor under study by
randomizing patients to
various regimens
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

However, The label
``randomized'' is not
equivalent to assurance of
high quality, nor does it
alone add validity to the
study. Thus, randomized
studies also need careful
assessment of their design,
methods, analyses, and
conclusions
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

Many prophylactic and
therapeutic studies tend to
be experimental in design.

This type of literature
(randomized, controlled
clinical trial) is considered
the focus of drug information
responses and therapeutic
decision.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

The investigators has
control over or can
manipulate the major factor
under study in order to
identify the relationship
between the cause and the
expected effects.

The investigators have
control over patient selection
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial


Blinding
If there are subjective elements
used to judge the effectiveness
of treatment, there is a
compelling rationale to blind the
investigators
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

If there are subjective
assessments of the patients’
response, there is a
compelling rationale to blind
the subjects.

i.e. in the assessment of a
new medication to relieve
pain, double blinding (both
subjects and investigators)
is necessary
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

If all of the outcome
variables are objective,
blinding, strictly speaking,
is unnecessary. Thus in
the assessment of a new
medication to relieve pain,
double blinding (both
subjects and
investigators) is
necessary.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial


Selection of the study
group
One should look for
possible sources of
selection that would make
the sample atypical or no
representative.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

It is interesting that even
such seemingly ``random''
allocation of cases such
as alternate days may
introduce an
unappreciated bias.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

Research centers may
have more than one team
that alternated coverage
every 24 hours. Patients
admitted on alternate
days, therefore, are cared
for by different teams of
physicians.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

A study that entailed
alternate-day
assignment to treatment
groups would entail, as
well, the factor of
differences in physician
practice style, a factor
that one could not
disentangle in analysis
of study results.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial


Nature of the control
group or standard of
comparison
We frequently encounter the
``historical control'' group
that, almost always, has a
``poorer'' result than the
contemporary group.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

The problem, of course, is
that the basic assumption
that the modality of
treatment under
investigation is the only
cause for the difference in
results is clearly
erroneous
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

For instances the
reduction in mortality in
trauma patients could be
related to many variables
such as diminution in
delay between injury and
treatment, change in
surgical training
experience, completely
new methods of
treatments and so on.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

In fact, the principal reason
for randomization in a study
is to attempt to distribute the
unknown and potentially
important variables equally
among groups to avoid
selection bias.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

We may also see this effect
if patients accrue slowly and
the study thus runs over
many years. Other aspects
of therapy may change and
have a greater impact on
outcome than the original
variable selected for study.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

True experiments
(experimental) provide the
highest level of evidence for
establishing cause-andeffect relationships.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

Other types of research can
also be used to solve
patient related problems,
and must be used at
situations such as:
1. When they are more
effective in answering
specific questions
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

For examples: In literature,
there are only small trials
related to the therapeutic
effects of streptokinase,
tissue plasminogen
activator, and aspirin in the
treatment of stroke
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial

Because solid conclusion
cannot be made from
these smaller studies, the
results, which may be
taken from meta-analysis
study (type of literature on
which previous small trials
were combined and
statistically reanalyzed),
may be more effective at
answering the question.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
Clinical Trial
2.
When they are the only data
available to answer a specific
questions, For example: it may
not be feasible to study the
toxicity of certain agents in
controlled clinical trials. and
epidemiological research such
as case-control or cohort
studies are alternative that
must be employed to answer
such a question.
Courtesy of outsourcing-pharma.com
http://www.outsourcingpharma.com/ClinicalDevelopment/CROs-Slowly-Shiftingto-Adaptive-Clinical-Trial-Designs
CASP Appraisal tools for Different
Study Designs – RCTs Appraisal Tool
I.


Appraisal Tool for Randomized controlled Trials:
Three broad issues which are considered when
appraising RCTs:
1. Are the results valid?
2. What are the results?
3. Are the results useful locally?
The tool is composed of 11 questions, the first 3 are
screening questions and if the screening questions are
answered “yes” then it worth proceeding to the other 9
questions otherwise the study is invalid and should stop
appraising it.
Critical Appraisal Skill Programme
(CASP) – RCT Appraisal Tool

Screening questions for RCTs:
Question
1. Did the trial address
a clearly focused
issue?
Hint
focused In terms of: the population
studied, the intervention given, the
comparator given and the outcomes
considered
2. Was the assignment How was this carried out?
of patients to treatment Was the allocation sequence concealed
randomized
from researchers and patients?
3. Were all the patients
who entered the trial
Was the trial stopped early?
properly accounted for Were patients analyzed in the groups to
at its conclusion?
which they were randomised ?
Yes
No
I can’t
tell
Critical Appraisal Skill Programme
(CASP) ) – RCT Appraisal Tool

Detailed Questions for RCTs appraisal
Continue next slide
Question
Hint
4. Were patients, health Think about : 1. patients? 2.health
workers and study
workers? 3. study personnel?
personnel “blind” to
treatment?
5. Were the groups
similar at the start of
the trial?
6. Aside from the
experimental
intervention, were the
Factors that might affect the outcome
such as age, sex, social class
Yes No
I can’t
tell
Critical Appraisal Skill Programme
(CASP) ) – RCT Appraisal Tool
7. How large was the
treatment effect?
•What outcomes were measured
•Is the primary outcome clearly specified
•What results were found for each
outcome
8. How precise was the
estimate of the
treatment effect?
What are the confidence limits?
9. Can the results be
applied in your context?
(or to the local
population)
D o you think that the patients covered
by the trial are similar enough to the
patients to whom you will apply this?, if
not how to they differ?
10. Were all clinically
important outcomes
considered?
Is there other information you would like
to have seen?
If not, does this affect the decision
11. Are the benefits
worth the harms and
costs?
Even if this is not addressed by the
review, what do you think
N of 1 Trial
N

of 1 Trial
N of 1 trial attempt to apply
the principles of clinical
trials such as
randomization and blinding
to individual patients.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial
N

of 1 Trial
It can be likened to a
crossover study conducted in
a single subject in that a
patient receives treatments in
pairs; one of the experimental
therapy and one of either
alternative treatment or
placebo; in a random order.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

Is usually consists of several
treatment periods that are
continued until effectiveness
is proven or refuted.

Is through randomization and
blindness of both physician
and patient the bias that may
occur due to the treatment
order, placebo effects, and
observer intention is reduced
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

Desired outcomes are
identified prior to initiation of
the study to ensure that
objective criteria that are
meaningful to both the
physician and patient are
used to assess treatment
efficacy
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

1.
No of 1 trials differ from case
reports in that
No of 1 trials provide more
objective information than case
reports, and are useful for
providing definitive information
for drug prescribing in
individual patients. This is
reflected in the fact that
publication of such studies has
increased in recent years.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial
2.
3.
No of q trials are prospective
(predefined methods, clearly
defined outcome measures,
randomized, blind) and follow
multiple period design
The purpose of such studies is
to compare effects of drug to
control during multiple
observation periods in a single
patient.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial


No of 1 trials are conducted:
When randomized, controlled
clinical trials are not
available, which is the case
for many diseases and
therapies.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

When the restrictive inclusion
criteria of the controlled trial
may make it difficult to apply
results from the trial to
individual patients routinely
encountered in clinical
practice.
 When the beneficial effects
of a particular treatment in
an individual patient are in
doubt.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial



No of 1 Trials are used to:
Determine whether a drug
therapy is effective in an
individual patient
Taken as a whole, a group of
N-of-l trials can help to
identify characteristics that
differentiate responders from
non-responders
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial


Trials of multiple doses can
identify the most effective
dose and the clinical
endpoints most influenced by
the drug. However, the
treatment should be of a
short half-life to allow multiple
cross-over periods without
carry-over effects
N-of-l trials may improve
appropriate prescribing of
drugs in individual patients
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
Example

Carbamazepine may be an option for
relief of pain in a patient with diabetic
neuropathy, but definitive information on
the efficacy of such treatment is limited.
Therefore, the investigators may conduct
an n-of-l trial to determine whether such
therapy is useful in this patient.
Courtesy of 123RF:http://www.123rf.com/photo_11159182_high-resolution-3d-illustration-of-icon-people-linked-bycommunication-lines-that-start-from-one-per.html
N of 1 Trial

N-of-l trials are especially
useful when long-term
treatment with a particular
drug may result in toxicity,
and the clinician wishes to
determine whether benefits
outweigh potential risks
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

In one study out Of 57 n-of l
trials completed, 50 (88%)
provided a definite clinical or
statistical answer to a clinical
question leading to the
conclusion by the authors
that n-of-l trials were useful
and feasible in clinical
practice.
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
N of 1 Trial

In another study out of the 34
patients completed n-of-l
trials evaluated over a 2-year
period, 17 (50%) were judged
to provide definitive results.

The overall, clinician
confidence in the therapy
was found to increase or
decrease depending on the
direction of trial results
Courtesy of
123RF:http://www.123rf.com/photo_111
59182_high-resolution-3d-illustration-oficon-people-linked-by-communicationlines-that-start-from-one-per.html
Stability Study
 Stability
Study (IN Vitro
study)

The purpose of such studies is
to evaluate stability of drugs in
various preparations such as
the ophthalmologic,
intravenous, topical, and oral,
under various conditions (e.g..,
heat, freezing, refrigeration,
room temperature).
Courtesy of defabar:
http://www.defabar.com/en/Stab
ility+studies.html
Stability Study

This type of research is
extremely important to the
practice of pharmacy as in the
following:
Courtesy of defabar:
http://www.defabar.com/en/Stab
ility+studies.html
Example

The pharmacists who prepare intravenous
solutions for use by patients at home often
want to know how long a drug admixed in a
particular solution is stable or if freezing
increases the length of time an admixture is
stable in order to determine how many
intravenous admixtures may be dispensed at
a time.
Courtesy of defabar: http://www.defabar.com/en/Stability+studies.html
Stability Study

It is also important for
pharmacists involved with
extemporaneous compounding
to know the length of time a
particular preparation is stable.
Courtesy of defabar:
http://www.defabar.com/en/Stab
ility+studies.html
Stability Study

Other In Vitro pharmaceutical
studies such as testing for the
effects of: for examples
antimicrobials or anti-cancers
activities in cultures, and
conducting tablets
disintegration testing for
assuring the quality of
pharmaceutical products.
Courtesy of defabar:
http://www.defabar.com/en/Stab
ility+studies.html
Survey Research
 Survey
Research
(Programmatic study)

The purpose of such studies is
to determine the impact and/or
economic value of new services
such as the services of
monitoring patients with chronic
disease at community
pharmacies or a new drug
information service or a new
patient education.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research

Its another type of true
experiment.

Required when there is a
need to Focus on the impact
and economic value of
programs and services
provided by pharmacists in
community and institutional
settings.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research

Programmatic research is
particularly important
because limited resources
and budget constraints
demand that only those
services that improve patient
care in a cost-effective
manner be implemented.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research

The economic value of many
widely available pharmacy
services, including drug
information, CPR team
involvement, and discharge
patient counseling, has not
been adequately studied and
hence this type of research
may be useful to document
the values of these services.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research

The methodology of studies
evaluating pharmacy
programs and services is
usually supplied with the
following information:
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research


Control group which could be
group of patients who do not
use the program or service,
or alternatively, periods
before and after
implementation of the
program or service may be
compared.
Clear definition to the
program or service under
investigation
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research


The focus of the
investigation which could be
either the patient, provider,
third-party payer, or society.
If the economic impact of the
service is among the
objectives, all economic
costs related to providing the
program or service is usually
estimated.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Survey Research

The outcome parameters
that are used to determine
effectiveness of the program
or service is usually well
defined.
Courtesy of: Washington State
University's Social & Economic Sciences
Center:Research
http://www.sesrc.wsu.edu/sesrcsite/
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
V.


Appraisal Tool for Qualitative Research:
Three broad issues are considered when appraising a
cohort study:
1. Are the results of the review valid?
2. What are the results?
3. Are the results useful locally?
The tool is composed of 10 questions, the first 2 are
screening questions and if the screening questions are
answered “yes” then it worth proceeding to the other 8
questions otherwise the research is invalid and should
stop appraising it.
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool

Screening Questions for Qualitative Research
Question
Hints
1. Was there a
clear statement
of the aims of
the research?
Regarding aim(s) consider:
• What was the goal of the research?
• Why it was thought important?
• Its relevance
2. Is a
qualitative
methodology
appropriate?
Consider
• If the research seeks to interpret or illuminate the
actions and/or subjective experiences of research
participants
• Is qualitative research the right methodology for
addressing the research goal?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool

Detailed Questions for Qualitative Research
Continue next slide
Question
Hint
3. Was the research
design appropriate
to address the aims
of the research?
Did the researcher justify the research
design (e.g.. have they discussed how they
decided which method to use)?
4. Was the
recruitment strategy
appropriate to the
aims of the research
Has the researcher explained how the
participants were selected?
Did they explain why the participants they
selected were the most appropriate to provide
access to the type of knowledge sought by
the study.
Are there any discussions around
recruitment (e.g.. why some people chose not
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
Continue next slide
Question
Hints
5. Was the
data collected
in a way that
addressed the
research
issue?
Consider
• If the setting for data collection was justified
• If it is clear how data were collected (e.g.. focus
group, semi-structured interview etc.)
• If the researcher has justified the methods chosen
• If the researcher has made the methods explicit
(e.g.. for interview method, is there an indication of
how interviews were conducted, or did they use a
topic guide)?
• If methods were modified during the study. If so,
has the researcher explained how and why?
• If the form of data is clear (e.g.. tape recordings,
video material, notes etc)
• If the researcher has discussed saturation of data
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
Question
6. Has the relationship
between researcher and
participants been
adequately considered?
Hints
Continue next slide
Yes No I cant
tell
 Did the researcher critically examine
their own role, potential bias and
influence during
(a) Formulation of the research
questions
(b) Data collection, including sample
recruitment and choice of location.
• How the researcher responded to
events during the study and whether
they considered the implications of any
changes in the research design
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
Continue next slide
Question
Hints
7. Have ethical
issues been taken
into consideration?
Consider
• If there are sufficient details of how the
research was explained to participants for the
reader to assess whether ethical standards
were maintained
• If the researcher has discussed issues raised
by the study (e.g.. issues around informed
consent or confidentiality or how they have
handled the effects of the study on the
participants during and after the study)
• If approval has been sought from the ethics
committee
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
Question
8. Was the
data analysis
sufficiently
rigorous?
Hints
Continue next slide
Yes No I cant
tell
Consider:
• If there is an in-depth description of the analysis
process
• If thematic analysis is used. If so, is it clear how
the categories/themes were derived from the data?
• Whether the researcher explains how the data
presented were selected from the original sample
to demonstrate the analysis process
• If sufficient data are presented to support the
findings
• To what extent contradictory data are taken into
account
• Whether the researcher critically examined their
own role, potential bias and influence during
analysis and selection of data for presentation
Critical Appraisal Skill Programme
(CASP)– Qualitative Research Appraisal Tool
Question
Hints
9. Is there
a clear
statement
of
findings?
• are the findings explicit?
• Is there adequate discussion of the evidence both for
and against the researchers arguments?
• Has the researcher discussed the credibility of their
findings (e.g.. triangulation, respondent validation, more
than one analyst) ?
•Are the findings discussed in relation to the original
research question ?
10. How
valuable is
the
research
• Does the researcher discuss the contribution the study
makes to existing knowledge or understanding? e.g.. do
they consider the findings in relation to current practice
or policy?, or relevant research-based literature?
• Do they identify new areas where research is
necessary ?
• Did the researchers have discuss whether or how the
findings can be transferred to other populations or
considered other ways the research may be used
Yes No
I cant
tell
Bioequivalence Study
 Bioequivalence

study
The purpose of this research
is to assess the
bioequivalence of two or
more products for formulary
selection and or therapeutic
indication
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

1.
It is important because:
Is increasing because of
the increasing number of
generic products.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study
2.
It is one of the five
conditions required to
document therapeutic
equivalence between
generic and brand product
(namely: safe, effective,
pharmaceutically equivalent,
bio-equivalent, adequately
labeled and manufactured
according to GMP).
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study
3.
Is required when the
pharmacist have to select
one from among several
apparently equivalent
products either: to use for
individual patients and/or to
include on formularies of
health care organizations
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The more skilled the
pharmacist is at interpreting
the data, the more
comfortable he or she will be
in selecting the appropriate
product for the specific
patient or organization.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Bioequivalence trials are
often conducted under
standardized conditions It
also conducted In a small
number of normal, healthy
adult volunteers because of
availability and lack of
confounding factors in this
population.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Data from healthy
volunteers, however, may
not reflect the population for
whom the medication is
prescribed
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Single or multiple doses may
be used with a cross-over
design (a parallel design can
be use for drugs having long
half life)
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The products are
bioequivalent when they that
are equivalent in rate and
extent of absorption (i.e. the
rate and extent of absorption
differ by -20% to +25% or
less).
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Cmax, AUC & Tmax are the
primary pharmacokinetic
parameters used to assess
the rate and extent of drug
absorption.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Courtesy of bpacnz better medicine http://www.bpac.org.nz/BPJ/2007/March/bioequiv.aspx
Bioequivalence Study

These criteria are selected
based on an arbitrary
medical decision that, for
most products, a -20%/+25%
difference in the
concentration of the active
ingredient in blood will not be
clinically significant.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Additionally, for approval of a
generic product, a
manufacturer must show that
a 90% confidence interval for
the ratio of the mean
response (AUC, Cmax) of its
product (test) compared to
that of the innovator
(reference) product is within
the limits of 0.8 to 1.25 (80%
to 125%).
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Conditions of Bioequivalence
subjects: The acceptable
age and weight range for
the subjects is required to be
defined in the methods.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Clinical parameters used to
characterize a normal,
healthy adult (e.g.., physical
examination observations,
hematologic evaluations) are
required and should be
described.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The subjects are usually
non smokers and may also
have limitations placed on
their caffeine because both
factos may affect blood
levels of the product
question.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The intake of food was
closely monitored and
controlled, because food can
impact the rate and
absorption of some products.

The sample collection times,
which should be based on
the half-life of the drug, is
defined in the method
section.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

All subjects should receive
the drug under the same
conditions, and all blood
levels should be taken at
the same time.

The confounding factors
such as: increased weight,
increased alcohol intake,
and initiation of smoking
were minimized between
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The confounding factors
such as: increased weight,
increased alcohol intake,
and initiation of smoking
were minimized between
treatment periods (crossover periods).
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

In the method section
evidence for crossover
study design, assay
validation, and consistent
conditions were usually
present.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The standard
bioequivalence (PK) study
is conducted using a twotreatment crossover study
design in a limited number
of volunteers, usually 24 to
36 adults. Alternately, a
four-period, replicate
design crossover study
may also be used.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html

Bioequivalence Study
Alternate study methods, such as
in-vitro studies or equivalence
studies with clinical or
pharmacodynamic endpoints, are
used for drug products where
plasma concentrations are not
useful to determine delivery of
the drug substance to the site of
activity (such as inhalers, nasal
sprays and topical products
applied to the skin).
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

The statistical methodology
for analyzing these
bioequivalence studies is
called the two one-sided
test procedure.

.
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Bioequivalence Study

Based on the opinions of
FDA medical experts, a
difference of greater than
20% for each of the above
tests was determined to be
significant, and therefore,
undesirable for all drug
products
Courtesy of HT Pharmaceuticals:
http://www.htpharmaceuticals.com/welcom
e.html
Introduction
Follow-Up (Cohort) Study
Case-Control Study
Cross-Sectional Study
Case & Case Series Study
Courtesy of Discovery Works, Inc: http://www.discoveryworks.net/
Introduction

When the investigator
cannot manipulate the
major factor under study,
he or she must rely on
what has been observed;
this study is an
observational study.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

Observational studies
should not be viewed as
inferior being inferior to
experimental (clinical trial)
studies. observational
studies can also provide
substantial, sound
medical evidence.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

In fact, a well-planned and
well-executed observational
study can be much more
informative than a weakly
designed and poorly
executed randomized
clinical trial study.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

data collection could be
either prospective or
retrospective.

The principal advantage of
prospective data collection
is that the researcher,
having clearly identified the
objectives, can ensure
collection of this relevant
information in a manner that
he or she can determine.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

On the other hand,
Retrospective analysis of
medical records depends
on what happens to
appear in the record, often
with no indication of the
manner in which the
information was obtained.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Intoduction

For example, sex, age, and
hospital outcome (survival
or death) are key data
elements that may not
appear for every patient in a
retrospective chart review.
Clearly, without a specified
protocol, one cannot
anticipate that a daily blood
gas, serum creatinine, or
any other intermittent
measurement dependent on
a specific order will appear
in the chart.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/

There are possible
sources of selection that
would make the sample
atypical or non representative.

Alternate days coverage by
different teams of
physicians may introduce
unappreciated bias.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

``historical control'' group
almost always has a
``poorer'' result than the
contemporary group. For
instances the reduction in
mortality in trauma patients
could be related to many
variables such as diminution
in delay between injury and
treatment, change in surgical
training experience,
completely new methods of
treatments and so on.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

We may also see this
effect if patients accrue
slowly and the study thus
runs over many years.
Other aspects of therapy
may change and have a
greater impact on outcome
than the original variable
selected for study.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

Observational studies
determine associations
rather than cause-and-effect
relationships.

In evaluating the results of
observational study it is
important to remember that
an association between
exposure and outcome
does not necessarily prove
causation.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Introduction

other factors not addressed
by the investigators that are
possibly related to both the
exposure and outcome
must be considered.

The author must state
clearly the populations
with which they are dealing:
groups studied and
population should be
carefully considered
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Example
Example

If three studies were conducted to evaluate
the effect of a treatment regimen in treating
a respiratory disorder. One might assume
that the failure to prove the hypothesis in the
third study was because they involved only
patients with early and moderate arterial
hypoxemia. The first two studies were
specifically excluded these patients and
concentrated on developing therapy for
those who had persistent hypoxemia despite
aggressive application of conventional
respiratory therapy.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Example
Example

Thus, a treatment regimen that reversed
hypoxemia in patients who were refractory
to the then ``conventional therapy'' of acute
respiratory insufficiency was found to be not
useful in another population that had only
moderate hypoxemia and did not have true
adult respiratory distress syndrome (ARDS).
If the authors do not state clearly the
populations with which they are dealing, the
readers can easily lose this important
distinction.
Courtesy of Discovery Works, Inc:
http://www.discoveryworks.net/
Follow-up (cohort) study
 Follow-up
(cohort)
studies

In a follow-up study, a
group of subjects exposed
to the factor of interest are
compared to an unexposed
group of subjects and
followed prospectively.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Example

drug A has become available for the
prevention of myocardial infarction in
patients with coronary artery disease;
however, there is some concern that
drug A actually increases death through
pro-arrythmic effects. A follow-up study
could compare subjects exposed to drug
A and those not exposed to drug A with
the primary endpoint being the incidence
of sudden death.
Courtesy of VENTURE ACCELARATOR PARTNERS INC: http://www.vapartners.ca/top-follow-up-b2b-sales-tactics/
Follow-up (cohort) study

Cohort studies are most
useful for examining
relatively common
diseases with shorter
biological intervals for the
development of disease.
This is because of the cost
to maintain follow-up, which
is one of the primary
disadvantages of cohort
studies
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

Optimal cohort studies do
the following:
 Clearly and
unambiguously stated the
research question
 Described in detail the
relevant inclusion and
exclusion criteria
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

Have similar exposed and
unexposed individuals in
terms of demographic
characteristics so that
susceptibility to the disease
state is equal except for the
presence of the risk factor
under investigation, and that
can be achieved if subjects
are randomized to exposure
or non-exposure.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study
 The outcome is similarly
assessed in both the
exposed and unexposed
groups.
 The same efforts to
measure outcomes are
made for both groups in
order to avoid
measurement bias.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

if there are different followup rates for the two
groups, In this case, the
outcome incidences would
reflect follow-up rates
rather than exposure to
the risk factor.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

Attributable risk and/ or relative
risk is calculated from the data
and provides information about
the incidence of outcomes

Attributable risk can also be
used to demonstrate a
protective effect (e.g.., folic acid
has been shown to decrease
the risk of colon cancer).
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

If relative risk is equal to 1,
the risk is the same for
both exposed and
unexposed subjects and if
it is less than 1, the risk is
less for individuals
exposed to the factor.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

It is important to remember
that relative risk gives an
idea of the size of the
effect, but does not provide
information about precision
or statistical significance of
the result.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Example

on a study on the effects of industrial
formaldehyde exposure on the
development of chronic respiratory illness
(i.e., chronic obstructive pulmonary
disease and emphysema) were assessed.

Risk for the development of respiratory
illness is 200/2000 or 0.10 for those
exposed to formaldehyde and 30/2000 or
0.01 for unexposed subjects.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Example

Attributable risk is the difference of these
two values or (0.10- 0.01) or 0.09, which
translates to 90 per 1000 exposures.

Relative risk is equal to the ratio of these
two numbers (0.10/0.01) or 10. In this
case, risk of respiratory illness is 10 times
greater in individuals exposed to
formaldehyde.
Courtesy of VENTURE ACCELARATOR PARTNERS INC: http://www.vapartners.ca/top-follow-up-b2b-sales-tactics/
Follow-up (cohort) study

Calculation of 95%
confidence intervals is
necessary for evaluation of
the statistical significance of
the results

Studies where loss to
follow-up exceeds 20% in
either the exposed or nonexposed cohort should be
interpreted with caution
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Follow-up (cohort) study

One of the primary
disadvantages of cohort
studies is the cost of followup. For example, if the
outcome is rare, it takes
many years for adequate
assessment of disease
development or to establish
disease-free status.
Courtesy of VENTURE ACCELARATOR
PARTNERS INC:
http://www.vapartners.ca/top-follow-up-b2bsales-tactics/
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
III.


Appraisal Tool for Cohort Studies:
Three broad issues are considered when appraising a
cohort study:
1. Are the results valid?
2. What are the results?
3. Are the results useful locally?
The tool is composed of 12 questions, the first 2 are
screening questions and if the screening questions are
answered “yes” then it worth proceeding to the other 8
questions otherwise the study is invalid and should stop
appraising it.
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool

Screening Questions for Cohort Studies
Question
Hint
1. Did the study
address a clearly
focused issue?
Issue focused in terms of:
The population studied
The intervention given
The outcome considered
Is it clear whether the study tried to detect a
beneficial or harmful effect?
2. Was the cohort
recruited in an
acceptable way?
Look for selection bias which might compromise
the generalisibility of the findings:
Was the cohort representative of a defined
Population?
Was there something special about the Cohort
Was everybody included who should have been
included?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool

Detailed Questions for Cohort Studies
Continue next slide
Questions
3. Was the exposure
accurately measured to
minimize bias?
Hints
Look for measurement or
Classification bias:
Did they use subjective or objective
measurements?
Do the measurements truly reflect
what you want them to (have they been
validated)?
Were all the subjects classified into
exposure groups using the procedure?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Continue next slide
Question
Hints
4. Was the outcome
accurately measured
to minimize bias?
Look for measurement or classification bias:
Did they use subjective or objective
measurements?
Do the measures truly reflect what you
want them to (have they been validated)?
Has a reliable system been established for
detecting all the case (for measuring
disease occurrence)?
Were the measurement methods similar in
the different groups?
Were the subjects and/ or the outcome
assessor blinded to exposure (Does this
matter)?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Continue next slide
Question
Hint
5. (a) Have the authors
identified all important
confounding factors?
List the ones you think might be
important, that the author missed
5. (b) Have they taken
account of the confounding
factors in the design and/or
analysis?
Look for restriction in design, and
techniques e.g.. modeling, stratified,
regression, or sensitivity analysis to
correct, control or adjust for
confounding factors
6. (a) Was the follow up of
the subjects complete
enough?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Continue next slide
Question
Hints
6. (b) Was
the follow
up of
subjects
long
enough?
Consider:
The good or bad effects should have had long
enough to reveal themselves
The persons that lost to follow-up may have different
outcomes than those available for assessment
In an open or dynamic cohort, was there anything
special about the outcome of the people leaving, or the
exposure of the people entering the cohort?
7. What are
the results
of this study
Consider:
What are the bottom line results?
Have they reported the rate or the proportion between
the exposed/unexposed, the ratio/the rate difference?
How strong is the association between exposure and
outcome (RR,)?
What is the absolute risk reduction(ARR)?
yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Continue next slide
Question
Hints
8. How precise are
the results?
Look for the range of the confidence
intervals, if given
9. Do you believe the
results
Consider:
Big effect is hard to ignore!
Can it be due to bias, chance or
confounding?
Are the design and methods of this study
sufficiently flawed to make the results
unreliable?
Bradford Hills criteria (e.g.. time sequence,
dose, dose-response gradient, biological
plausibility, consistency)
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Continue next slide
Yes No I can’t
tell
Question
Hint
10. Can the
results be
applied to local
Population
Was a cohort study the appropriate method to
answer this question ?
Could the subject covered in this study be
sufficiently different from your population to cause
concern?
Is it likely that your local setting differs much from
that of the study?
Can you quantify the local benefits and harms?
11. Do the
results of this
study fit with
other available
evidence
Critical Appraisal Skill Programme
(CASP) – Cohort Studies Appraisal Tool
Question
Hint
12. What are the implications
of this study for practice?
Consider:
One observational study rarely
provides sufficiently robust evidence
to recommend changes to clinical
practice or within health policy
decision making
For certain questions observational
studies provide the only evidence
Recommendations from
observational studies are always
stronger when supported by other
evidence
Yes No I can’t
tell
Case-control study
 Case-control

study
Case-control studies are a
type of observational study
that offer an epidemiologic
research alternative to
cohort studies because they
are less time consuming,
cost less and require a
smaller number of
participants than cohorts
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

In a case-control study,
participants (i.e., cases) with
a particular characteristic
(e.g.., disease) are
compared to a similar group
of participants without the
characteristic (i.e., controls)
to determine risk factors
associated with
development of the
characteristic.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Those factors that occur
more frequently in cases
than controls may be
associated with
development of the
characteristic.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Example
 For
our hypothetical product, drug A,
a case-control study design would be
useful in determining if more deaths
from arrhythmia occurred in groups
treated with drug A than those who
did not receive this agent.
Case-control study

Case-control studies are
commonly conducted
because they are relatively
inexpensive and simple to
perform.

Case-control studies are
more useful than follow-up
studies when diseases
occur infrequently (rare) or
many years after exposure,
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

This type of study design
is considered the most
efficient design for
studying rare diseases.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Because case control
studies are conducted in
the opposite direction of
randomized clinical trials
and follow-up studies, and
are designed to determine
cause rather than effect,
they are sometimes called
"trohoc" studies (i.e.,
"cohort" spelled
backward).
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

A prominent disadvantage
of the case-control study
design is that information
about the exposure and
outcome are collected
simultaneously, so it is
difficult to sort out the
temporal relationship of
the two ( which proceed
which)
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

For example, vaginal
bleeding is an early sign of
uterine cancer but it may
lead to prescribing of
progesterone. An
investigator may later
erroneously conclude that
use of progesterone was
associated with development
of uterine cancer when it fact
the cancer preceded
progesterone use
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control Study

It is extremely difficult to
ensure that both cases
and controls have equal
exposure to all factors,
other than the variable
under investigation, which
may contribute to disease
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control Study

Furthermore, it is often
difficult to determine if the
exposure preceded the
outcome, a situation
termed (Protopathic bias),
where the disease may
lead to exposure to the
risk factor rather than vice
versa.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Matching is often used to
ensure that cases and
controls are similar. With
matching, each case has
a control that is
comparable in terms of
demographic and
exposure characteristics
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

It is often difficult to
determine which variables
should be used for matching
cases to controls (e.g.., sex,
age, date of admission, etc.)
and such matching allows
assessment of only the risk
factor under investigation
and no other variables that
may have contributed to
disease.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Matching, however, may
have a negative impact on
the interpretation of study
results if cases and
controls are matched on a
factor that is itself related
to exposure.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Some experts have
suggested use of several
control groups selected on
the basis of different
criteria in an attempt to
reduce some of the biases
discussed.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study
Overall, due to the
retrospective nature of
case control studies, the
two major methodological
issues are:
1. appropriate selection of
the control group
2. accurate determination of
the level of exposure.

Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Cases and controls should
undergo the same criteria
for evaluation, because
presence of disease is more
likely to be found in
individuals who undergo
extensive diagnostic testing.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

Recall bias occurs when
patients are interviewed
regarding historical events

data collectors should be
blinded to the status of the
participants as cases or
controls.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control Study

During case-control studies,
odds ratios are calculated.
Odds ratio is an estimate of
risk ratio. The interpretation
is the same: greater than
one, increased risk; equal to
one, no effect; less than
one, protective effect.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Case-control study

As with cohort studies,
95% confidence intervals
should be calculated.
Courtesy of Nick and
Zuzu:
http://nickandzuzu.com/20
11/06/finish-start/
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
IV.


Appraisal Tool for Case Control Studies:
Three broad issues are considered when appraising a
cohort study:
1. Are the results valid?
2. What are the results?
3. Are the results useful locally?
The tool is composed of 11 questions, the first 2 are
screening questions and if the screening questions are
answered “yes” then it worth proceeding to the other 9
questions otherwise the study is invalid and should stop
appraising it.
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool

Screening Questions for Case Control Studies
Question
Hints
1. Did the study
address a
clearly focused
issue?
A question can be focused in terms of:
The population studied
The risk factors studied
Whether the study tried to detect a beneficial or
harmful effect?
2. Did the
authors use an
appropriate
method to
answer their
question?
Is a case control study an appropriate way of
answering the question under the circumstances?
(Is the outcome rare or harmful)
Did it address the study question?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool

Detailed Questions for Case Control Studies
Continue next slide
Question
Hint
3. Were
the cases
recruited in
an
acceptable
way?
We are looking for selection bias which might
compromise validity of the findings:
Are the cases representative of a defined population?
Were the cases representative of a defined
population?
Was there an established reliable system for selecting
all the cases?
Are they incident or prevalent?
Is there something special about the cases?
Is the time frame of the study relevant to
disease/exposure?
Was there a sufficient number of cases selected?
Was there a power calculation?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
Continue next slide
Question
4. Were the controls
selected in an
acceptable way?
Hints
We are looking for selection bias which
might compromise the generalisibility of the
findings:
Were the controls representative of defined
population (geographically and/or
temporally)
Was there something special about the
controls?
Was the non-response high? Could nonrespondents be different in any way/
Are they matched, population based or
randomly selected?
Was there a sufficient number of controls
selected
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
Continue next slide
Question
Hints
5. Was the
exposure
accurately
measured to
minimize bias?
We are looking for measurement, recall or
classification bias:
Was the exposure clearly defined and accurately
measured?
Did the authors use subjective or objective
measurements?
Do the measures truly reflect what they are
supposed to measure? (Have they been
validated?)
Were the measurement methods similar in the
cases and controls?
Did the study incorporate blinding where
feasible?
Is the temporal relation correct? (Does the
exposure of interest precede the outcome?)
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
Continue next slide
Question
Hints
6. (a) What
confounding
factors have the
authors
accounted for?
List the ones you think might be important, that
The author missed.
• Genetic
• Environmental
• Socio-economic
6. (b) Have the
authors taken
account of the
potential
confounding
factors in the
design and/or
in their analysis
Look for Restriction in design, and techniques
e.g.. modeling stratified-, regression-, or sensitivity
analysis to correct, control or adjust for
confounding factors
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
Continue next slide
Question
Hints
7. What
are the
results of
this
study?
•What are the bottom line results?
• Is the analysis appropriate to the design?
• How strong is the association between exposure
and outcome (look at the odds ratio)?
• Are the results adjusted for confounding, and
might confounding still explain the association?
• Has adjustment made a big difference to the
OR?
8. How
precise
are the
results?
• Size of the P-value
• Size of the confidence intervals
• Have the authors considered all the important
variables?
• How was the effect of subjects refusing to
participate evaluated?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Case Control Studies Appraisal Tool
Question
9. Do you
believe the
results?
10. Can the
results be
applied to the
local
population?
Hints
• Big effect is hard to ignore!
• Can it be due to chance, bias or confounding?
• Are the design and methods of this study
sufficiently flawed to make the results unreliable?
• Consider Bradford Hills criteria (e.g.. time
sequence, dose-response gradient, strength,
biological plausibility
Could the subjects covered in the study be
sufficiently different from your population to cause
concern?
Is it likely that your local setting differs much from
that of the study?
Can you quantify the local benefits and harms?
Yes
No I can’t
tell
Cross-Sectional Study

Cross-sectional or
"prevalence" studies can be
thought of as a "snapshot"
because data are collected
and evaluated at a single
point in time
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Cross-Sectional Study

A study is classified as
cross-sectional because
measurements are taken
at a single point in time,
even though the
observations may cover
a period of several months
or years.
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Cross-Sectional Study

the data of the following
study were collected from
13,586 adults at two visits:
one with questionnaire,
and the other to take blood
sample for cholesterol
measurement and to
record prescribed
medicine uses.
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Example

Example: UK study looked into the lipids
profile in English population and the use of
lipid lowering agents. Data are collected at
two home visits: One by interviewer to
administer the questionnaire, and one by a
nurse to record the use of prescribed
medicines and to take a blood sample for
lipids biochemistry. The purpose is determine
the prevalence of use of lipid lowering
agents and its relation to blood lipid
concentrations in English adults.
Courtesy of evanto MARKETPLACE: http://graphicriver.net/item/multi-layers/2756209
Example

The conclusion made is that despite the high
prevalence of dyslipidemia in English adults,
the proportion of adults taking lipid lowering
drugs in 1998 was only 2.2%. Rate of
treatment were low among high risk patients
eligible for primary prevention with
medications, and less than one –third of the
patients with established CV disease
received such treatment.
Cross-Sectional Study

A study which evaluate
opinions of a cross section
from the physicians on the
clinical services provided by
pharmacists is another
example

Cross-sectional studies are
most ideally suited for
generating, as opposed to
testing, hypotheses.
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Cross-Sectional Study

Cross-sectional studies are
relatively quick and easy to
perform and may be useful
for measuring current
health status or setting
priorities for disease
control.
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Cross-Sectional Study
As in other observational
trial designs, the research
question and the relevant
inclusion and exclusion
criteria must be clearly
and unambiguously
stated.
Also, selection of cases
must be clearly described
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Cross-Sectional Study

1.
2.
Problems that may occur during
cross-sectional studies include:
Errors in data collection: there
is no prior data for comparison
Transient effect that may
influence observations:
Occurrences present at the
time of study but would not be
identified if study was repeated
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Courtesy of evanto MARKETPLACE: http://graphicriver.net/item/multi-layers/2756209
Example

A good example of transient effects is student
evaluations of university professors. If a
professor chooses to have students evaluate a
course after a particularly grueling examination,
chances are the evaluations would be poor
based on students' response to the examination
just taken. However, if the evaluations were
administered after a curve had been applied for
final grades, students may reflect on the course
positively based on overall knowledge they
received from the instructor, rather than a single
negative experience.
Cross-Sectional Study

Transient effects may only
be uncovered through
retrospective evaluation of
the study by the
investigator.

The investigator must
perform a thorough
assessment of all factors
that may have impacted the
results of the trial.
Courtesy of evanto MARKETPLACE:
http://graphicriver.net/item/multilayers/2756209
Case Study & Case Series

In case reports, the
investigators reported their
observation on individual
patient on points that are
either not known or not
well documented.
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Example

A pharmacist may report a case or a
series of cases on patients with
rheumatoid arthritis with fatal drug-drug
interaction because of the concomitant
use of non steroidal anti-inflammatory
drug with his course of treatment that
include methotrexate.
Courtesy of Invertedi: http://www.invertedi.com/our-clients/
Case Study & Case Series

In contrast to n-of-1 trials
(discussed before), which
report results on a single
patient, a case study is an
observational study rather
than experimental: The
concepts of randomization
and blinding is not applied
to the individual patients
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Case Study & Case Series

Is usually retrospective and
Does not involve multiple
treatment periods

The results in case reports are
either:
 Not compared to a control
group, or
 May be compared to a group
of previously treated patients
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Case Study & Case Series

The design of and methods for
conducting a case study is not
well defined or agreed upon
and the interpretation of case
studies can be difficult. For
example, , a beneficial effects
attributed to the drug or
treatment under investigation
may actually be: a function of
spontaneous regression of the
signs and symptoms of the
disease or placebo effect
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Case Study & Case Series

1.
2.
3.
4.
Case studies are important
in:
Early recognition of drug
toxicities and teratogenicity.
Identifying treatments for
rare disorders where large
subject pools cannot be
identified.
New diagnosis
Formulation of hypotheses.
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Case Study & Case Series

The results can be applied
to patients in clinical
practice only when the case
studies or case series:
 Show a beneficial effect of
a drug or treatment in
diseases whose outcomes
are consistently grim.
 When all other treatments
have failed.
Courtesy of Invertedi:
http://www.invertedi.com/our-clients/
Introduction
Non-Systematic Review
Qualitative Systematic Review
Meta-Analysis
Courtesy of CHRITOPHER P.LONG: http://www.cplong.org/2013/11/the-disciplinary-economy-of-open-peer-review/
Introduction

Overviews are becoming
more prevalent in the
literature, and are being
relied on as an efficient
method for keeping up with
the large amount of
information presented to the
health care professional
each day.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

1.
2.
3.
The term “overview”
encompasses three very
different entities:
Non-systematic (narrative)
review
Systematic qualitative
review
Meta-analysis ( systematic
quantitative review)
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

Non-systematic reviews are
generally considered
tertiary (general)
literatures because they
provide much the same
information as found in
textbooks, but are
sometimes used like
secondary references
because they also contain
extensive bibliographies.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

On the other hand, Is
systematic reviews we
expect to find:
 Extensive literature search
is usually conducted.
 Use variety of resources
such as Medline, PubMed,
etc, to identify studies
related to the subject of
the review.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction
 Focus on clearly defined
population such as for
example “therapeutic and
adverse effects of inhaled
corticosteroids in pediatric
patients population”.
 Includes studies with valid
research methods.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction
 Examine reasons for
differences in study results
and conclusion as for
example the differences in
responses between the
two studies because the
later includes patients with
sever respiratory failure
where the drug is
ineffective in this
population.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction
• Focus on clinically
important outcomes
studies, and consider both
the benefit and risk of
drug therapy

Such skills are necessary
because of the poor quality
of many published narrative
reviews.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

Example of their
shortcomings: it was found
that out of 89 reviews, only
3 (3.4%) described the
methods used in the
literature search

Systematic reviews are
superior to non systematic
reviews in any given topic
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

Systematic reviews address
questions of:
1. Treatment
2. Causation
3. Diagnosis
4. Prognosis
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

All reviews, (narrative and
systematic alike), are
retrospective, observational
research studies and are
therefore subject to
systematic and random
error.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

It is not uncommon to find
that the conclusions of
general overviews,
systematic reviews, or
meta-analyses conflict with
one another.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

Explanations for conflicting
conclusions in systematic
reviews includes: 1)
differences in research
methodology, 2) differences
in study populations, 3) type
of intervention, or 4) study
endpoint, 5) as well as
chance.
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Introduction

The purpose of review
articles is to determine the
“truth” among conflicting
and variable primary
literature; however, this may
not be accomplished or may
conflict with results from
other reviews and readers
need to determine whether
studies included in the
review apply to their
situation
Courtesy of CHRITOPHER P.LONG:
http://www.cplong.org/2013/11/thedisciplinary-economy-of-open-peer-review/
Non-systematic (narrative) review

Review articles that
discussed treatment of
diseases or clinical aspect
of drug therapy.

Considered tertiary literature
because they provide much
the same information as
found in text books (Can be
used as a secondary
sources because it lead to
primary literature).
Courtesy of Snoring Mouthpiece Guide:
http://www.snoringmouthpieceguide.com/sno
remender-product-review-how-does-it-stackup-against-its-competitors/
Non-Systematic (Narrative) Review

Consist of an analysis and
interpretation of previously
conducted research studies.

Enable pharmacists to gain
insight into a topic or
question of interest and may
provide more current
information than textbooks.
Courtesy of Snoring Mouthpiece Guide:
http://www.snoringmouthpieceguide.com/sno
remender-product-review-how-does-it-stackup-against-its-competitors/
Non-Systematic (Narrative) Review

However, they may be
subject to the biases of the
authors and inaccuracies in
the literature search.

This type does not apply
systematic methods and
answers broad rather than
focused clinical questions.
Courtesy of Snoring Mouthpiece Guide:
http://www.snoringmouthpieceguide.com/sno
remender-product-review-how-does-it-stackup-against-its-competitors/
Example

For example: an article which reviewed the
physical and pharmacokinetic properties of
budesonide, outlining how they, safety data,
and use of different inhalation devices
enable budesonide to meet many of the
clinical requirements of an ideal inhaled
corticosteroid for the treatment of asthma
Courtesy of Snoring Mouthpiece Guide:
http://www.snoringmouthpieceguide.com/sno
remender-product-review-how-does-it-stackup-against-its-competitors/
Qualitative Systematic Review

The term “qualitative
systematic review” has been
applied to a summary of
results of primary studies
where the results are not
statistically combined
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Example

For example: In a study the authors elected
to review the data comparing all 5-ASA
agents to placebo, to sulfasalazine, and to
other 5-ASA products for mild to moderate
active UC. Separate MEDLINE and
EMBASE searches of all articles (English
and non-English language) from 1980- 2002
were performed with different combinations
of the following search terms: ulcerative
colitis, 5-aminosalicylate, mesalamine,
mesalazine, therapy, clinical trial,
balsalazide, olsalazine, and randomized.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Example

The bibliographies of all studies that focused
primarily on induction of remission were
manually searched, as were the proceedings
from the annual meetings of the American
Gastroenterological Association and the
American College of Gastroenterology from
1991–2002. Care was taken to exclude
abstracts that were subsequently published
in 212 VOL. 3 NO. 4 2003 Reviews in
Gastroenterological Disorders 5-ASAs and
Active UC continued full manuscript form.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Example

Multiple inflammatory bowel disease
investigators and the manufacturers of all of
the oral 5-ASA compounds available in the
United States were contacted to identify any
additional unpublished trials that compared 5ASA with sulfasalazine, placebo, or other 5ASA products. The selection criteria were as
follows: Two investigators independently
reviewed the titles and abstracts of all
citations identified by the literature search.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Example

Both investigators independently retrieved and
reviewed the manuscripts of potentially relevant
studies and applied detailed selection criteria:
(a) randomized trials with a parallel group
design; (b) study population of adults with mild
or moderate active UC; (c) comparison of an
oral 5-ASA (e.g., mesalamine, balsalazide, or
olsalazine) to placebo, sulfasalazine, or other
5-ASA products; and (d) evaluation of clinical
improvement or remission.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Example

Studies on rectal preparations, combination
therapies, maintenance of remission, or
therapy of Crohn’s disease were excluded. The
methodological quality and the results of each
study were independently abstracted using a
form that was developed by the investigators.
Quality assessment was performed using
standard criteria defined by Guyatt and
colleagues.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Example

colleagues.10 Results are reported according
to the a priori definition of the primary endpoint
described in the study. Post hoc and secondary
analyses were noted as such and were not
reported in the table. Disagreements were
resolved by discussion and consensus.
LEARNING MARKET: http://www.learningmarket.org/page.cfm/Link=138/t=m/goSection=5_35
Qualitative Systematic Review

1.
2.
For valid conclusions to be
derived from qualitative
systematic reviews, the
authors must:
Define the study population
or topic of interest.
Include only those studies
that used valid research
methods.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

The conclusions of the
qualitative systematic
reviews are likely to be very
different depending on the
patient population selected.

Poorly controlled,
nonrandomized, un-blinded
studies should be excluded
from the analysis.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

Poorly controlled,
nonrandomized, unblinded studies should be
excluded from the
analysis.

.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

Authors should use a variety
of resources to identify
studies for the qualitative
systematic review. Single
database is not likely to
capture all relevant studies;
a combination of databases,
study bibliographies, and
experts in the field should
be used to identify studies
for evaluation
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

Consideration should be
given to inclusion of
unpublished (e.g.., data on
file at the manufacturer or
personal communication
with investigators) and
published studies, because
it has been determined that
published studies are more
often of a positive nature
than unpublished studies, a
situation termed publication
bias.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

Because assessment of
studies may be subjective
and biased, two or more
authors should critique the
studies and all should
concur on which studies will
be included in the qualitative
systematic review:
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review

Data should be
summarized in table
format, outcomes
described in the qualitative
systematic review article
should be meaningful,
and, if the trial is a clinical
trial it should be clinically
important.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Qualitative Systematic Review


For example, improved
survival is a more
important endpoint than
reduction in tumor size in
breast cancer patients.
Authors should also
assess benefits versus
risks associated with the
therapy under review, if
possible.
LEARNING MARKET:
http://www.learningmarket.org/page.cfm/Li
nk=138/t=m/goSection=5_35
Meta-analysis

Quantitative systematic
review (Meta analysis),
has been described as a
systematic review that
uses statistical methods to
combine the results of two
or more studies.

It can be used to look at
both clinical trials and
epidemiological research
Meta-analysis

Such studies are now
widely used to provide
supporting evidence for
clinical decision-making.

They provide quantitative
and objective assessment.
Meta-analysis

The design is provide
greater insight into clinical
dilemmas than individual
clinical trials, simply
because it includes more
patients than the number
of patients in each
individual clinical trial.
Meta-analysis



Meta-analyses are
especially useful when:
previous studies have been
inconclusive or
contradictory.
sample size may have been
too small to detect a
statistically significant
difference between
treatment and control
groups.
Meta-analysis


Definitive trials cannot be
conducted.
While awaiting the results of
definitive trials.
Meta-analysis

Sacks and colleagues have
described the following
purposes for performing a
meta-analysis:

To increase the statistical
power for primary
endpoint and for
subgroups.
To resolve uncertainty
when reports disagree.

Meta-analysis



To improve estimates of size
of effect.
To answer new questions
not posed at the start of
individual trials.
To bring about
improvements in the quality
of the primary research.
Meta-analysis




Meta-analysis has been
used to address important
clinical questions such as:
Whether aspirin reduces the
risk of pregnancy-induced
hypertension.
Whether cholesterol
decreases mortality.
Whether fluoxetine
increases suicidal
ideations.
Meta-analysis


Methodological problems
with meta-analyses:
When results from multiple
trials are combined, the
biases of the individual
studies are incorporated
and new sources of bias
arise.
Meta-analysis

The quality of the metaanalysis depends on the
quality of the individual
studies used to develop the
meta-analysis.
Meta-analysis
Publication bias is a major
problem of meta-analyses
where it arises because
investigators are more likely
to publish studies that
demonstrate positive effects
of drugs. Therefore, studies
that show lack of efficacy
are less likely to be located
than those that demonstrate
beneficial effects of a drug.
Meta-analysis

or maybe because of the
inclusion of unpublished
studies which have not been
scrutinized through the
peer-review process.
Meta-analysis

Authors need to address the
validity of articles used in
the meta-analysis such as:
 Randomization techniques
 Compliance
 Blinding
 Intention-to-treat analyses
Meta-analysis

The studies should be
similar enough to allow
pooling of data, and hence
homogeneity should be
assessed using appropriate
statistical tests to document
similarities.
Meta-analysis

To reduce bias in the metaanalysis, only the methods
section of the individual
studies should be used to
determine which studies are
appropriate for inclusion.
Meta-analysis

Investigators collecting data
should be blinded to the
names of the original
articles' authors, place of
publication of the article,
and final results.
Meta-analysis

Appropriate statistical
analyses should be
undertaken such as: 95%
confidence intervals, which
provide the range of values
where the true value lies
95% of the time, should be
calculated.
Meta-analysis

Use of the above criteria when
conducting meta-analyses has
improved in recent years.
However, recently the usefulness
of meta-analysis has been
questioned when the results of
subsequent randomized,
controlled trials did not support
previously published metaanalyses on the same subject as
described above.
Meta-analysis

LeLorier and co-workers
have compared the results
of a series of large,
randomized controlled trials
with those of previously
published meta-analyses
examining the same
questions.
Meta-analysis

They found that the
outcomes of the 12 large,
randomized, controlled trials
studied were not predicted
accurately by the metaanalyses published
previously on the same
topics 35% of the time.
Meta-analysis

The randomized, controlled
clinical trials corresponded
to meta-analyses in terms of
population studied,
therapeutic intervention, and
at least one outcome.
Meta-analysis

In this study, 46% of
divergences in results
involved a positive metaanalysis being followed by a
negative randomized,
controlled trial while the
remaining 54% of identified
divergences involved a
negative meta-analysis
followed by a positive
randomized, controlled trial.
Meta-analysis



Reasons for divergences as
cited by the authors included
Heterogeneity of the trials
included in the metaanalyses.
and publication bias (i.e. the
tendency of investigators to
preferentially submit studies
with positive results for
publication).
Meta-analysis


Several points should be
considered when evaluating
meta-analyses:
A quality Meta-analysis
must clearly define the
clinical question to be
addressed by the analysis
Meta-analysis

Details of literature
searches that were
conducted to locate primary
research articles must be
given, i.e. MEDLINE,
EMBASE, CINAHL,
Biological abstracts on CD,
Cochrane Library……etc
Meta-analysis

Because computerized
searches may not locate all
of the relevant articles, other
resources such as
textbooks, experts in the
field, and reference lists
from clinical studies should
also be consulted.
Meta-analysis


Trials included in and excluded
from the meta-analysis should
be listed.
Strict standards should be
established prior to the initiation
of the meta-analysis to ensure
that the criteria used for the
inclusion of participants, the
administration of the principle
treatment, and the measurement
of outcome events are similar in
all trials studied
Meta-analysis


Types of patients, their
diagnosis, treatments and
therapeutic endpoints used
in the original clinical
studies should be given
The source of financial
support for the original
articles (because this may
represent study bias)should
also be addressed
Meta-analysis
Interpretation of results of
meta-analyses is limited
by:
 What studies were (or
were not) included.
 How well the studies
were (or were not) done.

And how homogenous
(or heterogeneous) the
studies were.
Example

The role of Ipratropium Bromide in the
emergency management of Acute Asthma
Exacerbation was studied (A Meta-analysis)
to determine whether the addition of inhaled
ipratropium to inhaled -agonist therapy is
effective in the treatment of adults with acute
asthma exacerbation.
Example

Published reports of randomized,
controlled trials assessing the use of
ipratropium and -agonists in asthma were
identified by a search of the MEDLINE,
EMBASE, CINAHL, biological Abstracts on
CD, the Cochrane Library, and Current
Contents databases.
Example

Bibliographies from identified studies and
from review articles were manually searched.
Published and unpublished reports in
English, French, and Italian were identified
and assessed for inclusion in the metaanalysis.
Example

Randomized, double-blind, placebocontrolled trials were selected in which
ipratropium was used as adjunctive therapy
to -agonists in adult patients with acute
asthma exacerbation presenting to a hospital
emergency department or similar acute care
setting. Qualified articles criteria: 1) target
population are adult acute asthmatic (exclude
< 18 y, exacerbation of COPD). 2)
Intervention similar to the study. 3) Design
(randomized, placebo controlled).
Example


Data were extracted independently by 2
reviewers.
For eligible trials, the mean percent change
in peak expiratory flow rate (PEFR), or forced
expiratory volume in one second (FEV1), and
their SDs were assessed in the ipratropiumtreated and control groups. The effect of
ipratropium on hospitalization rates and
adverse effects were also analyzed
Example

Data from 10 studies, reporting on a total of
1,377 patients with asthma, were pooled using
a weighted average method. Compared with
placebo, the use of ipratropium was
associated with a pooled 7.3% improvement in
FEV1 (95% confidence interval [CI] 3.8% to
10.9%), corresponding to an absolute
improvement in FEV1 in the ipratropium/ agonist group, which was 100 mL (95% CI 50
to 149 mL) above that seen for the group that
received agonist w/o ipratropiom.
Example

Similarly, the pooled estimate of treatment
effect in trials that reported data as PEFR
was 22.1% (95% CI 11.0% to 33.2%),
corresponding to an absolute peak expiratory
flow improvement of 32 L/min (95% CI 16 to
47 L/min) in favor of the ipratropium/ -agonist
combination group. When these data were
combined using effect size as a common
measure, the use of ipratropium was
associated with a summary effect size of .38
(95% CI .27 to .48).
Example

Effect sizes were negatively correlated with
baseline mean expiratory flows, suggesting
that studies enrolling patients with more
severe airflow obstruction showed greater
absolute benefits of combination
bronchodilator therapy. For the 3 trials
reporting hospital admission data (n=1,031),
patients receiving ipratropium had a relative
risk of hospitalization of .73 (95% CI .53 to
.99).
Example

The use of ipratropium was not associated
with any severe adverse effects when used in
conjunction with 2-agonists. The conclusion
made that there is a modest statistical
improvement in airflow obstruction when
ipratropium is used as an adjunctive
treatment to 2-agonists for the treatment of
acute asthma exacerbation.
Example

Although the clinical significance of this
improvement in airflow obstruction remains
unclear, it would seem reasonable to
recommend the use of combination
ipratropium/ -agonist therapy in acute adult
asthmatic exacerbations, since the addition
of ipratropium seemed to provide physiologic
evidence of benefit without risk of adverse
effects
Example

.[Stoodley RG, Aaron SD, Dales RE: The role
of ipratropium bromide in the emergency
management of acute asthma exacerbation:
A meta-analysis of randomized clinical trials.
Ann Emerg Med July 1999;34:8-18.]
Critical Appraisal Skill Programme
(CASP) – Systematic Review Appraisal Tool
II.


Appraisal Tool for Systematic Reviews:
Three broad issues are considered when appraising
RCTs:
1. Are the results valid?
2. What are the results?
3. Are the results useful locally?
The tool is composed of 10 questions, the first 2 are
screening questions and if these 2 screening questions
are answered “yes” then it worth proceeding to the other
8 questions otherwise the study is invalid and should
stop appraising it.
Critical Appraisal Skill Programme
(CASP) – Systematic Review Appraisal Tool

Screening Questions for Systematic Reviews
Question
Hint
1. Did the review
address a clearly
focused question?
Issue focused in terms of:
The population studied
The intervention given
The outcome considered
2. Did the author look
for the right type of
papers?
“the best sort of studies” would :
Address the reviews question
Have an appropriate study design
(usually RCTs for papers evaluating
interventions)
Yes
No I Can’t
tell
Critical Appraisal Skill Programme
(CASP) – Systematic Review Appraisal Tool

Detailed questions for systematic reviews
Continue next slide
Question
3. Do you think the
important, relevant
studies were included?
Hint
Look up for:
Which bibliographic databases were used
Follow up from reference lists
Personal contact with experts
Search for unpublished as well as
published studies
Search for non-English language studies
4. Did the review’s
The authors need to consider the rigour of
authors do enough to
the studies they have identified. Lack of
assess the quality of the rigour may affect the studies’ result.
included studies?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Systematic Review Appraisal Tool
Continue next slide
Question
Hint
5. If the results of the
review have been
combined, was it
reasonable to do so
•Were the results similar from study to
study?
•Were the results of all the included
studies clearly displayed?
Were the results of the different studies
similar?
•The reasons for any variations in results
are discussed
6. What are the overall
results of the review?
Are you clear about the review’s
“bottom line” results?
What are these? (numerically if
appropriate).
How were the results expressed (NNT,
odds ratio etc)
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP) – Systematic Review Appraisal Tool
Question
Hint
7. How precise are the
results?
Look at the confidence intervals, if given
8. Can the results be
applied to the local
population?
Could the patients covered by the
review be sufficiently different to your
population to cause concern?
Is it likely that Your local setting differ
much from that of the review?
9. Were all important
outcomes considered
Is there other Information you would like
to have seen?
10. Are the benefits worth
the harms and costs?
What do you think? (even if this was not
addressed by the review)
Yes No I can’t
tell
Pharmacoeconomic Study
Quality of Life Assessment Study
Introduction

Health outcomes research
encompasses the areas of
pharmacoeconomics and
quality of life assessments
Courtesy of RX OUTCOMES ADVISER:
http://rxoutcomesadviser.wordpress.com/tag/
health-outcomes/
Pharmacoeconomic Study

Allow determination of drug
or health care services that
provide the best outcome
for the money invested.

In pharmacy it determines
which drug to be included in
the formulary as well as
which services (drug
information, nutritional
support) to be included
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

There are 4 types of
pharmacoeconomic
evaluations:
1. Cost-minimization
2. Cost-benefit analyses
3. Cost-effectiveness
4. Cost-utility analyses
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

Cost and benefits are often
identified inappropriately in
pharmacoeconomic
research.

All possible alternatives,
beneficial and unwanted
outcomes needs to be
considered in addition to all
forms of costs
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study
 Cost-minimization


Look only to the cost
because the two drugs or
services are assumed to be
equal.
It is rare because
therapeutic equality is rare,
and because outcome is
not measured some people
believe that it is not
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

It can be used to decide
which drug (e.g..
antibiotic)can be included in
the formulary.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Example


The once daily aminoglycoside minimizes
hospital cost (97.63 dollar) versus (154.46
dollar) for the three times daily IV
administration of aminoglycoside.
In this study outcomes were not measured
because the two dosage regimens were
assumed to be equivalent.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study
 Cost-benefit
analyses

Place a monetary value on
both input and outcomes
 Results expressed as ratio
(benefit to cost) or in terms
of net cost or benefit
(benefit-cost).
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

Performed when the
objective of the study is to
obtain a single
measurement of the net
monetary value of a drug or
service.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

Methodologically difficult
because of difficulty in
converting therapeutic
outcome (e.g.. reduce
hospital stay) to monetary
term
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Example

pharmacokinetic services versus diabetic clinic
can be compared with the cost saving
(outcomes) associated with each service, even
though different type of outcomes are expected
for each alternative.

Pharmacokinetic service: total cost $90,000 and
revenue (benefit) $120,000 whereas Diabetic
clinic service total cost $ 100,000 and revenue
(benefit)$ 120,000.
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Example

Benefit to cost ratio =PKN services =
120,000/90,000 =1.33:1
= diabetic services = 135,000/100,000 =1.35:1

Net benefit = 120,000-90,000 = 30,000
= 135,000-100,000 =35,000

Results expressed as ratio (benefit to cost) or in
terms of net cost or benefit (benefit-cost).
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Pharmacoeconomic Study
 Cost-effectiveness

To designate the drug
therapy or service that is
least expensive.

Done when the purpose is
to determine relative
efficacy of various drugs or
services in achieving
desired outcome.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Pharmacoeconomic Study

Compare cost of drugs or
services to their therapeutic
outcomes.

It is the most common type
of pharmacoeconomic
studies.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Example

New cholesterol lowering agent versus the
current medication. The two treatments were
used for about 12 months. The costs of each
medication over this period were estimated to
be $1,000 and $1500 respectively. The
cholesterol level lower at the end of the period
by 25 mg/dl and 30 mg/dl respectively. The
average cost per reduction in cholesterol level =
1000/25= $ 40 per mg/dl and 1500/35 = $ 50
per mg/dl.
.
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Example

A marginal cost effectiveness is then
calculated (1500-1000)/30-25 =100 per
mg/dl.
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Pharmacoeconomic Study
 Cost-utility analyses
 Relate therapeutic outcome
to both cost and patient
preference.

Utility refers to the amount
of satisfaction obtained
from a particular therapy or
program (cost-utility
measure cost per unit of
utility.
Courtesy of Events Health:
http://www.eventshealth.com/event/3rdannual-health-outcomes-andpharmacoeconomics-research-conference/
Example

Vindesine and cisplatin (VP), versus
cyclophosphamide, doxorubicin and cisplatin
(CAP), versus Best Supportive Care (BSC) in
patients with lung cancer.

Survival were 214 days, 165 days and 112 days
respectively for the three treatments systems.

Quality of life were 0.19, 0.15 and 0.19
respectively
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Example



The costs of the three 10,000, 7000, and
5000.
The cost utility ratio: 52,000 versus 44,000
for the BSC
BSC at least as effective as chemotherapy
and less expensive
Courtesy of Events Health: http://www.eventshealth.com/event/3rd-annual-health-outcomes-and-pharmacoeconomics-researchconference/
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool
VI.
Appraisal Tool for Economic Evaluation:

Three broad issues are considered when appraising a
cohort study:
1.
Is the economic evaluation valid?
How were costs and consequences assessed & compared?
Will the results help on purchasing services for local people?
2.
3.

The tool is composed of 12 questions, the first 2 are
screening questions and if the screening questions are
answered “yes” then it worth proceeding to the other 10
questions otherwise the research is invalid and should
stop appraising it.
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool

Screening Questions for Economic Evaluation
Question
Hints
1. Was a welldefined
question
posed?
Is it clear what the authors are trying to achieve?
• What is the perspective?
• How many options are compared?
• Are both costs and consequences considered?
• What is the time horizon?
2. Was a
comprehensive
description of
the competing
alternatives
given?
Is there a clear decision tree (or similar given):
• Can you tell who did what, to whom, where and
how often?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool

Detailed Questions for Economic Evaluation
Continue next slide
Question
Hints
3. Does the paper
provide evidence that
the pragramme would
be effective (i.e. would
the pragramme do more
good than harm)?
Was an RCT or systematic review was
used?; if not consider how strong the
evidence was
(Economic evaluations frequently have to
integrate different types of knowledge
stemming from different study designs)
4. Were the effects of
the intervention
identified, measure and
valued appropriately?
Effects can be measured in natural units
(e.g.. years of life) or more complex units
(e.g.. years adjusted for quality of life
such as QALYs) or monetary equivalents
of the benefit gained (e.g.. $)
Yes
No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool
Question
Hints
5. Were all
important and
relevant resources
required and
health outcome
costs for each
alternative
identified,
measured in
appropriate units
and valued
credibly?
 Identified?
 Remember the perspective being taken
Measured accurately in appropriate units prior
to evaluation?
 Appropriate units may be hours of nursing
time, number of physician visits, years-of-life
gained etc.
 Valued credibly?
 Are the values realistic?
 How have they been derived?
 Have opportunity costs been considered?
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool
Continue next slide
Question
Hints
6. Were costs and
consequences adjusted for
different times at which
they occurred
(discounting)?
7. What were the results of • What is the bottom line?
the evaluation?
•What units were used (e.g.. cost/life
year gained, Cost/QALY, net benefit)?
8. Was an incremental
analysis of the
consequences and cost of
alternatives performed?
Yes No
I cant
tell
Critical Appraisal Skill Programme
(CASP)– Economic Evaluations Appraisal Tool
Question
Hints
9. Was an adequate
sensitivity analysis
performed?
• Were all the main areas of uncertainty
considered by changing the estimate of
the variable ?
• How would this change the result of the
economic evaluation ?
10. Is the programme
likely to be equally
effective in your context
or setting?
•Could the patients covered by the review
be sufficiently different to your population
to cause concern?
•Is it likely that your local setting differs
much from that of the review?
11. Are the costs
translatable to your
setting?
12. Is it worth doing in
your setting?
Yes No I can’t
tell
Quality of Life Assessment Study

Health-related quality of
life (HR-QOL) includes
many dimensions or
domains of health status
such as physical, social
and cognitive functioning,
mental health, symptom
tolerance, and overall
well-being.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

Each dimension of health
may be reported as:
1. Composite score.
2. Sub-score for each specific
dimension.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study


the Short-Form 36 (SF-36), a
generic quality of life (QOL)
instrument, measures eight
health concepts
These scales can be scored
individually (sub-score) or
grouped to provide summary
(composite) scores (overall
physical health and overall
mental health) and they are:
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study
vitality
2. physical functioning
3. bodily pain
4. general health perceptions
5. physical role functioning
6. emotional role functioning
7. social role functioning
8. mental health
1.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

The addition of HR-QOL
measures to clinical and
economic trials of
medical interventions
allows the evaluator to
assess the impact at the
level of the individual.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

The QO L reporting has
been increasing in the last
decade (from 1% in 1980 to
about 4% in 1997);
however, Sanders and
colleagues reported that
only 5% of all randomized
controlled trials and 10% of
cancer studies reported HRQOL measures.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

QOL is a relatively new
concept and the findings of
limited reporting practices
may reflect confusion
about the term "quality of
life" and the resultant lack
of a conceptual basis for
HR-QOL measures.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

1.
2.


There are two types of
instruments:
Generic instruments
Specific instruments
Generic instruments:
are available for general use in
different populations for
measuring changes in HR-QOL
across a broad spectrum of
diseases and treatments
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study


They are composed of a
standard set of health
dimensions.
One of the most commonly
used generic instruments is
the SF-36.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

Because this type of
instrument can be used for
different diseased and
patient populations. The
results derived from generic
instruments can be used to
compare impact of different
treatments in the same
disease, or impact of
treatments across diseases.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

They may not, however, be
able to detect QOL
differences within a specific
disease state.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

Specific instruments: such
as the Health Assessment
Questionnaire (MQ), are
required to obtain diseasespecific or populationspecific QOL information.
These types of instruments
are not suited for crosscondition comparison.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

When evaluating studies
that report HR-QOL
measures, there are several
issues the reader must keep
in mind:
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study
1.

Score versus clinical
significance:
For most HR-QOL
instruments, changes in
score that constitute
clinically important
differences are not known.
This makes interpretation
of HR-QOL results from
clinical trials difficult.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

The availability of a
benchmarking process for
determining clinically
important differences
would improve
interpretation of HRQOL
results from clinical trials.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Example

The Expanded Disability Status Scale is
used for grading levels of disability

in multiple sclerosis and ranges from 0 to 10.
Varying levels of disability are broken down
by 0.5 unit increments (i.e., "no disability"
falls into the 1 to 1.5 range and "minimal
disability" falls into the 2 to 2.5 range).
Courtesy of THE NEW PIONEERS: http://thenewpioneers.biz/tag/quality-of-life-indicators/
Example

If anew agent for the treatment of multiple
sclerosis demonstrates an improvement of
0.25 (i.e., causes that patient's score to
decrease from 1.5 to 1.25) on this scale, it is
difficult to determine whether or not this is a
clinically important difference.
Courtesy of THE NEW PIONEERS: http://thenewpioneers.biz/tag/quality-of-life-indicators/
Quality of Life Assessment Study
2.
Lack of reporting:
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Example

Gill and Feinstein evaluated how well HRQOL is being measured in the medical
literature. Seventy-five (75) articles were
randomly selected for inclusion in the
analysis.

Investigators conceptually defined HR-QOL
in only 11 (15%) of the 75 articles and
identified target domains in only 35 (47%)
Courtesy of THE NEW PIONEERS: http://thenewpioneers.biz/tag/quality-of-life-indicators/
Example

Reasons for selecting the HR-QOL instrument
were given in 27 (36%) and results were
aggregated into a composite "HR-QOL-score in
only 13 articles (17%).

The authors concluded that HR-QOL is a uniquely
personal perception, denoting the way individual
patients feel about their health status and/ or nonmedical aspects of their lives, and that most
measurements of HR-QOL in the biomedical
literature do not capture this important concept."
Courtesy of THE NEW PIONEERS: http://thenewpioneers.biz/tag/quality-of-life-indicators/
Quality of Life Assessment Study
3.
Lack in the consistency
and availability of
standards for measuring
HR-QOL
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Example

Of the 67 studies sampled in a bibliographic
study by Sanders and associates, 48 used
62 different pre-existing instruments and a
further 15 studies reported new measures.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study
4.

Validity of HR-QOL
measures across
different cultures or
subcultures is rarely
reported
Culturally defined factors
may impact patient HRQOL and its assessment.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

Accurate measurement of
HR-QOL is very difficult
without extensive validation
efforts in every cultural
population of interest. As
more experience is gained
with QOL studies, the
issues of clinical
significance and validated
instruments may be
resolved. Readers must
understand that this field is
growing rapidly and being
perfected.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study
5.

Response rates
When reviewing QOL
studies, response rates are
critical for reporting HRQOL as non-response can
introduce serious bias. The
fact that response rates
were unreported in almost
a half of the trials analyzed
by Sanders and co-workers
was a matter of concern.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

The authors also found that
selective reporting of
favorable or statistically
significant results was a
problem.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study


Important aspects of trials
reporting on HR-QOL data
and questions to ask when
reviewing trials that report
on HR-QOL including:
validity and the
applicability of the HRQOL
instrument to the disease
state or treatment under
investigation should be
documented
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

The HR-QO L instrument
should be sensitive to
changes in the patients'
status throughout the
clinical trial, and should
measure aspects of the
patient's lives that the
patients consider to be
important.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

if the HR-QOL instrument is
used multiple times, a training
effect may occur by which the
performance of either the
patient or assessor or both
improves or is modified simply
through the multiple use of the
instrument. The reviewer of
such a trial needs to ask if
this type of effect was
present and how it may
have influenced the results
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

Did all sites in a multi-center
trial performed the HRQOL
analyses.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study

1.
2.
Summary to the aspects
and questions to ask when
reviewing trials that report
on HR-QOL including:
Did the study design
stimulate clinical practice.
Were aspects of patients
lives that patients
considered important
measured.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Quality of Life Assessment Study
3.
4.
5.
Are the results biased by
the presence of training
effects.
If it is a multicenter trial, did
all of the sites evaluate
QOL
Is the QOL instrument used
valid for examining specific
disease in question.
Courtesy of THE NEW PIONEERS:
http://thenewpioneers.biz/tag/quality-of-lifeindicators/
Clinical Prediction Rule

A research study that aims
to find the tool that shows
how much each aspect of
a certain medical case (i.e.
physical examination, basic
laboratory and history)
contributes to either
prognosis, diagnosis or the
response to a certain
treatment.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule

They are mathematical
tools that helps clinicians in
decision making.

Idealy these rules should
have both high sensitivity
and specificity.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule

Sensitivity: is the ability of
the tool to identify as high a
proportion as possible of
patients who will actually
develop the outcome of
interest.

Specificity: is the ability of
the tool to exclude those
patients who will not
develop the outcome6
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule

However, sensitivity and
specificity are mutually
exclusive and in reality
there must be a trade off
between the two.

The establishment of a
prediction rule passes
through the following four
phases:
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule
1.
2.
3.
4.
Development: identification
of predictors.
Validation: testing of the
rule.
Impact analysis:
measurement of the
usefulness of the rule.
Implementation:
widespread acceptance
and adoption.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule

1.
Properly developed and
validated clinical prediction
rules have the following
advantages over human
clinical decision:
Statistical models can
accommodate many more
factors than the human
brain is capable of
considering.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule
2.
3.
Using the same data, a
statistical model always
give the same results
without inconsistencies or
disparities in contrast to
human judgment.
Several prediction models
have been shown to be
more accurate than
judgment alone.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Clinical Prediction Rule

1.
2.
3.
However, they have some
disadvantages:
Many clinicians still don’t
know how to use these
tools.
Might be considered as
user unfriendly.
The large number of
existing models available
for the same problem.
Courtesy of
http://arthritisrelie
ved.com/signs-ofarthritissymptoms/
Example

The CHADS2 score is a clinical prediction
rule That estimate the risk of stroke in
patients with non-rhematic atrial
fibrillation (AF), it is used to determine
whether or not treatment is required with
anticoagulation therapy or antiplatelet
therapy
Courtesy of http://arthritisrelieved.com/signs-of-arthritis-symptoms/
Example
Score
1
1
A
D
Condition
Congestive heart failure
Hypertension: blood pressure consistently
above 140/90 mmHg (or treated
hypertension on medication)
Age>=75 years
Diabetes mellitus
S2
Prior Stroke or TIA or Thromboembolism
2
C
H
Courtesy of http://arthritisrelieved.com/signs-of-arthritis-symptoms/
1
1
Example

In a prospective cohort study, Cage et al
tested the predictive accuracy of 5 stroke
risk stratification schemes on 2580
participants with nonvalvular AF who were
prescribed aspirin.All the schemes predicted
stroke better than chance, however, the
CHADS(2) identified primary prevention
patients who were at high risk of stroke (5.3
stokes per 100 patients-year) better than
other rules7 .
Courtesy of http://arthritisrelieved.com/signs-of-arthritis-symptoms/
CASP Appraisal Tool for Clinical
Prediction Rule Test
CASP Appraisal tools for Diagnostic
Study
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool
VII.
Appraisal Tool for Diagnostic Test:

Three broad issues are considered when appraising
a cohort study:
1.
Are the results of the study valid?
What are the results?
Will the results help me and my patients/population?
2.
3.

The tool is composed of 12 questions, the first 2 are
screening questions and if the screening questions
are answered “yes” then it worth proceeding to the
other 10 questions otherwise the study is invalid
and should stop appraising it.
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool

Screening Questions for Diagnostic Test
Question
Hints
1. Was there a
clear question
for the study to
address?
A question should include information about
• The population
•The test
•The setting
• The outcomes
2. Was there a
comparison
with an
appropriate
reference
standard?
Is this reference test(s) the best available indicator
in the circumstances?
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool

Detailed Questions for Diagnostic Test
Continue next slide
Question
3. Did all patients get the
diagnostic test and
reference standard?
Hints
•Were both received regardless of the
results of the test of interest
• Check the 2X2 table (verification bias)
4. Could the results of
the test have been
• Was there blinding?
influenced by the results • Were the tests performed independently
of the reference standard • (Review bias)
5. Is the disease status
of the tested population
clearly described?
• Presenting symptoms
• Disease stage or severity
• Co-morbidity
• Differential diagnoses (Spectrum Bias)
Yes No I can’t
tell
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool
Question
Hints
6. Were the methods for
performing the test
described in sufficient
detail?
Was a protocol followed?
7. What are the results?
•Are the sensitivity and specificity and/or
Are the results presented in such a way
likelihood ratios presented?
•that we can work them out?
8. How sure are we
about the results,
consequences and cost
of alternatives
performed?
• Could they have occurred by chance?
• Are there confidence limits?
• What are they?
Continue next slide
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool
Continue next slide
Question
Hints
9. Can the result be applied Do you think your patients/population
to your patients/the
are so different from those in the study
population of interest?
that the results cannot be applied?
Such as age, sex, ethnicity and
spectrum bias
10. Can the test be applied
to your patient or
population of interest?
• Resources and opportunity costs
• Level and availability of expertise
required to Interpret the tests
• Current practice and availability of
services
11. Were all outcomes
important to the individual
or population considered?
•Will the knowledge of the test result
improve patient wellbeing?
• Will the knowledge of the test result
lead to a change in patient
management?
Yes
No
I cant
tell
Critical Appraisal Skill Programme
(CASP)– Diagnostic Test Appraisal Tool
Question
12. What would be the
impact of using this test on
your patients/population?
Hints
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study
VIII.
Appraisal Tool for CPR study:

Three broad issues are considered when appraising
a cohort study:
1.
Are the results of the study valid?
What are the results?
Will the results help locally?
2.
3.

The tool is composed of 11 questions, the first 3 are
screening questions and if the screening questions
are answered “yes” then it worth proceeding to the
other 10 questions otherwise the study is invalid
and should stop appraising it.
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study

Screening Questions for CPR study
Continue next slide
Question
Hints
1. Is the CPR clearly
defined?
• Is the type of patients to whom the CPR will
be applied clearly defined?
• Are the variables included in the rule clearly
defined?
• Is the outcome relevant and is it clinically
reasonable? (The outcome can be expressed
as a probability or as course of action)
2. The population
from which the rule
was derived included
an appropriate
spectrum of patients?
• Is it adequate the way the patients were
selected?
• Is the spectrum of patient to whom the rule
will apply is well represented?
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study
Question
3. Was the rule
validated in a
different group
of patients?
Hints
• It is not good enough that the rule had a good
performance on the patient group used to
derive it. The rule should be validated in a set
of patients different from those who served
to derive the rule
• The validation was done in a group of patients
similar to the one used to derive it
Yes No I cant
tell
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study

Detailed Questions for CPR study
Question
Continue next slide
Hints
4. Were the predictor
variables and the
outcome evaluated in a
blinded fashion?
• People evaluating the outcome know the
predictor variables?
• People evaluating the predictor variables
know the outcome
5. Were the predictor
variables and the
outcome evaluates in
the whole sample
selected initially?
• Are exclusions and drop outs well
described and do the authors discuss the
reasons for them?
• Sometimes the outcome cannot be
measured in the same way in all patients
6. Are the statistical
methods used to
construct & validate
the rule clearly
described?
• Were all important variables included and
the positivity criteria explained?
• Is the statistical method adequately
described?
• Was reliability of the rule considered?
Yes No
I cant
tell
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study
Question
Hints
Continue next slide
Yes No I cant
tell
7. Can the
performance of
the rule be
calculated?
•Performance results can be presented as: Sens,
Sp, + LR, -LR, ROC curve, calibration curves etc.
8. How precise
was the
estimate of the
treatment
effect?
•Did they try to refine the rule with other variables
to see whether the precision could be improved or
the rule simplified?
•What is the sample size and the number of
variables included in the CPR?
•Is the rule robust? Has there been any attempt to
refine it?
Critical Appraisal Skill Programme
(CASP)– Clinical Prediction Rule (CPR) Study
Question
Hints
9. Would the prediction
rule be reliable and the
results interpretable if
used for your patient?
• Is your setting too different from that of
the study?
10. Is the rule
acceptable in your
case?
• The ease of use and the availability of the
rule and the costs
• If the rule is reasonable from a clinical
point of view
11. Would the results of
the rule modify your
decision about the
management of the
patient or the
information you can
give to him/her?
• In addition to your opinion, might there be
studies analysing the impact (in monetary
terms or health results) of the rule?
• If nothing will change, the rule is at best
useless in terms of benefit to the patients.
• How the initial estimation has changed
after applying the rule, and the effect it has
had on the action threshold
Yes No I cant
tell
References
1.
2.
Bandolier “Evidence based thinking about
health care”, 2004. Glossary index.
http://www.medicine.ox.ac.uk/bandolier/glo
ssary.html, 17/12/2013
Stipp D, June 2003. A Little Poison Can
Be Good For You The received wisdom
about toxins and radiation may be all wet.
Fortune.http://money.cnn.com/magazines/f
ortune/fortune_archive/2003/06/09/343948/
index.htm, 17/12/2013
References
3. Schulz KF, Altman DG, Moher D, for the
CONSORT Group. CONSORT 2010
Statement: updated guidelines for reporting
parallel group randomised trials. BMJ
2010;340:c332.
4. Shiell A, Donaldson C, Mitton C, Currie G.
Health economic evaluation. J Epidemiol
Community Health 2002;56:85-88.
Because
we’ve
always
done it this
way!
oDefinitions
Background
oTerminology
o
Definitions

“Evidence based medicine is the conscientious, explicit, and
judicious use of current best evidence in making decisions
about the care of individual patients. The practice of
evidence based medicine means integrating individual
clinical expertise with the best available external clinical
evidence from systematic research” (Sackett, 1996)1.

“the use of mathematical estimates of the risk of benefit
and harm, derived from high-quality research on
population sample, to inform clinical decision-making in
the diagnosis, investigation or management of individual
patients.“ (Greenhalgh, 2010)2.
Definitions

“Evidence based medicine is the process of
systematically finding, appraising, and using
contemporaneous research findings as the basis for
clinical decisions” (Rosenberg et al, 1995)3.
Background


Common sources used by clinicians to make a certain
decision include:
1. Experience, for example adverse drug reaction.
2. Reasoning and Intuition
3. Advice from colleagues
4. Published Evidence
Only through the use of published evidence the use of
costly, ineffective or harmful interventions can be
reduced.
Background

The concepts of EBM were laid by Dr. Archibald L.
Cochrane in 1972 when his influential book:
“Effectiveness and Efficiency: “Random Reflections on
Health Services”, was first published. Dr. Cochrane is
honored by giving his name to the Cochrane database
of systematic reviews and the Cochrane Collaboration.

These concepts were embodied into practical methods
by groups of scientists in Canada, USA and UK during
the 1980s and 1990s.

The establishment of the Cochrane Collaboration is in
1992 by the UK government is considered to be one of
the main factors in the spreading of EBM4.
Background

EBM has its basis in clinical
epidemiology5, it incorporate epidemiology
into clinical practice and aim to bridge the
gap between them. Epidemiology is the
study (or the science of the study) of the
patterns, causes, and effects of health and
disease conditions in defined populations.
It is the cornerstone of public health, and
informs policy decisions and evidencebased medicine by identifying risk factors
for disease and targets for preventive
medicine.
Background
 EBM
reflects shift to concentrate on
relevant patient problems (“just in
time” education) which is:
 Memorable
 Up to date
 Relevant to the clinician practice
Background

•
The purpose of the implementation of EBM is to achieve
acceptable Clinical Effectiveness.
Clinical Effectiveness aims to ensure that each individual
patient receives the appropriate and best available treatment.
It is composed of three elements:
1. Evidence Based Medicine (EBM).
2. Clinical Expertise
3. Patient factors (i.e. concurrent disease(s) cultural and
Religious background, insurance and financial situation,
race….etc)
Background

“The practice of evidence-based medicine is a process
of lifelong, self-directed, problem-based learning in
which caring for one's own patients creates the need
for clinically important information about diagnosis,
prognosis, therapy and other clinical and health care
issues.”14
Background
Elements of Clinical Effectiveness
EBM
Clinical
Expertise
Patient
Factors
Clinical Effectiveness
Background

Clinical Effectiveness itself is an element of the bigger
concept of Clinical governance which is the systematic
approach of maintaining and improving the quality of
patient care within a health system:
Background
Education &
Training
Risk
Management
Elements of
Clinical
Governance
Clinical
effectiveness
Clinical
governance
Openness
Clinical audit
Research &
Development
Background

“Evidence-Based Practice” or “Evidence-Based
Healthcare” extends the application of EBM to include
other allied medical professions i.e. pharmacy
technician and medical laboratory technician.

EBM skills expected from all clinicians 13:
 Information mastery: finding the best evidence for
every day practice
 Have at fingertips “just in time’ information at point of
care for clinical decision making either web based
and/or using Personal Digital Assistant
 Evaluate expert-based information, including
colleagues, CME, presentations, reviews and
guidelines
Background

Only a small percentage of clinicians need to be able to
do the following:
 Critical Appraisal and Interpretation of Research on:
Therapies, Diagnostic Tests, Prognosis
 Critical Evaluation and Interpretation of: Systematic
Reviews, Including Meta-analysis, Decision Analysis,
Practice Guidelines, Pharmaceutical Advertising,
Including Pharmaceutical Representatives
 Assigning Levels of Evidence to Research Findings
 Teaching Level 1 Skills
 Produce written Communication of Research
Findings for
physicians and patients (Slawson et al, 2005)
Terminology
The following are examples of terms used in EBM are
obtained from Bandolier
(http://www.medicine.ox.ac.uk/bandolier/), an online
journal about evidence-based healthcare.
 Event rate
The proportion of patients in a group in whom the event
is observed. Thus, if out of 100 patients, the event is
observed in 27, the event rate is 0.27 or 27%. Control
event rate (CER) and experimental event rate (EER)
are used to refer to this in control and experimental
groups of patients respectively.
The patient expected event rate (PEER) refers to the
rate of events we would expect in a patient who
received no treatment or conventional treatment.
Terminology

Experimental event rate (EER)
The rate at which events occur in an experimental
group. It may be represented by a percentage (say
50%) or as a proportion (when it is 0.5).

Control event rate (CER)
The rate at which events occur in a control group. It
may be represented by a percentage (say 10%) or as a
proportion (when it is 0.1).
Terminology

Absolute risk reduction/increase
The absolute arithmetic difference in rates of bad
outcomes between experimental and control
participants in a trial, calculated as the
experimental event rate (EER) and the control
event rate (CER), and accompanied by a 95% CI.
Depending on circumstances it can be reduction in
risk (death or cardiovascular outcomes, for
instance, in trials of statins), or an increase (pain
relied, for instance, in trials of analgesics).
Terminology

Confidence interval (CI)
Quantifies the uncertainty in measurement. It is usually
reported as 95% CI, which is the range of values within
which we can be 95% sure that the true value for the whole
population lies. For example, for an NNT of 10 with a 95% CI
of 5 and 15, we would have 95% confidence that the true
NNT value was between 5 and 15.

Number needed to harm (NNH)
This is calculated in the same way as for NNT, but used to
describe adverse events. For NNH, large numbers are good,
because they mean that adverse events are rare. Small
values for NNH are bad, because they mean adverse events
are common.
Terminology

Number needed to treat (NNT)
The inverse of the absolute risk reduction or increase
and the number of patients that need to be treated for
one to benefit compared with a control. The ideal NNT
is 1, where everyone has improved with treatment and
no-one has with control. The higher the NNT, the less
effective is the treatment. But the value of an NNT is
not just numeric. For instance, NNTs of 2-5 are
indicative of effective therapies, like analgesics for
acute pain. NNts of about 1 might be seen by treating
sensitive bacterial infections with antibiotics, while an
NNT of 40 or more might be useful, as when using
aspirin after a heart attack.
Terminology

Odds
A ratio of the number of people incurring an event to the
number of people who have non-events.

Odds ratio
The ratio of the odds of having the target disorder in the
experimental group relative to the odds in favour of
having the target disorder in the control group (in
cohort studies or systematic reviews) or the odds in
favour of being exposed in subjects with the target
disorder divided by the odds in favour of being exposed
in control subjects (without the target disorder).
Terminology
Relative risk
The ratio of risk in the treated group (EER) to risk in the
control group (CER).
RR = ERR/CER
It used in randomized trials and cohort studies Also use
this.
 Relative risk reduction (RRR)
The relative risk reduction is the difference between the
EER and CER (EER-CER) divided by the CER, and
usually expressed as a percentage.

Terminology

Statistical power
The ability of a study to demonstrate an association or
causal relationship between two variables, given that
an association exists. For example, 80% power in a
clinical trial means that the study has a 80% chance of
ending up with a p value of less than 5% in a statistical
test (i.e. a statistically significant treatment effect) if
there really was an important difference (e.g. 10%
versus 5% mortality) between treatments. If the
statistical power of a study IS low, the study results will
be questionable (the study might have been too small
to detect any differences). By convention, 80% is an
acceptable level of power
Terminology

P-value
The probability (ranging from zero to one) that the
results observed in a study (or results more extreme)
could have occurred by chance. Convention is that we
accept a p value of 0.05 or below as being statistically
significant. That means a chance of 1 in 20, which is
not very unlikely. This convention has no solid basis,
other than being the number chosen many years ago.
When many comparisons are being made, statistical
significance can occur just by chance. A more stringent
rule is to use a p value of 0.01 ( 1 in 100) or below as
statistically significant, though some folk get hot under
the collar when you do it.
Five Steps of EBM
1.
2.
3.
4.
5.
Formulation of an answerable and specific
question.
Searching and finding the best evidence.
Critical appraisal of the evidence to assess
its validity, relevance and recency.
Application of the results of critical
evaluation into practice.
Evaluation of the practice performed.
Five Steps of EBM

When available, National and
international guidelines Are the
outcomes of the first three steps and
health practitioners have only to do the
last two steps (implementation and
evaluation of performance). However,
these guidelines are not always
available and the healthcare practitioner
has to perform all these five steps in
many cases.
oP.I.C.O.
Mnemonic
Formulation of an Answerable
and Specific Question

The question needs to be both relevent
to the patient’s problem and phrased in
a way that direct the search to relevant
and precise answers.
P.I.C.O Mnemonic

The P.I.C.O. mnemonic from the the Medical Literature Searching
Skills/Cochrane Library Tutorial webpage at oxford university
website7 can be used to help:
P
Population/Patient: Relevant patients, this may include one or more
of many factors like age range, condition, gender, race, etc.(i.e.
Smokers, Post-menopausal women, diabetics of afro-Caribbean
origin, asthmatic children from 5-10 years old, etc).
I
Intervention/Indicator: Management strategy, diagnostic test or
exposure of interest (i.e. drug, lab test, smoking etc).
C
Comparator/Control: Alternative management strategy or test or
exposure.
O
Outcome: Patient-relevant consequences of intervention (increased
risk of lung cancer, average reduction of systolic blood pressure,
etc).
P.I.C.O. Mnemonic
 Note: The search results may be further narrowed
down by Restriction of the search certain years of
publication (i.e. between 2008 and 2013).
 Example: A patient asked you which is more
effective in the management of common cold, a
loratadine/pseudoephedrine combination or vitamin
C 500mg chewable tablets?. Formulate an
answerable question in order to begin searching in
the literature.
P.I.C.O. Mnemonic

Answer:
Population/Patient
Patient with common cold
Intervention/Indicator
Vitamin C 500mg
Control/Comparator
Loratadine/Pseudoephedrin
e combination
outcome
Symptomatic management
From the above table a properly formulated search question
would be:
In patients with common cold, is vitamin C 500mg once
daily more effective in the symptomatic management
than Loratadine/Pseudoephedrine combination once
daily?
oDatabases
oGrading
of Evidence
oSources of Evidence
Databased

Many databases can be used to identify relevant, valid
and up to date evidence, some of them are free like
Medline/Pubmed data base (printed subset is named
Index Medicus) while others require subscription like
Embase and IDIS databases.

The databases of most value to EBM are those which
provide high quality evaluated evidence like the
Cochrane Library database.

The Cochrane Library database is a database of
Medicine and healthcare sciences provided by the
Cochrane Collaboration and other organizations.
Databases

Cochrane Reviews collection within the Cochrane
Library is of particular importance since it include
systematic reviews and meta-analyses which provide
the strongest evidence.

It is subscription-base database. However, access to
the abstracts and plain language summaries of the
articles is free.
Grading of evidence

Evidence is divided into levels or grades
according to the risk of bias in the
evidence. The following is the Scottish
Intercollegiate Guidelines Network’s
grading of evidence8 .
1++
1+
1-
2++
• High-quality meta-analyses, systematic reviews of RCTs, or RCTs with a very
low risk of bias
• Well-conducted meta-analysis, systematic reviews, or RCTs with a low risk of
bias
• Meta-analyses, systematic reviews, or RCTs with a high risk of bias
• High-quality systematic reviews of case-control or cohort studies
• High-quality case-control or cohort studies with a very low risk of
confounding or bias and a high probability that the relationship is causal
2+
• Well-conducted case-control or cohort studies with a low risk of confounding
or bias and a moderate probability that the relationship is causal
2-
• Case-control or cohort studies with a high risk of confounding or bias and a
significant risk that the relationship is not causal
3
4
• Non-analytic studies; for example, case reports, case series
• Expert opinion
Searching and Finding the Best
Evidence

The following are definitions of the sources of evidence
Meta-analysis
Systematic
review
Reviews
Randomized
controlled Trials
(RCT)
• A systematic review that uses quantitative methods to
summarise the results. It is where we pool all the
information we have from a number of different (but
similar) studies.
• A primary literature review that use scientific methods to
reduce or eliminate bias the compromise research
internal validity (information identification, collection and
summary).
• Summarization and deduction of conclusions from
research result, i.e. editorials and text books.
• Individuals are randomly assigned to either an
intervention group or a control group. The two groups are
identical for significant variables and they are followed for
certain end point.
Cohort Studies
• Individuals are divided according to whether they are
exposed to a certain agent or not into cohort and control
groups and the two groups are then followed up for
certain outcomes.
Case-control
studies
• Individuals are divided according to whether they have a
certain outcome (i.e. disease) or not into case and
control groups and the two groups are analyzed
retrospectively to look for differences in exposure
between them.
Cross-sectional
studies
• Surveys and interviews of samples from study
populations at one point in time .
Case reports
• Repots based on individual patients or subjects. A group
of related cases collected together called case series.
Expert opinion
• A consensus of experience from the good and the great.
oTool
Used
Critical appraisal of the evidence

It is the process of systematic assessment of evidence
for its validity, relevance and results.

There are appraisal tools available online like CASP
(Critical Appraisal Skill Program-Oxford University )
appraisal tool which address broad issue like:
 Validity of the evidence
 Appropriateness of methods
 The results
 Presentation and summarization of results
 Applicability of results locally
 Relevance
Critical appraisal of the evidence

More information about Critical Appraisal will be given
in a separate lecture.

When there is a local, national or international clinical
guidelines which is evidence based and answer
sufficiently the question of interest, then the first three
steps of EBM are already done for you and you need
now to start from the fourth step.
oEBM
in Clinical rounds
oImplementation of Guidelines
Evidence into -practice chart
(modified from Health Development Agency – NHS report:
Getting evidence into practice public health)11
EBM in Clinical Rounds

Evidence Cart

In 1998, Sackett and Straus12 examined via a descriptive
feasiblity study the introduction of a cart that contains
critically appraised secondary sources (i.e. Cochrane
Library, MEDLINE, JAMA Rational Clinical Examination
series and other sources and text books) in addition to
means for projecting and printing them, the carts was
introduced into clinical rounds at John Radcliffe in Oxford
in April 1997.

It was found that making evidence readily available to
physicians in a busy medical service increased the extent
to which evidence was sought and incorporated into
patient care decision.
(Sackett & Straus, 1998)
EBM in Clinical Rounds

18 (25%) of successful searches led to
a new diagnostic skill, an additional test
or a new management decision, and 16
(23%) led to a change in a previous
clinical skill, diagnostic test or treatment
decision.
Implementation of Clinical
Practice guidelines

According to Evans and Rawaf3, the key
elements for successful implementation
of CPG are:
 Board support and clear leadership
 Provision of a dedicated resource
(guidelines’ manager)
 Multidisciplinary team support
 Systematic approach to financial planning
and implementing guidelines
 Evaluation of uptake and feedback
EBM Organasations Nationally
and Internationally
National and Gulf Center for Evidence Based Health
Practice (NGCEBHP)
 NGCEBHP is officially recognized by the GCC ministers of
Health as an Evidence Based Health Practice referral center
for Saudi Arabia and the Gulf region. A decree issued with
this recognition during the 57th meeting held in Geneva on
19th of May 2004 (Decree 6 item 3).


Also, it is recognized as the supporting center for various
health care sectors in Evidence Based Practice all over the
Kingdom of Saudi Arabia based on resolution number (7/16)
dated 10/06/2012 from Council of Health Services.
EBM Organasations Nationally
and Internationally

The NGCEBHP aims to play a pivotal role in the
production aand dissemination of EBM across all areas
of healthcare.

NGCEBHP current location is in the Deanship building
at King Saud bin Abdulaziz University of Health
Sciences (KSAU-HS), Riyadh.

Website: http://ngcebm.ksau-hs.edu.sa/
EBM Organasitions Nationally
and Internationally
Sheikh Abdullah S. Bahamdan Research Chair for
Evidence-Based Health Care and Knowledge
Translation
 Was founded in 2008 and is located in King Khalid
University Hospital (KKUH) in King Saud university.
 The objectives of this chair is to conduct EBHC-related
research, including systematic reviews of research
results, evidence-based clinical practice guidelines and
adapt them for local use and encourage the writings of
books, translation and publish research in this field

International & National guidelines
Clinical Practice Guidelines

Also called Clinical Guidelines or Medical Guidelines,
are documents that guide decisions regarding
diagnosis, management and treatment in certain areas
of healthcare.

Current clinical guidelines work within the paradigm of
Evidence-based medicine (EBM); they Are the output
of the first three steps of EBM ( they identify, evaluate
and summarize high quality evidence)

Clinical guidelines can be produced either nationally or
internationally by governments’ bodies or medical
associations.
Clinical Guidelines

The aims of clinical guidelines are9:
 provide recommendations for the treatment and care
of people by health professionals.
 be used to develop standards to assess the clinical
practice of individual health professionals.
 be used in the education and training of health
professionals.
 help patients to make informed decisions.
 improve communication between patient and health
professional.

In English speaking countries, the following guidelines
are most popular.
Guidelines

In the U.S.A clinical guidelines are collected in the
database of National Guidelines Clearinghouse (NGC)
which is maintained as a public resource by the Agency
of Healthcare Research and Quality (AHRQ).

NGC receives guidelines that are made under the
auspices of medical specialty association; relevant
professional society; public or private organization;
government agency at the Federal, State, or local level;
or health care organization or plan.
Guidelines

There are inclusion criteria that must be met by
submitted guidelines to the NGC and the guidelines
must have developed within 5 years prior to
submission.

Website: http://www.guideline.gov/index.aspx
Guidelines

In the UK, the 2 most famous bodies that issue guidelines
are:
1-)National Institute of Health and Care Excellence (NICE)
 was set up in 1999 as a special health authority under the
name National institute of Clinical Excellence. Then, after
joining with the Health Development Agency in 2005 it began
developing health guidance and the name was changed to
National Institute of Health and Clinical Excellence.

Finally in 2013, it became non departmental public body and
the its name was changed to its recent name to reflect its
new added responsibilities in social care.
Guidelines

NICE serves the English NHS and the Welsh NHS
(NHS=National Health Service).

Website: http://www.nice.org.uk/
Guidelines
2-)Scottish Intercollegiate Guidelines Network (SIGN)
 It was established in 1993 and it develops guidelines
for National Health Service (NHS) of Scotland. Its now
a part of Healthcare Improvement Scotland.

The guideline recommendations are graded
according to the strength of the supporting
evidence so that to help healthcare practitioners to
prioritize recommendation for local
implementation.

Website: http://www.sign.ac.uk/index.html
Guidelines
In Canada, the Guideline Advisory Committee(GAC)
provided guidelines evaluations in Canada between
1997 and 2008 before it joined Center of Effective
Practice (CEP). Currently it does not undertake reviews
without funding. Website:
http://www.effectivepractice.org/
 The Canadian Medical Association has its database for
clinical practice guidelines named Infobase:


http://www.cma.ca/index.php/ci_id/54316/la_id/1.ht
m
Guidelines

In Saudi Arabia there is, up to now, no single official
body responsible for the development and
dissemination of unified evidence-based CPG to all
healthcare practitioners in Saudi Arabia.
Guidelines
G-I-N Network (Guidelines International Network)
 The Guidelines International network, G-I-N, is a global
network, founded in 2002. It has grown to comprise 94
organisations and 117 individual members representing 43
countries from all continents (October 2013). The network
supports evidence-based health care and improved health
outcomes by reducing inappropriate variation throughout the
world. Website: http://www.g-i-n.net/


Here in Saudi Arabia, the Chair of Evidence-Based
Healthcare and Knowledge Translation (EBHC-KT),
College of Medicine, King Saud University, Riyadh and King
Saud bin Abdulaziz University of Health Sciences (KSAUHS), National Guard Health Affairs both are members in G-IN network.
Guidelines


1.
2.
3.
G-I-N Network (Guidelines International Network)
Three principal aims:
Providing a network and partnerships for guideline
organisations, implementers, end-users, researchers,
students and other stakeholders
Assisting members in reducing duplication of effort and
improving the efficiency and effectiveness of evidence-based
guideline development, adaptation, dissemination and
implementation
Promoting best practice through the development of
opportunities for learning and building capacity, and the
establishment of high quality standards of guideline
development, adaptation, dissemination and implementation.
Guidelines



AGREE Collaboration (The Appraisal of Guidelines
for Research andEvaluation)
Established in 1998 by European Union (EU) and included
researchers and policy makers from inside and outside the
EU with the purpose of developing a common Instrument
of guidelines appraisal.
The AGREE Collaboration defined quality of guidelines as
the confidence that the potential biases of guideline
development have been addressed adequately and that
the recommendations are both internally and externally
valid, and are feasible for practice.The assessment
includes judgments about the methods used for
developing the guidelines, the components of the final
recommendations, and the factors that are linked to their
uptake.
Guidelines
AGREE Collaboration (The Appraisal of Guidelines
for Research and Evaluation)
 The result of the Collaboration’s effort was the original
AGREE Instrument, a 23-item tool comprising 6 quality
domains.
 The original AGREE Instrument has been updated and
methodologically refined. The AGREE II is now the new
international tool for the assessment of practice guidelines.
 The AGREE II consists of 23 key items organized within 6
domains followed by 2 global rating items (“Overall
Assessment”). Each domain captures a unique dimension of
guideline quality.
 The following table shows the general structure and content
of the AGREE II instrument10.

Domain
Description
1. Scope and Purpose
Concerned with the overall aim of the guideline,
the specific health questions, and the target
population (item 1-3)
2. Stakeholder
Involvement
Focuses on the extent to which the guideline was
developed by the appropriate stakeholders and
represents the views of its intended users (items46)
3. Rigour of Development Relates to the process used to gather and
synthesize the evidence, the methods to formulate
the recommendations, and to update them (7-14)
4. Clarity of Presentation
Deals with the language, structure, and format of
the guideline (items 15-17)
5. Applicability
Pertains to the likely barriers and facilitators to
implementation strategies to improve umptake, and
resource implications of applying the guideline
(item 18-21)
6. Editorial Independence
Concerned with the formulation of
recommendations not being unduly biased with
Detailed Structure of AGREE II

Domain 1. Scope and purpose
1. The Overall objective(s) of the guideline is (are)
specifically descirbed.
2. The clinical question(s) covered by the
guidelines is (are) specifically described.
3. The patients th whom the guidelines is meant to
apply are specifically described
Detailed Structure of AGREE II

Domain 2. Stakeholder Involvement
4. The guideline development group includes
individuals from all the relevant professional
groups.
5. The patients’ views and preferneces have been
sought.
6. The target users of the guideline are clearly
defined
7. The guideline has been piloted among end users
Detailed Structure of AGREE II

Domain 3. Rigour Development
8. Systematic methods were used to search
for evidence
9. The criteria for selecting the evidence are
clearly described
10. The methods for formulating the
recommendations are clearly described
11. The health benefits, side effects, and risks
have been donsidered in formulating the
recommendation
Detailed Structure of AGREE II
12. There is an explicit link between the
recommendations and the supporting
evidence.
13. The guideline has been externally reviewed
by experts prior to its publication.
14. A prodcedure for updating the guideline is
provided.
Detailed Structure of AGREE II

Domain 4. Clarity of Presentation
15. The recommendations are specific and
unambiguous.
16. The different options for management of the
condition are clearly presented.
17. Key recommendations are easily identifiable.
Detailed Structure of AGREE II

Domain 5. Applicability
18. The guideline is supported with tools for
application.
19. The potential organizational barriers in applying
the recommendations have been discussed.
20. The potential cost implications of applying the
recommendations have been considered.
21. The guideline presents key review criteria for
monitoring and/or audit purposes.
Detailed Structure of AGREE II

Domain 6. Editorial Independence
22. The guideline is editorially independent
from the funding body.
23. Conflicts of interest of guideline
development members have been reorded
Impact of EBM
There are three levels of impact4:
1. Strategic level
 Bodies involved in issuing and appraising guidelines at
national or international levels use the principles of EBM
together with economic analysis and technological
assessment to inform guidelines and to decide which
treatments should be available at the level of national
health systems
1. Tactical level
 Hospitals and primary care organizations implement EBM
in preparing hospital formularies and hospital guidelines.

Impact of EBM

Individual level
 EBM combines with clinical expertise and patient
factors are used by clinicians to assess risk-benefit
ratios of various treatments at the individual patient
level
Reference
1.
Sackett DL, Rosenberg WM, Gray JA,
Haynes RB, Richardson WS (January
1996). "Evidence based medicine: what
it is and what it isn't". BMJ 312 (7023):
71–2.
http://www.bmj.com/content/312/7023/7
1
2.
Greenhalgh, Trisha. How To Read a
Paper: The Basics of Evidence-Based
Medicine. Wiley-Blackwell, fourth
Reference
3.
Rosenberg W, Donald A. Evidence based
medicine: an approach to clinical
problem-solving. BMJ 1995; 310:11221126. http://www.bmj.com/content/
310/6987/1122
4. Belsey J. What is Evidence Based
Medicine. What is…?series 2009, 2nd
edition.
http://www.medicine.ox.ac.uk/bandolier/painres/
download/whatis/ebm.pdf
References
5. Doi,
S.A.R. (2012). Understanding evidence
in health care: Using clinical epidemiology.
South Yarra, VIC, Australia: Palgrave
Macmillan.
6. Miquel Porta (2008). A Dictionary of
Epidemiology. Oxford University Press. pp.
10–11.
7.http://learntech.physiol.ox.ac.uk/cochrane_tut
orial/
cochlibd0e84.php
References
8. Scottish Intercollegiate Guidelines
Network. Sign 50: A Guideline Developer’s
Handbook, 2011 edition, Edinburgh.
http://www.sign.ac.uk/pdf/sign50.pdf
9. National Institute for care and health
excellence (NICE) website:
http://www.nice.org.uk/aboutnice/whatwedo
/aboutclinicalguidelines/about_clinical_guid
elines.jsp
Referneces
10. Brouwers M, Kho ME, Browman GP,
Burgers JS, Cluzeau F, Feder G, Fervers B,
Graham ID, Grimshaw J, Hanna S,
Littlejohns P, Makarski J, Zitzelsberger L for
the AGREE Next Steps Consortium. AGREE
II: Advancing guideline development,
reporting and evaluation in healthcare. Can
Med Assoc J. 2010. Dec 2010; 182:E839842
References
11. Kelly MP, Speller V, Jane Meyrick for
Health Development Agency. Getting
evidence into practice in public health.
National Istitute for Health and Care
Excellence (NICE) 2004:
http://www.nice.org.uk/aboutnice/whowear
e/
aboutthehda/evidencebase/keypapers/
evidenceintopractice/getting_evidence
_into_practice_in_public_health.jsp
References
12. Sackett DL, Straus SE, 1998. Finding and
applying evidence during clinical rounds:
the “evidence cart”. JAMA 280(15):1336-8.
13. Slawson DC, Shaughnessy AF. Teaching
evidence-based medicine: should we be
teaching information management
instead? Acad Med. 2005 Jul;80(7):685-9.
14. Bordley, D.R. Fagan M, Theige D. Evidencebased medicine: a powerful educational tool
for clerkship education. Am J Med. 1997
May;102(5):427-32.