Dr. Maureen Markle-Reid, RN, MScN, PhD
Associate Professor and Acting Assistant Dean (Research),
School of Nursing, Associate Member, Clinical Epidemiology and Biostatistics
Nursing Graduate Seminars
October 4, 2011







Define evidence-informed decision-making and describe its importance to nursing practice
Identify the parts of a relevant, answerable question
Describe the different types of questions relevant to nursing practice
Define the term critical appraisal and discuss its relevance to nursing practice, policy and research
Identify criteria appraisal criteria for health care intervention studies
Develop skill in applying the criteria for critical appraisal of an intervention study to determine the quality and applicability of the research.

“The conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients”
(Sackett et al, 1996)

A Model for Evidence-Based Clinical Decisions
Health care resources
Patient preferences and actions
Clinical Expertise
Clinical Decision
Research evidence
Clinical state, setting, and circumstances
[Haynes, R.B., Devereaux, P.J., & Guyatt, G.H. (2002). Clinical expertise in the era of evidence-based medicine and patient choice. ACP J Club, 136, A11-14]

Why is evidence-based practice important?
 Demonstrate that nursing actions and decisions are clinically appropriate and result in positive outcomes for clients, their families and health care system as a whole
 Demonstrate professional accountability to clients
 Provide evidence for the cost-effectiveness of nursing care

Steps to Evidence-Informed Decision-Making:
1.
Define: Formulate a focused answerable question from the practice situation based on your information needs
2.
Search: Efficiently search for research
3.
Appraise: Critically and efficiently appraise the research sources
4.
Synthesize: Interpret/form recommendations for practice based on the literature
5.
Apply: Apply the results to the client/population
6.
Implement: Decide whether (and plan how) to implement the adapted evidence into practice
7.
Evaluate: Evaluate the effectiveness of implementation efforts

Formulate a Focused Answerable Question

P opulation / Situation

I ntervention / Exposure

C ounter intervention

O utcome

T imeframe



Client group or clinical scenario of interest
Single patient or group of patients with a particular condition or health care problem, e.g.,:
 A person with a health condition
 People with hypertension (a group of people with a particular condition
 Primary health care for the elderly (an aspect of health care delivery)



Interventions can be:
 Therapeutic
 Preventive
 Diagnostic
 Organizational
If exploring the meaning of a phenomena, the question may involve a situation rather than an intervention

:
 Clinical decisions involve choosing between alternative courses of action (or no action)



What is the outcome, or the effect we are hoping to achieve by the using the intervention?
May be more than one outcome that is important to the question



What is the period of time over which the question occurs?
What is the optimal time to measure a change in the outcome(s)?

Step 1:Formulate a focused answerable question
 Formulating Answerable Clinical Questions (Centre for Evidence-
Based Medicine, Mount Sinai Hospital)
 The Well-Built Clinical Question (Duke University Medical Center
Library and Health Sciences Library, UNC-Chapel Hill)
 Formulating Patient Centered Questions (University Library,
University of Illinois at Chicago)
 Asking Focused Questions (Centre for Evidence-Based Medicine,
Oxford)

Types of questions relevant to nursing practice






Therapy/Health care interventions : What is the effectiveness of different interventions?
Causation and harm: What might be causing disease/ill health/adverse effects?
Diagnosis or assessment : Does a diagnostic test differentiate between people with and without a condition?
Prognosis: What are potential future outcomes of a condition?
Economic evaluation : What is the cost-effectiveness of different interventions?
Meaning : Describing, exploring and explaining aspects of health and illness.

Different clinical questions require evidence from different research designs
Type of Question Suggested Best Type of Study Design
RCT>cohort>case control>case series Therapy/Health care intervention
Diagnosis
Prognosis
Harm (Causation or etiology)
Economic evaluation
Meaning of illness
Prospective, blind comparison to a gold standard
Cohort study>case control>case series
RCT>cohort>case control>case series
Economic analysis
Qualitative
16
Nursing Graduate Seminar
2009

What type of research design will you use?
All studies
Theoretical
(PS)
Qualitative
Descriptive
(PO)
Survey
(cross sectional)
Q1
Experimental
Analytic
(PICO or PECO)
Q2
Observational analytic
Q3
Cohort study (Randomised)
Parallel group
(Randomised) crossover
Cross-sectional
(analytic)
Case-control study
[Glasziou, P., & Heneghan, C. (2009). A spotter’s guide to study designs. EBN Notebook, 12, 71-72 ]

Step 2: Collect the best evidence:
Hierarchy of Pre-Processed Evidence
EXAMPLES:
SYSTEMS:
Computerized decision support
SUMMARIES: :
Evidence-Based Guidelines
~ Evidence-Based Texts
SYNOPSES OF SYNTHESES :
DARE ~ Evidence-Based
Abstract Journals
SYNTHESES (Systematic
Reviews): Cochrane Database of Systematic Reviews
SYNOPSES OF SINGLE
STUDIES :
Evidence-Based Abstract
Journals
SINGLE STUDIES:
Clinical Queries
[Adapted from Haynes, R.B. (2007). Of studies, summaries, synopses, and systems: The “5S” evolution of information services for evidence-based healthcare decisions. Evidence-Based Nursing,
10, 6-7]

Step 3: Critically appraise the literature for validity and applicability
What is critical appraisal?
 Critical appraisal is the process of assessing and interpreting evidence by systematically considering its validity, results and relevance to an individual's work.

Relevance of Critical Appraisal to Nursing
Practice, Policy and Research

Use the literature more effectively in answering clinical questions to guide clinical practice

Distinguish stronger evidence from weaker evidence – identify high quality research

Identify the methodological strengths and limitations, results and relevance of the studies in answering a clinical question

Three Steps in Using an Article from the
Health Care Literature
 Are the results valid?
 What are the results?
 Are the results applicable (and useful) to my client care population?
[DiCenso, A., Guyatt, G., & Ciliska, D. (2005). Evidence-based nursing: A guide to clinical practice ]

Are the results valid?

Are the study methods sufficiently rigorous to ensure that the study results represent an unbiased estimate of the true effect?
OR

Are the study methods sufficiently biased to lead to a false conclusion?

Are the results valid?

Final assessment of validity is never a yes/no decision

Validity as a continuum ranging from strong studies to weak studies

Evaluation of the validity of a study involves some subjectivity

What are the results?

Size and precision of the estimate of effect

Are the results applicable (and useful) to my client care population?

Can you apply the results to patients in your clinical setting?

Were all important outcomes considered?

Are the likely intervention benefits worth the potential harm and costs?

Applying appropriate criteria
 User Guides to Evidence-Based Practice
 19932000: Evidence based medicine working group: “User guides to the Medical Literature” in JAMA
 2002: Guyatt et al. “User guides to the medical literature: A manual for evidence based clinical practice”
 2005: DiCenso, Guyatt , & Ciliska. “Evidence-based nursing: A guide to clinical practice”

Applying appropriate criteria

Critical Appraisal Forms:







Critical Appraisal of Articles on CAUSATION
Critical Appraisal of Articles on THERAPY/INTERVENTIONS
Critical Appraisal of SYSTEMATIC REVIEWS
Critical Appraisal of Articles on PREVALENCE AND INCIDENCE
Critical Appraisal of QUALITATIVE RESEARCH
Critical Appraisal of Articles on PROGNOSIS http://www.cche.net/usersguides/prognosis.asp
Critical Appraisal of GUIDELINES [AGREE: Appraisal of Guidelines
Research and Evaluation Instrument] http://www.agreecollaboration.org/pdf/agreeinstrumentfinal.pdf

Example: The Effectiveness of a Nurse-Led
Interprofessional Team Approach to Fall Prevention in Older Home Care Clients at Risk of Falling
Markle-Reid, M., Browne, G., Gafni, A., Roberts, J., Weir, R., Thabane, L., Miles, M.,
Vaitonis, V., Hecimovich, C., Baxter, P., & Henderson, S. (2010). The effects and costs of a multifactorial and interdisciplinary team approach to fall prevention for older home care clients “at risk” for falling: A randomized controlled trial. Canadian
Journal on Aging, 29(1), 139-161

Critical Appraisal Criteria for Health
Care Intervention Studies
Screening Questions:
1.
Did the study ask a clearly focused question?
Consider if the question is ‘focused’ in terms of:



The population studied
The intervention given
The outcomes considered

Research Question:

`What is the effectiveness of a 6-month nurse-led interprofessional team approach to fall prevention compared with usual home care services in older home care clients “at risk” of falling with respect to the number of falls and fall risk factors
(slip or trip, health-related quality of life and function, depressive symptoms, nutritional status, gait and balance, cognitive function, fear of falling)?





P Frail older adults using home care services at risk for falling
I Nurse-led interprofessional team approach to fall prevention
C Usual home care services
O Number of falls, slip or trip, health-related quality of life and function, depressive symptoms, nutritional status, gait and balance, cognitive function, fear of falling
T 6 months

Critical Appraisal Criteria for Health
Care Intervention Studies
Screening Questions:
2.
Was this a randomized controlled trial (RCT) and was it appropriately so?


Consider:
Why this study was carried out as an RCT
If this was the right research approach for the question being asked

Critical Appraisal Criteria for Health
Care Intervention Studies


Critical Appraisal Criteria for Health
Care Intervention Studies
3.
How were participants allocated to intervention and control groups?




Consider:
How participants were allocated to intervention and control groups. Was the process truly random?
Whether the method of allocation was described
How the randomization schedule was generated
Were the groups comparable at baseline on characteristics that might explain the outcome?

Treatment
Population
Eligible Sample
Baseline Measures
R n=
Ineligible
Control
Follow-up Measures Follow-up Measures




Random allocation of participants to groups ensures that groups are similar in all respects except exposure to the outcome;
Prospective design ensures that exposure to the intervention precedes the development of the outcome;
Greater likelihood that participants, health care providers, and outcome assessors can be blinded.





May not be ethical or feasible;
Volunteer bias limits generalizability;
Problems associated with a longitudinal design:


Time
Expense

Dropouts
Final results may not be available for several years.

Critical Appraisal Criteria for Health
Care Intervention Studies
4.
Were participants, staff and study personnel
‘blind’ to participants’ study group?
Consider:

The fact that blinding is not always possible



If every effort was made to achieve blinding
If you think this matters to the study
The fact that we are looking for ‘observer bias’

Critical Appraisal Criteria for Health
Care Intervention Studies
5.
Was follow-up complete?
Consider:


If any intervention-group participants got a control group option or vice versa
If all participants were followed up in each study group


Was there loss to follow-up bias?
If all of the participants’ outcomes were analysed by the groups to which they were originally allocated (intention to treat analysis)

Eligible Clients n=267
Refused 141
Unable to Contact 13
Language 4
158
Informed consent from 40.8% of eligible applicants n=109
R
Group Size
Group 1 n=54
Interventions
Dropouts @ 6 month followup: 17 clients (15.6%)
Analyzed @ 6 month followup: 92 clients (84.4%)
Interprofessional Falls
Prevention Team n=5 (9.3%) n=49 (90.7%)
Group 2 n=55
Usual Home Care
Services n=12 (21.8%) n=43 (78.2%)

Critical Appraisal Criteria for Health
Care Intervention Studies
6.
Were participants in both groups followed up and data collected in the same way?
Consider:
 Was data collected in the same way and at the same time interval for both groups?


Multiple sources of data:


In-person interviews
CCAC data


RAI-HC data
Data from service provider agencies

Measurement of Clinical Outcomes: Baseline and 6-months

Outcomes
Effects:
Frequency and Context of Falls (Falls Surveillance Report)
Functional Health Status and Quality of Life (SF-36 Health Survey)
Cognitive Status (SMMSE)
Depressive Symptoms (CES-D)
Gait and Balance (POMA)
Nutritional Status (SCREEN II)
Environmental Safety (HOME FAST)
Perceived Self-Efficacy (MFES)
Costs: Health Services Utilization (HSSU and CCAC Utilization Data)

Critical Appraisal Criteria for Health
Care Intervention Studies
7.
How are the results presented and what is the main result?
Consider:
 What are the results?


How large this size of result and how meaningful it is
How you would sum up the bottom-line result of the trial in a few sentences


At 6 months, there was no difference in the mean number of falls between groups;

Subgroup analyses showed that the intervention was effective in reducing falls in men, 75-84 years of age, with a fear of falling, or a negative history of falls;

Greater reduction in number of slips and trips in the intervention group;

Greater improvement in role functioning related to emotional health in the intervention group;

No additional cost from a societal perspective, e.g. both interventions cost the same.

Critical Appraisal Criteria for Health
Care Intervention Studies
8.

How precise are these results?
Consider:
 If the result is precise enough to make a decision
If a confidence interval were reported. Would your decision about whether or not to use this intervention be the same at the upper confidence limit as at the lower confidence limit?

9.
Critical Appraisal Criteria for Health
Care Intervention Studies
Were all important outcomes considered so the results can be applied?
Consider whether:



How the participants could be different from your population in ways that would produce different results
Your local setting differs from that of the trial
Is the intervention feasible in your setting?
Consider outcomes from the point of view of the:
 Individual



Policy maker and professional
Family caregivers
Wider community

Critical Appraisal Criteria for Health
Care Intervention Studies
9.
Were all important outcomes considered so the results can be applied?
Consider whether:
 Any benefit reported outweighs any harm and/or cost
 Policy or practice should change as a result of the results of this trial

Methodological Strengths

Study design: randomized controlled trial

Follow-up period was appropriate

Follow-up was complete (<20% loss to follow-up)

Groups comparable at baseline

Methodological Limitations
 Small sample size
 Volunteer bias
 Potential for recall bias related to reporting falls
 Strategies for collecting data regarding the number of falls

Other Resources






Evidence-Informed Decision-Making Resource Modules posted on Avenue to Learn
Bandolier Evidence-Based Health Care http://www.medicine.ox.ac.uk/bandolier/
Tutorials:
 Introduction to Evidence-Based Medicine (Duke University Medical Center Library) http://www.hsl.unc.edu/services/tutorials/ebm/index.htm
Directories:


Evidence Based Medicine (EBM) Tookit (New York University School of Medicine.
The Frederick L. Ehrman Medical Library): www.urmc.rochester.edu/hslt/miner/digital_library/evidence_based_resources.cfm
 Resource Guide for Evidence-Based Practice (University of Alberta Libraries) http://www.library.ualberta.ca/subject/evidence/guide/index.cfm
McKibbon, A. (1999). PDQ: Evidence-based principles and practice. Hamilton, Ontario:
B.C. Decker Inc.
Evidence Based Practice Tips http://www.ebmtips.net/risk001.asp