Developing and Evaluating Decision Aids
Annette O’Connor and MJ Jacobsen, 2000
Index
SECTION I.
Decision aids to assist patients in participating in choice
about Professional Care Options . . . . . . . . . . . . . . . . . . . . 3
A. What is a decision aid? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
B. When do you need a decision aid? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.3
C. Do decision aids work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.4
D. How do you develop a decision aid? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1. Assess need. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.4
2. Assess feasibility. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.5
3. Define the objectives of the aid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
4. Identify the framework of decision support. . . . . . . . . . . . . . . . . . . . . . . 7
5. Select the methods of decision support to be used in the aid . . . . . . . . . . .8
6. Select the designs and measures to evaluate the aid. . . . . . . . . . . . . . . . 12
7. Plan dissemination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
Example of developing a decision aid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17
Measures used to evaluate decision aids . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
20
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
Annotated Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.33
2
SECTION II
Developing a schedule for Adaptation and Feasibility Testing
of Decision Aids . . . . . . . . . . . . . . . . . . . . . . . . . ..38
3
A. What is a decision aid?
Patient decision aids or “shared decision making programs” are interventions for
preparing patients for decision making about professional care options. They are meant to
supplement rather than to replace the counseling provided by health practitioners. According
to the Cochrane Collaboration (1), decision aids for professional care options are defined as:
..interventions designed to help people make specific and deliberative choices among options
by providing (at the minimum) information on the options and outcomes relevant to the
person’s health status. Additional strategies may include providing: information on the
disease/condition; the probabilities of outcomes tailored to a person’s health risk factors; an
explicit values clarification exercise; information on others’ opinions; and guidance or
coaching in the steps of decision making and communicating with others. Decision aids may
be administered using various media such as decision boards, interactive videodiscs, personal
computers, audiotapes, audio-guided workbooks, pamphlets, and group presentations.
Excluded from the definition of decision aids are passive informed consent materials,
educational interventions that are not geared to a specific decision, or interventions designed
to promote compliance with a recommended option rather than a choice based on personal
values.
Decision aids have been developed for: a) medical therapies for atrial fibrillation,
benign prostatic hypertrophy; low back pain, cancers of the breast and lung, leukemia,
lymphoma, circumcision, and ischemic heart disease; b) diagnostic tests such as amniocentesis
and screening for colon and prostate cancers; c) preventive therapies such as Hepatitis B
vaccine and hormone therapy at menopause; d) clinical trial entry decisions; and e) end-of life
decisions such as resuscitation in seniors (1,2).
B. When do you need a decision aid?
The need for a decision aid may depend on whether the potential health care strategy is
considered a “standard of care”, a “guideline”, or an “option” . For example, insulin for Type
I diabetes or antibiotics for a bacterial infection would be considered standards of care
because there is strong evidence of their effectiveness and strong agreement among patients
that these are valued interventions. Therefore, there is relatively less discussion about whether
the patient should take this intervention and more discussion on how they should take it. In
the case of guidelines, there may be more uncertainty for patients and practitioners. Although
the evidence about the outcomes of the interventions is known, there is less agreement among
patients regarding values: not everyone agrees that the benefits outweigh the risks. For
example, there is good evidence that amniocentesis for pregnant women over 35 is effective in
detecting abnormalities, but not all women choose the procedure because their values differ.
Interventions are classified as options of care when evidence on outcomes is known or
unknown, and agreement on values may be even more variable, or unknown. An example of
an option that emerged in 1998 is Tamoxifen to prevent breast cancer in healthy but high risk
women. One large trial showed a benefit in reducing the risk of breast cancer and two smaller
trials did not show a benefit. Moreover, the benefits in the large trial were also accompanied
by an increased risk of endometrial cancer and vascular events such as thrombophlebitis and
4
pulmonary embolism. When the large trial results were published by the U.S. National Cancer
Institute on the Internet, the following information was provided:
“Based on the [trial] ..results, should women who are at increased risk of breast cancer
take tamoxifen? Women at increased risk of breast cancer now have the option to consider
taking tamoxifen to reduce their chances of breast cancer. ...the decision is an individual one in
which the benefits and risks of therapy must be considered. The balance of these benefits and
risks will vary depending on a woman’s personal health history and how she weighs the
benefits and risks. Even if a woman is at increased risk of breast cancer, tamoxifen may not be
appropriate for her”.
C.
Do decision aids work?
Evaluation studies from a Cochrane systematic overview of trials (1) and two general
reviews (2,3) have shown that decision aids improve decision making by:
• reducing the number of patients who are uncertain about what to do;
• increasing patients’ knowledge of the problem, options, and outcomes;
• creating realistic personal expectations of outcomes;
• improving the agreement between choices and patients’ values;
• reducing decisional conflict and the factors contributing to decisional conflict such as
feeling informed, clear about values, and supported in decision making; and
• increasing participation in decision making without adversely affecting anxiety.
The impact of decision aids on satisfaction with decision making is more variable. More
research is needed on which decision aids work best with which decisions and which types of
patients. We also need to evaluate their acceptability to practitioners and diverse groups and
cultures, their impact on client-practitioner communication, and their effects on compliance
with choice, health-related quality of life and appropriate use of services.
D. How do you develop and evaluate a decision aid?
Steps in Developing and Evaluating a Decision Aid
1. Assess need.
2. Assess feasibility.
3. Define the objectives of the aid.
4. Identify the framework of decision support.
5. Select the methods of decision support to be used in the aid.
6. Select the designs and measures to evaluate the aid.
7. Plan dissemination.
As summarized above, there are seven steps we use when developing our decision aids in
Ottawa (2). When considering each step, there are several questions developers need to ask
themselves and there are certain methods that can be used to answer the questions that are
posed. These are described briefly below.
5
1. Is there a need for a decision aid?
Needs assessment involves the compilation of evidence about the nature of the decision
difficulty, the numbers affected, practice and preference variation, availability of aids
elsewhere, and demand for the aid. Methods for needs assessment are varied and data are
obtained from primary and/or secondary sources. It is important that needs are defined from
the perspective of potential users, both patients and practitioners. The perspective of
managers and third party payers may also be necessary. Useful question here include:
What are the decision making needs of patients and practitioners?
Needs are assessed using key informant interviews, focus groups, or surveys. They
elicit patients’ and practitioners’ perceptions of: decisions perceived as important &
difficult; usual roles and decision making practices; barriers & facilitators in providing
or accessing decision support, and potential strategies for overcoming barriers.
What makes the decision difficult?
This can be determined by examining published systematic overviews, decision
analyses, and preference studies. These types of studies may indicate whether the
decision is characterized by: small or uncertain benefits; uncertain or material risks; the
need to make value tradeoffs between benefits and risks; and variation in patients’
preferences for outcomes.
Are sufficient numbers affected and how are they affected?
This question can be answered by reviewing data bases, population surveys, and
statistics on demographic characteristics of the population, morbidity, and mortality.
Is there sufficient variation in utilization?
Utilization information can be obtained from practice atlases, utilization data, and
practice variation studies
Are there decision aids available to meet these needs?
The answer to this question is obtained from a review of published overviews and
reports. It is also useful to contact centers that produce aids (see Appendix A).
Is there a demand for decision aids and what methods are preferred?
Market surveys of practitioners and patients are useful to determine demand for an aid.
2. Is it feasible to develop a decision aid?
Feasibility is assessed to determine that the aid can be developed with available evidence
and resources and delivered and updated in a timely, accessible and acceptable manner. Useful
question here include:
Are there adequate resources?
6
Developers need to assess their financial resources to undertake such work. They also
need to ensure they have experts with the external credibility, access to dissemination
networks, and commitment to making ongoing updates. If these are lacking,
developers need to link to established evidence evaluation and dissemination networks.
Is there enough evidence of benefits and risks to incorporate into a decision aid?
Information is obtained from systematic overviews with appraisals of the quality of
evidence. It is also helpful to review ongoing trials and talk to experts to determine
how quickly the evidence expected to change.
Can delivery be accessible and acceptable to users?
The answers to these questions can be elicited in focus groups and market surveys.
3. What are the objectives of the decision aid?
The objectives of the decision aid should be stated explicitly. They influence the selection
of the framework, intervention strategies and evaluation methods. The objectives of the
decision aid should identify what will be achieved. The objectives should be clear, specific,
and measurable, and relevant for the situation. Preferably, they should be stated from the
perspective of the user. They can range from general to specific and from short-term to longterm.
An example of a broad general objective is: “To improve the decision making of patients
and their practitioners who are considering options for treating benign prostatic hypertrophy.”
An example of a specific objective is: “To improve patients’ knowledge of options and
outcomes regarding treatments for benign prostatic hypertrophy.”
The short-term or long-term objectives depend on what ends are viewed as desirable (2).
Some see “evidence-informed patient choice” as the desired end. This means that patients
make a choice that is informed by the scientific evidence about the potential benefits and
harms of the available options. It is based on the belief that we have a basic moral obligation
to provide individuals with sound information as well as choice about their health care.
Therefore, if a person makes a choice based on adequate knowledge of options and realistic
expectations of potential benefits and harms, then the desired end has been achieved.
Some would argue that evidence-informed choice is not sufficient unless it leads to
other beneficial outcomes such as greater clinical effectiveness, health gain, individually
appropriate utilization, reduced expenditures on inappropriate interventions, reduced
litigation, etc. The short-term objective of making an informed choice is therefore viewed as a
means to another desirable end.
Reflection Exercise: What is a good decision?
Think about your views on this issue.
What would convince you that a person had made a good decision?
What would convince you that a person had used a good decision making process?
Note: For further readings on what experts have to say, reference 5 has several points of view.
7
Now consider the objectives you would like to achieve in your decision aid.
DEFINING THE OBJECTIVES OF A DECISION AID
Write the objectives of your decision aid in your own words:
Are there any objectives listed below (T) that apply to your situation.
9 Improve decision making of patients and practitioners.
9 Improve patients’/practitioners’ knowledge of the clinical problem, options, outcomes, variation in
patient/practitioner opinions and practices.
9 Create realistic expectations of outcomes, consistent with available evidence.
9 Clarify patients’ personal values for outcomes and promote congruence between patients’ values and choice.
9 Reduce patients'/practitioners' decisional conflict (uncertainty) about the course of action to take.
9 Promote implementation of choices.
9 Improve patients’/ practitioners’ satisfaction with decision making.
9 Other, specify
9 Improve outcomes of decisions.
9 Promote patients’ persistence or compliance with choice.
9 Reduce patients’ distress from consequences of decision.
9 Improve patients’ health-related quality of life.
9 Promote informed use of resources by patients/practitioners.
9 Other, specify
4. Which framework will drive the development of the decision aid?
Depending on the objectives, several frameworks are available to guide decision aid
development. Compete references to these frameworks are provided in the annotated
bibliography at the end of this section.
Charles, Gafni and Whelan focus on features distinguishing shared decision making from
other models of decision making.
Entwistle defines evidence-informed choice and outlines different criteria for evaluation
Hersey & Lohr have a health services research and informatics perspective.
Llewellyn-Thomas places decision support in a broader sociopolitical context and expands on
the types of preferences one can elicit from patients.
Mulley places shared decision making in the context of outcomes research.
The Ottawa framework emphasizes the preparation of both the patient and practitioner using a
decision making behaviour framework that separates the effects of each decision support
method.
Rothert et al also describe the mutual roles of patients and practitioners in decision making,
with a focus on information and values.
8
These ‘decision support’ frameworks may need to be supplemented by others. For
example, if decision making behaviour is strongly influenced by biological factors (e.g.
addictions), other constructs need to be added. If a key challenge is not only deliberating
about options, but also implementing options and maintaining behavioural change, models of
change and behaviour such as Prochaska’s stages of change or the Precede Procede Model
may be relevant.
THE OTTAWA FRAMEWORK OF DECISION SUPPORT
The Ottawa Framework identifies several determinants of health care decisions that may be suboptimal and are
potentially modifiable by decision aids. Patients and practitioners may have problems with:
a) perceptions of the decision (e.g. inadequate knowledge, unrealistic expectations of outcomes, unclear values,
high uncertainty or decisional conflict);
b) perceptions of others (e.g. biased or limited perceptions of the variation in others’ opinions and practices;
social pressures, inadequate support), and
c) personal and external resources to make the decision (e.g. limited skills in shared decision making).
Decision aids are designed to address these problematic determinants of choice by providing accurate,
balanced, and tailored information, clarifying patients’ values, and augmenting skills in shared decision
making. For example,
-inadequate knowledge may be improved by providing information on options and outcomes;
-unrealistic expectations (perceived probabilities of outcomes) may be re-aligned by presenting probabilities of
outcomes that are tailored to the patient’s clinical risk and by describing outcomes so that they are easy to
imagine and identify with;
-unclear values are addressed by describing outcomes in familiar, simple, and experiential terms so as to better
judge their value and by providing the opportunity to weigh the benefits versus the risks;
-biased perceptions of the variation in others’ opinions may be corrected by presenting all options, and in
some cases, by providing examples of others’ choices and statistics on variation in choices;
-limited skills in shared decision making skills may be improved by providing structure, guidance or coaching
in deliberating about the personal issues involved in the choice and in communicating preferences.
These methods of decision support need to be adjusted to another important determinant of decisions,
the patients’ and practitioners’ characteristics
As a consequence of these interventions, patients presenting with uncertainty or decisional conflict caused by
these problems may become more certain about what to choose and may be more likely to implement these
choices.
Based on the Ottawa framework, one can hypothesize that decisions aids will improve the
determinants of choice so that decisions are more likely to be: 1) informed (i.e. based on better knowledge
and realistic expectations); 2) consistent with personal values; and 3) implemented. Moreover, patients’
comfort with the decision making process (e.g. decisional conflict, self-confidence and satisfaction with
decision making) may be improved.
Based on the results of other educational interventions designed to promote realistic expectations of
outcomes and informed active involvement in one’s care, it is also reasonable to hypothesize that patients may
be more likely to persist with decisions, to report less distress with the consequences of their decisions, and to
experience improved health-related quality of life.
5. Which methods will be included in the decision aid?
In selecting the decision support methods, the developer needs to determine how much
emphasis will be placed on preparing the patient and the practitioner. The specific decision
support methods, content, and delivery methods depend on the nature of the decision, the
needs of the users, feasibility constraints, and the objectives of the decision aid. The ‘essential
content’ in decision aids is still a matter of debate.
Information about clinical condition, options and outcomes. Most decision aids start with
a description of the clinical situation that has stimulated the need to consider certain options
and outcomes. Patients need to know about the conditions or diseases they face, common
9
manifestations and complications. The health care options are then described including: what
they include, the method and duration of delivery, and the patient’s involvement in their use.
Next, the outcomes of each option are described in sufficient detail for patients to
understand what it is like to experience such an outcome. This is important because people
will often underestimate the likelihood of an option that they cannot imagine or identify with.
Moreover, people cannot judge the value of an outcome that is unfamiliar. In many decision
aids, the functional impact of the outcome is described (e.g. how the patient can be expected
to respond physically, emotionally, and socially). The evidence to support the description of
outcomes can be found in quality of life studies. We usually select the impacts that are most
frequently reported and most important to the patients. In the absence of quality of life data, a
panel of experienced patients and clinicians can be helpful in describing outcomes. Some
decision aids that are delivered by video include interviews of patients describing what it is like
to experience the outcomes.
Presenting Probabilities of Outcomes. One of the consistent benefits of decision aids is to
create realistic expectations of outcomes. This is achieved by presenting probabilistic
information about the likelihood of benefits and risks. Patients who are unaware who have
unrealistic expectations (e.g. over-estimate the benefits and underestimate the risks) can be
helped with this information.
The numerous issues in presenting risk information are summarized in a recent monograph
of the National Cancer Institute (Journal of the National Cancer Institute Monograph- Risk Communication and
Decision Making. Number 25, 1999, p. 67-80). Generally, it is advisable to use numbers to describe
probabilities because people do not agree on what is meant when words are used (e.g. high
probability, small chance). Numbers are better at creating realistic expectations.
Whether numbers should also be accompanied by graphic illustrations (100 faces, bar
charts, pie charts) is still being examined. We find 100 faces most helpful. The pie charts do
not distinguish between small probabilities associated with various options (e.g. 50% versus
60%). The bar charts are difficult to interpret by people with less education. When displaying
the 100 faces, we shade the number expected to experience an outcome and leave the other
faces unshaded. This allows us to frame the message about chances of outcomes in both
positive and negative terms. Many studies have shown that one gets different responses
depending on what is emphasized: whether the chances of something happening (5% will get
this complication) versus the chances of something not happening (this complication may
occur, but in 95% of cases a person will remain free of complications). We shade the numbers
in a row rather than randomly because it is difficult for people to notice small differences in
probabilities between options using random shading.
Values Clarification Exercises. Patients clarify their values in two possible ways. First
the descriptions of outcomes provided vicarious experience from which to judge their value.
Second, some decision aids ask patients to explicitly consider the personal importance of each
benefit and risk.
In those decision aids that use explicit valuing approaches, some handle probabilities and
values for outcomes separately by asking patients to value each outcome via formal utility
assessments. They then combine the values for outcomes with their associated probabilities
using expected utility decision analysis to arrive at a recommendation for the patient. Others
ask patients to value treatments, by considering both probabilities and values together, using
probability tradeoff tasks; relevance charts; and "weigh scale" exercises. The purposes of these
valuing exercises are to structure and provide insight into how values affect personal decision
making.
Whether it is necessary to have an explicit values clarification exercise has not been
determined. One study showed that the addition of a ‘weigh scale exercise’ did not benefit
women who were not going to change their current practices. Whether it is beneficial as a
10
communication tool between patients and practitioners and whether it is beneficial for patients
who are planning to change practices remains to be seen.
We normally use a values clarification exercise in our decision aids, usually in the personal
worksheet. The benefits and risks are placed on a balance scale so that patients can: 1) add
other potential benefits and risks that are important to them; 2) indicate the personal
importance of each by shading or checking those that are affecting their decision making; and
3) communicate their values efficiently to their practitioners and others involved in decision
making.
Information About Others’ Opinions. Whether decision aids should include information
about others’ decisions is still open for debate. One survey of patients’ decision-making needs
indicated that some, but not all thought this information was essential for decision making.
Some developers of decision aids include no examples in order to remain neutral. Others
provide balanced information on different points of view.
We usually include examples of how different patients deliberate about options. The
examples are composites of cases in our clinics. The decision aid presents a situation
exemplifying a patient choosing each of the options, including one in which no treatment or
test is chosen. Patients learn not only what others choose, but also the reasoning behind the
choices. The different cases reinforce the notion that decision making is variable and should
be individualized according to a person’s own situation and personal values.
Guidance and Coaching in Decision Making and Communication. Guidance and
coaching has been found to be helpful in promoting better coping strategies, health practices,
and outcomes. Whether it is helpful in decision aids is still under investigation. We provide
structure and guidance in decision making by showing patients the steps in decision making
and by providing suggestions for what to do in a follow-up discussion with the practitioner.
The steps are usually presented after patients receive the general information on options. This
helps them to personalize the information in their own situation. The steps usually include:
• Considering personal benefits and risks;
• Clarifying personal values;
• Listing current health practices;
• Listing questions;
• Indicating preferred role in decision making; and
• Indicating current leaning or predisposition toward the options.
We use a personal worksheet for patients to consider these steps and illustrate with examples
of how others complete the worksheet. The worksheet provides a focus for communication
and discussion at a follow-up visit. It can also be used to discuss varying views among family
members.
Methods of Delivering Decision Aids. Decision aids can be delivered in various forms:
decision boards, interactive videodiscs, personal computers, audiotapes, audio-guided
workbooks, pamphlets, and group presentations. The method used depends on the
preferences of the users and the resources and expertise of the developers. The efficacy of
different methods are under active investigation. Most developers use more than one delivery
method.
Preparing the Practitioner. There are several methods available for preparing the
practitioner. We normally use a manual or practice guideline that summarizes the scientific
information regarding the decision. The practice guidelines are those developed by a
provincial or national group (e.g. Cancer Care Ontario, Society of Obstetrics and
Gynecology). We have also held or contributed to local and national continuing education
workshops that summarize the evidence regarding the options and efficacy of the decision aid
in preparing patients for decision making.
11
Now it’s time to consider the methods you wish to include in your decision aid. These charts
may be helpful in summarizing your views for discussion with other members of the team.
PATIENT DECISION SUPPORT METHODS
check T those you wish to include
Information
9 About Clinical Problem, specify ______________________________________________________
_________________________________________________________________________________________
_________________________________________________________________________________________
____________________________________________________________________
9 About Options, specify ______________________________________________________
_________________________________________________________________________________________
_________________________________________________________________________________________
____________________________________________________________________
9 About Consequences of Options, specify _______________________________________________
_________________________________________________________________________________________
_________________________________________________________________________________________
____________________________________________________________________detail in describing
outcomes: 9 define outcomes; 9 describe physical, emotional, social impact; 9 use narrative/scenario styles
9 About probabilities of Outcomes: , specify _____________________________________________
display? 9 numerical; 9 frequencies/ percents; 9 graphic pie charts; 9 100 people 9 qualitative (low,
moderate, high); tailoring?: 9 not tailored; 9 stratified by personal risk factors;
9 Evidence for statements: 9 level of evidence; 9 references included/not
9 Suggested readings: 9 included; 9 not included
Values Clarification
9 implicit only; 9 explicit, specify:
__________________________________________________________________ ___________
9 marking important pros/cons (stars, circling, highlighting); 9 weigh scale exercise; 9 decisional balance
sheets; 9 other
Information on Others specify _______________________________________________________
9 none; 9 cases of different choices; 9 statistics on variation in patients' decisions or practitioners' opinions;
9 recommendations from clinical societies
Guidance/Coaching specify _______________________________________________________
9 steps of decision making; 9 communicating with practitioners
9 tips on managing consequences of choices; 9 tips on maintaining behaviour change
Delivery specify _______________________________________________________
9 person to person counseling; 9 telephone counseling; 9 group counseling
9 generic tools; 9 decision board; 9 audio-guided workbook; 9 interactive videodisc; 9 linear video
9 computer based tool; 9 other
12
PRACTITIONER DECISION SUPPORT METHODS
check T those you wish to include
Type of DecisionSupport
9 clinical algorithms;
9 patient materials
9 other
Information:
9 scientific evidence re decision;
9 rationale for decision aid;
9 efficacy of decision aid;
9 timing and use in practice;
9 scientific references.
9 other
Delivery
9 manual; 9 video; 9 lecture; 9 workshop; 9 hot-line; 9 academic detailing
9 other
6. Which designs and measures will be used to develop and evaluate the
aid?
Development and evaluation depend on the objectives of decision aids. Developers need
to make decisions about: the sampling and design architecture; the criteria for evaluation; and
the measurement tools that will be used to operationalize the criteria.
In previous papers we have made the following comments about development design and
evaluation. The standard methods we use to develop decision aids include: 1) drafting and
redrafting by an expert inter-disciplinary panel of health services researchers, education and
communication specialists, and practitioners; 2) assessment and revision by panels of
practitioners and patients who are experienced with the decision; 3) pilot testing with
practitioners and patients at the point of decision making; 4) conducting evaluative studies
using before/after and randomized trial designs.
Our evaluations of decision aids distinguish between improved decision making and
improved outcomes of decisions because: a) decisions that depend on client's values cannot be
judged as right or wrong; and b) good decisions can still result in bad outcomes due to
variable nature of clinical outcomes. Therefore, we define a good decision as one that is
informed, consistent with personal values, and acted upon, in which participants express
satisfaction with decision making. We are also evaluating the impact of decision aids on
compliance with choice, quality of life, and appropriate use of resources.
Base on completed reviews, we have the following insights and recommendations for
evaluative studies.
Gaps in Research. There are several gaps in research on decision aids. More research is needed
on: a) how decision aids perform for different clinical decisions; b) their acceptability to practitioners;
c) their acceptability to diverse patient groups; d) their impact on patient-practitioner communication;
e) their downstream effects on persistence with the decision, distress, regret, and health-related quality
of life; and f) their optimal strategies for disseminating and implementation strategies.
Methodological Problems. Most evaluation studies are fraught with methodological difficulties.
They cannot be double-blind. Those that randomize patients rather than practitioners have
contamination problems that narrow the differences that will be detected. Those that randomize
practitioners need to be very large, because of cluster sampling. Moreover, they may have selection
biases because clinicians, knowing their assignment, may: a) be more or less enthusiastic about
13
recruiting patients; and b) recruit different types of patients. Despite the researchers’ best efforts, it is
very difficult in a real world setting to present the decision aid at the appropriate time to patients who
are eligible to consider all of the options in the aid. Furthermore, efficacious interventions may have no
impact if either patients or practitioners, or both, are extremely polarized toward one of the options at
baseline. When post-intervention measures are administered after the consequences of the choice are
known, it is very difficult to avoid having the outcome color their evaluation of satisfaction with the
decision making process and the decision.
Recommendations:
Future studies should:
• examine the impact of decision aids on a broader range of decisions with a more comprehensive
range of patient and practitioner outcomes;
• select patients who are at the point of decision making for whom the choices in the aid are relevant;
• measure patients’ and practitioners’ baseline predispositions toward the choices;
• have sample sizes large enough to detect clinically meaningful differences in decisions among the
undecided subgroup of patients;
• measure patients’ perceptions of practitioners’ opinions;
• have a usual care arm and describe clearly what usual care comprises;
• describe clearly what was in the decision aid and how it was used in the diagnostic/treatment
trajectory.
With this background, it is time to consider designs and evaluations. These charts may be useful to
identify the stage you are in and to record you views for discussion with the team.
AT THIS STAGE, WHICH DESIGN IS APPROPRIATE?
Development Panel
Participants: researchers, clinicians, educators, patients, opinion leaders;
Methods: iterations of drafts, feedback, revisions, feedback etc.
Review Panels
Participants: potential users (practitioners, patients who have already made decisions)
Methods:
focus groups, personal interviews, questionnaires to elicit acceptability, etc.
Sample size: n = 10 per group
Pilot Studies
Participants:
patients at the point of decision making
Designs where: X=decision aid intervention; C=control intervention; O=observation of effects;
R=randomization
post-test only, usually with pre-established criteria for success (e.g. 70% find aid acceptable)
sample size about 10
XO
14
before/after study- baseline questionnaire, decision aid, post-test questionnaire
sample size about 30 per group
OXO
Trials
Participants: patients at point of decision making; practitioners
Designs
quasi experiment
OCO
OXO
Randomized trial
unit of randomization 9 patient;
ROXO
ROCO
9 practitioner
AT THIS STAGE, WHICH MEASURES ARE APPROPRIATE
Criteria For Evaluation
Measurement Tools
EARLY STAGE (e.g. post-test only)
Acceptability
Acceptability Questionnaires–Barry tool, Ottawa tool
PILOT STUDIES (e.g. before/after)
Knowledge
Knowledge/Comprehension test
Expectations of outcomes
Probability scales- numbers and words
Clarity of values
Values subscale of Decisional Conflict Scale
Decision
Choice Question (option x, option y, unsure); choice predisposition
Decisional conflict
Decisional conflict scales for patients and providers
TRIALS
Knowledge, Expectations, Clarity of Values, Decision, Decisional Conflict as above
PLUS
Realistic perceptions of others
Perceptions of % of practitioners/patients choosing options; subjective
norms
Skill in decision making
Self-efficacy scale, Implementation data
Satisfaction with decision making Decision Satisfaction Inventory; Satisfaction with Decision); Satisfaction
with Preparation for Decision Making
Use of decision aid
Diary, Utilization data
Participation according to needs Congruence between preferred and actual role in decision making Degner
scale, Strull Question, Deber questionnaire
Persistence with decisions
Survey of decision over time; Refills, implementation data
Health related quality of life
Generic (e.g. SF 12) and Condition-specific
Use of resources
Analysis of utilization data
Costs
Consult health economist. See Hersey and Lohr Framework and Nease &
Owens cost-effectiveness model
A description of the measures we use and samples are appended in this section.
15
7. How should the decision aid be disseminated?
Dissemination involves the targeted distribution and promotion of the use of the decision
aid. Six key elements of research transfer and use (2) are presented: potential adopters;
practice environment; the evidence-based innovation (e.g. the decision aid); strategies for
transferring the evidence into practice; evidence adoption; and outcomes. These elements are
systematically monitored prior to, during, and following any research transfer efforts.
The data generated by monitoring is used to:
1)
2)
3)
4)
identify potential barriers and supports to research use associated with the potential
adopters, the practice environment, and evidence based innovation;
provide direction for selecting and tailoring transfer strategies;
track the progress of the transfer effort, and
assess the adoption of the evidence and its impact on outcomes of interest.
Although dissemination is identified as a final step, it should be addressed early in the
development process so that the aid is acceptable to potential users and has a greater potential
for adoption. Therefore, dissemination questions can be posed during the needs and feasibility
phases. Development and review panels can include potential users (practitioners and
patients) and partners who may assist with dissemination (consumer groups, health
professional organizations, disease foundations, and public education agencies).
16
References to Section I
1. O’Connor A, Rostom A, Fiset V, Tetroe J, Entwistle V, Llewellyn-Thomas H, Holmes-Rovner M, Barry M, Jones J.
Decision Aids for Patients Facing Health Treatment or Screening Decisions: A Cochrane Systematic Review. British
Medical Journal 1999;319:731-734. Contact Paola Rio, Co-ordinator of the Consumers and Communication Review Group
(rio@hna.ffh.vic.gov.au) for updates of the cochrane overview.
2. O’Connor AM, Fiset V, DeGrasse C, Graham I, Evans W, Stacey D, Laupacis A, Tugwell P. Decision aids for patients
considering health care options: Evidence of efficacy and policy implications. Journal of the National Cancer Institute
Monograph No 25., 1999 p.67-80. Overview of decision aids-points to gaps, methods issues, policy issues.
3. Research Triangle Institute. Consumer health informatics and patient decision-making. (AHCPR Pub. No. 98-N001).
Agency for Health Care Policy and Research: Prepared by James Hersey Jennifer Matheson and Kathleen Lohr at the
Research Triangle Institute. 1997. A gold mine for defining some of the directions the research should be taking.
Prepared by James Hersey and Kathleen Lohr at the Research Triangle Institute. Submitted to Denise Doherty and
Terrence Shannon at the AHCPR.
4. British Medical Journal- Patients as Partners in Care 1999;vol 319.
5. Effective Clinical Practice - July-August 1999;2(4):163-170.
6. Journal of the National Cancer Institute Monograph- Risk Communication and Decision Making. Number 25, 1999
7. O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E.
A decision aid for women considering hormone therapy after menopause: Decision support framework and evaluation.
Patient Education and Counselling, 1998;33(3):267-279.
8. O'Connor AM, Tugwell P, Wells G, Elmslie T, Jolly E, Hollingworth G, McPherson R, Drake ER, Hopman W,
MacKenzie T. Randomized trial of a portable, self-administered decision aid for post-menopausal women considering
long-term preventive hormone therapy. Med Dec Making. 1998;18 (3):295-303.
9. Man-Son-Hing M, Laupacis A, O’Connor A, Biggs J, Drake E, Yetisir E, Hart R & SPAF3 Investigators. A randomized
trial of a decision aid for patients with atrial fibrillation. JAMA 1999;282:737-743. This is a good example of using a
decision aid to debrief patients regarding the results of a clinical study they just participated in.
10. O’Connor AM, Drake ER, Fiset V, Graham I, Laupacis A & Tugwell P. The Ottawa Patient Decision Aids.
Effective Clinical Practice 1999;2(4):163-170. Descibes the Ottawa Approach and summarizes results to date. Part of a
special edition focused on decision making.
17
APPENDIX: Example of How to Apply the Steps
Example of developing a decision aid for postmenopausal hormone therapy (See
Reference 7-10).
1. Need.
In terms of need, there was general agreement that:
• the decision was difficult, due to the uncertainty of outcomes with long-term use, the need to balance
benefits against risks, and variation in opinions of whether the benefits outweighed the risks;
• sufficient and growing numbers were involved in making this decision (at least 10% of the population);
• there was considerable variation in utilization and preferences both from the practitioners' and clients'
perspectives;
• there was sufficient demand for an aid; and
• women and practitioners wanted a balanced presentation of the benefits and risks based on current
evidence and a guide to help streamline the process of deliberation between women and practitioners.
2. Feasibility.
In terms of feasibility, there was general agreement that:
• we had the resources, expertise, and networks to develop, evaluate and disseminate the aid;
•
we had commitment to update the evidence via our linkage to the International Cochrane Collaboration
which has an ongoing process of summarizing evidence of benefits and risks;
• although the evidence was expected to change as new studies emerged, we extended the shelf life of the
aid by using pooled evidence from meta-analyses and letting users know of upcoming trials in 10 years
which will provide higher quality evidence of long term benefits and risks; and
• we could deliver the aid in a manner that was acceptable and accessible to women and practitioners.
3. Goals and Objectives.
The goals of the decision aid were to improve decision making and the outcomes of the decisions from both the
patient and practitioner perspective. The specific objectives were to improve:
• decision making so that it is informed (based on adequate knowledge and realistic expectations),
consistent with personal values and implemented
• satisfaction with decision making
• persistence with choice
• health related quality of life
• appropriate use of health care resources.
4. Framework of Decision Support.
We used the Ottawa decision support framework
5. Decision Support Methods
Content and delivery methods: We structured the content of the decision aid according to the Ottawa
framework and American College of Physician counselling guidelines. There were two parts, a woman's takehome audio-workbook, and a practitioner manual.
The woman's take-home audio-workbook prepared the women for a follow-up visit to discuss the issue with her
practitioner. A 40 minute audiotape guided a woman through a 32-page illustrated booklet and personal
worksheet providing information on:
I. General Information on Diseases of Aging, Alternatives, Benefits, and Risks
• CHD, osteoporosis, endometrial cancer, and breast cancer including definition, incidence, median age of
onset, mortality rates, and the physical, emotional and social impact of these diseases; and major risk
factors;
• prevention and early detection strategies;
• HRT regimens; benefits and risks including effects on CHD, osteoporosis, menopausal symptoms,
endometrial cancer, breast cancer, side effects, contraindications, and other reasons women decide not to
take HRT;
• probabilities of disease with and without HRT according to risk of disease and hysterectomy status;
18
II. Steps in Weighing Her Own Benefits and Risks
Guidance in completing a Personal Worksheet to:
• identify her personal lifetime benefits and risks of HRT tailored to her hysterectomy status, and risk of
CHD, osteoporosis and breast cancer; review her menopausal symptoms status and menstrual history; and
consider other issues important in the decision;
• clarify her values using a "weigh scale" to rate her perceived importance of each benefit and risk;
• identify her current health practices in promoting healthy bones, heart, and breasts;
• list her questions;
• identify her preference for participation in decision making; and
• indicate her predisposition or "leaning" toward taking HRT.
III. Suggested Steps for a Follow-up Visit with her Practitioner
The completed Personal Worksheet provides a focus for discussion for the woman to:
• Review possible benefits and risks with her practitioner to verify & fill in gaps
• Discuss personal values by showing the practitioner her weigh scale
• Make a decision considering benefits, risks, personal values, and preference for decision participation
• Plan the next steps
The booklet included illustrative icons to represent each concept. The text was adjusted to a grade 8 reading
level, but was comprehensible to those with less than Grade 8 reading because of the accompanying audiotape
and illustrations. The data describing lifetime risks and benefits were those used in the ACP guidelines from
the overview of Grady et al. A woman used the aid in a self-paced, active way, responding to checklists and
writing in her opinions. In order to familiarize herself with the steps in weighing the benefits and risks, she
was shown how four other women completed the steps before being asked to complete her own assessment.
The four different cases also reinforced the notion that decision making is variable and can be individualized
according to a woman’s health history, values, current health practices, and preferences for decision
participation.
The practitioner's decision support materials included: 1) a guidelines for using the decision aid; 2) a manual
describing the scientific evidence of benefits and risks; practice guidelines; and suggested counseling
strategies; and 3) a prescription and management algorithm for those choosing HRT including patient
assessment; selection of hormonal treatments; and follow-up surveillance.
6. Development and Evaluation Design and Measures.
We developed and evaluated the decision aid in several phases. The early prototypes of the decision aid for
hormone replacement therapy (HRT) were developed using an iterative process with a: 1) development panel
of interdisciplinary researchers, educators, and practitioners; and 2) review panels comprising of potential
users including post-menopausal women, medical and nursing practitioners, and educators. These groups
guided its development using criteria such as need, feasibility, and appropriateness and acceptability of the
objectives, content, and decision support methods. The methods used to elicit the review panel’s opinions
included focus groups, meetings, interviews, and surveys.
We then proceeded to evaluate the acceptability of the decision aid with 40 women. Based on their comments
we revised the aid.
Next we conducted a before/after study of 94 postmenopausal patients from 6 family practices in downtown
Ottawa Canada. We found that compared to baseline, patients who used the take-home aid significantly
(p<0.001): improved their general knowledge of alternatives, benefits, and risks; had more realistic personal
expectations of outcomes with and without HRT; and reduced their decisional conflict scores, with the greatest
improvements observed in feeling certain, informed, clear about values, and supported in decision making.
We then conducted a randomized controlled trial of an audioguided workbook versus a general educational
pamphlet briefly describing HRT benefits, risks, side effects and likely beneficiaries in general terms. We
found that women exposed to the decision aid had significantly (p<0.04): more realistic personal expectations;
and less decisional conflict, particularly in the area of feeling informed, clear about values, and supported in
decision making. They also found the intervention more acceptable (p<0.05). There were no differences
between groups in general knowledge about HRT.
19
We are just completing a randomized controled trial of involving 40 family practitioners and 200 women. We
randomized 40 Ottawa family physicians to prepare women for counseling using either: 1) a pamphlet briefly
summarizing options and outcomes and likely beneficiaries in general terms; or 2) an audio-guided workbook
with detailed information on options, steps in decision making, examples, and personal worksheet
summarizing personal: outcome probabilities; values; practices; questions; preferred participation roles; and
predispositions. Patient and physician questionnaires were administered at baseline, after the take-home
intervention, and 1 week, 2 months and 9 months after counseling. Evaluation measures include: the quality of
patient-practitioner communication; efficiency and satisfaction with the decision support process; knowledge
and expectations; decisional conflict; satisfaction with the decision; persistence with decisions; distress from
treatment side effects; and quality of life.
We found that both intervention groups were comparable at baseline. Post-intervention, both groups had
improved knowledge, but those using the complex decision aid reported more: realistic expectations, shared
decision making, and satisfaction with decision making. Physicians using the audio-workbook were also more
satisfied with women’s preparation for decision making. Other outcomes are being analysed. Our conclusions
is that simpler aids are not as useful as complex aids in preparing patients for decision making about hormone
therapy.
7. Disseminate
Our dissemination strategy is currently in the planning stages. We have begun to assess the
barriers/facilitators of use and are evaluating potential implementation strategies.
20
Examples of Measures Used in the Ottawa Decision Aids
Evaluation Measures
Summary Table
Choice predisposition
Decision
Knowledge
Expectations
Values
Satisfaction with preparation
for decision making
References
21
Summary Table
Definition
Measure
Psychometric Properties
Populations
1. Demographic & Clinical
Characteristics
clients’: age, gender, education, marital
status, ethnicity, occupation, locale,
Statistics Canada Survey
diagnosis & duration of condition,
Validated in national surveys, previous
studies
General public, HRT, genetic testing,
treatments for breast cancer, end
stage COPD, atrial fibrillation,
schizophrenia
Survey questions previous studies
health status (physical, emotional,
cognitive, social)
Generic & Disease-specific quality of
life measures
practitioners’: age, gender, ethnicity,
clinical education and specialty,
practice locale, years of experience
Survey questions previous studies
22
2. Perceptions of Decision
Choice Predisposition
degree to which person is leaning
towards or away from an alternative at
baseline
Decision
stated choice among options
Definition
Decisional Conflict
uncertainty about course of action to
take arising from factors inherent in the
decision (uncertainty of outcomes, or
the need to make value tradeoffs
between benefits and risks) and
modifiable factors (inadequate
knowledge, unrealistic expectations,
unclear values and norms, and
inadequate social support).
Knowledge
cognizance of the clinical problem,
alternatives & rationale, and main
benefits, risks, and side effects
15-point scaled anchored by the
options ‘prefer A’ and ‘prefer B’ with
‘unsure’ at the midpoint (16)
Test-retest coefficient > 0.90 (16)
Correlates to values and expectations
(9)
Sensitive to change (9)
Question eliciting decision (16)
regarding an option (accept, decline,
unsure) or options (prefer A, prefer B,
..., unsure)
Test-retest > 0.90 (16)
Measure
Psychometric Properties
HRT, genetic testing, treatments for
breast cancer, end stage COPD,
atrial fibrillation, schizophrenia
Populations
Decisional Conflict Scale (1)
16-item Likert scale response format
Subscales: certainty, informed, clear
values, supported, quality decision
Test-retest & alpha coefficients >0.80
(1)
Discriminates between those delaying
and making decisions (1,15). Norm:
scores 2.0 (out of 5) or less associated
with making a decision
(effect size 0.43-0.82)
Sensitive to change following decision
aid (4, 5,9) & discriminates between
different interventions (7,8,10)
HRT, genetic testing, treatments for
breast cancer, end stage COPD,
atrial fibrillation, schizophrenia
Knowledge test (5,7,9,10) of
information in decision aid using
true/false/unsure response format
Content validity because information in
decision aid evidence-based and verified
by panels of clinicians and patients
HRT, treatments for lung cancer,
breast cancer, end stage COPD,
atrial fibrillation
23
Expectations, Realistic
perceived personal likelihood or
probability of outcomes with and
without treatment
an expectation is considered realistic if
it corresponds to the probabilities
presented in a decision aid for the
person s particular risk category (plus
or minus a pre-specified acceptable
range of responses )
Values
desirability or personal importance of
benefits & risks
Probability scales ranging from 0% to
100%; number of intervals depends on
degree of precision required for
particular decision outcomes
(9,10,17,18)
Test-retest coefficients > 0.80;
Alpha > 0.70
Content validity based on evidence
Sensitive to change following decision
aid (effect size = 1.0) (9)
Discriminates between patients in
different clinical risks categories (17)
Discriminates between decision aid and
controls (10,7) (effect size = 1.0)
HRT, atrial fibrillation,
schizophrenia
11-point rating scales (9,10) anchored
by 0 (not at all important to me in this
decision) and 10 (extremely important
to me in this decision)
Measures discriminate between those
making different decisions (9)
HRT, genetic testing, treatments for
breast cancer, end stage COPD,
atrial fibrillation, schizophrenia
24
Definition
Measure
Psychometric Properties
Populations
3. Perceptions of Others
Perceptions of the Variability/
Norms in Decision Making
Perceptions of the variability in:
a) choices made by other patients faced
with the decision; b) opinions of
important others (subjective norms)
perceptions of what important others
think is the appropriate choice. For the
client, important others may include
their spouse, family, peers, and
practitioner(s). For the practitioners, it
may include the client, professional
peers, and personal network.
Decision Participation Preference
actual and preferred role in decision
making (client controlled, shared with
practitioner, practitioner controlled,
other controlled)
1. Specific estimates (0%- 100%) of
the proportion of other patients
choosing an option. 2. Importance of
this information for personal decision
making. 3. General perceptions or
variability in qualitative terms using
Likert scale.
Currently being evaluated in study with
cardiac patients
Cardiac patients considering
autologous transfusion (ongoing)
Degner’s (19) Control Preferences
Scale with 5 response items ranging
from I prefer to make the decision to
I prefer that my doctor makes the
decision
Reliability and validity established in
several studies (19)
Cancer Rx
25
Definition
Measure
Psychometric Properties
Populations
4. Resources for Decision Making
Self-efficacy
confidence in one's abilities in decision
making, including shared decision
making
Satisfaction with preparation for shared
decision making
satisfaction with the way the decision
aid prepared patients for decision
making and communication with their
practitioners at a consultation visit to
discuss a particular treatment/
diagnostic decision
Decision Self-efficacy Scale- 11 item
summated rating scale with a 3 point
response scale consisting of 1 (a lot
confident), 2 (a little confident) and 3
(not confident). Focuses on the client's
social role in interacting with the health
team and extracts clients' judgements
of their abilities to obtain information
and express concerns and opinions
about treatment options, as well as
make informed choices.
alpha coefficient > 0.80
discriminates between those who make
and delay decisions (15)
Persons considering HRT and
treatments for schizophrenia
Patient and practitioner versions of 11item scale eliciting satisfaction with aid
in preparing patient for consultation
visit: organizing thoughts,
communicating values, stating
questions, participating in decision
making, improving efficiency and
quality of visit
alpha coefficients > 0.90
discriminates between interventions such
as general pamphlet and decision aid
(effect size =1.7) (20)
HRT (ongoing study)
26
Acceptability of decision aid
user s rating of comprehensibility of
each component in the decision aid
responses to questions on the length,
pace, amount of information, and
balance: a. 100 mm visual analogue
scale anchored by "poor" and
"excellent"; b. structured response
categories (eg. too long, just right, too
short; slanted toward option A,
balanced, slanted toward option B
Amniocentesis, HRT, lung cancer
27
Choice Predisposition
This question is designed to determine an individual’s choice predisposition. Make sure to ask this question at baseline as many
respondents already have polarized views.
[My opinion about hormone therapy]
Before using the education program, we want to know what your opinion is of using hormone therapy.
If your doctor asked you right now to make a choice about using (or continuing to use) hormone therapy, please show where
you would be on the scale below, by placing a check in the box # % # .
If you wanted to take hormones, you would check # % # far to the left.
If you did not want to take hormones, you would check # % # far to the right.
If you were not sure, you would check # % # in the middle.
Yes
Hormones
Unsure
No
Hormone
© O’Connor, 1996, University of Ottawa
_________________________________________________________________________________________________________
Decision
This question is designed to determine an individual’s choice post-intervention.
[My thoughts on the best choice for me]
Now that you have reviewed the information about volunteer-donated blood transfusions and self-donated blood
transfusions, which choice (U) looks the best for you?:
‘
Volunteer-donated blood transfusions
Reason/comments:
‘
Self-donated blood transfusions
Reason/comments:
‘
I'm not sure
Reason/comments:
28
© O’Connor, 1996, University of Ottawa
Knowledge
Knowledge is defined as cognizance of the clinical problem, the alternatives and rationale for their use, and the main benefits, risks
and side-effects important to the decision. Knowledge is tested using a true/false/unsure response format. The questions asked cover
some of the basic information about options, benefits and risks that patients should be aware of when making a decision. The
attached 20-item scale taps an individual s knowledge of hormone therapy. response format.
[What I know about hormone therapy]
Here are some questions about hormone replacement therapy. Don t worry if you can’t remember everything . . . we did not expect you to
memorize the information you received. But it would help us learn what things impressed you enough that you can recall them readily. If you
wish, you may refer back to the decision support strategy you reviewed.
Below are listed some statements about hormones taken after menopause. Please show whether you think they are true, false, or you are not sure
by circling the word beside each statement.
1.
2.
3.
4.
Hormone therapy can be given:
Early in menopause
True
False
Unsure
Well past the menopause
True
False
Unsure
For 10-20 years
True
False
Unsure
Protection from breast cancer
True
False
Unsure
Protection from broken hips from osteoporosis
True
False
Unsure
Protection from diabetes
True
False
Unsure
Protection from heart disease
True
False
Unsure
Increases risk of breast cancer
True
False
Unsure
Increases risk of broken bones from osteoporosis
True
False
Unsure
Increases risk of diabetes
True
False
Unsure
Increases risk of heart disease
True
False
Unsure
Breast tenderness
True
False
Unsure
Fainting
True
False
Unsure
Irritability
True
False
Unsure
Bloating
True
False
Unsure
Hot flushes
True
False
Unsure
Headache
True
False
Unsure
Menstrual bleeding
True
False
Unsure
Benefits of taking long-term hormone therapy are:
Risks of using hormone therapy are:
Some side effects of hormone therapy are:
29
Insomnia
True
False
Unsure
Weight gain
True
False
Unsure
30
Expectations
Expectations are the patients perceived personal likelihood or probability of outcomes with or without treatment. These are
probabilities ranging from 0% to 100%; the number and range of intervals depend upon the probabilities and the degree of precision
required for particular decisions. These outcomes have content validity because they are evidence-based, have been shown to
discriminate between interventions, and to be responsive to change following decision aids in several contexts. We vary the levels
of the scale according to the risk information presented in the decision aid.
The next questions deal with the your opinions on the number of women with risk factors similar to yours who may get
certain diseases based on their decision not to take or to take hormones.
First, lets consider how many may have diseases if they decide not to take any hormones.
A. Hip Fractures
For women with risk factors similar to yours, how many women out of 100 may get HIP FRACTURES in their lifetime if
they decide NOT to take any hormones
(Check |_| one)
|_| don’t know
|_| 0 out of 100 Nobody who has risk factors similar to mine will get hip fractures in their lifetime
|_| 5 out of 100
|_| 10 out of 100
|_| 15 out of 100
|_| 20 out of 100
|_| 25 out of 100 One quarter
|_| 30 out of 100
|_| 35 out of 100
|_| 40 out of 100
|_| 45 out of 100
|_| 50 out of 100 Half the women who have risk factors similar to mine may have hip fractures in their lifetime
|_| 55 out of 100
|_| 60 out of 100
|_| 65 out of 100
|_| 70 out of 100
|_| 75 out of 100 Three quarters
|_| 80 out of 100
|_| 85 out of 100
|_| 90 out of 100
|_| 100 out of 100 Everybody who has risk factors similar to mine will get hip fractures in their lifetime
©O’Connor, 1996, University of Ottawa
31
Values
While values don’t change much in response to an intervention, they are good predictors of choice. This scale taps into an
individual’s values for the benefits and risks of treatment.
[Things that are important to me when making a decision about hormone therapy]
Below are listed some things women consider when making a decision about hormone therapy. Please show how important these are to you by
circling a number from 0 (not at all important to me) to 10 (extremely important to me).
How important is protection from heart disease to you when making a decision about hormone therapy?
0
1
not
at all
important
2
3
4
5
6
7
8
9
10
extremely
important
to me
How important is protection from broken hips from osteoporosis to you when making a decision about hormone therapy?
0
1
2
3
4
5
6
7
8
9
10
not
extremely
at all
important
important
to me
What other positive factors are important to you when making a decision about hormone therapy? (Please specify any positive factors you
have considered and rate their importance to you)
a) _______________
0
1
2
3
4
5
6
7
8
9
10
b) _______________
0
1
2
3
4
5
6
7
8
9
10
c) _______________
0
1
not
at all
important
2
3
4
5
6
7
8
9
10
extremely
important
to me
How important are the side effects of therapy to you when making a decision about hormone therapy?
0
1
not
at all
important
2
3
4
5
6
7
8
9
10
extremely
important
to me
How important is the risk of breast cancer to you when making a decision about hormone therapy?
0
1
not
at all
important
2
3
4
5
6
7
8
9
10
extremely
important
to me
What other negative factors are important to you when making a decision about hormone therapy? (Please specify any negative factors
you have considered and rate their importance to you)
a) _______________
0
1
2
3
4
5
6
7
8
9
10
b) _______________
0
1
2
3
4
5
6
7
8
9
10
c) _______________
0
1
not
at all
important
2
3
4
5
6
7
8
9
10
extremely
important
to me
32
© O’Connor, 1996, University of Ottawa
33
Satisfaction with preparation for decision making
Most satisfaction measures are not very discriminating, but this 11-item scale produces large effect sizes.
The following questions refer to the audiotape, booklet [and personal worksheet] you used in preparation for your counseling visit at CHEO.
Please give your opinion about how the material prepared you for the visit by circling the number that shows how much it helped you. A ‘0’means
it did not help at all and a ‘4' means it helped a great deal.
How much did the audiotape, booklet [and personal worksheet] .....
1.
Help you to organize
your own thoughts about
the decision?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
2.
Help you to consider
what you think of
the pros and cons of
each option?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
3.
Help you to identify
the questions you
need to ask?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
4.
Help you to consider
how involved in this
decision you want to be?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
5.
Help you to know what to
expect at the visit with
your [doctor/counselor]?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
6.
Prepare you to communicate
your opinions?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
7.
Prepare you to make a
better decision?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
8.
[Will] Make the follow-up
visit run more smoothly?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
9.
Affect your relationship
with your doctor/counselor?
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
10.
[Will] Improve the way time
is/was spent during the
visit
0
Not at
All
1
Very
Little
2
Somewhat
3
A Lot
4
A Great
Deal
Graham & O’Connor, 1996, University of Ottawa
34
References to studies that have used some of the scales
1.
O'Connor, A. M. (1995). Validation of a Decisional Conflict Scale. Medical Decision Making, 15(1), 2530.
2.
O’Connor, A., Llewellyn-Thomas, H., Sawka, C., Pinfold, S., To, T., Harrison, D. (1997). Physicians’
opinions about decision aids for patients considering systemic adjuvant therapy for axillary-node negative
breast cancer. Patient Education and Counseling,30:143-53. Survey to find out from clinician’s
perspectives what outcomes would convince them to use a decision aid.
3.
Bredeson, C. (1997). Use of a disease specific decision aid to determine the minimal clinically important
difference between autologous bone marrow transplantation versus standard salvage therapy for patients
with high risk low grade non-Hodgkin’s lymphoma. Unpublished master’s thesis, University of Ottawa,
Ottawa, Ontario, Canada.
4.
Engler-Todd L, Drake E, O’Connor AM, Surh L, Hunter A. (1997) Evaluation of a decision aid for
prenatal testing for women of advanced maternal age. Journal of Genetic Counseling, 6(4): 439.
5.
Fiset, V. (1998). Evaluating the effectiveness of a decision aid for patients considering treatment options
for stage-4 non small cell lung cancer. Unpublished master’s thesis, University of Ottawa, Ottawa,
Ontario, Canada.
6.
Man-Son-Hing M, Laupacis A, O’Connor A, Wells G, Lemelin J, Wood W, & Dermer M. (1996)
Warfarin for atrial fibrillation: The patient’s perspective. Archives of Internal Medicine, 156: 1841-1848.
7.
Laupacis,A., Hing, M., O’Connor, A., Biggs, J., Hart, R. And the Stroke Prevention in Atrial Fibrillation
III (SPAF 3) Investigators. (1998, June). A randomized trial of an audiobooklet (AB) decision aid in
patients with atrial fibrillation (AF). Interactive poster presented at the 14th Annual Meeting of the
International Society for Technology Assessment in Health Care (ISTAHC), Ottawa, Ontario, Canada.
(Abstracts, p. 41).
8.
Morgan, Matthew. (1997). A randomized trial of the ischemic heart disease shared decision making
program. Unpublished master’s thesis, University of Toronto, Toronto, Ontario, Canada.
9.
O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H,
Graham I, Drake E. (1998) A decision aid for women considering hormone therapy after menopause:
Decision support framework and evaluation. Patient Education and Counselling, 33(3):267-279.
10. O'Connor AM, Tugwell P, Wells G, Elmslie T, Jolly E, Hollingworth G, McPherson R, Drake ER,
Hopman W, MacKenzie T. (1998) Randomized trial of a portable, self-administered decision aid for postmenopausal women considering long-term preventive hormone therapy. Medical Decision Making,
18(3):295-303.
11. O'Connor, A. M., Pennie, R. A., & Dales, R.E. (1996). Framing effects on expectations, decisions, and
side effects experienced: The case of influenza immunization. Journal of Clinical Epidemiology, 49(11),
1271-1276.
12. Rothert ML, Holmes-Rovner M, Rovner D, Kroll J, Breer L, Talarczyk G, Schmitt N, Padonu G, Wills C.
(1997) An educational intervention as decision support for menopausal women. Research in Nursing and
Health, 20:377-387.
13. O’Connor A, Wells GA, Tugwell P, Laupacis A, Elmslie T, Drake E. Randomized Trial Of Explicit
Versus Implicit Values Clarification Techniques Used in a patient decision aid. Health Expectations
(Forthcoming). Abstract in Medical Decision Making, 1998; 18(4).
14. Ajzen I, and Fishbein M. Understanding attitudes and predicting social behaviour. Englewood Cliffs:
Prentice Hall, 1980.
35
15. Bunn H, and O’Connor AM. (1996). Validation of client decision making instruments in the context of
psychiatry. Canadian Journal of Nursing Research. 28(3):13-27.
16. O’Connor A, Tugwell P, Wells G. (1994) Testing a portable, self-administered decision aid for post
menopausal women considering long-term hormone replacement therapy to prevent osteoporosis and heart
disease. Medical Decision Making, 14(4);438 (Abstract).
17. O’Connor, A., and Pennie, R. (1995). Reliability and validity of measures used to elicit health
expectations, values, tradeoffs and intentions to be immunized for Hepatitis B. Journal of Clinical
Epidemiology, 48(2): 255-262.
18. O’Connor, A., Davies, B., Dulberg, C., Buhler, P., Nadon, C., McBride, B., and Benzie, R. (1993).
Psychometric properties of a health risk attitude measure for use with pregnant smokers. Medical Care,
31(7): 658-662.
19. Degner L, Sloan JA, Venkatesh P. (1997) The control preferences scale, Canadian Journal of Nursing
Research, 29(3):21-43.
20. Unpublished data from an ongoing trial: A. O’Connor et al. Randomized trial of a decision aid versus a
pamphlet in preparing post-menopausal women and their family physicians for decision making regarding
hormone therapy. Funded by the Arthritis Society of Canada. A patient and practitioner version of a tool
entitled Satisfaction with preparation for decision making is being used. See Appendix A for a copy of
the instrument. The alpha coefficients exceed 0.90. Mean scores are 2 out of 5 (sd=0.56) for individuals
prepared with the pamphlet compared to 3 out of 5 (SD 0.55) for those prepared with the decision aid.
This difference is highly significant (P=0.001); the effect size is 1.7.
21. Sawka et al. (1998) Health Expectations,1: 23.
ANNOTATED BIBLIOGRAPHY ON DECISION AIDS
Reviews of the Efficacy of Decision Aids
O’Connor AM, Fiset V, DeGrasse C, Graham I, Evans W, Stacey D, Laupacis A, Tugwell P. Decision aids for patients
considering health care options: Evidence of efficacy and policy implications. Journal of the National Cancer Institute
Monograph No 25., 1999 p.67-80. Overview of decision aids-points to gaps, methods issues, policy issues.
Research Triangle Institute. Consumer health informatics and patient decision-making. (AHCPR Pub. No. 98-N001).
Agency for Health Care Policy and Research: Prepared by James Hersey Jennifer Matheson and Kathleen Lohr at the
Research Triangle Institute. 1997. A gold mine for defining some of the directions the research should be taking.
Prepared by James Hersey and Kathleen Lohr at the Research Triangle Institute. Submitted to Denise Doherty and
Terrence Shannon at the AHCPR.
O’Connor A, Rostom A, Fiset V, Tetroe J, Entwistle V, Llewellyn-Thomas H, Holmes-Rovner M, Barry M, Jones J.
Decision Aids for Patients Facing Health Treatment or Screening Decisions: A Cochrane Systematic Review. British
Medical Journal 1999;319:731-734. Contact Paola Rio, Co-ordinator of the Consumers and Communication Review Group
(rio@hna.ffh.vic.gov.au) for updates of the cochrane overview.
Special Issues of Journals Focused on Patient Decision Making, Forthcoming
British Medical Journal- Patients as Partners in Care 1999;vol 319.
Effective Clinical Practice - Patient Decision Making (journal sponsored by American College of Physicians, American
Society of Internal Medicine and Alliance of Community Health Plans) July-August 1999;2(4):163-170.
Journal of the National Cancer Institute Monograph- Risk Communication and Decision Making. Number 25, 1999, p. 6780
Health Expectations, March 2000
36
Situations requiring decision aids
Eddy DM. A Manual for Assessing Health Practices & Designing Practice Policies The Explicit Approach. Philadelphia:
American College of Physicians. 1992. Distinguishes between standards, guidelines, and options, definitions useful in
identifying when decision aids are appropriate.
American College of Physicians. Guidelines for counselling postmenopausal women about preventive hormone therapy.
Ann Intern Med. 1992;117(12):1038-41. An example of a practice guideline in which assessing patient preferences is
recommended in determining optimal treatment strategy.
Kassirer JP. Incorporating patients’ preferences into medical decisions. N Engl J Med. 1994;330(26):1895-6. Raises
important issue re when a patient preferences need to be incorporated in medical decision making.
Examples of Decision Aids from Different Centers
Foundation for Informed Medical Decision Making (Dartmouth/Massachussetts General)
BarryMJ, Cherkin DC, Chang Y, Fowler FJ, Skates S. A randomized trial of a multimedia shared decision-making
program for men facing a treatment decision for benign prostatic hyperplasia. Disease Management and Clinical
Outcomes, 1997; 1(1):5-14. The first RCT of the shared decision making programs developed by the Foundation for
Shared Decision Making located at Dartmouth University, New Hampshire, but including a network of researchers across
North America. See also:
Spunt BS, Deyo RA, Taylor VM, Leek KM, Goldberg HI, Mulley AG. An interactive videodisc program for low back pain
patients. Health Educ Res 1996;11(4):535-41.
Liao L, Jollis JG, DeLong ER, Peterson ED, Morris KG, Mark DB. Impact of an interactive video on decision making of
patients with ischemic heart disease. J Gen Intern Med 1996;11(6):373-6.
Wagner EH, Barrett P, Barry MJ, Barlow W, Fowler FJ Jr. The effect of a shared decision making program on rates of
surgery for benign prostatic hyperplasia. Pilot results. Med Care 1995; 33 (8):767-70.
Flood AB, Wennberg JE, Nease RF, Fowler FJJ, Ding J, Hynes LM. The importance of patient preference in the decision
to screen for prostate cancer. Prostate Patient Outcomes Research Team. J Gen Intern Med 1996;11:342-9.
Morgan, Matthew. A randomized trial of the ischemic heart disease shared decision making program. Unpublished
master’s thesis, University of Toronto, Toronto, Canada. 1997.
Bernstein SJ, Skarupski KA, Grayson CE, Starling MR, Bates ER, Eagle KA. A randomized controlled trial of
information-giving to patients referred for coronary angiography: Effects on outcomes of care. Health Expectations
1998;1(1):-61.
Michigan State University
Rothert ML, Holmes-Rovner M, Rovner D, Kroll J, Breer L, Talarczyk G, Schmitt N, Padonu G, Wills C. An educational
intervention as decision support for menopausal women. Research in Nursing and Health, 1997;20:377-387. Trial of
different methods of delivering decision support (written brochure, guided discussion in group, guided discussion in group
plus personalized decision exercise).
McMaster University
Levine MN, Gafni A, Markham B, MacFarlane D. A bedside decision instrument to elicit a patient’s preference
concerning adjuvant chemotherapy for breast cancer. Ann Intern Med. 1992;117(1):53-8. Preliminary evaluation of
decision boards, personally delivered by physicians or nurses, being used at McMaster University in Hamilton Ontario
Canada to promote shared decision making. See also:
Whelan TJ, Levine MN, Gafni A, Lukka H, Mohide EA, Patel M et al. Breast irradiation post lumpectomy: development
and evaluation of a decision instrument. J Clin Oncol 1995;13(4):847-53.
Sebban C, Browman G, Gafni A, Norman G, Levine M, Assouline D,et al. Design and validation of a bedside decision
instrument to elicit a patient's preference concerning allogeneic bone marrow transplantation in chronic myeloid leukemia.
Am J Hematol 1995;48:221-7.
New England Medical Center
37
Pauker SP, and Pauker SG. The amniocentesis decision: Ten years of decision analytic experience. Birth Defects.
1987;23(2):151-69. One of the first “bedside” decision aids using formal utility assessments that are incorporated in a
decision tree.
Nijmegen Institiute for Cognition and Information
Stalmeier PFM, Unic IJ, Verhoef LCG, Van Daal WAJ. Evaluation of a shared decision making program for women
suspected to have a genetic predisposition to breast cancer. Med Decis Making 1999;19:230-241. Uses a formal decision
analytic approach.
Queen’s University
Brundage MD, Cosby RH, Feldman-Stewart D, Gregg R, Dixon P, Youssef Y, et al. A pilot study of a decision aid for
patients with locally-advanced non-small cell lung cancer. Med Decis Making 1998:18(4);483 (Abstract). Uses the
probability tradeoff approach to clarify personal values.
United Kingdom Kings Fund on Informed Choice
A series of studies are being funded to evaluate decision supporting interventions.
University of Manitoba
Davison BJ, Degner LE. Empowerment of men newly diagnosed with prostate cancer. Cancer Nurs 1997;20:187-96
Davison JB, Kirk P, Degner LF, Hassard TH. Information and Patient Participation in Screening for Prostate Cancer.
Patient Education and Counseling Forthcoming June 1999.
University of Ottawa/Loeb Research InstituteWWW.LRI.CA and click on Ottawa Health Decision Center
O'Connor AM, Tugwell P, Wells G, Elmslie T, Jolly E, Hollingworth G, McPherson R, Drake ER, Hopman W,
MacKenzie T. Randomized trial of a portable, self-administered decision aid for post-menopausal women considering
long-term preventive hormone therapy. Med Dec Making. 1998;18 (3):. Ottawa approach to prepare practitioner and
patient for decision making; patient take-home materials include booklet, audiotape, and personal worksheet; the
worksheet provides a focus for discussion at a follow-up visit.
Man-Son-Hing M, Laupacis A, O’Connor A, Biggs J, Drake E, Yetisir E, Hart R & SPAF3 Investigators. A randomized
trial of a decision aid for patients with atrial fibrillation. JAMA 1999;282:737-743. This is a good example of using a
decision aid to debrief patients regarding the results of a clinical study they just participated in.
O’Connor AM, Drake ER, Fiset V, Graham I, Laupacis A & Tugwell P. The Ottawa Patient Decision Aids. Effective
Clinical Practice 1999;2(4):163-170. Descibes the Ottawa Approach and summarizes results to date. Part of a special
edition focused on decision making.
University of Rochester
Dolan JG, Kerr JE. Colorectal cancer screening: A multicriteria decision analysis.
(Abstract).
Med Decis Making 1998:18(4);488
Dolan JG. Are patients capable of using the analytic hierarchy process and willing to use it to make clinical decisions?
Med Decis Making 1995;15(1):76-80.
University of Toronto
Llewellyn-Thomas HA, McGreal MJ, Thiel EC, Fines S, Erlichman C. Patients’ willingness to enter clinical trials:
measuring the association with perceived benefit and preference for decision participation. Soc Sci & Med.
1991;32(1):35-42. An example of using a probability tradeoff technique to clarify a person’s values regarding entering a
clinical trial.
Sawka CA, Goel V, Mahut CA, Taylor GA, Thiel EC, O’Connor AM. Development of a patient decision aid for choice of
surgical treatment for breast cancer. Health Expectations 1998;1(1):23-36. Uses the Ottawa Approach. See also: Goel V,
Sawka C, Thiel E, Gort E, O’Connor A. A randomized trial of a decision aid for breast cancer surgery. Med Decis Making
1998:18(4);482 (Abstract).
Conceptual Frameworks of Decision Support
Charles C, Gafni A, Whelan T. Shared decision-making in the medical encounter: What does it mean? Soc Sci Med,
1997:44:681-692. McMaster Approach. Useful examination of what constitutes shared decision making.
38
Entwistle VA, Sowden AJ, Watt IS. Evaluating interventions to promote patient involvement in decision making: By what
criteria should effectiveness be judged? J Health Serv Res Policy 1998; 3(2):100-7. From the York University Center for
Dissemination, UK. Illustrates that evaluation criteria depend in part on the the type of involvement of the patient and
practitioner. For an examination of the ethical underpinnings of informed choice, see also: Entwistle VA, Sheldon TA,
Sowden A, Watt IS. Evidence-Informed patient choice. Int J Technol Assess Health Care 1998;14:2.
Llewellyn-Thomas, H. Presidential Address. Med Dec Making. 1995;15(2):101-6. A framework for understanding the
factors that influence decisions and preferences, placed in a broader sociopolitical context. Toronto Approach. Very good
for distinguishing between preferences for outcomes, time, treatments, control in decision making etc.
Mulley A. Outcomes research: Implications for policy and practice. In: Smith R, Delamother T, editors. Outcomes in
Clinical Practice. London: BMJ Publishing Group: 1995. p. 13-27. Foundation for Informed Medical Decision Making
approach. Focus on outcomes research.
Keeney RL, Raiffa H. Decisions with multiple objectives: preferences and value tradeoffs. New York: John Wiley and
Sons; 1976. Illustrates the decision analytic perspective.
O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E. A
decision aid for women considering hormone therapy after menopause: Decision support framework and evaluation.
Patient Education and Counselling, 1998;33(3):267-279. Ottawa Framework to guide decision support strategies used in
decision aids and their evaluation. Illustration of how to operationalize the framework using a HRT decision aid. Defines
determinants of decisions, decision support interventions addressing determinants, and potential effects on the quality of
the decision process, the decision, and outcome of the decision. See also www.LRI.CA
Research Triangle Institute. Consumer health informatics and patient decision-making. (AHCPR Pub. No. 98-N001).
Agency for Health Care Policy and Research: Prepared by James Hersey Jennifer Matheson and Kathleen Lohr at the
Research Triangle Institute. 1997. The framework has a strong health services research perspective.
Rothert M, Talarcyzk GJ. Patient compliance and the decision making process of clinicians and patients. J Compliance
Health Care 1987;2:55-71. From Michigan State University. Describes the mutual roles of patients and practitioners.
Measures for Evaluating Decision Aids
Overview of issues:
See essays written by several leading authors on defining “a good decision” in Effective Clinical Practice 1999;2(4):163170.
Edwards A, Elwyn G. How should ‘effeciveness’ of risk communication to aid patients’ decisions be judged? A review of
the literature. Medical Decision Making 1999; 19 Forthcoming oct/dec issue. Describes the measures used in published
studies including cognitive, affective, and behavioural measures.
Entwistle VA, Sowden AJ, Watt IS. Evaluating interventions to promote patient involvement in decision making: By what
criteria should effectiveness be judged? J Health Serv Res Policy 1998; 3(2):100-7. Excellent discussion of evaluation
issues.
O’Connor AM, Fiset V, DeGrasse C, Graham I, Evans W, Stacey D, Laupacis A, Tugwell P. Decision aids for patients
considering health care options: Evidence of efficacy and policy implications. Journal of the National Cancer Institute
(Forthcoming September 1999) Discusses evaluation issues and mentions several of the tools in use.
O’Connor A, Llewellyn-Thomas H, Sawka C, Pinfold S, To T, Harrison D. Physicians’ opinions about decision aids for
patients considering systemic adjuvant therapy for axillary-node negative breast cancer. Patient Edu Counsel 1997;30:14353. Describes criteria for evaluation that would convince a random sample of oncologists that a decision aid was
‘effective’.
Decisional Conflict
Dolan, J.G., Markakis, K.M., Beckman, H.B., & Gleeson, M.L. (1996). Further evaluation of the provider decision process
assessment instrument (PDPAI): a process-based method for assessing the quality of health providers’ decisions (abstract).
Medical Decision Making, 16(4), 465. Tool to assess practitioner’s decisional conflict, based on one developed by
O’Connor for patients. See also www.LRI.CA
O'Connor, A. M. (1995). Validation of a Decisional Conflict Scale. Medical Decision Making, 15(1), 25-30. Keywords:
decisional conflict scale; conceptual framework decisional conflict; 16 items; 3 subscales: uncertainty Re decision; factors
39
contributing to uncertainty; perceived effective decision making; test-retest 0.81; internal consistency 0.78-0.92;
discriminant validity high; population immunization , breast screening. See also: Bunn H, O’Connor AM.Validation of
client decision making instruments in the context of psychiatry. Can J Nurs Res 1996;28(3):13-27.
Decision Making Style, Preference
Degner L, Sloan JA, Venkatesh P. The control preferences scale. Can J Nurs Res 1997;29(3):21-43. Overview of a
measure used to elicit preferences for participation in decision making.
Kaplan, S. H., Greenfield, S., Gandek, B., Rogers, W. H., & Ware, J. E. (1996) Characteristics of physicians with
participatory decision-making styles. Annals of Internal Medicine, 124(5), 497-504.
Krantz, D.S., Baum, A., & Wideman, M.V. (1980). Assessment of preferences for self-treatment and information in health
care. Journal of Personality and Social Psychology, 39, 977-990. Keywords: decision aid tool; Krantz Health Opinion
Survey; preferences for Tx approaches; two subscales; measure preferences for information and for behavioural
involvement in medical care; validity: construed, criterion, discriminant; reliability: test-retest.
Lerman, C.E., Brody, D.S., Caputo, G.C., Smith, D.G., Lazaro, C.G., & Wolfson, H.G. (1990). Patients' Perceived
Involvement in Care Scale: relationship to attitudes about illness and medical care. Journal of General Internal Medicine,
5, 29-33. Keywords: decision aid tool; perceived involvement in care scale; attitudes to illness and management of illness
(Dr. facilitation of patient involvement, level of info exchange; participation in decision making) ; reliability/validity;
internal consistency; .
Strull WM, Lo B, Charles G. Do patients want to participate in decision making? JAMA 1984;252:2990-4. Describes a
measnure for eliciting preferences for participation in decision making.
Satisfaction with decision, decision making
BarryMJ, Cherkin DC, Chang Y, Fowler FJ, Skates S. A randomized trial of a multimedia shared decision-making
program for men facing a treatment decision for benign prostatic hyperplasia. DMCO, 1997; 1(1):5-14. Describes the
Decision Satisfaction Inventory tool in detail.
Guyatt, G. H., Mitchell, A., Molloy, D. W., Capretta, R., Horsman, J., Griffith, L. (1995). Measuring patient and relative
satisfaction with level of aggressiveness of care and involvement in care decisions in the context of life threatening illness.
Journal of Epidemiology, 48, 1225-1224. Keywords: decision aid tool; indices; patient satisfaction index (23 items) which
measures patient and relative satisfaction with aggressiveness of Tx and degree of participation in decision making;
relative of competent patient satisfaction index (34 items); relative of incompetent patient satisfaction index (29 items);
102 elderly patients and 153 relatives, recruited 8 nursing homes; intraclass correlations 0.86-0.94; correlations with
global ratings high (0.59-0.75)
Hollen, P.J. (1994). Psychometric properties of two instruments to measure quality decision making. Research in Nursing
& Health, 17, 137-148. Keywords: decision aid tool; decision making quality scale, (DMQS) version x2 (self and other);
decision making quality inventory; (DMQI) (version V2- teens, parents); conceptual basis; Janes and Mann conflict model;
DMQS 7 criteria; canvassing of alternatives and objectives; evaluation of consequences; search for info; unbiased
assimilation of new info; reevaluation of consequences; planning for implementation contingencies, (7 items); DMQI
decision making style (5 types) through 6 stages of decision making (24 items), acceptability, reliability, content validity
described.
Holmes-Rovner, M., Kroll, J., Rothert, M. L., Schmitt, N., Rovner, D. R., Breer, L., Padonu, G., & Talarczyk, G. (1996).
Patient satisfaction with health care decisions. The Satisfaction with Decision Scale. Medical Decision Making, 16(1),
58-64. Keywords: decision aid tool; satisfaction with decision tool; global satisfaction with decision and 3 attributes of
effective decision; differentiates satisfaction with decision from satisfaction with provider, desire to participate in decision;
6 items; 250 women hormone replacement therapy; feasibility high; correlation with decisional conflict scale / confidence
in decision scale.
McCusker, J. (1984). Development of scales to measure satisfaction and preferences regarding long-term and terminal
care. Medical Care, 22, 476-493. Keywords: decision aid tool; measuring attitudes toward medical care of chronically and
terminally ill patients and families; conceptual framework: Breslau twelve scales, three versions;( internal consistency,
discriminant validity, convergent validity) evaluated for each scale.
Sutherland, H. J., Lockwood, G. A., Minkin, S., Tritchler, D. L., Till, J. E., & Llewellyn-Thomas, H. A. (1989).
Measuring satisfaction with health care: a comparison of single with paired rating strategies. Social Science & Medicine,
28, 53-58. Keywords: decision aid tool; comparisons of 2 techniques of measuring patient satisfaction with health care;
paired comparisons; ratings on VAS; breast cancer patients; test-retest reliability; convergent reliability.
40
Roberts, C. S., Cox, C. E., Reintgen, D. S., Baile, W. F., & Gilbertini, M. (1994). Influence of physician communication
on newly diagnosed breast patients' psychologic adjustment and decision- making. Cancer, 74, 336-341. Keywords:
decision aid tool; cancer diagnostic interview scale; patient perception of surgeon behaviour; info-giving or interpersonal;
18 items; internal consistency 0.92
41
SECTION II
Developing a schedule for Adaptation and Feasibility Testing of Decision Aids
You have considered many of the issues about using decision aids. Now its time to plan the next steps.
Below is a chart to summarize some of the tasks that need to be done in making progress toward achieving your goal. Think about what need to
be done, who should do it, and when the task should be started and completed.
Schedule
Task
Person Responsible
Projected Start Date
Projected Completion
Date
42