1
Appendix A: Patient Reported Outcomes Research Essentials
2
Patient-reported outcomes (PRO) research focuses on assessments of health-related quality of
3
life,1 function, personal satisfaction, and well-being, among other topics. PRO research is based
4
on the development, administration, interpretation, and use of PRO measures (PROM), also
5
known as instruments. Each PROM is developed to measure patient responses in a specific
6
domain or set of domains. A domain is a physical, mental, or social state, which may be further
7
divided into meaningful subdomains. Examples of domains include the following: physical
8
functioning, occupational / role functioning, pain intensity, anxiety, or satisfaction with social
9
role.2–4 This conceptual framework provides the basis for understanding how patient-reported
10
11
outcomes relate to one-another and to the patient.
PROMs may be “generic” instruments (which measure general domains in health-related
12
quality of life) or disease or condition “specific” instruments (such as an instrument measuring
13
physical or social function related to blepharoptosis). The number of questions or “items”
14
comprising each instrument varies from very few to many. For each item, subjects record their
15
level of agreement or disagreement. When response scales are balanced with an equal number of
16
equally “spaced” (requiring psychometric calibration) positive and negative positions, they may
17
behave as interval-level measures rather than ordinal-level measures.5,6 These techniques allow
18
experts to build instruments and draw conclusions based on individual items and respondents,
19
rather than at the test-level or population-level alone.7 Finally, although most readers have a
20
basic scientific appreciation for terms like “reliability” and “validity,” these terms have formal
21
psychometric definitions in PRO research (main Table 1). When used appropriately in a well-
22
designed study, they become powerful tools for quantitatively assessing otherwise qualitative
23
concepts.
24
25
26
Appendix B: Evidence-Based Practice and Patient-Reported Outcomes
27
A useful framework has been developed to help Oculofacial clinicians incorporate PRO literature
28
into daily evidence-based practice (Figure 1). This novel framework combines insights from
29
evidence-based medicine experts8–10 and PRO experts in plastic surgery.1 The evidence-based
30
practice model generally starts with a clinical problem raised while treating a patient
31
(“IDENTIFY”). The clinician formulates a specific question designed to address the problem
32
(“ASK”). The question is used to fuel a literature search (“ACQUIRE”). Evidence is then
33
reviewed for validity and usefulness (“APPRAISE”). Appraisal of PRO evidence requires
34
evaluating not only study methodology, but also instrument development and results. Once the
35
evidence is deemed valid and applicable, it is used in conjunction with clinical experience and
36
patient preferences to make medical decisions (“APPLY”). Clinical outcome or performance is
37
then evaluated (“EVALUATE”) and the cycle begins again.
38
39
Appendix C: How to initiate PRO research
40
This primer was written to save other interested investigators the time and logistical effort of
41
breaking into the uncharted domain of oculofacial PRO research.
42
WHO: A strong team will combine both clinical and outcomes experts. A motivated team
43
of clinical investigators is necessary for study conception and design, as well as instrument
44
development and data collection. PRO experts have the requisite experience to operationalize the
45
project using formal methods in psychological measurement theory. Even if the clinical
46
investigator does not intent to develop a new measure, the PRO expert can offer guidance on
47
how to use existing instruments, how to interpret data, etc. The authors suggest consulting their
48
affiliated or local post-secondary institution, seeking potential collaborators in the following
49
departments: Medical Social Sciences, Psychology, and / or Educational Psychology. Additional
50
team members to consider include medical librarians, statisticians, and trained patient
51
interviewers.
52
WHAT to do and What to avoid (Logistics): The clinician is encouraged to compile a
53
library of relevant resources, which might include texts on Measurement Theory and Health-
54
Related Quality of Life,11–14 as well as tutorials and reviews of PRO in related specialties,15–18
55
and representative high-quality generic and disease-specific PRO measures.19–22 Define study
56
goals early. Communicate with the Institutional Review Board. Involving PRO experts --
57
especially early in the study setup -- is essential, as their omission may jeopardize quality,
58
credibility and usefulness of the entire project. Do not wait until later in the process to involve
59
these experts -- it may be too late or too expensive to recover from unguided mistakes. The
60
Patient Reported Outcomes Measurement Information System (http://www.nihpromis.org/)
61
offers a wealth of information and may serve as a springboard for your next step.
62
HOW: Developing a novel instrument is not always necessary, and accomplishing this
63
properly requires substantial commitments of time, money, and energy.16,17,23 However, it is
64
instructive to review the processes involved in item / instrument development (Figure 2).
65
66
Figure Legends:
67
Figure 1: Framework for incorporating patient-reported outcomes into evidence-based practice.
68
The upper rectangular area frames a general approach to implementing an evidence-based
69
approach to clinical medicine. The lower rectangular area specifically addresses appraisal of
70
studies involving patient-reported outcomes.
71
72
Figure 2: General stepwise approach to patient-reported outcomes instrument development.
73
74
Figure 3: Growth of Outcomes Research. Between 1990 and 2012, 98,520 articles classified
75
with the MeSH heading “Quality of Life” had been published, resulting from an annual
76
growth rate 365 papers per year (approximately 365 more papers were published each
77
new year, compared with the previous year). Combining MeSH headings of “Eyelids,”
78
“Orbit,” and “Lacrimal Apparatus,” 19,782 articles had been published over the same
79
period, with a growth of 22.5 papers per year.
80
81
Supplementary References
82
83
1. Davis Sears E, Chung KC. A guide to interpreting a study of patient-reported outcomes. Plast
Reconstr Surg 2012;129:1200–7.
84
85
2. Revicki DA, Chen W-H, Harnam N, et al. Development and psychometric analysis of the
PROMIS pain behavior item bank. Pain 2009;146:158–69.
86
87
88
3. Rose M, Bjorner JB, Becker J, et al. Evaluation of a preliminary physical function item bank
supported the expected advantages of the Patient-Reported Outcomes Measurement Information
System (PROMIS). J Clin Epidemiol 2008;61:17–33.
89
90
91
4. Teresi JA, Ocepek-Welikson K, Kleinman M, et al. Analysis of differential item functioning
in the depression item bank from the Patient Reported Outcome Measurement Information
System (PROMIS): An item response theory approach. Psychol Sci Q 2009;51:148–180.
92
93
94
5. Carifio J, Perla RJ, Lowell M. Ten Common Misunderstandings, Misconceptions, Persistent
Myths and Urban Legends about Likert Scales and Likert Response Formats and their Antidotes.
J Soc Sci 2007;3:106–116.
95
6. Jamieson S. Likert scales: how to (ab)use them. Med Educ 2004;38:1217–8.
96
97
7. Sacristán JA. Patient-centered medicine and patient-oriented research: improving health
outcomes for individual patients. BMC Med Inform Decis Mak 2013;13:6.
98
99
100
8. Guyatt G, Rennie D, Meade M, Cook D. USERS ’ GUIDES TO THE MEDICAL
LITERATURE EVIDENCE-BASED CLINICAL PRACTICE. Second. New York, NY:
McGraw Hill Medical; 2008.
101
102
9. Schardt C, Mayer J. Introduction to evidence based practice. Duke Univ Med Cent Libr UNC
Heal Sci Libr 2010. Available at: http://www.hsl.unc.edu/services/tutorials/ebm/.
103
104
10. Buckingham J, Saunders D, Fisher B. Evidence based medicine toolkit. Univ Alberta,
Canada 2013.
105
106
11. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development
and use. (2nd, ed.). Oxford University Press; 2008:364.
107
108
12. McDowell I. Measuring health: a guide to rating scales and questionnaires, third edition.
Oxford University Press; 2006:768.
109
110
13. Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human
Sciences, Second Edition. New York, NY: Lawrence Erlbaum Associates; 2007:360.
111
112
14. Spilker B. Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd ed. Lippincott
Williams & Wilkins; 1995:1312.
113
15. Alsarraf R. Outcomes instruments in facial plastic surgery. Facial Plast Surg 2002;18:77–86.
114
115
116
16. Pusic AL, Lemaine V, Klassen AF, et al. Patient-reported outcome measures in plastic
surgery: use and interpretation in evidence-based medicine. Plast Reconstr Surg 2011;127:1361–
1367.
117
118
17. Cano SJ, Klassen A, Pusic AL. The science behind quality-of-life measurement: a primer for
plastic surgeons. Plast Reconstr Surg 2009;123:98e–106e.
119
120
18. Thoma A, McKnight LL. Quality-adjusted life-year as a surgical outcome measure: a primer
for plastic surgeons. Plast Reconstr Surg 2010;125:1279–87.
121
122
19. Mangione CM, Lee PP, Gutierrez PR, et al. Development of the 25-item National Eye
Institute Visual Function Questionnaire. Arch Ophthalmol 2001;119:1050–1058.
123
124
125
20. Terwee CB, Gerding MN, Dekker FW, et al. Development of a disease specific quality of life
questionnaire for patients with Graves’ ophthalmopathy: the GO-QOL. Br J Ophthalmol
1998;82:773–9.
126
127
128
21. Alsarraf R, Larrabee Jr. WF, Anderson S, et al. Measuring cosmetic facial plastic surgery
outcomes: a pilot study. Arch facial Plast Surg Off Publ Am Acad Facial Plast Reconstr
Surgery, Inc Int Fed Facial Plast Surg Soc 2001;3:198–201.
129
130
22. Mistry N, Rockley TJ, Reynolds T, Hopkins C. Development and validation of a symptom
questionnaire for recording outcomes in adult lacrimal surgery. Rhinology 2011;49:538–45.
131
132
23. FDA. Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product
Development to Support Labeling Claims. Silver Spring, Maryland; 2009.
133
134