The Analysis and
Interpretation of
Pain Clinical Trial Outcomes:
Enhancing Understanding
John T. Farrar, MD, PhD
University of Pennsylvania
CCEB
Surrogate Outcomes
Only three “real” outcomes
• Birth
• Death
• Quality of life
CCEB
Changing the State of the Brain
What do
you see?
CCEB
Why Do We Care
• RCTs - important for most medical therapy
• Did not need an RCT for introduction of
penicillin
–Pneumococcal pneumonia
–No penicillin - Last week 9/10 people died
–With penicillin – This week 1/10 people died
• Corollary – if you identify the right group,
measurement and design issues statistics
will be less controvertial
CCEB
Outline of the Presentation
• Measurement must be appropriate
• Handling of missing data is important
• Part 1: How do patients report pain
• Part 2: Analysis
• Part 3: Interpretation
CCEB
Pain is a Subjective Experience
• No “objective” direct measure
• Not easy to relate to an underlying
neurologic process in an individual
• Depend on subjects to accurately
report their experience
CCEB
• Creates inter-person variation in the
reporting of pain that is unavoidable
• Creates observer discomfort about
the validity of the measure
Pain Measures - Intensity Scales
0__1__2__3__4__5__6__7__8__9__10
|____________________________________________|
|
|
Least
Worst
None Mild Moderate Severe Excruciating

Intra-person reliability – Good
Inter-person reliability – Poor
CCEB
How do Patients Decide
If a Treatment is Useful
• Does the treatment make my symptoms
better now?
• Are there any side-effects?
• Is the pain relief “good enough”?
>>>> Am I better overall?
>>>> Should I take something else?
CCEB
Global Rating of Quality of Life
Overall how would you rate your quality of life:
over the last ______:
0 1
Worst
It can be
CCEB
2
3
4
5
6
7
8
9
10
Best
it can be
Global Change in Quality of Life
How has your quality of life changed over the
last ______:
(or - since the last _____:)
Very
Much
Worse
CCEB
Much
Worse
A little
worse
No
change
A little
better
Much
Better
Very
Much
better
Another View on Scales
How Do Patients Use a Numeric Scale
(Acute Pain)
• Study data: Randomized clinical trial of oral
trans-mucosal fentanyl versus placebo
• Method: Re-analysis of data set stratified on
baseline pain intensity score
• Population: 89 cancer pain patients with
acute breakthrough pain
• Results =>
CCEB
Data Collection Instrument
• Baseline
– Pain Intensity 1_2_3_4_5_6_7_8_9_10
• At 15, 30, 45 and 60 minutes
– Pain Intensity 1_2_3_4_5_6_7_8_9_10
– Pain Relief 0 (none) 1(slight) 2(mod.) 3(lots) 4(comp.)
• Second rescue medication - Time________
– Overall Performance
» 0 (none) 1(slight) 2(moderate) 3(lots) 4(complete)
CCEB
Raw Change in Pain Intensity
Compared to Global Performance Scale
0
-2
4
Raw PID
5
-4
6
7
-6
8
9
-8
10
-10
Poor
Fair
Good
Very Good
Global Performance Scale
CCEB
Excellent
Percent Change in Pain Intensity
Compared to Global Performance Scale
00%
% PID
Raw
PID
-2
-20%
4
5
-40%
-4
6
7
-60%
-6
8
9
-80%
-8
10
-100%
-10
Poor
Poor
Fair
Fair
Good
Good
Good
Very
Very
GoodExcellent
Excellent
Global Performance Scale
CCEB
How Do Patients Use a Numeric Scale
(Chronic Pain)
• Study data - RCTs of pregabalin in multiple
diseases
• Method – Compared measured pain
intensity (0-10 NRS) and patients global
impression of change (PGIC)
• Population - Data on 2,724 subjects from
10 clinical trials of diabetic neuropathy (3),
postherpetic neuralgia (3), chronic low
back pain (2), fibromyalgia (1) and
osteoarthritis (2).
CCEB
Reduction of Pain Diary Scores from Baseline to Endpoint
2
1
0
Raw Change Score
-1
BP = 4
BP = 5
BP = 6
BP = 7
BP = 8
BP = 9
-2
-3
-4
-5
-6
-7
-8
Very Much Worse /
Much Worse
Minimally Worse
No Change
Minimally Improved
PGIC Category
Much Improved
Very Much
Improved
Percent Reduction of Pain Diary Scores from Baseline to Endpoint
30
20
10
Percent Change Score
0
-10
BP = 4
BP = 5
BP = 6
BP = 7
BP = 8
BP = 9
-20
-30
-40
-50
-60
-70
-80
Very Much Worse /
Much Worse
Minimally Worse
No Change
Minimally Improved
PGIC Category
Much Improved
Very Much
Improved
Clinically Important Differences
for the 0-10 NRS
• Used the global response levels as the metric
of a clinical importance response
• Compared change in 0-10 NRS measure over
time to this standard
• Determined the clinically important change
cut-off by calculating:
–Sensitivity, specificity and accuracy
–Receiver Operator Characteristic (ROC)
analysis
CCEB
Studies of Duloxetine
• Secondary analysis of 5 studies
–Diabetic neuropathy – 3
–Fibromyalgia – 2
• Total number of patients – 1600
• Study period – 12 weeks
• Pain measures – 0-10 NRS
–Worst, least, average
• Patient global impression of change
CCEB
BPI Average Pain Percentage Change Score
Improvement
A
80
70
60
50
40
30
20
10
0
-10
-20
-30
-40
-50
-60
-70
-80
B
Study 1:
Study 2:
Study 3:
Study 4:
Study 5:
Very
much
worse/
Much
worse
A little
worse
No
change
A little
better
Much
better
DPNP
DPNP
DPNP
FM
FM
Very
much
better
80
70
60
50
40
30
20
10
0
-10
-20
-30
-40
-50
-60
-70
-80
PGI Category
Duloxetine (N=1077)
Placebo (N=533)
Very
much
worse/
Much
worse
A little
worse
No
change
No
change
Very
much
better
D
Male (N=636)
Female (N=974)
A little
worse
Much
better
PGI Category
C
80
70
60
50
40
30
20
10
0
-10
-20
-30
-40
-50
-60
-70
-80
Very
much
worse/
Much
worse
A little
better
A little
better
PGI Category
Much
better
Very
much
better
80
70
60
50
40
30
20
10
0
-10
-20
-30
-40
-50
-60
-70
-80
18-49 (N=434)
50-59 (N=547)
60-69 (N=431)
70+ (N=198)
Very
much
worse/
Much
worse
A little
worse
No
change
A little
better
PGI Category
Much
better
Very
much
better
Receiver Operator Response Curve
Percentage Pain Intensity Difference
CCEB
Clinically Important Values
BPI average
pain
(N=1610)
Model
BPI average
pain
(N=1610)
BPI worst
pain
(N=1612)
Raw change
Very much better
-3.5
-4.0
Raw change
Much or very
much better
-2.5
-3.0
Raw change
A little, much, or
very much better
-2.0
-2.0
Percentage
change
Very much better
-51%
-51%
Percentage
change
Much or very
much better
-34%
-34%
Percentage
change
A little, much, or
very much better
-23%
-21%
Part 1: Conclusion
• Patients use the 0-10 NRS scale
primarily as a percent scale and is
best analyzed as a percent change
from baseline pain
• A 30-35% improvement on the
0-10 NRS pain intensity scale is a
reasonable cut-off point for a clinically
important change
CCEB
Two Groups Randomized:
Both centered at 20% change at end of the study
16
14
12
10
Control
Combined Treatment
8
6
4
2
Percent change from baseline
CCEB
145%
138%
130%
123%
115%
108%
100%
93%
85%
78%
70%
63%
55%
48%
40%
33%
25%
18%
10%
3%
-5%
-13%
-20%
-28%
-35%
-43%
-50%
0
Actually Bimodal Distribution
16
14
12
10
Control
Non-Responders
Combined Treatment
8
6
4
2
Percent change from baseline
CCEB
145%
138%
130%
123%
115%
108%
100%
93%
85%
78%
70%
63%
55%
48%
40%
33%
25%
18%
10%
3%
-5%
-13%
-20%
-28%
-35%
-43%
-50%
0
Actually Bimodal Distribution
16
14
12
10
Control
Responders
Non-Responders
Combined Treatment
8
6
4
2
Percent change from baseline
CCEB
145%
138%
130%
123%
115%
108%
100%
93%
85%
78%
70%
63%
55%
48%
40%
33%
25%
18%
10%
3%
-5%
-13%
-20%
-28%
-35%
-43%
-50%
0
CCEB
1.15
1.08
1
0.93
8
0.85
10
0.78
0.7
0.63
0.55
0.48
0.4
0.33
0.25
0.18
0.1
0.03
-0
-0.1
-0.2
-0.3
-0.4
-0.4
-0.5
16
14
12
Control
Responders
Non-Responders
Combined Treatment
6
4
2
0
Study Efficiency
Mean vs Dichotomous Analysis
16
14
12
10
Control
Responders
Non-Responders
Combined Treatment
8
6
4
2
145%
138%
130%
123%
115%
108%
100%
93%
85%
78%
70%
63%
55%
48%
40%
33%
25%
18%
10%
3%
-5%
-13%
-20%
-28%
-35%
-43%
-50%
0
Efficiency = 1.145 (T-test N=30, Chi-sq N=26)
CCEB
Mean of Control Group and Low Probability Responders = 15%
Mean of the Treated Group = 34%; Cut-off = 33%
Group Mean Results – PID
Oral Transmucosal Fentanyl Citrate (OTFC)
Pain Intensity Differences
(with imputed values)
Better
^
|
|
|
|
|
|
|
^
p<.001 at all time points
4
3
2
OTFC
Placebo
1
0
0
15
30
45
60
Minutes
CCEB
Farrar JT, et al Oral transmucosal fentanyl citrate: randomized, doubleblinded, placebo-controlled trial for treatment of breakthrough pain in cancer
patients. Journal of the National Cancer Institute 1998; 90(8): 611-6
OTFC Study Outcomes:
Relative Risk Comparison
PID >33% %Max TotPAR
Relative Risk
Conf. Interval
p-value
Relative Risk
Conf. Interval
p-value
1.89
1.59 – 2.17
<0.0001
1.56
1.30 – 1.93
<0.0001
PR >1
Perf >1
1.73
1.42 – 1.95
<0.0001
1.56
1.34 – 1.95
<0.0001
At 60 minutes
CCEB
OTFC Looked at Density Plots
Density
Function
Cumulative
Distribution
Function
OTFC Placebo Proportion of Responders
(at different cut off points - for 30 minutes)
Cumulative Distribution of Responders Graph
100.0%
Proportion of Responders
90.0%
80.0%
70.0%
60.0%
OTFC #1
50.0%
Placebo
40.0%
30.0%
20.0%
10.0%
0.0%
0.0%
CCEB
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
Percent of Pain Intensity Difference
80.0%
Percentage Pain Intensity Difference
90.0%
100.0%
Mean Value Does Not Provide a
Unique Answer to the Clinical Question
• Mean value for the change in pain intensity over
time is 10%. This would be observed if:
• 1) every patient in the treatment group
improved by 10%, or
• 2) if 50% of the treatment group
got better by 20% and 50% had no improvement, or
• 3) if 50% of the treatment group
got better by 40% and 50% got worse by 20%.
CCEB
FDA Primary Data Analysis
• Data source
–Neuropathic pain RCTs (n=15)
–Indications
» Post-herpetic neuralgia (n=7)
» Diabetic peripheral neuropathy (n=8)
–Pharmaceuticals
» Pregabalin (n=11)
» Gabapentin (n=2)
» Duloxetine (n=2)
–Primary outcome measure
CCEB
» Change in 0-10 NRS pain score
Representative Data
Method
• Each RCT was analyzed using absolute and
percent change of the mean pain score.
• List of the analytic methods compared for each
trial for between group analyses (active treatment
vs. placebo)
– T-test
– Wilcoxon rank sum test
– Kolmogorov-Smirnov test
– ROC based - AUC comparison
– Ordinal logistic regression
– Log rank
CCEB
T=
RS
pbo
p150
0.2445 0.1865
rank
4
2
p600
0.0003 0.0006
rank
2
5
total rank
6
7
pbo
p150
0.1542 0.1101
rank
5
4
p600
0.0002 0.0003
rank
3
4
total rank
8
8
T = T-test with equal variance
RS = Wilcoxon rank sum test
ROC = AUC comparison
Test
ROC
KS
dif
0.1847 0.5362
1
6
0.0003 0.0003
4
3
5
9
pctchg
0.1071 0.2919
3
6
0.0002 0.0005
2
6
5
12
OL
LR
0.2230 0.2664
3
5
0.0011 0.0001
6
1
9
6
0.1055 0.0795
2
1
0.0004 0.0000
5
1
7
2
KS = Kolmogorov-Smirnov
OL = Ordinal logistic regression
LR = Log rank
Rank Totals
rank totals
T=
RS
117
109
Test
ROC
KS
dif
64
165
OL
LR
108
130
99
125
pctchg
rank totals
105
127
79
156
T = T-test with equal variance
RS = Wilcoxon rank sum test
ROC = AUC comparison
KS = Kolmogorov-Smirnov
OL = Ordinal logistic regression
LR = Log rank
CCEB
Conclusions
• Tools for measuring pain have high interperson variability and lower intra-person
variability
• Mean values do not provide a unique
answer to the clinical question of how many
people get better
• Responder analysis accurately reflect the
number of people in each treatment group
that reach a level of change in that study
CCEB
Conclusions (cont)
• To the degree that the test group is an
accurate representation of the general
population the response rates in the
treated group will reflect what the
clinician is likely to see, regardless of
the reason for the response.
• Both mean value and the responder
analysis provide useful information and
should be presented.
CCEB
THANK YOU
Research Group
Additional Collaborators
•
•
•
•
•
•
•
•
•
•
•
•
•
•
CCEB
Chris Rowan
Kevin Haynes
Andrea Troxel
Brian Strom
Rosemary Polomano
Robert Dworkin
Dennis Turk
Nathaniel Katz
Michael Rowbotham
Russel Portenoy
John Messina
Michael Poole
Mitchell Max
Jesse Berlin