FUNDAMENTALS OF MEDICAL RESEARCH
Ed J. Gracely, Ph.D.
Family, Community, and Preventive Medicine
Multivariate statistics
July 16, 2014
Handout 4 of 4
Goals: After this session, you should be able to:
1.
2.
3.
4.
Explain the concept of “holding a variable constant” to identify independent associations.
Explain how multivariate techniques help identify independent associations.
Interpret the results of a multivariate analysis.
Describe the chief limitations with multivariate techniques.
A) The concept of holding variables constant
Ex: An agricultural researcher who is trying to see the effect of amount of fertilizer on plant growth
will make sure that different fertilizer quantities are compared between groups of plants that have
the same amount of water, exposure to sunshine, soil conditions, and temperature. In this way,
because the other potential influences on growth are held constant, the effect of the fertilizer can
directly be observed.
Ex: A medical researcher testing the benefits of treatment A vs. treatment B will often randomly
assign subjects to groups. The assignment at random will not make every subject the same on
other relevant variables but should make the two groups similar. For example, they should, on
the average, end up similar in mean age, mean severity, percent male and female, etc... Thus
there should be no important differences between the groups that could influence the results
other than the treatment given to each group.
This is a basic principle of science: To see the effect of one variable, hold constant or make groups
comparable on (as much as possible) the other variables that could affect the outcome.
In the above examples the researcher is manipulating conditions directly. This kind of research is the
most powerful, when it can be done. The techniques in this session are mainly applicable to designs
in which direct manipulation and assignment of independent variables is not possible.
Ex: A researcher is studying children 4 - 8 years old. The researcher finds a highly significant
association between height and skill on a task, p < 0.001.
In truth, height has nothing to do with ability on this task.
Q: Why might there be an association between height and skill in these children? a
Q: If the researcher looks at children who are all exactly the same age (4 years, 6 months, for
example) would you expect to see a relationship between height and skill? Vote: Y /N. b
1
Q: If the researcher looks only at the children who are exactly 3 feet tall, would you expect to
see an association between AGE and ability? Vote: Y/N c
Key point: In observational research, where manipulation and assignment to conditions are
not possible, one important way to "hold constant" variables is by selection of subjects.
If I want to study the effect of height on a skill task, I would try to get subjects who
were the same age, sex, and perhaps weight and even intelligence level, while varying
in height. Then I could see the effect of height un-confounded by those other variables.
A variable that predicts the outcome when considered alone, but no longer predicts
when other variables are held constant, probably has little or no independent effect on
the outcome.
Ex: A researcher finds that compliance with medical recommendations is higher among people
who have a higher income and concludes that having money is conducive to following
recommendations.
Suppose that in truth, income has no direct effect on compliance. Income is merely
indicating level of education which in turn indicates understanding of the recommendations.
Q: If the supposition is true, and if the researcher carefully tested everyone's understanding
of the regimen and the need for it, then found a group of subjects who all fully
understood, would there still be differences in compliance due to income differences
within that well-informed subgroup? Vote: Y / N. d
Q: Suppose a group of subjects was selected who were all at the same income level. Would
level of understanding still predict compliance even in this group? Vote: Y / N. e
Why is this important in a multivariate class? Because multivariate statistics control variables
too, in ways that have logical similarities to what we did above. These techniques are said
to statistically "control" variables or "hold them constant".
The difference is that multivariate statistics do this mathematically, without actually
requiring the researcher to restrict levels of the variables being measured.
B) Some important terms
1) Multivariate: a wide variety of different techniques, difficult to categorize. Virtually all
techniques that incorporate several variables simultaneously in the analysis are considered
multivariate, except for analysis of variance.
These techniques are used to statistically control variables, as mentioned above. Many of them
also produce sophisticated prediction equations that can be used to combine several variables in
predicting an outcome.
2) Univariate: Other designs. t-tests, all ordinary ANOVAs, almost all non-parametric tests, simple
linear correlation/regression etc...
2
Three common techniques used for controlling variables
1) Multiple linear regression: Predicting a numeric variable from two or more predictor
variables. This provides a regression (prediction) equation with a multiplier ("regression
weight") for each of the predictors. Severity = Duration x 2.5 + Age x 1.3 + Initial
severity x 0.8 + 2, for example.
2) Logistic regression: a method employed when the dependent variable has exactly 2
levels, such as yes/no, success/failure, live/die. The primary measure of association is an
odds ratio.
3) Proportional hazards (Cox Model): a method used when the dependent variable is time
till some event occurs (often, but not always, death) especially when not all subjects in the
study reach the endpoint. The event can be good (for example release from the OR) or bad
(death, relapse). The time variable can be a true time (days, years) or some other measure
of process elapsed (number of tries). They key is that some subjects get a true time of
event whereas others ("censored observations") had not yet experienced the event when
last seen. Survival curves, Kaplan-Meier curves for example, are a univariate aspect of the
same kind of data. The primary measure of association is a hazard ratio, interpreted
mostly like a relative risk.
These three regression-type techniques are all interpreted in the same way. The effect of each
variable is reported with all other variables controlled (statistically "held constant"). They also
all give equations that can be used to *predict* the outcome variable from the predictors.
Thus, a variable that is significant (p < 0.05) as a predictor in a multivariate analysis may
be considered to have an association with the dependent variable that is not due to its
associations with the other predictors.
Q: Does this prediction independent of the other (included) variables prove that the significant
predictor is a cause of the dependent variable? f
Key point: When several predictors of an outcome are used together in a multivariate analysis, the
p values (and other statistics) for each predictor are calculated as if all of the other predictors
were constant. Thus the effect of each variable is reported in a way that is independent of its
associations with the other variables.
A variable that is statistically significant as a predictor in a multivariate analysis may be
considered as having an association with the outcome, that is NOT dependent on any of the
other variables. It is a candidate for being a cause of the outcome (although this cannot be
proven by statistical analyses of this sort).
A variable that is significant univariate, but NOT significant multivariate, is probably only
predictive because of its associations with other predictors.
Ex: Using the height and skill example, a researcher reports: "When placed into a multivariate
analysis, specifically a
(because the dependent variable was numeric), age was
predictive (p < 0.001), while height was not (p = 0.75). This indicates that (choose all that apply): g
3
a.
b.
c.
d.
e.
f.
When height was held constant, age still predicted significantly.
When height was held constant, age no longer predicted significantly.
When age was held constant, height still predicted significantly.
When age was held constant, height no longer predicted significantly.
Age appears to be predicting independently of height.
Height appears to be predicting independently of age.
Ex Orchard TJ. et al., Diabetes Research & Clinical Practice. 34 Suppl: S 165-71, 1996 Oct. studied
predictors of and outcomes of diabetic autonomic neuropathy (DAN).
For predicting 2-year mortality they say: "Mortality was increased four-fold in those
with DAN (P = 0.005), although this difference no longer was significant after
adjustment for baseline nephropathy or hypertension." h
Q: The DAN and mortality relationship is best described as:
a. DAN is predicting completely independently of the other variables studied.
b. DAN predicts largely because it is a marker for other serious conditions.
c. The authors should rely on the univariate results, which are more significant.
Q: Your patient, who has DAN but not nephropathy or hypertension, has read that DAN
patients have a substantially elevated mortality, even over the next few years. He asks
whether he should be concerned about his short-term mortality risk. You reply, based
on the above: i
a.
b.
Yes, sadly, you should be. Your DAN puts you at significantly elevated risk of
early mortality.
No, in the short term, increased mortality in DAN patients is mainly related to their
also having other dangerous conditions, which you don't.
For predicting DAN, they say:
"Duration of diabetes, the cardiovascular risk profile (hypertension, elevated LDL
cholesterol and triglycerides), and other complications (e.g. nephropathy) were all
univariately associated with subsequent DAN (P < 0.01). Smoking status and
hemoglobin A1 (HbA1) were less strongly, related (P < 0.05). [Multivariate
analysis...] showed diabetes duration and HbA1 to be significant independent
predictors."
Q: Assuming the time to development of DAN was an outcome, what procedure? j
Q: Based on these results, what conclusion might you draw about the relationship between
nephropathy and the development of DAN? k
4
C) Limitations of these methods (all of the above)
1.
All multivariate techniques make assumptions about the form of the relationships
involved. Multiple linear regression assumes linear relationships among variables, for
example. The assumptions must be met if the analysis is to work properly.
Ex: If age and skill were associated in a curvilinear fashion, perhaps because the latter starts
increasing slowly, then accelerates (for a few years), then the regression weight for height
would not correctly reflect the effect of holding age constant. This could lead to incorrect
conclusions, perhaps that height has a small, but real, causal relationship to skill.
2.
3.
4.
5.
These techniques only adjust for the variables included in the study, of course.
There can always be another variable that is the real determinant of an association,
unknown to you.
These techniques cannot help with direction of causality. In some cases this is
not an issue, while in others it is.
These techniques cannot establish even the fact of causality between two
variables.
Approximations (like "3 years old") and crude categories (like "former smoker")
reduce the capacity of multivariate statistics to fully correct for the effect of
possible confounders. There is a big difference between a former 2-pack a day
smoker who quit last year and someone who never smoked more than a few
cigarettes a day and quit 18 years ago! Simple errors in measurement ("Oops, I
meant to say I was 18, not 28, when I started smoking...") also reduce the
correction.
Ex: Suppose we knew the age of children in my earlier example only as "3 years old". This
could include kids from 3.0 - 3.99 years, plus a few whose parents rounded up from 2.5. In
this sample, would you expect to an association between height and skill on my hypothetical
task? l
D) Some graphical examples of a lot of stuff we've done
Ex: In this open-label study involving 247 patients with metastatic melanoma and BRAF V600 mutations, we
evaluated the pharmacokinetic activity and safety of oral dabrafenib (75 or 150 mg twice daily) and trametinib
(1, 1.5, or 2 mg daily) in 85 patients and then randomly assigned 162 patients to receive combination therapy
with dabrafenib (150 mg) plus trametinib (1 or 2 mg) or dabrafenib monotherapy. NEJM Nov 1, 2012
5
An example of commonly-seen Kaplan-Meier curves
The graph shows 3 curves, each for a different type of treatment. The footnote (not shown) provides no real
information about the graph, just about the clinical aspects. The arrows are mine, in case you have a black and
white copy! The vertical bars are “censored” cases, those still progression free at the time shown, which was their
last visit.
Q: The authors report hazard ratios: what statistical analysis would have been used to get those?
Q: Which curves are different from control? m
Q: How can you tell from the hazard ratio if each is significantly different from control? n
Q: What is the median? How would you read the median from the figure itself? o
Q: At 15 months, 40% of the combination group were progression-free (estimated) but it shows only 1 still at risk.
Why? And what happened at 17 months in that group? p
6
Here we see the main analysis (higher dose combined versus control) repeated for various subgroups. The main
point is to see if subgroups change the effect. Sometimes a p-value for interaction is given (testing whether the
effect is greater in one subgroup than another). Here they don’t. Clearly the effect is in the same direction for all
subgroups, and either significant or close to significant in all of them as well.
Q: For the age breakdown, the HR’s are the same. Why is the CI so much wider in the older group?
q
QUIZ:
1. Which of the following are common purposes of multivariate analyses? (Pick all that apply):
a.
b.
c.
d.
e.
f.
2.
3.
Avoid the assumptions of simpler tests (like a linear association or normal distribution).
Control confounding.
Provide prediction equations with multiple predictors.
Provide smaller p-values than are seen in univariate tests.
Identify associations that are independent of other variables.
Increase and strengthen confounding effects.
A researcher finds that variables A and B significantly predict the outcome, with a simple correlation. Variable
C is not significant as a predictor. In a multivariate analysis only A still predicts. Which is the most likely to be
causally associated with the outcome?
A B C
Hair loss found to predict mortality in a certain population. Suppose that hair loss is really just a marker for
high levels of a certain hormone, which is the real cause. If we looked at a group of men, all with the same
level of that hormone, what would you expect to see?
1. Hair loss is an even better predictor of mortality in that group.
2. Hair loss still predicts, but no better than before.
3. Hair loss is not a mortality predictor in that group.
7
4. T/F: If a variable predicts significantly in a multivariate equation you may safely infer that it causes the
outcome.
5. The prediction of times till death, especially when some subjects are still alive at the end of the study, is best
accomplished by:
1. Multiple regression
2. Proportional hazards
3. Logistic regression
4. Chi-square contingency table analysis
Quiz Answers
Quiz: 1: b, c, e 2: A 3: 3 4: F 5: 2
FMR.4_of_4 Multivariate.doc
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
p
q
Height and skill are both related to age
No. If all kids were exactly the same age, and age was the underlying reason for the height with skill association, the latter
association should be eliminated.
Yes. Holding height constant would not eliminate the effect of age.
No. If all fully understood, income should no longer matter, under the conditions of the example.
Yes. Even when all were the same income, variations in understanding would still matter.
No. Causality is hard to establish!
Multiple linear regression // (a), (d), and (e)
(b)
(b)
Proportional hazards
Nephropathy is a marker for poor control and duration. It predicts DAN because of these, not because nephropathy causes
DAN.
Yes, could be some association because of the wide age range.
Both of them have small p-values and are significant versus control.
If the hazard ratio confidence interval doesn’t include 1 it is significant. Both are.
The dashed line is the 50% line. Where each curve crosses that line is the median, that is, 50%, progression-free survival.
The survival curve does not change when someone is lost to followup or when the study ends for that subject. Only the
number at risk is reduced accordingly. So at the very end, while many were still progression-free (at last visit) only 1 was
still being followed, and that 1 had a relapse. In the other two groups the last patient see was still progression-free.
The width of the CI is largely a function of the sample size in each group.
8