Reading: Chapter 6
Homework 3 – I extend the due date to Wednesday, Nov. 25 th by noon.
Extra office hours: Monday, 11-12.
I. Detecting Hidden Bias Using Known Effects
In an observational study that attempts to control for all confounders, the key assumption is no unmeasured counfounders (no hidden bias)
( ,
T i
C i
)
independent of Z given X .
A sensitivity analysis says how sensitive the study’s conclusions are to hidden bias, but does not address how much hidden bias there is.
We will examine two techniques for collecting data that can help to reveal hidden bias: (1) collecting data on “known” effects; (2) multiple control groups.
Efforts to detect hidden bias and sensitivity analysis complement one another in their strengths and limitations. It is difficult if not impossible to detect small hidden biases, but a sensitivity analysis might indicate that small biases would not materially alter the study’s conclusions. All studies are sensitive to

sufficiently dramatic biases, but dramatic biases are the ones most likely to be detected.
Basic idea of using known effects to detect hidden bias: Suppose that the treatment is known to affect an outcome. Then if we see differences between the treatment and control groups on this outcome, this must mean that there are differences between the treatment and control group on unmeasured covariates and thus there is hidden bias.
Examples:
(1) Acute Stress as a Cause of Sudden Death from Coronary
Disease and Cancer : Is the risk of rapid death from coronary disease increased by acute stress? Trichopoulos et al. (1983,
Lancet ) compared coronary mortality in Athens in the period immediately following the 1981 earthquake to time periods immediately before the earthquake and to corresponding time periods in 1980 and 1982. After the earthquake, they claim that
“psychological stress was unquestionable, intense and general.”
They found higher rates of coronary mortality immediately following the earthquake than in comparison time periods.
It is plausible that acute stress acutely increases the risk of death from coronary disease, but not particularly plausible that acute stress quickly causes many deaths from cancer. Trichopoulos et al. conducted a parallel analysis of cancer mortality following the earthquake, finding no increase in risk. So the association of mortality with stress appears where an effect caused by acute stress is plausible, but not where a causal effect is less plausible.

(2) Anger and Curiosity as Causes of Myocardial Infarction : Do brief bouts of anger increase the risk of myocardial infarction
(MI). Mittleman et al. (1995, Circulation ) compared anger in the two hours before an MI to the same two hours on the previous day; this is a case-crossover study. They measured anger using the State-Trait Personality Inventory in each of the two time periods. Using Wilcoxon’s signed rank test, they found anger was more often reported in the two hours before an
MI than in the corresponding two hours on the previous day, the difference being highly significant, P=0.001.
However, Mittleman et al. were concerned that their measures of anger were baesd on recall, and a patient’s recall may itself be affected by the occurrence of an MI. Perhaps the events leading up to an MI appear to a patient to be more important than the events of the less dramatic, perhaps uneventful, previous day.
As a partial check of this possibility, Mittleman et al. also obtained measures of curiosity from the State-Trait Personality
Inventory in both time periods. They strongly doubted curiosity causes MI. They found no sign of differences in curiosity using the signed rank test, P=0.20. So the level of anger is associated with the timing of the MI, but the level of curiosity is not. They wrote, “The specificity observed for anger…as opposed to curiosity on the STPI subscales…argue against recall bias.
(3) Abortion and Crime : Theorizing that unwanted children, perhaps childrens of unwed teenage mothers, might possibly be more likely to commit crimes as teens or adults, Donohue and
Levitt (2001, Quarterly Journal of Economics ) investigated the sharp decline in crime rates in the 1990s and their relationship to the legalization of abortion by the Supreme Court decision in

Roe v. Wade . Among many analyses, their study includes a est for hidden bias using a known effect. Instead of a different outcome, they consider the effect on a subgroup that cannot be affected by the law change.
If legalized abortion reduced crime simply by eliminating future criminals, then the effect of legalization should be confined to certain age cohorts. The availability of legal abortion in 1973 should not affect the cohort born in 1971, but might affect the cohort born in, say 1975. In one of many analyses, Donohue and Levitt relate abortion rates by state to two outcomes:, namely (i) arrest rates by state for age cohorts that might be affected and (ii) arrest rates by state for cohorts that cannot be affected. Whether or not, adjustments are made for covariates, they find that arrest rates for cohorts that cannot be affected are unrelated to abortion rates, whereas arrest rates for cohorts that can be affected are negatively associated with abortion rates.
They write, “In high abortion states, only arrests of those born after abortion legalization fall relative to low abortion states.”
(4) Unrelated health conditions in the study of methylmercury in fish : Skerfving et al. (1974, Environmental Research ) studied whether eating fish contaminated with methylmercury causes chromosome damage. The outcomes of interest was the percentage of cells exhibiting chromosome damage. Pairs were matched for age and sex. control.cu.cells=c(2.7,.5,0,0,5,0,0,1.3,0,1.8,0,0,1,1.8,0,3.1) exposed.cu.cells=c(.7,1.7,0,4.6,0,9.5,5,2,2,2,1,3,2,3.5,0,4); library(exactRankTests) wilcox.exact(exposed.cu.cells,control.cu.cells,paired=TRUE)

Exact Wilcoxon signed rank test data: exposed.cu.cells and control.cu.cells
V = 84, p-value = 0.04712 alternative hypothesis: true mu is not equal to 0
In the absence of hidden bias, there’s evidence that eating large quantities of fish containing methylmercury causes chromosome damage.
Testing hidden bias using a known effect: Skerfving et al. described other health conditions of these subjects including other diseases such as hypertension and asthma, drugs taken regularly, diagnostic X-rays over the previous three years, and viral diseases such as influenza. control.other.health.conditions=c(rep(0,8),2,rep(0,3),2,1,4,1); exposed.other.health.conditions=c(0,0,2,0,2,0,0,1,1,2,0,9,0,0,1,0);
These are outcomes since they describe the period when the exposed subjects were consuming contaminated fish. However, it is difficult to imagine that eating fish contaminated with methylmercury causes influenza or asthma, or prompts X-rays of the hip or lumbar spine.
> wilcox.exact(control.other.health.conditions,exposed.other.health.conditions)
Exact Wilcoxon rank sum test data: control.other.health.conditions and exposed.other.health.conditions
W = 112.5, p-value = 0.5257 alternative hypothesis: true mu is not equal to 0

There is no evidence of hidden bias.
Questions: (1) When does such a test have a reasonable prospect of detecting hidden bias? (2) If no evidence of hidden bias is found, does this imply reduced sensitivity to bias in the comparisoins involving the outcomes of primary interest; (3) If evidence of bias is found, what can be said about its magnitude and its impact on the primary comparisons?
Power of the test of hidden bias: Let y denote the outcome for which there is a known effect of zero. For a particular unobserved covariate u , what unaffected outcome y would be useful in detecting hidden bias from u ?
Basic result: The power of the test of whether y is affected by the treatment increases with the strength of the relationship between y and u . If one is concerned about a particular unobserved covariate u , one should search for an unaffected outcome y that is strongly related to u .
Consider again the study of legalized abortion and crime by
Donohue and Levitt (2001). In their study, the unaffected outcome y is the arrest rate by state for cohorts that are too old to have been affected by the Roe v. Wade decision in 1973. Of course, arrest rates vary by state for many reasons, some of which were not observed. If u is an unobserved variable describing the states, then the power of the test for hidden bias using y is greatest when u and y are strongly associated. If arrest rates for older cohorts y are related to an unobserved u in much the same way that arrest rates for younger cohorts are

related to u , then a test for hidden bias will have high power to detect bias from a variable u that is related to the outcome of interest.