The Role of Statistical Science in
Guiding Health Policy
Dalene Stangl1
Don Berry2
Giovanni Parmigiani1
1Institute
of Statistics and Decision Sciences
2MD Anderson Cancer Center
The Role of the Statistician in
Policy Analysis and Research
 Sir Claus Moser’s, 1975 ASA meeting, the
“foremost responsibility (of the statistician) is to contribute to more
enlightened and efficient ‘decision making’ … through the fullest possible
exploitation of our skills in analyzing and interpreting the data.”
 Dorothy Price, Director, National Center for Health Statistics, (1976),
The American Statistician, “The Role of Statistics in the Development of Health Care Policy”
“…. As in other areas of social policy, health statisticians and health data are
increasingly expected to provide keys to rational decision making. To
accomplish this goal, the statistician and decision maker need to interact to
an increasing degree.”
 John Tukey,
1976, Am. J. of Epidemiology
“…those statisticians for whom opportunity and a natural bent combine to
offer experience and the development of expertise ought, in the public
interest, become as much policy makers as their roles allow.”
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Quote Commonalities
 All refer to importance of decision-making
 Recommend more involvement of statisticians
 All statements were made 20-25 years ago
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Focus of past 25 years
 1970s and 1980s
Develop coordinated, systematic data base
 “At NCHS we are searching for innovations to enhance data production with minimal
increased demand on resources and to provide data in a timely fashion…. We are being
asked to produce more data, which is more relevant, with resources that are not growing
commensurately. We are being asked to aid in the interpretation and analysis of the data
as well.”
 1990’s
Statistics and Policy (B.D. Spencer ed., 1997)
 no mention of decision theory
“The statistical basis of public policy: a paradigm shift is
overdue” (Lilford and Braunholtz, 1996)
 Bayesian methods superior to conventional methods.
 Primary advantages
 Utility functions were
given
sentence
+ reference
The Role
ofone
Statistical
Science
in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
‘Typical’ Bayesian Solution
 Hospital Profiling
Outcome ex. - mortality rate at time t (adjust case-mix)
 Classical - Z-scores
 Bayesian - Posterior probabilities of ‘excess’ mortality
Implicit rather than explicit decision analysis
How is decision-making embedded in the analysis?
 choice of outcome
 time point
 relative performance measure versus national guideline of performance
 quality thresholds
 posterior tail areas
Sufficient?
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Why Insufficient?
 Needs explicit decision-theoretic framework
 Two proposals
 relinquish automatic constant utilities embedded in
p-values and posterior probabilities
 present statistical output in ways that increase the
possibility and probability of applying a wide diversity
of utility functions
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Prescriptive Perspective
 “Making Health Policy Decisions: Is Human Instinct
Rational? Is Rational Choice Human.” Paltiel and
Stinnett, Chance, 1996
Approach formal analysis from a prescriptive perspective, I.e.
aim to provide decision-makers with information that can help
them to make better choices but stop short of telling them
what to do.
“By being forced to consider this issue explicitly, people may,
whatever their final decision, benefit from scrutinizing and
coming to grips with values to which they had previously given
little thought.”
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Proposal 1: relinquish automatic constant
utilities embedded in p-values and posterior
probabilities
    i
 threshold, D
 two decisions
 d0 accept - sufficient quality
 d1 reject
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
 d1 is better than d0 if >D
 d0 is better than d1 if >D
 U(d,) measures the worth/utility of d when the
uncertain value is 
Figure 1.
i
sufficient
Dnot sufficient
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Constant and Linear Utility
Figure 2. Constant Utility Function
U(d0,)
U(d1,)
D
sufficient
Figure 3. Linear Utility Function
U(d0,)
i
U(d1,)
 i
D
not sufficient
sufficient
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
not sufficient
Compromise Utility
Figure 4. Compromise Utility Function
U(d0,)
U(d1,)

i
D
sufficient
not sufficient
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Loss of declaring sufficiency
Figure 7. Loss of Declaring Sufficiency
L()=U(d1,)-U(d0,)

i
D
sufficient
not sufficient
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Uncertainty in 
 Calculate expected loss
L()p()
 Declare sufficient quality iff negative
 Balances cost/benefits in a simple,
comprehensive way
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Specifying Loss
 Not the statistician’s loss function
 Statistician can help decision-maker articulate
value judgements in a way that allows coherent
procedure
Figure 7. Loss of Declaring Sufficiency
L()=U(d1,)-U(d0,)

i
D
The Role of Statistical Science in
sufficient
not
Guiding
Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
sufficient
Extensions
 other loss functions
 multivariate outcomes - Tan and Smith (1998)
 prior elicitation
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Proposal 2: Predictive Distributions for
General Outcomes
 Hospital Profiling
excessive mortality at time t as measured by 1.5 x
median mortality across all hospitals
predictive survival curves across time
 Advantages
 allow diversity of utility functions
 metric upon which values are easily understood
 incorporate QUALYs
 Disadvantages
 requires event times
 harder to model The
event
times than dichotomous
Role of Statistical Science in
outcomes
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
Other Thoughts
Meta-Analysis in Medicine and Health Policy (Stoto)
Attitudes of Policy World
Graduate Education
Attitudes of Statisticians
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
References












Berger, J.O. and Delampady, M. (1987). Testing precise hypotheses (with discussion). Statistical Science
2: 317-352.
Berger, J.O. and Sellke, T. (1987). Testing a point-null hypothesis: the irreconcilability of p-values and
evidence ( with discussion). J. Amer. Statistica Assoc. 82:112-139.
Berger. R.L. and Hsu, J.C. (1996). Bioequivalence trials, intersection-union tests and equivalence
confidence sets (with discussion). Statist. Sci. 11:283-319.
Bernardo, J.M. and Smith, A.F.M. (1994). Bayesian Theory. Wiley. Chichester.
Casella, G. and Berger, R.L. (1987). Reconciling Bayesian and frequentist evidence in the one-sided testing
problem. J. Amer. Statist. Assoc. 82: 106-111.
Cochran, W.G. (1976). The role of statistics in national health policy decisions. American Journal of
Epidemiology 104(4):370-379.
Lilford, R.J. and Thornton J.D. (1992). Decision logic in medical practice. Journal of the Royal Collegeof
Physicians of London, 26(4):400-412.
Lilford, R.J. and Braudholtz, D. (1996). The statistical basis of public policy: a paradigm shift is overdue.
British Medical Journal 313(7057):603-607.
Lindley, D.V. (1985) Making Decisions. Wiley, Chichester.
Lindley, D.V. (1997) The choice of sample size (with discussion). The Statistician 46: 129-166.
Lindley, D.V. (1998) Decision Analysis and Bioequivalence Trials. Statistical Science 13(2): 136-141.
Lindley, D.V. and Singpurwalla, N.D. (1991). On the evidence needed to reach agreed action between
adversaries, with application to acceptance
The Rolesampling.
of Statistical
J. Amer.
Science
Statist.
in Assoc. 86: 933-937.
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.
References continued...





Normand, S., Glickman, M., Gatsonis, C. (1997). Statistical methods for profiling providers of medical care:
issues and applications. J. Amer. Statist. Assoc. 92(439):803-814.
Paltiel, A.D. and Stinnett A.A. (1996). Making health policy decisions: Is human instinct rational? Is
rational choice human? Chance 9(2):34-39.
Rice, D. (1977). The role of statistics in the development of health care policy. The American Statistician
31(3):101-106.
Tan S.B. and Smith, A.F.M. (1998). Exploratory thoughts on clinical trials with utilities. Statistics in
Medicine 17:2771-2791.
Tukey, J.W. (1976). Discussion of: “Role of statistics in national health policy decisions.” American Journal
of Epidemiology 104(4):380-385.
The Role of Statistical Science in
Guiding Health Policy, JSM,
Monday August 9, 1999,
Baltimore, Md.