Elicitation
Some introductory remarks by
Tony O’Hagan
Welcome!
• Welcome to the third CHEBS dissemination
workshop
• This forms part of our Focus Fortnight on
“Elicitation”
• Our format allows plenty of time for discussion
of the issues raised in each talk, so please feel
free to join in!
Two strands
• The talks today address elicitation in two quite
different contexts
› Elicitation of expert knowledge in the form of
probability distributions of unknown parameters
› Elicitation of people’s preferences for health states, in
order to formulate utilities
• A key objective of this FF is to bring these two
strands together, to see what each can learn
from the other
Eliciting probabilities
• Bayesian statistics concerns the updating of
beliefs from prior to posterior
• In principle, specification of prior beliefs is
therefore a key component of any analysis
• In practice it is rarely done seriously
› Given sufficient data, it doesn’t matter much
› It is difficult and time consuming to do properly
› It doesn’t lend itself to nice papers in leading
statistics journals
Not necessarily ‘prior’
• Eliciting expert opinion probabilistically does not
have to be thought of as ‘prior’ to some data
• Expert information is a crucial input to risk
analyses, etc, where there is no prospect of
getting more data
• This is the area where elicitation methodology
has been developed most
Elicitation in C-E analysis
• Elicitation of prior beliefs for Bayesian analysis
of clinical trial and other data
› Often, sample sizes are not very large and prior
information is appreciable
• Elicitation of expert knowledge of inputs to
economic models
› Needed for sensitivity or value of information analysis
• Fully Bayesian synthesis of information
› Use posterior distributions for model inputs where
possible
Challenges
• Regulatory context
› Elicitation will need to be robust, defensible
• Correlations
› How to elicit dependence?
› Or how to structure/model the parameters so that
the structure induces the right dependence?
Eliciting preferences
• What is the ‘effectiveness’ for which ‘we’ are
willing to pay?
• We can only compare costs across the whole
health care spectrum, and allocate budgets
rationally, if effectiveness is on a common scale
• Ultimately, we are willing to pay for health
improvements that yield
› Longer life and/or
› Better quality of life
QALYs
• The Quality-Adjusted Life Year seems to be the
gold standard
› Health providers’ willingness to pay can be reduced
to how much they would pay for one QALY
• Elicitation then involves asking people how they
value different states of health
› This is in order to know how to ‘adjust’ a life year
• Various challenges seem to arise, both
theoretical and practical
Questioning the QALY
• Is two years of one person’s life worth one year
of each of two people’s?
• If two years of poor health is worth one year of
perfect health, are four years of poor health
worth two of perfect health?
• Whose preferences/values?
› Individuals vary wildly
› Systematic differences between sectors of population
› What does society or the health provider think?
Practical challenges
• Choice of instrument: SG versus TTO
• Health state is massively multidimensional
› Can’t value every possible combination
› Relevant dimensions depend on medical condition
• How to quantify uncertainty in valuations
• Deferred benefits valued differently from
immediate benefits
› Does that make sense from a societal perspective?
Some commonalities
• Different instruments yield different answers
› SG is also used to elicit individual probabilities
› But probabilistic elicitation is more about distributions
• A multidisciplinary perspective is needed
› There is a lot of psychology involved
• Questions of whose judgements are needed
› Individual versus community
• Results are subject to uncertainty
› Always need to extrapolate from limited elicited data
• Fundamental duality