The New Zealand WHOQOL-BREF with national
items: Focus group work, item selection,
confirmatory factor analysis, and Rasch analysis
Chris Krägeloh, Paula Kersten, Patricia Hsu, & Joanna Feng
Positive Psychology Conference,
AUT, Akoranga Campus
8 June 2013
Positive Psychology Conference
8 June 2013
Activities of the NZ WHOQOL Group
• Conducts research in the area of QOL, especially
around the WHOQOL tools (WHOQOL-BREF and
other modules, such as WHOQOL-SRPB, WHOQOLDIS, WHOQOL-OLD).
• We recently validated the WHOQOL-BREF for use in
the New Zealand general population.
• We developed a NZ version of the WHOQOL-BREF,
with 5 additional national items.
Positive Psychology Conference
8 June 2013
Activities of the NZ WHOQOL Group
• Monitor the use of the WHOQOL tools in NZ to avoid
mis-use.
• Provide general support and advice on the use of
the tools; provide general guidelines on scoring and
interpretation.
• Maintain a current database of WHOQOL data
collected in NZ for the purpose of providing reference
values for future studies and use in clinical settings.
Positive Psychology Conference
8 June 2013
The NZ WHOQOL Group in PCRC
• The NZ WHOQOL Group has recently joined the
Outcomes Research Cluster of the Person-Centred
Research Centre at AUT.
• This cluster has demonstrated expertise on the use
of outcome measures in health research.
• Through alignment with this cluster, WHOQOL work
in NZ will have access to national and international
research networks and contribute to the development
and use of outcome measure in health research
contexts.
Positive Psychology Conference
8 June 2013
Purpose of the present talk
1. Background on the initial focus group work for
content validation of the WHOQOL-BREF for use
in New Zealand.
2. Development of additional optional national items
for the WHOQOL-BREF.
3. Validation of the national items: their alignment with
the four domains of the WHOQOL-BREF.
Positive Psychology Conference
8 June 2013
• WHOQOL = World Health Organization Quality of Life
Scale.
• Health-related QOL assessment.
• Developed in the 1990s cross-culturally in 14 countries.
Expanded to over 35 country versions since.
• 2005 estimate of 123 researchers in 67 centres and 39
countries involved in studies using WHOQOL
instruments.
Positive Psychology Conference
8 June 2013
Rationale behind original WHOQOL development
• Need for measurement of health beyond traditional
morbidity and mortality to include impact of disease and
impairment on daily activities and behaviour.
• Desire to find out what patients and clients felt about
themselves to supplement what experts thought.
• Introduces a humanistic element to health care to balance
mechanistic medical approaches.
• Unsatisfactory and culturally biased translations of similar
UK and North American measures.
Positive Psychology Conference
8 June 2013
Definition of QOL
“individuals’ perception of their position in
life in the context of the culture and value
systems in which they live and in relation
to their goals, expectations, standards and
concerns. It is a broad ranging concept
affected in a complex way by the person’s
physical health, psychological state,
personal beliefs, social relationships and
their relationship to salient features of their
environment” (WHOQOL Group, 1995)
Positive Psychology Conference
8 June 2013
Field centres involved in the development of WHOQOL-100
•
•
•
•
•
•
•
•
Melbourne, Australia
Zagreb, Croatia
Paris, France
Delhi, North India
Madras, South India
Beersheba, Israel
Tokyo, Japan
Tilburg, The Netherlands
•
•
•
•
•
•
•
Panama City, Panama
St. Petersburg, Russian Federation
Barcelona, Spain
Bangkok, Thailand
Bath, UK
Seattle, USA
Harare, Zimbabwe
Additional centres involved in the WHOQOL-BREF
• Hong Kong
• Leipzig, Germany
• Mannheim, Germany
• La Plata, Argentina
• Porto Alegre, Brazil
Positive Psychology Conference
8 June 2013
Properties of the WHOQOL
• Comprehensive multidimensional profile.
• Subjective perceptions: “How satisfied are you with your
ability to walk?”
• Objective approach “How well can you walk?”
• Subjective approach decided upon.
• Cross-culturally developed with many languages.
• Standardised 5-point Likert rating scale covering 4
dimensions - intensity (how much), frequency (how
often), evaluation (how satisfied), capacity (are you able).
Positive Psychology Conference
8 June 2013
Developmental process
1. Concept clarification (expert review)
2. Qualitative pilot (incl. focus groups)
3. Developmental pilot (300 questions)
4. Field test (series of studies on smaller scale)
Positive Psychology Conference
8 June 2013
Versions and features
• WHOQOL-100
= core generic instrument
• WHOQOL-BREF = abbreviated 26-item version
• WHOQOL-8 = eight-item version
• Additional modules = HIV, OLD, SRPB, DIS
• Some country versions have optional additional
national items.
The WHOQOL-BREF is
the most widely used
WHOQOL instrument.
It is also the most widely
used QOL tool in the
words.
Positive Psychology Conference
8 June 2013
Up until recently, research studies in New Zealand
using the World Health Organization Quality of Life
questionnaire WHOQOL-BREF have been using the
Australian or British versions.
In 2008, Prof. Rex Billington and colleagues founded
the New Zealand WHOQOL Group. One of the early
goals of this group was to validate the instrument for
use in New Zealand and thus develop a New Zealand
version.
Positive Psychology Conference
8 June 2013
Positive Psychology Conference
8 June 2013
Validation of the New Zealand WHOQOL-BREF:
Random sample of participants from the national
electoral role to obtain ratings for the existing 26
WHOQOL-BREF items.
Sent out 3,000 questionnaires with self-addressed
return envelopes. 710 questionnaires were returned
(response rate approximately 24%).
Young people were underrepresented and
supplemented by additional purposive sampling,
increasing the total number to 808.
Validation of the New Zealand WHOQOL-BREF:
 Conducted confirmatory factor analysis (CFA) with
the following specifications:
 Promax rotation
 Two types of extraction methods:
Diagonally-weighted least squares (DWLS)
with polychoric correlations
Positive Psychology Conference
8 June 2013
Validation of the New Zealand WHOQOL-BREF
The results suggest a good fit:
RMSEA=0.072 (criterion for excellent fit <0.060)
CFI=0.966 (criterion for very good fit >0.950)
SRMR=0.067 (criterion <0.080).
Note that, unlike many other studies, a CFA method was
used that is more appropriate for Likert-scale data, and
this type of method often yields worse fit indices.
Nevertheless, the fit indices were comparable (if not
higher) than those reported elsewhere.
Positive Psychology Conference
8 June 2013
Validation of the New Zealand WHOQOL-BREF:
 Some ceiling effects for Items 23 (condition of
living place) and 25 (transport)
 The Rasch model fit the data well and confirmed
the CFA results.
 There were some issues with DIF (differential item
functioning) by demographic variables, and it is
recommended to analyse the WHOQOL-BREF
domain scores using nonparametric statistics.
Positive Psychology Conference
8 June 2013
1. Focus groups and new items
Conducted 13 focus groups with general community
members, people with disabilities, and health experts.
These focus groups explored themes about what
participants described as elements of QOL.
WHOQOL team also wrote 24 potential new items
based on themes that had emerged from those focus
groups.
Positive Psychology Conference
8 June 2013
2. Importance ratings
24 new items were sent out with the WHOQOL-BREF
to a random sample of 3,000 participants from the
national electoral role.
Participants rated each item in terms of importance.
Candidates for new items with a mean importance
rating of <4 were discarded.
Positive Psychology Conference
8 June 2013
Items were excluded if they exhibited the following:
 ceiling or floor effect
 low correlation with total QOL and likely parent domain
 decreased reliability of the domains
 does not discriminate between sick and well
 duplicates an existing core item
 results in unacceptable CFA fit indices if included
Positive Psychology Conference
8 June 2013
Items excluded were, for example:
• How satisfied are you that you eat healthily?
• To what extent do you feel you have individual
freedom?
Positive Psychology Conference
8 June 2013
Final selected new national items
Psychological domain:
• To what extent do you feel you have control over your life?
• To what extent are you able to manage personal
difficulties?
• To what extent do you feel respected by others?
• How satisfied are you that you are able to meet the
expectations placed on you?
• Social domain:
• To what extent do you have feelings of belonging?
NB: you can include
the NZ national items
in domain score
calculations.
However, as you
present them, please
make it clear that your
scores contain
national items.
Positive Psychology Conference
8 June 2013
The role of Rasch analysis
• The outcome measures research cluster has
expertise in modern psychometric theories
and methods.
• One of these is Rasch analysis, which we
have used in our validation work of the NZ
WHOQOL-BREF.
• It also has many other advantages, as
illustrated by the following example:
Ordinal vs interval measures
Toilet Use
Dependent
□0
Needs some help but can do something alone (wipe, undress)
□1
Independent (on & off, dressing, wiping)
□2
Feeding
Unable
□0
Needs some help cutting, spreading butter etc.
□1
Independent (Able to eat normal food served without aid)
□2
Ruler used with permission from Prof Alan Tennant, Leeds University
Ordinal vs interval measures
Toilet Use
Dependent
□0
Needs some help but can do something alone (wipe, undress)
□1
Independent (on & off, dressing, wiping)
□2
Feeding
Unable
□0
Needs some help cutting, spreading butter etc.
□1
Independent (Able to eat normal food served without aid)
□2
Ruler used with permission from Prof Alan Tennant, Leeds University
Ordinal vs interval measures
Toilet Use
Dependent
□0
Needs some help but can do something alone (wipe, undress)
□1
Independent (on & off, dressing, wiping)
□2
Feeding
Unable
□0
Needs some help cutting, spreading butter etc.
□1
Independent (Able to eat normal food served without aid)
□2
Ruler used with permission from Prof Alan Tennant, Leeds University
Positive Psychology Conference
8 June 2013
Ordinal-level data are problematic
• The assumptions of ordinal-level data are frequently
broken by researchers and health professionals, as
ordinal data should not be used in arithmetic
operations, such as:
–
–
–
–
calculation of means
counting up totals
change scores
minimal clinically important difference
• Under some circumstances, you can come to the
wrong conclusion that the patient has plateaued when
in fact s/he has not.
• Rasch analysis can reveal under which circumstances
breaking these assumptions can be problematic, and
can provide ways to convert these ordinal-level
measures to an interval scale.
Thank you!