Development of the
Valuing Questionnaire
Dr Matthew Davies &
Dr Matthew Smout
School of Psychology
University of Adelaide
ACT’s emphasis on values
“…creative hopelessness, defusion, connecting with self,
acceptance, contact with the present moment, and valuing
must lead to actual concrete differences in the client’s
behaviour that service those valued ends”
(Strosahl et al., 2004)
“While ACT allows for symptom alleviation, it is not a main
target or the therapeutic goal. Rather the focus is on what
we call broadband outcomes. Such outcomes are about
helping the client move in life directions that they truly care
about”
(Eifert & Forsyth, 2005)
“All ACT techniques are eventually subordinated to helping the
client live in accord with his or her chosen values.”
(Hayes et al.,1999, p.205)
Values assessment in ACT
research to date
 Values measures not used as primary
outcome measures in most ACT studies
 Not all studies include values measures
 Those that do usually operationalise as
quality of life/satisfaction with life: state
not behaviour
Existing values measures
 Outside ACT
 Rokeach Values Questionnaire
 Schwartz Values Survey
 Values In Action (VIA)
 Inside ACT





VLQ
Bull’s Eye
Chronic Pain Values Inventory (CPVI)
Survey of Life Principles
Personal Values Questionnaire (PVQ)
Difficulties with individual domains
methods
 Having domains considered individually do not represent
how values work – i.e., out of context of choice, promote
intellectualised / idealised choosing; ranking would be
more realistic but creates psychometric limitations
 Therefore, empirically all domains get high importance
ratings: little differentiation (McCracken & Yang, 2006)
 Also high social desirability
 Difficult to weight/combine across domains meaningfully.
 Time is limited; valuing as a verbal-linguistic activity is relatively
unlimited. e.g.,Low consistency in one domain due to high
consistency in another domain may still reflect living relatively
ideal valued living
 One behaviour may serve more than one value; one valued
domain may be served by many behaviours
 Assessing by domain increases number of items; makes
it difficult to capture other ACT-specific qualities of values
in a brief measure
Methodology: Item generation
 Searched key ACT texts for “value” references to comprehensively
cover the concept from an ACT perspective









Hayes et al (1999) Acceptance and Commitment Therapy: An experiential approach to
behavior change. New York: Guilford
Hayes & Strosahl (2004). A practical guide to acceptance and commitment therapy.
New York: Springer.
Wilson & Dufrene (2009). Mindfulness for two: An acceptance and commitment therapy
approach to mindfulness in psychotherapy. Oakland: New Harbinger.
Dahl et (2009) The Art and Science of Valuing in Psychotherapy: Helping Clients
Discover, Explore, and Commit to Valued Action Using Acceptance and Commitment
Therapy. Oakland: New Harbinger.
Eifert & Forsyth (2005). Acceptance and commitment therapy of anxiety disorders. A
Practitioner's Treatment Guide to Using Mindfulness, Acceptance, and Value-Guide
Behavior Change Strategies. Oakland CA: New Harbinger.
Luoma et al (2007). Learning Act: An Acceptance & Commitment Therapy Skills-Training
Manual for Therapists (Context / Nhp Context / Nhp).
Ciarrochi & Bailey (2008). A CBT-Practitioner's Guide to ACT: How to Bridge the Gap
Between Cognitive Behavioral Therapy and Acceptance and Commitment Therapy.
Oakland: New Harbinger.
Zettle (2007). Act for Depression: A Clinician's Guide to Using Acceptance and
Commitment Therapy in Treating Depression. Oakland CA: New Harbinger.
Plus self help books by Hayes & Smith (2005), Forsyth & Eifert (2008), Strosahl &
Robinson (2008)
Methodology: Item generation
 Apparent key features of values from ACT standpoint:
 Clarity
 Priority
 Chosen; Pliance not main source of control
“Justification of actions involves fitting them into existing culturally supported verbal
networks…in many ways this is the exact opposite of valuing” (Hayes et al., 1999,
p.208)
 Enactment
“To some degree, we engage in an act of valuing each time we do something that is
purposive or instrumental” (Luoma et al., 2007, p.131)
 Persistence
“If the client bases living entirely on the absence of emotional or cognitive obstacles,
then valued directions cannot be pursued in a committed fashion (Hayes et al,
1999, p.210).
 Engagement/vitality
“living in accord with one’s values lends an inherent sense of purpose and vitality that
makes confronting monsters a legitimate and honourable undertaking” (Hayes et
al., 1999, p.209)
 Consistency/stability
“Values are more abstract and global than concrete verbal goals and thus provide a
kind of verbal glue that makes sets of verbal goals more coherent” (Hayes et al,
1999, p. 207); “Basic values seldom change” (Hayes et al., 1999, p.259)
Methodology: Item generation
 Brainstormed a list of items to tap each facet.
 Initially gave list of 64 items to friends/family to rate readability
(clear, not sure, unclear) & provide comments
 A list of the 64 items then given to one group (n=4) of ACT experts
to rate how well each item represents valuing on a scale of 0 to 4.
 Items deleted/amended on feedback, new items generated, 79item list provided to 3 other experts to rate for representativeness
(Thanks to Patty Bach, Rob Zettle, Robyn Walser, Russ Harris, Joe
Ciarrochi, Kirk Strosahl, Jennifer Plumb)
 Final list of 70 items.
 Questionnaire starts with the statement:


“1. Please read each statement carefully and then select the number
which best describes how much the statement was true for you
DURING THE PAST WEEK, INCLUDING TODAY.
Likert Scale from 0 (Not At All) to 6 (Completely)
Methodology: validation
 Administered to n=658 university students: ~50%
random split for exploratory/confirmatory
 Validation measures







Depression Anxiety Stress Scales (DASS) (symptom measure)
Satisfaction With Life Scales (SWLS)
Valued Living Questionnaire (VLQ)
Positive and Negative Affect Scales (PANAS)
Mindfulness Attention Awareness Scale (MASS)
Acceptance and Action Questionnaire II (AAQ II)
Ryff’s (1989) Wellbeing Scales






Autonomy
Environmental Mastery
Personal Growth
Positive relations with others
Purpose In Life
Self-acceptance
Demographics
Exploratory (n=366)
Confirmatory (n=292)
Age:
20.4 (sd 4.5)
20.2 (sd 3.9)
Gender:
69% Female
69% Female
Ethnicity:
68.3% White
21.6% Asian
71.6% white
19.5% Asian
Language at home:
80.8% English
81.2% English
Employed:
39%
57%
40%
49%
Course year level:
87% First year
88% First year
Student status:
92% Full time
94% Full time
Relationship status:
61.5%
34.3%
59%
36%
How should a valuing questionnaire
be validated? Self-report measures
Strongest positive associations
 other values measures: VLQ
 satisfaction with life
 well-being
 ?quality of life
Moderate positive associations
 Measures of mindfulness
 Measures of psychological flexibility
Low correlations
 Affect states
 Psychological and physical symptoms
Exploratory Factor Analysis
• Deleted Multivariate outliers (n=59)
• Principal Axis Factoring
• 11 Eigenvalues > 1
• 2 factors accounted for 51% variance
• Promax (oblique) rotation
 Factor
Rotation Sums
Squares Loading =valued
25.6% variance
1 – of“Successful
living”
 40/70 variables loaded > .32 on this factor
 10 variables loaded .8 -.85; Cronbach α = .95
 I continued to get better at being the kind of person I want to be
 I did things that brought me closer to what really matters to me in
my life
 I made progress in the areas of my life I care most about
 Most of the things I did were very important to me
 I was proud about how I lived my life
 I lived my life in a manner I would be happy to be remembered for
 I’ve been able to stay focused on the things that are most important
to me in life
 I was active and focused on the goals I set for myself
 I was in touch with what was important to me
 My behaviour was a good example of what I stand for in life
Factor 2 – Disrupted valued living
 Rotation Sums of Squares Loading = 19.4% variance
 30/70 variables loaded > .32 on this factor
 10 variables loaded .7 -.81, Cronbach α = .93










I tried to work towards important goals, but something always got in the way
I had unpleasant thoughts and emotions that stopped me from achieving my goals
Difficult thoughts, feelings or memories got in the way of what I really wanted to do
It seemed like I was just ‘going through the motions’, rather than focusing on what
was important to me
I spent a lot of time trying to please other people
I found it hard to follow through on important activities because of unpleasant
thoughts and feelings
Most of what I did was to please other people, rather than doing what’s important to
me
I was basically on “auto-pilot” most of the time
When things didn’t go according to plan, I gave up easily
I found it hard to stay focused and engaged with what I was doing
Confirmatory Factor Analysis
Fit Indices
 CMIN/DF = 1.914 (< 3 OK)
 NFI = .945 (.9 - .95 OK, > .95 ideal)
 GFI = .912 (should be > .9)
 AGFI = .877 (should be > .9)
 CFI = .973 (closer to 1 better)
 RMSEA = .058 (< .06 good, closer to .05 better)
 TLI = .966 (close to 1 better, can exceed 1)
Validation with Existing Measures
Symptom Measures –
DASS-21 & PANAS
Depression
Anxiety
Stress
Pos.
Affect
Neg.
Affect
VQ - Successful
-.527
-.214
-.297
.716
-.270
VQ – Disrupted
.706
.560
.599
-.463
.645
Well-Being & Life Satisfaction
VQ – Success
VQ - Disrupted
Autonomy
.393
-.483
Env. Mastery
.646
-.718
Personal Growth
.454
-.602
Pos. Relations
.433
-.524
Purpose in Life
.640
-.716
Self-Acceptance
.681
-.649
Satisfaction With Life
.639
-.513
Ryff’s Scales
Valued Living Questionnaire
VQ – success
VQ - disruption
VLQ Importance
.314
-.147
VLQ Consistency
.462
-.312
VLQ Composite
.475
-.287
Mindfulness & Acceptance
VQ – Success
VQ - Disruption
Mindful Awareness (MAAS)
.449
-.482
Acceptance (AAQ2)
-.400
.701
Alison Christie – Honours thesis




N = 555
Age: M = 32.9, sd 14.87
69.5% Female
Country:
 Australia = 78%
 North America = 15.2%
 Europe = 4%
 Education:




<12 years schooling: 5%
Year 12 = 39%
University = 44%
Other professional = 7%
 Income:




< $20,000 = 19%
$20,000-$69,999 = 29%
$70,000-$129,999 = 28%
> $130,000 = 24%
Christie cont..
• Tested a 35-item version (2 items dropped – did not
load). EFA: 2 factors accounted for 52.2% variance
(n=553)
VQ – Success
VQ - Disruption
CPVI (Mean Importance)
.209
.355
CPVI (Mean success)
.421
.597
CPVI (Mean diff: Import-succ)
-.283
-.352
All significant p < .001
Christie cont..
• Tested a 35-item version (2 items dropped – did not
load). EFA: 2 factors accounted for 52.2% variance
(n=553)
VQ – Success VQ - Disruption
Core value 1 Importance (PVQ)
.167
.285
Core value 1 Success PVQ)
.342
.506
All significant p < .001
Preliminary sensitivity to
change - Dewar
 N=24 government department workers & friends
attending an “ACT for the general public” pilot 5session program
 VQ was a 12-item verson (top 3 loading items of an
early 4-factor PCA) summed (with reverse coded items
reversed)
(n=24)
M
Sd
VQ Baseline
43.75
8.17
VQ Post-test
46.79
6.42
MAAS Baseline
59.00
9.68
MAAS Post-test
63.38
9.60
t(23) = -2.2, p = .038
t(23) = -2.15, p = .042
Summary of Valuing
Questionnaire Findings
 General (rather than domain-specific) values measure
 Captures two facets of valuing (r = -.516)
 Factors are internally reliable
 Items generated in consultation with ACT experts and
literature
 Convergent Validity demonstrated with established
ACT & Non-ACT measures, although is VQ functioning
as mood measure in current form or is it that existing
ACT Values measures are problematic?
How should a valuing questionnaire
be validated? Future work
 ?Test more “behavioural” item version
 Comparing questionnaire responses to other means of capturing
valued behaviour (e.g., diaries, EMA)
 Correlation between Valuing Questionnaire responses and size of
domain-specific values-behaviour correlations
 Developing norms for clinical and high-achieving/ “successful”
populations
 Sensitivity to change: in general; to values-focused interventions
AND OF COURSE FINALISING THE STATISTICAL ANALYSES ON
THE CURRENT DATA SET
Primary outcome measures in ACT studies
17 studies: 8 measure distress/symptoms; 2 measure willingness, 4 measure
problem behaviours, 3 objective/indirect measures of problem behaviour.
0 directly measure changes in valued living.
Use of values measures in ACT studies
since Hayes et al (2006)





Bricker et al (2010) telephone-delivered ACT smoking: Intent-to quit &
smoking cessation. No values measures
Hayes et al (2010) acceptance-based behaviour therapy for GAD: VLQ,
“What percentage of the time did you feel you were spending time on the
things that are important to you?” (0-100)
 % time spent in things important: 48.31% → 71.55% over therapy
 Valued action was not a significant predictor of quality of life after
acceptance controlled
Hernandez-Lopez et al (2009): ACT v CBT smoking. No values measures
Johnson et al (2010): ACT self-help for chronic pain (n=14)
 Chronic Pain Values Inventory: discrepancy between importance and
success across various life domains
 Both ACT and control group reduced values-discrepancy scores over
time, no differences
Lillis et al (2009): 1-day mindfulness & acceptance workshop for obesity.
Used weight-related quality of life measure (ORWELL). 3-mo FU better
QOL for intervention group.
Use of values measures in ACT studies
since Hayes et al (2006)









Lungdren et (2008): n=18 RA ACT v yoga for drug-refractory epilepsy.
Used SWLS & WHO-QOL (both groups improved equally on these
measures)
Wicksell et al (2009) ACT v TAU chronic pain whiplash. Improvements in
SWLS
Dalrymple & Herbert (2007) n=19 uncontrolled ACT social anxiety. Used
VLQ (importance-consistency discrepancies) & QOLI, both improved over
time.
Forman et (2007) 101 Randomly assigned ACT v CBT. Both groups
improved on SWLS, QOLI
Forman et (2007) acceptance v control-based strategies coping with food
cravings. No values measures
Gregg et (2007): RCT Education + ACT v Education. No values
measures
Lappalainen et (2007) CBT and ACT in trainees. No values measures
Lungdren et (2008) mediators of ACT on epilepsy in n=28 South Africans.
Values attainment and persistence in face of barriers (measured by
BullsEye) correlated with seizures, SWLS, WHO-QOL, wellbeing
(magnitude .61 - .73)
Luoma et (2008) 6-hour group workshop for reducing self-stigma for
people in addiction Tx. Used Quality of Life Scale but no values specific
measures.
Existing values measures
 Values In Action (VIA)
 Designed to measures 24 “character strengths”
(e.g., creativity, vitality, kindness)
 Individual rates agreement with statements
describing behaviour representative of strength
(e.g., “am able to come up with new and different
ideas”) A = very much like me – E = very much
unlike me
 High item numbers (e.g., 213-item version used by
MacDonald et al, 2008)
 Trait-like in standard form: encourages personality
assessment
Existing values measures
Valued Living Questionnaire (VLQ)
(Wilson et al., 2010)
 Individual rates each of 10 valued domains 1-10 for
Importance and how consistently has lived according to how
much they value the domain:










Family relations
Marriage/couples/intimate relations
Parenting
Friendships/social
Employment
Education/training
Recreation
Spirituality
Citizenship/Community Life
Physical Well-being
Existing values measures
 Bull’s eye (Lungdren et al, 2005):
 Individual places x to represent how
consistent his/her actions are to
his/her valued direction
 Distance from bull’s eye =
discrepancy in valued living
 Final Bullseye = psychological barriers
to valued living. Individual places X
representing persistence in pursuing
values in face of barriers
 Test-retest reliability r = .86
 Mediation of follow-up changes in
seizures, personal wellbeing from
ACT for epilepsy (Lungdren et al.,
2008)
VLQ
(Wilson et al., 2010)
 Validation with n=338
undergraduates and Butcher
Treatment Planning Inventory
 Importance range limited: mean
ratings from 7.43 – 9.46
 Importance-consistency product
normally distributed
 One factor accounts for 35%
variance: item-factor loadings
ranged .39 - .65.
 Internal consistency importance
α = =.77 and consistency α =.75
: interpretation needs validation.
VLQ
(Wilson et al., 2010)
 Consistency subscale correlation with ‘% time on things that
are important to you’ = .4 (Hayes et, 2010)
 Very low correlation with AAQ r = -.14
 Correlation with MCSD r = .25
 ?VLQ ‘consistency’ wording confusing when individuals rate
domain low importance
CPVI
McCracken & Yang, 2006:
 6 domains (family, intimate relations, friends, work, health,
growth & learning)
 Rate importance 0 (not at all) - 5 (extremely important)
 Rate success 0(not at all) – 5 (extremely successful)
 Importance-success discrepancies unrelated to pain intensity
 α = .82 (success, discrepancy)
CPVI
CPVI cont..
 Discrepancy score is sensitive to
change (Johnston et al., 2010)
 Moderate relations to general
health (SF-36) .36-.53 & unique
variance after Acceptance &
mindfulness in 7/8 SF-36 scales
(McCracken & Vellerman, 2010)
 Prospectively predict pain-related
anxiety and impairment (Vowles &
McCracken, 2008)
CPVI
McCracken & Keogh (2009)
Survey of Life Principles
(see Ciarrochi & Bailey, 2008)
 52 items representing 10 universal values cluster of Schwartz
 Each rated 1-9 for importance, pressure to hold & 1-5 putting into
practice & success in implementing.
 Norms for university students for Importance, Pressure, Success,
Activity for each item.
Personal Values Questionnaire
(PVQ)
(Blackledge et al)
 9 values domains (full version); 3 value domains (family, couple
relationships, friendships)
 Individual writes down their value statement in each domain and
then rates on 1-5 scale:









Importance
Commitment to living the value
Desire to improve progress
Success in living the value in last 10 weeks
Social pressure to hold value
Guilt/shame if value was not important
Importance free of social pressure
Extent the value makes life meaningful
Experience fun and enjoyment when living consistently with value