Developing quick
expenditure
questions
Findings from focus groups, cognitive
interviews and a split-ballot experiment
23rd April 2014
Focus Groups
Aims
To examine what people include when asked about how
much their household spends
To explore whether people can recall how much their
household spends
 Is it easier to give a ‘total figure’ or ‘spending break down’?
 Timeframes?
 Sensitivity?
To explore how finances are managed in different
households and implications for questionnaire design
To examine views on existing survey questions on spending
from a respondent perspective.
2
Methodology
 Six groups in three geographical areas (Manchester,
Reading and London)
 90 minutes
 £30 incentive
 Participants varied:





Age group
Sex
Household income
Household composition
Subjective financial wellbeing
 45 participants in total: 6-9 per group
3
Problems with ‘household’ spending
1
Clarification required over what to include
“Do you mean stuff in the house? Or just what you
spend your salary on generally?”
(Female, Group One)
Question needs to be clear we are interested in all
forms of spending.
Not just spending on items in the house
4
Problems with ‘household’ spending
2
 All respondents felt they could give an estimate of own total spending (to the





nearest £200?) BUT
They may not know what other people in the household spend (e.g. lodgers,
adult children)
Someone else may keep track of household finances (e.g. spouse)
Couples felt they could give an approximation of their partner’s total spend
Respondents in shared households felt they could not give any details of
other household members spending beyond shared bills
Suggests questions should be asked of the ‘benefit unit’ not the household
“If you just share a flat with them or whatever…well, you wouldn't have a clue
what they're spending their money on. They could spend it on anything!”
(Male, Group One)
5
Ease of recall varied
Participants varied in terms of how easy they thought it would
be for them to report how much they spent in total in the last
month
Easy if:
 You keep records or check statements
 Outgoings are similar each month
 Income is roughly equal to expenditure
“Exactly what goes in comes out, pretty much every month.”
(Female, Group Four)
Difficult if:
You don’t keep track or you ‘spend without thinking.’

6
Types of expenditure
Easier categories to recall
More difficult categories to recall
Fixed regular bills such as mortgage, rent, Expenses that vary month by month (such as
council tax and things paid by standing orders.
petrol)
Large ‘one-off’ purchases such as a new car or a
new TV
Items the participant has spent nothing on (e.g.
non-smokers found it very easy to say how
much they spent cigarettes)
Regular food shop (provided you buy a similar
amount each time or have a food budget)
Essentials
Fixed subscriptions such as gym membership.
7
Smaller items paid (costing less than £10).
Irregular but frequent spends (topping
mobiles phones, topping up the gas meter)
up
Food shopping if you don’t budget for it or buy
different amounts each time.
Food bought outside of the home/ topping up the
main regular shop
Impulse buys/ spur of the moment purchases/
treats.
Leisure activities that have no fixed cost, e.g.
going to the pub, buying craft materials,
gambling.
Other findings
 Avoid single categories for socially undesirable
behaviours (such as tobacco, alcohol or gambling).
 Preference for reference periods varied:


Usual versus average
Month v weekly v annual
 Keep questions short and to the point
 Keep examples to a minimum
 Avoid the term ‘utilities’ as it is ambiguous
 Do not ask separate categories on food and non-food
grocery shopping
8
Outcome of focus groups
Overall findings were positive:

People willing and able to provide some information on spending.
Different approaches to questions were favoured by different
individuals and not easy to predict and all have pros and cons.
Four formats of question proposed:




Total spending in the last month
Essential spending and non-essential spending in the last month
Spending breakdown
Income and surplus
New definitions of household and best person to collect
information from to be tested.
9
Cognitive
interviewing
What is cognitive interviewing?
 Qualitative interviews exploring the question and answer process:




Understanding
Recall
Judgement
Response
 How do people go about answering these questions?
 How easy or difficult?
 What is being included or excluded?
 Methods



11
Administer questions (mimic survey conditions)
Think aloud
Probing concurrent and retrospective
Sample and Recruitment
 Small purposive sample


Round 1= 14 cases
Round 2= 14 cases
 Variation in:





12
Household compositions
Sex
Income
Employment status (including self-employed)
‘Budgeting style’ (R2 only)
Round One
 Four versions of expenditure questions tested:
1.
One-shot question (Total spending in last month)

2.
3.
Two-part question (Essential and non-essential spending)
Breakdown question (Eleven spending types plus ‘other’)

4.
Self-rated accuracy questions
Total and adjustment
Income minus surplus (plus Qs on spending using savings or credit)

Total and adjustment
 Every question tested in the same order…

13
General to more detailed
Overarching issues
 Knowledge of others
 Varied interpretation of ‘last month’:




Previous calendar month
Current calendar month (including future days)
Budgeting month
Usual month
Inclusion and exclusion criteria of ‘spending’





14
Servicing debts
Saving and investments
Work expenses
Remittances and child maintenance
Housing benefit and council tax benefit
One-shot question
Strategies
Adding
15
Total
Income
minus
surplus
One-shot: How easy was it?
It is something that you
do every month or every
week. You spend the
same thing on the same
things.
16 .
It would take me ages to
work out exactly how much.
Problems with One-Shot
 Forgetting examples

Aural presentation
 Forgetting ‘other’ types of spending
 Calculation problems
Accuracy questions
 Answers to accuracy questions were inaccurate (!)

17
Range question tended to produce reports of greater inaccuracy.

Unclear how ‘nearest £50’ converts to range

Evidence of social desirability
Two-part question: Essential and nonessential
 Similar problems as detected at one-shot question

Calculation, forgetting examples, forgetting types of
spending etc
 Overlap between essentials and non-essentials (not a
problem)
18
Breakdown question
 7/14 cases £200+ different from one-shot
 Breakdown more accurate

Showcard prompted items to be remembered
 Breakdown less accurate


Over-estimating cost (reference period issues)
Double-counting
 Adjustment/ reconciliation question useful for
certain groups
19
Income minus surplus question

7/14 cases £200+ different from one-shot

Worked best for those with consistent income who
spent it all each month

Problematic where:



20
Income irregular or complex
Income sensitive
Surplus kept in current account
Respondent Preference
Breakdown
It’s broken up into so
many different categories
it forces you to be more
accurate. You probably
get a much more
accurate reading from
that question.
21
One-shot
I spend the same sort of
amount every month and
I know how much I was
going to spend out in that
month.
Recommendations from R1
 Continue to assess benefit unit spending
 Explore ‘last month’ vs. ‘last 30 days’ reference periods

Trial a question on whether spending last month was ‘typical’
 Retain ‘One-shot’ question


Trial ‘example showcard’
Trial directing Rs to different strategies
 Retain ‘Breakdown’ question


22
Provide clarifications on inclusions/ exclusions
Retain adjustment question
 Drop ‘two-part’ and ‘income minus surplus’ questions
Round two : Split Sample
Dummy
questionnaire
Dummy
questionnaire
23
One-shot with
‘example
showcard’
One-shot with
‘strategy
showcards’
Breakdown with
adjustment
Breakdown with
adjustment
Comparison
and probing
Comparison
and probing
One-shot: Strategy showcard
24
One-shot: Strategy showcard

25
One-shot versus breakdown
Strengths
One-shot
 Quick
 Flexible answering
strategies
 Preferred by Rs who ‘knew’
total spend
Breakdown  Recall of certain items
improved
 No calculations
Exclude money put into
savings
26
Weaknesses
 Adding strategy: items
forgotten/ errors in calculation
 Income minus surplus:
Included money put into
savings
Time consuming
May know ‘total’ but not
‘breakdown’
More intrusive
Recommendations from R2
 Continue to use benefit unit expenditure
 Use ‘last month’ period
 Retain ‘One-shot’ question for piloting
 Drop ‘strategy showcard’
 Keep ‘example showcard’ (with refinements)
 Keep check on ‘typical’ spending
 Retain ‘Breakdown’ question for piloting
 Keep adjustment/ reconciliation question
27
Adapting questions for survey usage…
 Placement of questions:


Benefit unit level information does not fit neatly within existing Qn
structures…
Finding the ‘person most knowledgeable’ in practice…
 Mixed-mode administration:

Face-to-face/ telephone/ web mode

28
Display of example ‘showcard’ in web mode