Subjective Well-being User Consultation Key findings Slide Pack April 2013 Contents • • • • • • • About the Research The Headlines Context Unpacking the Headlines Understanding ‘Subjective Well-being’ Displaying and illustrating the findings Building credibility and trust in the research 2 About the Research 3 Aims of the Research • To investigate how members of the public respond to ONS subjective well-being releases • To determine which methods of presentation are easiest to understand, interpreted most accurately and viewed as most appealing • To inform the development of future subjective well-being statistical bulletins 4 What did we do? • Carried out 3 rounds of focus groups – Round 1 – 3 focus groups – Round 2 – 3 focus groups – Round 3 – 2 focus groups • Pre-group task – Each group was asked to bring along an example of research that was clear and easy to understand • Tested different materials at each round – Modified the materials between each round drawing on the findings from the groups 5 How were the groups composed? Round 1 Round 2 Round 3 Group 1 (up to 8 people) Age - 50+ Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: Degree and above Group 4 (up to 8 people) Age - 50+ Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: A level or equivalent Group 7 (up to 8 people) 18-45 Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification: mix of both A level only and Degree and above Group 2 (up to 8 people) 18 - 30 Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: A level or equivalent Group 5 (up to 8 people) 18 - 30 Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: Degree and above Group 8 (up to 8 people) 46+ Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification: mix of both A level only and Degree and above Group 3 (up to 8 people) 31 - 49 Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: A level or equivalent Group 6 (up to 8 people) 31 - 49 Gender: 4 male and 4 female At least two who were Asian or Black Educational qualification level: Degree and above 6 59 people took part…… Gender Educational level Male 29 Degree 27 Female 30 A-Level 32 Age Ethnic minority group 46 18 - 30 19 White 31 - 49 21 Black/Caribbean 9 50 + 19 Asian/Indian 4 7 The Headlines 8 Key Learning: Engaging the Public • Make it engaging and interesting: – Bring the findings alive – ‘a story’ or organise around themes – ‘So what?’- draw out the relevance for the audience – Provide a connection a ‘hook’ into the findings – Offer a ‘layered’ approach – enabling people to access different types of summary • Ensure it is accessible: – Be concise and clear – Use simple straight forward language (avoid jargon) – Simple headings – Clear layout – Use a consistent style – Consider the font size – Use colour (in a consistent way) • A ‘picture paints a thousand words’ – Provide ‘visuals’ to illustrate the findings (not too much variation in style) 9 Some considerations…. • Need to cater for different tastes – Different formats work with different people – Preferences vary for words/bulleted statements, tables with percentages or more visual displays – Pie, bar charts, line graphs and info-graphics seemed more popular than tables – Younger respondents appeared to read the bar charts and tables more easily and were more receptive to info-graphics • Try to personalise the message – make it resonate in some way – People only tend to look at research that is relevant to them and their circumstances – or a topic that interests them • Make it really simple and easy to engage with – If it doesn’t interest people they are less likely to make the effort to look at it • People are wary and sceptical about research – Fuelled by their mistrust of marketing research – Or statements about what is good or bad for you 10 Context 11 Materials tested • Key findings document summarising the headline messages from the research – As a comparison we also tested a 'journalistic' style variant featuring one aspect of the findings (Round 3 only) – A quiz to assess understanding of the key findings document (Round 3 only) • Making sense of the language and terms – E.g. choosing between different statements and options • Making sense of visual options – E.g. tables, line graphs, bar charts, info-graphics and maps • Mocked up statistical release (Round 3) 12 Developing the materials • The examples tested were mocked up versions specifically intended to test reactions to different approaches to writing and presentation – They were developed in response to the feedback we received from the groups – They were not based on ONS publications nor consistent with ONS house style • That said, the Key Findings document used at Round 1 was closely based on the First Annual Statistical Release produced in July 2012 – It was adapted to be shorter and simpler looking more like a 2 page research summary of key findings in bullet points (See Annex) • At Round 2 we simplified and reduced the words used to describe the key findings and organised the material around the 4 key questions – This produced a pretty good version of the text • At Round 3 we incorporated graphics, bar charts and line graphs to illustrate the findings and test reactions to different types of display – And made some minor revisions to the text 13 Participants’ research findings • Pre group task – participants were asked: – To bring an example of survey findings that are clear and easy to understand • They chose research findings that: – Were of interest, or relevance or that they had a personal connection with – Examples: A product (cosmetics, cars, phones, baby food) or assessment of a service or experience (transport, care work, employee or residents’ satisfaction) or aspect of their lifestyle (pensions, sex) or a health concern (depression, obesity, cancer) • They liked the way they were presented because they were: – Communicated in short simple statements/an article and occasionally a video – Drew out the conclusions or made the comparisons easy to see – Typically involved some kind of (colourful) visual display – which reflected their personal preferences for a table, pie or bar charts, use of graphics and info-graphic etc. • They found their research on: – The internet (e.g. BBC, via Google), in magazines (e.g. Which, Good Housekeeping, Esquire), in newspapers (e.g. Daily Mail, Metro, Guardian and Evening Standard) 14 Unpacking the Headlines 15 Rounds 1 – 3:Participants’ Views of the Key Findings Document • • • • • • • Generally participants were not that interested in the subject and questioned the value of it – It’s telling you nothing just stating the obvious – ‘people with decent jobs, housing and money are quite happy and that those who are unemployed without those facilities etc. aren’t; it’s obvious! ’ Although by Round 3 we seemed to have engaged some participants in the topic Throughout the research they often struggled to see the relevance of the topic – Indicating that they were not interested in their own or other people’s well-being And a few viewed the research as a waste of money – they questioned: – The value and relevance of asking people about their feelings ‘yesterday’ – Whether the underlying motive was for the government to encourage people to feel happy and contented with their lot That said, by Round 2 we arrived at a pretty good version of the words – Which people found reasonably straightforward and less ‘convoluted’ And by Round 3 we arrived at what appeared to be a good combination of words and visuals which people found easy to understand and engage with – There were still concerns about the language being rather official in tone (e.g. like a business report) – And they questioned the range of different graphics, bar charts and line graphs used – Suggesting that a more limited range of options would be easier to fathom And they liked the use of colour as it made the document easier to engage with 16 Our best guess at the Key Findings document This is based on a combination of the best features of the two versions we tested at Round 3 17 How’s life in the UK ? Findings about personal well -being from a UK survey Background The Annual Population Survey asks a cross section of 165,000 adults over the age of 16 how they are feeling about their lives in response to the following four questions: Q1. Q2. Q3. Q4. Overall, Overall, Overall, Overall, how satisfied are you with your life nowadays? to what extent do you feel the things you do in your life are worthwhile? how happy did you feel yesterday? how anxious did you feel yesterday? People are asked to rate their answers on a scale of 0 to 10 where ‘0’ is ‘not at all’ and ‘10’ is ‘completely’. Interviews were carried out on different days of the week and during every month of the year between April 2011 and March 2012. Q1 Are we satisfied with our lives? • The majority of people (75.9 per cent) are satisfied with their lives – and rate their life satisfaction as 7 or more out of 10. Generally, younger people (aged 16-19) and older people (aged 65-79) are the most satisfied with their lives. People in their 40s and 50s have the lowest levels of satisfaction with their lives which may be linked to having greater work and family commitments at this stage of life. 18 19 Q3 How happy were we feeling yesterday? • Just over 70 per cent of people reported feeling pretty happy yesterday – rating their answers as 7 or higher. Figure 1 compares how people of different ages rate their life satisfaction, their levels of happiness and whether they think they are doing worthwhile activities. Figure 2. Average UK ratings for life satisfaction, worthwhile and happiness yesterday, by age group Ratings High Low 1. 2. 3. 4. 5. Adults aged 16 and over were asked 'Overall, how satisfied are you with your life nowadays?', 'Overall, to what extent do you feel the things you do in your life are worthwhile?' and ‘Overall, how happy did you feel yesterday?’ where nought is 'not at all' and 10 is 'completely'. Data from April 2011 to March 2012. All data weighted. Non-respondents not included. Sample based on 165,000 respondents in the UK (England, Scotland, Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics 20 Q4 How anxious were we feeling yesterday? • About 40 per cent of people reported feeling quite anxious yesterday – rating their answers as more than 5 out of 10. Figure 3. Average rating for anxious yesterday, by age group Ratings 0 Low 1. 2. 3. 4. 5. Adults aged 16 and over were asked ‘Overall, how Anxious did you feel yesterday?' where nought is ‘not at all’ and 10 is 'completely'. Data from April 2011 to March 2012. All data weighted. Non-respondents not included. Sample based on 165,000 respondents in the UK (England, Scotland, Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics 21 Do levels of happiness and anxiety vary over time? During the last year our levels of happiness peaked in April 2011 and March 2012 – in both these months we basked in a mini heat-wave and enjoyed record levels of sunshine. In April 2011 we also enjoyed an extra day off to mark the Royal Wedding Figure 3. Average happiness ratings for each month (UK), April 2011- March 2012 Ratings Anxiety was at its highest level in May 2011. During this month the volcano in Iceland erupted and Osama Bin Laden was killed by US armed forces sparking fears of a possible terrorist reprisal. Figure 4. Average anxiety ratings for each month (UK), April 2011- March 2012 Ratings 1. 2. 3. 4. 5. Adults aged 16 and over were asked ‘Overall, how happy did you feel yesterday’? and ‘Overall, how anxious did you feel yesterday’? Data from April 2011 to March 2012 All data weighted. Non-respondents not included. Sample based on 165,000 respondents in the UK (England, Scotland Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics 22 How’s life for different groups in society? • Women were more likely than men to report being satisfied with their lives, doing worthwhile activities and feeling happy yesterday. But they were also more likely than men to report feeling 'anxious yesterday'. • Unemployed people were twice as likely as those who were in work to have lower levels of life satisfaction. • On average, adults who were married, in a civil partnership or cohabitating reported higher levels of well-being than people who were single, widowed or divorced. • Being in poorer health is related to higher levels of anxiety. • 'Black/African/Caribbean/Black British' people reported the lowest average levels of 'life satisfaction'. People from the 'Arab' group reported the highest average level of anxiety. 23 Their Recommendations for the Key Findings • • Keep it short and simple – providing a summary 2/3 pages maximum It should present short succinct and concise messages – • They need to know why the research has been funded and why it should matter to them – – • • Ratios are probably easier than fractions and percentages On balance there is a preference for the actual rating (or average rating) rather than an interpretation (low or high etc.) They questioned the use of technical notes under the charts: – – • So question numbers next to sections are probably helpful And there is a need to keep the headline message and visual display together so the text is not divorced from what is being illustrated. Use plain simple English - no jargon (e.g. use average rather than mean) Avoid combining percentages, proportions and ratings. – • On balance they seemed to prefer line graphs and bar charts but this did vary Label axes appropriately There is a need for more layout and sign posting – – • • Explain how it could affect their lives Explain the relevance of the findings - and whether it is actually measuring anything meaningful Simple and clear use of visual displays – and only where they are adding to the message: – – • Avoid including words in brackets that repeat what is being said in a different way as people skip over these They either ignored them or misunderstood them and thought they were unnecessary – ‘makes your head hurt’ If they are to be used then they need to be easy to understand And provide additional more information about the research that people can read if they want 24 How best to engage the public? • Most people will only want to engage with a short article – Highlighting an interesting aspect of the findings • So the most effective way to engage them will be through more of a 'journalistic' approach – Using some kind of ‘hook’ to lure them into the findings – And possibly whet their appetite for more • It would help to link this article to the Key Findings document – For those who want to find out more about the research • But they are unlikely to want to access the Statistical Bulletin – As they see this as catering for the more technical and interested audience 25 We explore reactions to an example of the more ‘journalistic’ approach This was tested at Round 3 and was universally liked for being interesting and easy to engage with 26 Can’t get no satisfaction? Recent findings from the latest ONS research on personal well-being show that people in the UK are actually pretty satisfied with their lives. The research includes responses from 165,000 people across the UK who were asked to rate their personal well-being by answering four questions on the Annual Population Survey. In response to the question- ‘Overall, how satisfied are you with your life nowadays?”- the majority of people are (76%) rated their life satisfaction as 7 or more out of 10. Interestingly, the findings also show that levels of satisfaction vary a lot depending on age. Younger people (aged 1619) and older people (aged 65-79) are the most satisfied with their lives. People in their 40s and 50s have the lowest satisfaction- possibly linked to greater work and family commitment at this stage of life. So life is good when you’re young and free- and we should all be looking forward to a happy retirement. 27 Can get some satisfaction • Participants really liked this style of writing and thought it was a good engaging summary – They started to become interested in the findings – And they said it might encourage them to want to read on – Although participants who thought they were unlikely to be looking forward to a happy retirement said the last sentence made them cross • They easily related to the friendlier and less stuffy language. • They liked the catchy heading ‘can’t get no satisfaction’ heading – Although there was some concern that it was rather negative in tone and did not actually represent the positive findings • They also suggested that it would be helpful to break the text up with headings and consider the layout and font used – Applying the learning from the Key Findings document • There were mixed views about the use of the speedometer – They either liked it or would have preferred a bar/pie chart or line graph 28 Understanding ‘Subjective Well-being’ 29 Subjective Well-being • Confusion about the meaning of ‘subjective’ – – • • It is not a word people use – they guessed at its meaning or said they did not know what it meant ‘Personal’ works better for some but not all and they were less keen on ‘individual’ Well-being – has connotations with your health (but is a bit vague and generic word) Preference for ‘Happiness’ or ‘how happy people are’ or ‘how you feel about yourself’ ‘Well, you wouldn’t say to your mother how’s your well-being today, would you? It’s just ridiculous. It’s just official language.’ ‘Well I suppose it’s taking a look at yourself and talking about how happy you are’ ‘Your personal idea of…how you’re feeling and health and so on’ ‘How you feel you are’ It’s a very glib phrase….are they talking about healthcare, are they talking about your general health, are they talking about your personal relationships, are they talking about your relationships with your kids... ‘The feel good factor.’ ‘Well, my lifestyle, what I do, how I do it and my contentment I guess out of the way I live my life.’ ‘Subjective well-being makes no sense’ ‘Subjective is the opposite of objective’ ‘How you feel about yourself.’ ‘Classic example of jargon’ 30 Displaying and Illustrating the Findings 31 Key principles for visuals • Different people respond to different types of visual display so you need to provide a range of displays (e.g. some prefer a table and others prefer pie and bar charts, line graphs and info-graphics). – – • The visuals need to be simple and clear so that you can make sense of what it means with a glance – • Use contrasting colours They made different suggestions for which colours to use (e.g. Red for anxiety and green or yellow for life satisfaction) Intuitively people want you to use the same scale throughout the bulletin but they quickly appreciate the value of altering the scale where it makes sense to do so – – • Label the axis clearly Colour code all visuals systematically – – • They were not interested enough to work harder to understand the findings Use clear simple headings, labelling and headings – • But people struggle if you ‘mix and match’ too much Once you’ve got the formula from the first one it’s straightforward because you know what you’re looking at if they are similar They were concerned that changing the scale could confuse and potentially distort the findings by appearing to make any differences look larger than they are Also some people continued to be confused by the scale for anxiety being inverted There is a need to draw out the conclusions for the reader so they can read the text alongside the visual display 32 Final version of the table Table 1 Headline results for each of the four personal well-being questions United Kingdom Percentages Very low (0-4) Life satisfaction Worthwhile Happy yesterday 6.6 4.9 10.9 Very High (610) Anxious Yesterday Low 21.8 (5-6) Medium (7-8) High (910) Mean 49.8 48.6 39.3 26.1 31.4 31.8 7.4 7.7 7.3 17.5 15.1 18 High (4-5) Average Rating Medium (2-3) 18.1 23.5 Low (0- Average 1) 36.6 3.1 Table notes: 1. Adults aged 16 and over were asked 'Overall, how satisfied are you with your life nowadays?', 'Overall, to what extent do you feel the things you do in your life are worthwhile?', 'Overall, how happy did you feel yesterday?' and 'Overall, how anxious did you feel yesterday?' where nought is 'not at all' and 10 is 'completely'. 1. Data from April 2011 to March 2012. 1. All data weighted. 1. Non-respondents not included. 1. Sample based on 165,000 respondents in the UK (England, Scotland, Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics. 33 Views about Tables • • • • • • Tables produced much more polarised views – either turning people off or appealing to those who are comfortable with numbers and percentages and find them easy to read – ‘too much hard work It requires a bit of brainpower just to decipher it’ – They are giving you too much information – ‘it reminds me of something that you’d have in school. Literally it looks like something out of a textbook.’ It was said that unless you are really interested in the findings then you are unlikely to make the effort to work out what is in a table – ‘It does make me think of something like from school…It’s not a quick glance thing; it’s something you have to look at in a bit more detail.’ – And for this reason bar charts and other displays were often preferred for being easier and quicker to read But despite it being viewed as too ‘numbers heavy’ it was sometimes preferred to dense text They seemed to like the use of colour in the table They found it helpful to combine the anxious yesterday findings with the other findings – As you could easily see how the scales are inverted On balance it seemed helpful to include the average ratings in the table – But this did contribute to some people finding the table too complex and busy 34 Preferred Line-graph Figure 2. Average UK ratings for life satisfaction, worthwhile and happiness yesterday, by age group Ratings High Low 1. 2. 3. 4. 5. Adults aged 16 and over were asked 'Overall, how satisfied are you with your life nowadays?', 'Overall, to what extent do you feel the things you do in your life are worthwhile?' and ‘Overall, how happy did you feel yesterday?’ where nought is 'not at all' and 10 is 'completely'. Data from April 2011 to March 2012. All data weighted. Non-respondents not included. Sample based on 165,000 respondents in the UK (England, Scotland, Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics 35 Views about Line-graphs • Generally people seemed to find line graphs easy to read and quick to engage with – You can see the result immediately – the peaks and troughs – and it only needs minimal text for explanation • But others found them rather dry and dull – And reminiscent of Economics A-level • The age line-graph and time-line aroused discussion and helped to engage people with the findings – As they started to find a connection with the findings • They highlighted the importance of keeping the axes simple and labelling them appropriately – And not overloading the graph with too much information • They also stressed the need to use colour appropriately – As blue and purple were felt to be too close to one another for an instant snapshot. 36 Preferred Bar Chart Figure 3. Our relationships and levels of life satisfaction United Kingdom 0–4 (Very low) Percentages 5–6 (Low) 60 7–8 (Medium) 9–10 (High) 50 40 30 20 10 0 Married Cohabiting Single Widowed Divorced Notes: 1. 2. 3. 4. Adults aged 16 and over were asked 'Overall, how satisfied are you with your life nowadays?' where nought is 'not at all' and 10 is 'completely'. 'Married' includes people in Civil Partnerships, 'Cohabiting' includes same sex couples (but not in a Civil Partnership), 'Widowed' includes surviving civil partners and 'Divorced' includes people who are separated and separated/dissolved Civil Partnerships. Data from April 2011 to March 2012. Sample based on 165,000 respondents in the UK (England, Scotland, Wales and Northern Ireland). Source: Annual Population Survey, Office for National Statistics 37 Views about Bar Charts • Participants are familiar with bar charts and like them for this reason – And tend to find them clear and easy to read • They appeared to prefer side by side bars rather than bars stacked on top of each other – But this depends on how many categories are being presented – As they were concerned about the size of the bars being large and easy to read • They highlighted the importance of using contrasting colours – Rather than variations of blue or green that are particularly challenging for people with visual impairments • They sometimes questioned the value of using a bar chart – Suggesting a line graph might have been more appropriate – As they felt this would help to simplify the findings • They also highlighted the need to label scale points on the axis appropriately – E.g. high and low and number of children 38 The Preferred Info-graphic 39 Views about Info-graphics • Views seemed to differ by age – The older age groups were less keen on the info-graphics than the youngest group • • That said the use of graphics was clearly seen as a helpful way of telling the story and breaking up the text Of all the visuals the ‘speedometer’ seemed to receive the most positive response until Round 3 where it received a mixed review – ‘I did like …. moving away from the standard pie chart, bar graph or whatever, I like something a little bit different, it makes me want to look at the stat a bit more if it’s the speedo or whether it’s … just something a bit different, makes it nice to look at’ – Concerns were expressed about it being hard to read and misleading (partly because it looks too much like a pie chart so people want to read it in this way and they therefore found the scale and use of colour misleading) • They highlighted the importance of keeping info-graphics – As simple as possible – And using appropriate colours preferring pastel shades over the brighter colours • They also suggested other ways of using graphics such as smiley and frowning faces as keys for the visual displays 40 Views about the use of Maps • Views were more mixed about the use of maps • They were concerned about keeping the maps simple and clear – As they found the colours hard to distinguish between and the content too busy – They therefore suggested changing the colours and scale – And using a clearer key for the maps • Maps can be a good way to present the data but they need to be clear – And some people were not that interested in the detail in the map – Others said it switched them off • When they look at the map it made them question how far the survey represents people from across the UK • They suggested presenting a regional map as this might more easily enable the comparisons they are looking for – But they were interested in being able to compare where they live with other places 41 Building Credibility and Trust in the Research 42 What do Participants want to know about the Research? • In order for participants to trust the research and to take it seriously – They need to know: – Why it is being funded – How it will be used – And basic facts about how it was carried out • They suggested that this could be briefly provided in the introduction – with more of the detail in some kind of technical annex • The next ‘About the Study’ slide contains the key information that they are interested in – We developed this from the feedback at Rounds 1 and 2 – And tested this at Round 3 • The final slide provides a summary of the key information they wanted to know and questioned throughout the research 43 About the Study Since April 2011, the Office for National Statistics (ONS) have included four questions on the Annual Population Survey asking people to rate different aspects of their well-being. The survey includes a cross-section of 165,000 adults over the age of 16 in the UK. The questions are: Q1. Overall, how satisfied are you with your life nowadays? Q2. Overall, to what extent do you feel the things you do in your life are worthwhile? Q3. Overall, how happy did you feel yesterday? Q4. Overall, how anxious did you feel yesterday? People are asked to rate their answers on a scale of 0 to 10 where ‘0’ is ‘not at all’ and ‘10’ is ‘completely’. Interviews are carried out every day of the week except Sundays and during every month of the year. The survey findings are analysed alongside information about people’s characteristics and circumstances to help form a picture of what strengthens our well-being or weakens it. The questions can also be used to monitor how changing circumstances affect our well-being. All of this information will be used by policy-makers to help make better decisions, taking into account what the likely impacts on personal well-being will be. 44 Participants wanted to know… • • • Who has done the research – are they an independent body? The purpose and reason for the research – why is it relevant to them - ‘so what’? Also, the wanted to know more about the context of how the research was carried out – – – • • Where was the research carried out How large and representative the sample is – – • For those who are interested they want to see the evidence for it including a cross section of people They were interested in knowing how representative it is of age, geography, employment and financial status How the findings will be used – – • They questioned the value of asking people about their ‘feelings yesterday’ and using this as an indicator of general feelings So it would be helpful to explain that the survey is designed to provide robust time comparisons And specifically to say that people are being interviewed throughout the year Whilst the final paragraph in the ‘About the Study’ document was helpful for providing credibility for the research The idea that the findings could be used to shape policy aroused concern as they believed the data was too subjective and high level to be a robust source for policy decisions Other information they were interested in: – – – Comparisons of how the results vary over time Comparisons of how the UK compares with other countries Explanation about what makes people satisfied with their lives and reasons why they are satisfied. 45