2011 Census: Content
Workshop
Office for National Statistics
February 2008
Welcome
Glen Watson
Census
Purpose
• Provide update on funding for fourth page of
individual questions
• Allow key users another chance to comment
on ONS priorities for topics
• Focus on the new topics/questions that ONS
proposes for inclusion
• NB: Presentation focuses on Census in
England & Wales
Agenda
•
•
•
•
•
•
•
•
•
•
•
10.10 Introduction
10.40 Question testing
11.00 Second residences and visitors
11.40 Coffee
11.55 Migration
12.35 National identity
13.00 Lunch
13.45 Income
14.15 UK views on content
14.30 Prioritisation of topics
16.00 Close
Introduction
Peter Benton
Census
Census provides statistics on:
• Population units:
– people and housing &
– key demographics (age, sex, marital status, ethnicity)
• Population structures:
– households, families
• More detailed characteristics :
– eg religion, labour market status, industry, qualifications,
health/disability; etc
• Key requirement to ‘Get the Count Right’
– 2011 Census aim : “maximise overall response rates
and minimise non-response in specific areas and
among particular population subgroups”
Key lessons from 2001 Census
• Need to gather information on more than just
usual residents
– Include visitors
• Need to understand coverage
– Comparisons census vs admin data
• Need to improve enumeration of Communal
Establishments
Mobile population & complex lifestyles
• More people at more than one address
– Weekday residences for work
– Holiday / weekend homes
– Children of divorced parents
• International migration
– More ‘short term visitors to UK’ – resident or not?
• Plus familiar issues of students, armed
forces, prisons, hospitals, hotels, hostels
• Risk of undercount, overcount, or wrong location
• Need to count right people, in right place
– and be able to demonstrate this
Work done so far – 2005 consultation
• Formal 3 month consultation (May – Aug)
• Publication March 2006
• Over 2000 responses from 500 users
• Scoring of user requirements
• Evaluation of ONS considerations & other
reasons for collection
• Allocation of topics to one of three categories
Evaluation of topics
Category 1 Topics likely to
be included
Question testing to
refine questions
Category 2 Topics under
consideration
Investigation of
alternative sources
Question design &
testing
Category 3 Topics not to be Topics will not be
included
considered further
Criteria
• Strength of user need
– Resource allocation
– Policy use
– Research
• Small geographies
• Alternative sources
• Multi-variate analysis
• UK comparability
• Continuity with 2001
• ONS considerations
– Data quality
– Public acceptability
– Respondent burden
– Operational concerns
(eg costs, space)
• Other reasons for
collection
– Operational purposes
– Use for coding
Key requirement: Robust estimate of population count
Summary of user requirements from 2005
• Most 2001 topics
• New topics including:
- income
- language
- second residences
- national identity
• Little expressed requirement for number of
employees & hours worked
• More than 3 pages of questions!
• Difficult trade-offs to be made
Work done so far – topic groups
• Eight topic groups
• Topic experts from within ONS (except DIUS for
•
•
•
•
qualifications)
UK-wide membership
ONS Harmonisation
Methodology
Census
Topic-specific consultation
• Further specific consultation with users to better
understand needs/priorities:
– Ethnicity, identity, language & religion
– Population definitions – PDWG
– Migration
– Demographics
– Housing
– Income
– NS-SEC
– Disability
Other consultation and question testing
Cognitive question testing
2005 - 2008
Census Advisory Groups
May & Dec 2006, Nov 2007
Statistical Heads of Profession
Sept 2006, June & Nov 2007
Open meetings (London, Cardiff,
Sheffield)
March 2007
Postal Test
April 2007
Focus groups
April/May 2007
2007 Census Test
May 2007
Census Test Evaluation Survey
May 2007
Omnibus Survey testing of new questions
Sept - Dec 2007
Fourth page of individual questions
• Would allow inclusion of more topics
• Research carried out on effects of longer
questionnaires
• Split sample postal test 2007 – 3 vs 4
pages
• Comparison with other censuses around
world
• Cost analysis of fourth page
Changes since 2005 evaluation
• Increased requirement for information on migration
– Inter-Departmental Taskforce on Migration 2006
– Migration consultation
• Planned Disability Survey
• Question testing carried out on category 2 topics
• 2007 Test and Postal Test
• Priorities for content of 3 & 4 pages discussed:
– Open meetings March 2007
– Advisory Groups
– Heads of Profession
Questions/Comments
• Any questions or comments?
Question testing overview
Ruth Wallis
Data Collection Methodology
Why pre-test questionnaires?
• Find out if questionnaires meet survey
objectives
• Gain an estimate of reliability and validity of
answers
• Identify sources of potential non-sampling
error
What is cognitive interviewing?
A face-to-face interview to test questions or a
questionnaire, which aims:
• to understand how the respondent fulfils the
task of answering questions
• to detect any actions or understandings that
are not what the designer intended
Cognitive steps in answering a question
• Comprehend
• Retrieve
• Form judgement
• Edit answer
Identifies problems
• Uncovering ‘hidden error’
• Understanding the question
• Remembering or recalling the information
• Selecting a response
• Reactions to sensitive questions
Features of cognitive testing
• Observation
• Think aloud
• Concurrent probes
• Retrospective probes
• Paraphrasing
• Vignettes
• Card sorting tasks
Cognitive testing programme
• Phase 1: question development & testing for 2007
Census Test
• Oct 2004 to Apr 2006
– The questions tested were developed using:
• 2001 Population Bases and Definitions
questions
• ONS’ harmonised questions
• new user requirements
• A range of respondents were interviewed to
ensure all criteria of interest were covered. In
total:
– 56 interviews
– 2 focus groups
Cognitive testing programme
• Phase 2: development & testing for Rehearsal
• Jan 2007 to Aug 2008
– New format testing questions in context of the
whole questionnaire
• Five waves of cognitive testing:
– Wave 1: 23 respondents (Nov 2006 - Jan 2007)
– Wave 2: 69 respondents (Apr & May 2007)
– Wave 3: 77 respondents (Jul – Sept 2007)
– Wave 4: 27 respondents (Oct & Nov 2007)
– Wave 5 is under way
• Whole-of-questionnaire testing (Apr – Jun 2008)
Ethnicity focus groups
• Aims:
– explore public opinions surrounding
classifications of ethnic groups
– unpack attitudes towards colour labels, in
particular ‘Black’
– understandings of the term ‘ethnicity’
– inform development of ethnicity question
• 12 focus groups, 109 participants
– Caribbean, African, Any other black
background, mixed background
– London, Birmingham, Manchester, April & May
2007
• Report due soon
Questionnaire design principles
• Navigation & layout
– In West, people start reading at top left, after
that, we guide them with elements of layout
– Essential to have a clear path for eye to follow
through the questionnaire
– Clear numbering
– Consistent even columns & vertical layout
– Text set left and ragged right to provide patterns
for eye to grab hold of
Questionnaire design principles
• Minimise clutter
– Text, symbols, graphics should be kept to a
minimum, as every mark on the page must be
processed by respondent
– Maximise white
space
to make it easier to
140,000
households
read and more appealing
– Keep respondent relaxed
Second Residences & Visitors
Peter Benton
Census – Design Authority
Background
• Information on second residences has not been
collected in any previous census
• Information on visitors was collected in the 1991
Census, but not in a structured way in 2001
• Enumeration base for 2011 is ‘usual residents and
visitors’ – some information will be collected from
everyone at their Census night address.
Second residences consultation
• Over 80 responses received from a range of users
– including DCLG, DfT, Welsh Assembly Government, Local
Authorities, Demographics User Group, Market Research Society
• Uses include:
– inform service provision
• plan health, waste and public transport services
– resource allocation
• provide information on where people are likely to use resources
– understand the housing market
• dynamics of housing market and profiling housing demand
– inform housing affordability and development policies
• impact of second residences on availability of affordable housing
• Category 2 – more work required to develop suitable
questions and assess respondent burden
Second residences in coverage
assessment
• Duplicate returns from different locations:
– e.g student counted at both term-time and parents’
address, people with second residences for the working
week, children of divorced parents
– matching process to search for duplicates
– use second residences information to help matching
process
Second residences in processing of
outputs
• Census estimate of the population using the Mid-
Year Population Estimate base:
– definitional differences between where people are
counted
– use second residences information to estimate where
people spend the majority of their time and adjust for
difference
• Alternative population bases:
– e.g. weekday population, out of term population
Second residences in 2007 postal test
• 2.5% of respondents said they had a second residence
• A further 1% of respondents said they had a second
residence outside the UK
• Of those who said they had a second residence:
– 87% entered the address
• 69% of those entered the full postcode
• 11% of postcodes were half completed
• 20% of postcodes were left blank
• Of those that entered an address, the highest frequency of
location was London (13%), followed by West Sussex (9%)
Second residence question
A number of versions of this
question have been tested.
An additional question on length of
time spent at address was tested,
but respondents found it difficult to
answer.
The current question has proved
successful in testing.
Visitors consultation
• Over 40 responses received from a range of users
– including DCLG, DfT, DfES, Welsh Assembly Government, Local
Authorities
• Uses include:
– inform service provision
• measure the total pressure on services such as transport and health
– resource allocation
• essential for DCLG’s daytime net inflow and foreign visitors indicators
• DCLG still use visitor information from 1991 Census as not
collected in 2001
• Category 1
Visitors in communal establishments
• Visitor information will not be collected from visitors
in communal establishments
– too complex
– could jeopardise the count of usual residents
• This means that there will not be a full visitor count
– only visitors in households
– no persons present output base
• Primary purpose of collecting visitor information is
to get the best usual residents estimate
Visitor information for Quality Assurance
• A sample of visitors will be matched back to their usual
residence
– check whether they were missed where they usually live
• In CCS areas, will also be matched to the CCS in the
households they were visiting
– check whether they were mis-classified
• This will provide a source of data to assess the quality of,
and potentially adjust, the census population estimates
– important when defending the census results
• In non-CCS areas, the mis-classification rate will be used in
the QA
• A full match may be done in certain LAs if the QA suggests
concerns
Visitors in the 2001 Census
• From a sample of 7 Enumeration Areas:
– 9.5% of people recorded as visitors were actually usual
residents
– 1.5% were of no fixed abode and should have been
recorded as usual residents
– 17% of visitors were overseas visitors
– 67% of visitors were from the UK
• 20% of these UK visitors were missed at their usual residence
– 4.5% of visitors did not have an address recorded
• It is estimated there will be around 2.1 million
visitors on Census night in 2011
Visitor questions
In testing, respondents who had visitors generally had a good understanding
of who should be included in this question.
When the visitors questions were located in the household section of the
form, many respondents filled in the questions for usual residents.
Locating the questions on the back page of the form has proved more
successful.
Questions/Comments
• Any questions or comments?
Migration
Chris Smith
Jonathan Smith, Pamela Spicer, Richard Pereira
ONS Centre for Demography
ONS External Consultation; Preliminary Work
• May 2005 ONS published consultation document
‘The 2011 Census: Initial view on content for
England and Wales’
• Following on, ONS published an information paper
‘The 2011 Census: Assessment of initial user
requirements on content for England and Wales’
Key Migration Requirements Noted
• Address One Year Ago
• Country of Birth
• Clear need for additional migration data
Specific Migration Consultation
• In October 2006 a consultation document on
external user needs sent to data users
• 61 responses were received from:
– 12 Central Government Departments
– 34 Local Government
– 10 Academics
– 5 other groups/organisations
October 2006 Consultation: Key Migration Issues
Noted
• Address One Year Ago
• Country of Birth
• Citizenship
• Date of Arrival
Oct 2006 Consultation: Issues
• Address One Year Ago
– Identifies internal migrants and international in-migrants
in previous year
• Country of Birth
– Continuity with previous censuses essential
– To profile the large numbers of migrants from EU
‘Accession’ countries
– House of Commons Library said CoB a ‘poor proxy’ for
Citizenship; increased ‘blurring’ between CoB &
Citizenship.
Oct 2006 Consultation: Issues
• Citizenship
• Has never been included in UK censuses
• Is most easily defined by passport eligibility
• The National Statistics Quality Review (NSQR) on
International Migration Statistics recommended
inclusion of a Citizenship question in future
censuses
Citizenship: Potential Stated Uses
• Defines right of movement, eligibility to vote and
access to employment/welfare benefits
• Collection would allow citizens of other EU
countries to be measured at local level
• UK is one of only three OECD countries not
collecting Census information on citizenship
Oct 2006 Consultation: Issues
• Date of Arrival
• Date of Arrival for those born overseas not included in past
UK censuses
• NSQR on International Migration Statistics (2003)
recommended a question on year of (last) entry;
• Year of entry is included in the LFS, but data at local level
are limited.
Oct 2006 Consultation: Issues (cont.)
• Current CoB question makes it impossible to differentiate
migrants living in UK for many years from recent arrivals
• Date of Arrival for those born overseas would allow
reconciliation of stock/flow estimates of migration
• For those who arrived in UK in previous year, month of
arrival & intended length of stay should be collected
• Inter-Departmental Taskforce on Migration highlighted
importance of accurate data on short-term migrants for
local areas
Task Force on Migration Statistics
• Set up by National Statistician to improve
migration estimates
• Necessary due to current importance of migration
in population change
• Key recommendations in December 2006
included:
– More information about migrants in the country
– Use linkage for better information
– Provide better indicators of migrant numbers
– More coherent reporting
Census 2011: ONSCD Requirements;
• Key Priority is ‘capture’ of the ‘12 month+’ usually
resident population
• Enumeration Base of Usual Residents Plus
Visitors agreed in February 2005
• In the last two years much public discussion of
short-term migration from ‘Accession’ countries
Census 2011: ONSCD Requirements (cont.)
• Short-term migrants not uniformly distributed:
many LAs concerned about insufficient data;
• ONSCD concerned to:
– meet more recent requirements for information
on short-term migration
– ensure Census provides an appropriate base
for mid-year estimation/projection
Specific ONSCD Requirements
Requirements for Flexible Base Estimation
Residence Bases:
– Usually resident population length of stay 12
months+
– Short-term migrants temporarily resident in the
UK intending to stay for:
1-3 months
3-12 months
6-12 months
ONSCD: Required Residence Bases
• Priority is headcount of the usually resident
population (i.e. those people intending to be
resident for 12 months+)
• Headcount estimates required of short-term
migrants from the Census
ONSCD: Required Residence Bases (cont.)
• Year/month of arrival and address 12 months ago
only provide information on length of stay up to
census day
• Intended total length of stay would allow
distinctions to be drawn
• Intended length of stay questions are currently
being developed
• Implication = all international visitors > 1 month
complete all individual questions
Migration – date of entry
This question was based upon the question used in the
Labour Force Survey (LFS).
Generally respondents were able to answer this question
well and many were quite certain that the date written was
correct.
Migration - citizenship
This question was based upon the question used in the
General Household Survey (GHS).
Respondents during testing showed a general
understanding of the concept of passport entitlement.
Migration – intention to stay
Respondents certain of their circumstances or
visa entitlements generally found this question
easy to answer
Migration – intention to stay
This question has since been revised during
testing to take into account the 1 month usual
residence rule; it is therefore still being tested.
National Identity
Hamish MacPherson
Ethnicity, Identity & Inequalities
Background
• The national identity question has never been asked in a
census and was first introduced as part of the 2001 Labour
Force Survey (LFS)
 Numerous groups expressed dissatisfaction with the ethnic
group question in the 2001 Census as it did not provide a
tick-box for respondents to identify themselves as ‘Welsh’,
‘Scottish’ or ‘English’ but only allowed for ‘British’
 As a result of campaigning, ONS have given a public
commitment that National Identity will be asked in the 2011
Census
Salience of sub-British national identities
varies between places of residence
Majority of most ethnic minority groups identify
with one or more GB national identity
Proportion who consider their identity to be British, English, Scottish or
Welsh: by ethnic group, 2004, GB
Ethnic Group, National Identity, Religion
and Language Consultation
• 2011 Census Stakeholders Consultation 2006/07:
Ethnic Group, National Identity, Religion and
Language (EILR), England and Wales
• Provided a detailed picture of user needs to help
decide which questions should be asked in the
2011 Census.
• 606 responses to the consultation.
–
–
–
–
central and devolved government
expert, community and special interest groups
Local and regional government
local Service providers
User need
Majority of respondents to the 2006/07 Census
Stakeholders Consultation stated they needed a
national identity question:
• 58 per cent of respondents in England and Wales
• 75 per cent of respondents in Wales
Meets user requirements by:
• Allowing previously unavailable responses
• Improving public and political acceptance of the
•
•
•
•
questionnaire
Encouraging response rates and improving quality
of information from the ethnic group question
Providing a better understanding of local
populations and communities
Providing a measure of community cohesion
Measuring how people from different areas of the
UK perceive their national identity
Users require the outputs to effect a
variety of outcomes
• Grant allocation by central government
• Local resource allocation and service provision
• Meeting statutory obligations under legislation on
race relations and equal opportunities
• Informing policy development and monitoring
• Enabling monitoring and regulatory bodies to
function effectively
• Product marketing and delivery for private
organisations
Cognitive testing
• Tendency to interpret ‘national identity’ in terms of
legal status, often referring to passports and
nationality
• Others interpreted “what do you consider your
national identity to be” more subjectively referring
to how they feel, language, birthplace, residence
• Some overlap with ethnicity in how people
understand concept
Combining national identity and ethnicity
would not work in England and Wales
A single ethnic group question which included
concepts of national identity would not work in
England and Wales:
• Operational constraints
• Methodology
• Clarity
• Acceptability
• Analysis
National Identity – version 1
• Tested in cognitive
testing wave 1-3
question.
• Based on LFS
question
• Some respondents
do not appear to
notice the instruction
‘Tick all boxes that
apply’
National Identity – wave 3 conclusions
• Ideas and concepts of national identity centre largely
around legal status and country of birth, with some
respondents expanding this idea to include affiliation and
culture also.
– “Proud of being Irish, where you’re born and raised.”
– “What I perceive to be my nation, what I affiliate myself
with.”
• Evidence suggests respondents are reading all answer
options and are more likely to tick a specific nation i.e.
English, Scottish, Welsh or Northern Irish instead of British
despite explicitly expressing their ‘Britishness’.
– “I do often consider writing British but because the
option for Welsh was there it was easy because I would
consider myself Welsh before British and probably not
really perceive being British as a national identity.”
National Identity – version 2
•
Wording changed to
emphasise affiliation
•
“Tick all that apply”
instruction removed
due to space
limitations, assuming
respondents will infer
this from new wording
National Identity – version 3
Wording made clearer
Outstanding issues to
improve question clarity:
• Alternative wording:
“What do you consider
your national identity to
be?” allows room for a
“Tick all that apply”
instruction
• Examples for the “Other”
may assist respondents
• Ordering of “British” tick
box
Income
Elizabeth McLaren
Census – Questionnaire Design
Income
• Census topic consultation
• Topic-specific consultation
• Question testing prior to 2007 Test
• Findings from 2007 Test
• Findings from Test Evaluation Survey
• Findings prior to 2001
• Why not household income?
Census topic consultation
• Over 100 responses (2005)
• Wide range of potential uses
–
–
–
–
Measure of deprivation
Indicator of social exclusion
Estimation of local market sizes
Measure of affluence
• User need score – category 1
• ONS category 2
– More work required to develop suitable question to collect accurate
information & assess public acceptability
• Consultation on priorities (2007)
– More demand for other 4th page topics eg carers
Topic-specific consultation
• November 2005
• 30 responses
• Preference for:
- gross income
- closed question (7-10 bands)
- individual income to be collected
• Sources of income not supported
• Users supported inclusion of income – but not at
expense of poor response rate or poor quality
answers
Income question testing prior to 2007 Test
• Pre 2007 cognitive testing
• Banded vs exact income
• Gross vs net income
• Individual vs household
• Current vs time period
• Sources of income
Findings from 2007 Census Test
• 2.7% significant difference in response rates
•
•
•
•
•
•
between income and non-income areas
Note: Estimated £2 million cost for follow-up per
1% drop in response rate
Non-response to level of income high (9%)
Non-response higher for unemployed, ethnic
minorities, less educated
No sig. adverse public reaction notified through
contact centre or field staff
No sig. difference in requests for individual forms
No sig. difference in household under-coverage
2007 Census Test – key constraints
Key constraints affecting income evaluation:
• Voluntary
– relied on public’s good will to complete a return
– 46.1% response rate
• Publicity and media coverage
– limited
Findings from Test Evaluation Survey
• 16% refused to answer income questions
• 15% unhappy about answering 1 or more questions
–
–
–
–
58% unhappy about income level
48% unhappy about income sources
9% unhappy about ethnicity
2% unhappy about qualifications
• Comments from respondents
–
–
–
–
“should be left to Revenue, nothing to do with ONS”
“invasion of privacy”
“irrelevant to concept of Census”
“intrusive”
• 64% retest reliability for income level
• 77% retest reliability for sources of income
Findings prior to 2001
• Cognitive interviews 1995-97
• 1997 Test
– 2.8% difference in response rates but not statistically
significant
– 22% item non-response for income
• 1999 Rehearsal
Why not household income?
• Public acceptability
– Respondents unhappy about answering on behalf of
others
• Quality of data
– Respondents not including everyone in household
– Respondents don’t know income of others
– Household income better derived from individual
incomes
• Limited space on housing page
• International experience
– Other countries collect individual income
Questions/Comments
• Any questions or comments?
UK views on content
Elizabeth McLaren
UK views on content
• No final decisions on content from any country
• Final consultations taking place
• Each country subject to views of respective
parliaments/Ministers
• Further question testing to be carried out eg intended
length of stay
Possible UK differences for 4 page content
Topic (new)
England Scotland NI
& Wales
Second residences
Intended length of stay


National identity

*
Month/year of entry into UK





Month/year of entry for those who
have lived outside NI for continuous
period of 1 year or more
Nature of disability/illness
Voluntary work
*partially
incorporated within ethnicity question



Possible UK differences for 4 page content
Topic (2001)
England Scotland NI
& Wales

Hours worked
Part-time/full-time work


Study place address

Transport to study place
address
Religion brought up in


Housing questions
• General UK agreement
• GROS considering a household income question
• NISRA considering accessibility for disabled
question
• ONS considering number of bedrooms question
Priorities for content
Glen Watson
Table exercise
• Prioritise questions into low vs medium vs high
priority
• Construct questionnaire for four pages
• Construct questionnaire for three pages
• Feedback views to group
–
–
–
–
Where agreement on priority for topics
Where disagreement on priority for topics
What has been fitted on three pages of questions
What has been fitted on four pages of questions
Way forward
Glen Watson
Timetable
Decision on funding for fourth page
March 2008
Topics for Census agreed by RGs
April 2008
Cognitive question testing
April – June 2008
Postal Test and Omnibus Survey
April – June 2008
Census Advisory groups
April/May 2008
White Paper published with topics
Autumn 2008
Questionnaires for 2009 Rehearsal agreed
Autumn 2008
Census Advisory groups / roadshows
Autumn 2008
Census Rehearsal
Autumn 2009
Questionnaires for 2011 Census agreed
Autumn 2009
Census questionnaires approved by Parliament
Spring 2010
Census day
Spring 2011
Contact details
censustopics@ons.gsi.gov.uk
Census Questionnaire Design and Content
Office for National Statistics
Segensworth Rd
Titchfield
Hampshire
PO15 5RR