Harmonisation: Understanding
user experiences to maximise
the value of statistics
Rachel Leeser
Senior Research and Statistical Analyst - Social Exclusion Intelligence,
Greater London Authority
rachel.leeser@london.gov.uk
Emma Emery
User Engagement Manager, Royal Statistical Society (UK Stats Authority
secondment) e.emery@rss.org.uk
What we’re going to talk about…
• Reminder of Code of Practice & link between
harmonisation, user engagement and maximising value
of stats
• Examples of current harmonisation issues
• Rachel’s experience using income & earnings statistics
• Consideration of impact of lack of harmonisation/info
about harmonisation
• What the GSS should be doing & examples of GSS
harmonisation work
Harmonisation in 2015 EU peer review
report - recommendation
15.The United Kingdom Statistics Authority and the
National Statistician as Head of the Government
Statistical Service should strengthen their efforts to
harmonise United Kingdom statistics in order to
ensure coherence and comparability in the
production of official statistics within the United
Kingdom, over time and among producers of
statistics. (European statistics Code of Practice,
Principles 10, 14 and 15, coordination.)
For full details see p24 of the report here:
http://ec.europa.eu/eurostat/documents/64157/4372828/2015-UKreport/d44f7d3f-64c1-4450-8a37-bfadb8542607
Harmonisation: Relevant aspects of the
UK Code of Practice for Official Statistics
(1)
‘Promote comparability within the UK and
internationally by, for example, adopting
common standards, concepts, sampling frames,
questions, definitions, statistical units and
classifications (including common geographic
referencing and coding standards). Make the
reasons for any deviations from standard
models publicly available.’
Harmonisation: Relevant aspects of the
UK Code of Practice for Official Statistics
(2)
‘Ensure that official statistics are disseminated in forms that
enable and encourage analysis and re-use. Release
datasets and reference databases, supported by
documentation, in formats that are convenient to users.’
‘Seek to balance quality (for example, accuracy and
timeliness) against costs (including both costs to
government and data suppliers), taking into account the
expected uses of the statistics.’
• This translates into a fair bit of work for statistics producer teams –
some more than others - but is important for ensuring that statistics
are as useful and valuable as possible
• Ongoing communication with statistics users essential in truly
adhering to these aspects of the Code of Practice
Harmonisation in assessment
reports - example
NI Housing and Benefits stats. User feedback reported:
‘Concerns raised by housing reports users included uncertainty about
the definitions of some classifications, a lack of coherence between
specific statistics and those from other sources and a lack of
explanation behind the data.’
‘The availability of local data and the coherence of benefits statistics
with those from DWP were cited as being particularly useful.’
Recommendations:
(Sugg) ‘Provide more information about the availability of
comparable housing statistics for England, Scotland,
Wales and the Republic of Ireland.’
(Sugg) ‘Work with DWP to provide comparisons and
analysis of benefits statistics in Northern Ireland and in
GB that will aid user interpretation.’
Other examples of lack of harmonisation
affecting users…(1)
Some Census questions differ between UK countries – e.g
religion, resulting in questions over the impact on the
data - is it why Scotland measured a substantially lower
proportion of Christians & a substantially higher
proportion of ‘no religion’ than E&W census?:
Other examples of lack of harmonisation
affecting users…(2)
• Other Census issues – different age bands used by UK
countries & timeliness varies. Analysis by age bands for
whole of UK difficult.
• NHS health data e.g. waiting times not comparable
between UK countries
• Differences between SOC2000 & SOC2010 causing
break in time-series – can mean longitudinal series
impossible e.g. Certificate of Sponsorship data
To harmonise or not to harmonise
Income, Earnings, Deprivation & Poverty statistics - a user’s perspective
• Deprivation, Earnings, Income, Poverty
– Nebulous concepts
– Constantly evolving
– Variation between areas/commmunities
• Accuracy
• What would harmonisation look like?
Earnings – what counts?
• “Basic”, “Total”, “Net”, “Gross”, “Average”
–
–
–
–
–
What to include?
Overtime
Bonus/honorarium
Tips
In kind payments/benefits – accommodation, meals, pension contributions,
discounts, perks
– Deductions – travel, uniforms
– Variable contract or zero hours
– Self-employed
• Hourly, Weekly, Annual etc
• Sick pay, leave, bank holidays
• Main job, additional jobs
Earnings – Sources
• ASHE Annual Survey of Hours and
Earnings
– Employer survey of jobs, not people
• LFS – Labour Force Survey
– (APS) self-reported hours and earnings
• FRS – Family Resources Survey
– (HBAI) self reported weekly
•
•
•
•
EHS – English Housing Survey
WAS – Wealth and Assets Survey
STES – Short Term Earnings Survey
Monthly Wages and Salaries Survey
– (AWE) employer reports, best for bonuses
Earnings – why do we want to know?
• Comparative analysis, eg
– Equalities
– Regional
– Occupation, industry
• Proxy
• Spending potential
– with other (inferred/imputed?) income
• Living Wage
Earnings – deriving meaning
• Co-variables
– geography
– full-time/part-time, self-employed
– ………….
• Household size/characteristics
• ??costs directly associated with
working
– travel
– childcare
Deprivation
• No direct measure of something we can
call “Deprivation”
• Indicators
– Variation over time and space
• Indices
– CLG/Wales/Scotland/NI
– Census
– Harmonise domains?
• Flexibility rather than harmonisation
Income
• Sources
FRS, Small Area Estimates, WAS, GDHI, Effects of taxes and Benefits,
LCF, Commercial datasets……………..
• Gross or net
– Net of what?
– Housing Costs
• Banded – equal or variable
– Ability to relate to household members (equivalisation)
– Ability to relate to expenditure
• Particularly housing affordability
• Fuel costs etc
• Transfer payments
• Reporting – HBAI reports for people in households,
ONS small area estimates report for households
Poverty
• Wealth or income?
• Relative or “absolute”
– 60% of median of what?
• Equivalisation scales
• Disability (this applies elsewhere too!)
• Distinction between poverty and
deprivation?
– Child poverty measures in particular
To harmonise or not to harmonise
- some questions to ponder
• What are the merits of:
– flexibility vs consistency?
– transparency and detail vs easy?
– change vs continuity
• Local, national or international comparability?
Do they have to be either/or?
So… potential impact of lack of
harmonisation/info about harmonisation:
• Confusion for users of statistics
• Difficult or impossible to answer research questions
• Time consuming for users to identify the differences &
consider the implication for use
• Perception that official statistics are complicated or
confusing can lead to lack of trust
• Statistics/datasets don’t get used as much as they might
be if greater harmonisation – therefore, not as valuable,
or having as much impact as they could be
• BUT in some cases steps to increase harmonisation
may not be welcomed by users!
Examples of GSS promoting
harmonisation & responding to users
• ONS responded to user feedback on the 2001 Census and put in
place for 2011 outputs:
 Common table specifications for the key & quick statistics tables (users
previously had to do the table matching themselves)
 Standard bulk delivery file format for UK (users previously had to create a
common format themselves prior to analysis work)
• Personal well-being – ONS developed 4 standard questions &
added them to the Harmonisation Programme Standards to
encourage use by other research organisations. But EU-SILC uses
different questions, meaning comparisons with other countries not
straightforward.
• Disability – harmonised survey questions created following GSS
project involving stakeholders. More info:
http://www.ons.gov.uk/ons/guide-method/harmonisation/primary-setof-harmonised-concepts-and-questions/index.html Will they/are they
being used?
Actions for you….
• Ensure external communication and researching the
uses & impact of statistics are key elements of the
production process
• Encourage increased UK/international harmonisation
through collaboration with other statistics producer
teams
• Consider the impact of methodological changes on
longitudinal series that are valuable for research
• Make use of community sites such as StatsUserNet to
investigate users’ experiences of statistics & inform
balancing of priorities
• Ensure info available about extent of harmonisation &
comparability, and deviation from common standards
Useful references….
•
•
•
•
•
GSS Guidance on assessing & improving comparability:
https://gss.civilservice.gov.uk/statistics/presentation-anddissemination/comparing-official-statistics-across-uk/
GSS Primary set of harmonised concepts and questions:
http://www.ons.gov.uk/ons/guide-method/harmonisation/primary-set-ofharmonised-concepts-and-questions/index.html
National Statistics Harmonisation Group:
https://gss.civilservice.gov.uk/about/governance-and-structure/nationalstatistics-harmonisation-steering-group/
Quality, Methods & Harmonisation Tool:
https://gss.civilservice.gov.uk/statistics/methodology-quality/quality2/quality-resources/quality-methods-and-harmonisation-tool/
GPT spreadsheet guidance:
https://gss.civilservice.gov.uk/blog/2014/11/new-gss-spreadsheet-guidancereleased/
Get in touch!
• ONS harmonisation team for advice on improving
harmonisation of statistical products:
harmonisation@ons.gsi.gov.uk
• Emma Emery or the GSS Good Practice Team about
improving external communication & engagement:
e.emery@rss.org.uk /
goodpracticeteam@statistics.gsi.gov.uk
• Rachel Leeser on user experiences:
Rachel.Leeser@london.gov.uk
• And remember about StatsUserNet!
www.statsusernet.org.uk @StatsUserNet