Integrated Approach
Processing
SNA seminar in the Caribbean
Marie Brodeur
Director General, Industry Statistics Branch, Statistics Canada
St. Lucia
February, 2014
Why A Centralized Process?
 Best Practices
 Standardization of Processes
• Cross Survey Comparisons
• Enterprise Centric Processing/Coherence
Analysis
 Efficient use of Resources
 Transportable Knowledge Across Survey
Programs
2
UES Post-Collection Processing
Records from
Collection
Pre-Grooming
Edit & Imputation
Allocation /
Estimation
3
Tax
Data
Data Service
Center
Business
Register
Subject Matter
Review & Correction
Tool
Collection
 Precontact (Dec-Jan)
– Mostly for Business Register (BR) births; verification of contact
information (name, address, …)
– By phone (in a few cases, a letter or a fact sheet is sent)
 Mail-out of questionnaires (Jan-March)
– 2 or 3 mail-out dates
 Follow-up in case of non-response for some units
(begins about a month after mail-out)
– Phone call, remail or fax
 Mail-back of questionnaires
 Verifications of received questionnaires / Edits
– Is the questionnaire complete or are some key variables
missing? (Edit follow-up by phone in some cases)
4
Centralized Collection
Pre-Contact
Mailout
Receipt
(75% target)
Capture / Imaging
5
Prioritize
Edit / Verification
“Clean”
Records
Delinquent
Follow-Up
Use Of Tax Data
 Validation (comparison)
 Verify dubious collected data against the
equivalent tax data record
 Imputation
 One of the methods used for non-response
 Estimation
 Direct Data Replacement
 Calibration Estimates
 Update Business Register
 Allocation of survey data (use tax revenues, salaries
and expenses)
Centralized Processing Systems
And Databases
 Develop centralized systems
• Move away from stand-alone
• Single point of access for security
 Integrated Questionnaire Metadata System
 Edit and imputation
 Allocation and Estimation
 Data Warehouse
Enterprise Portfolio Managers
 Top 350 enterprises in Canada
 Status
• Platinum, Gold, Silver, Bronze
 Personal visits
 Enterprise Profiling
 Coordination of mail-out and collection
 Enterprise/ Establishment coherence
 Holistic Response Management
• Strategic Response Unit
• Escalation Process / Statistics Act
8
What Is E & I?
 Editing
• Verify that parts add-up to total
• Ensure that there are no missing values where parts
add up to total
• There must be consistency between related
variables
 Imputation
• Changing values in fields which fail edit rules with a
view to ensuring that the resulting data satisfy all
edit rules. In practice, reported data will rarely be
changed
• Impute for missing data or partially responded data
• Impute entire records in the case of total nonresponse
9
Why Is E&I Necessary?
 To produce a complete and consistent data file
that accounts for all sampled units
 Both units that did not respond to the survey
must be imputed and units that did not provide
a complete response must be imputed
 Correct erroneous responses
10
E&I Terminology
 Data Group
• Groupings (defined by SM) of records that will be
kept together for imputation purposes
• These groupings are based on multi dimensions:
 industry (NAICS)
 geography (province)
 Data groups that will be used for a specific survey will
depend on:
• initial sample design (number of units sampled and
the level of stratification used)
• number of records that respond to the survey (a
minimum of 5 or 10 records are required)
11
BANFF E & I System
 Impute for missing key variables as
specified by subject matter (i.e. total
revenue, total expenses)
 Impute for other missing variables:
• Apply Historical Trend
• Apply Current Year Trend
• Use donor (for partial imputation)
12
BANFF Algorithms
 DIFTREND - Historical trend imputation
 CURRATIO - Current ratio imputation
 PREVALUE – Value from the previous period for
the same unit is imputed
 PREAUX – Historical value of a proxy variable for
the same unit
 CURAUX – Current value of a proxy variable for the
same unit
13
Allocation - Definition & Purpose
Definition:
 Allocation is the distribution of survey and administrative
data from their acquisition level (Collection Entity) to the
targeted statistical units (Establishments or Locations)
as defined on the survey frame.
Purpose:
 To provide fully-processed micro data on a fiscal year
basis, for establishments or locations in-sample for the
UES
 Determine the distribution of value added by province
14
Sample Survey Allocation
SAMPLE
Collection/
Processing
Allocation
Establishment 1
Establishment 1
Establishment 2
Establishment 2
Questionnaire 1
Establishment 3
Establishment 3
Establishment U
Establishment 4
15
Questionnaire 2
Establishment 4
Overview of the IBSP Rolling Estimates Approach
Sampling
Multi-Mode
Collection
Active Management
Follow-Up
Manual Editing
Rolling
Estimates
Quality Indicators
and Scores
16
Statistics Canada • Statistique Canada
Automated
Processing
Editing
Imputation
Estimation
Interpretation &
Dissemination
2016-07-23
Active Management – Strategy Settings
 A subset of all Key Estimates is selected
 All Key Estimates are:
• Ranked from the most to the least important
• Weighted relatively using an importance factor
• Assigned a Quality Target
 Targets are set in line with the importance factor.
 Active Collection ends for a Key Estimate when the Quality Indicator meets
the Quality Target.
 Active management and sampling strategies are
coherent by design.
17
Statistics Canada • Statistique Canada
2016-07-23
Active Management – Definitions
 Quality Indicator (QI)
• QI= Sampling CV & Imputation CV & Pseudo Relative Bias
 Measure of Impact (MI) Score
• Impact of a unit on the QI for a given estimate
• Units imputed from a poor model or with reported/imputed values
far from their predicted values will have high MIs.
18
Statistics Canada • Statistique Canada
2016-07-23
Empirical Study – RY2011 Prototype
 Parallel run for 47 Business Surveys
 Four Rolling Estimates iterations
 Total CV calculated for all key estimates (8,600) at each iteration
19
Statistics Canada • Statistique Canada
2016-07-23
20
Statistics Canada • Statistique Canada
2016-07-23