The implementation of
tools to support the data
quality of the survey frame
Mario Ménard
November 2008
Outline
 Data sources to populate and
maintain the survey frame
 Quality Assurance Survey
 Interceptor Module
 Survey Frame Assessment Tool
2
Statistics Canada • Statistique Canada
2016-03-18
Data Sources used to maintain the frame
• Legal name, business address,
main activity description,
geographical location, size
measures etc.
Canadian
Revenue
Agency
(CRA)
• 99% of businesses have only
one operating entity
• Identification of inactive
businesses
NAICS
Coding
• Data collection centers transmit
survey feedback to the Register
• Interception of legal name,
business status and NAICS
changes
Central
Survey
Frame
Business
Register
• 400 000 new businesses each year
• 45 % are coded automatically based
on the description provided at the time
of registration
• # of misclassified • 55 % requires manual intervention
businesses
3
Statistics Canada • Statistique Canada
Survey
Feedback
• Review the validity of
the survey feedback
• Updates business
structure and its
characteristics
Profilers
and Frame
Analysts
• Review the updates performed
on business structures and its
characteristics
2016-03-18
Quality Assurance Survey
 QAS measures the number of inactive businesses and the
quality of NAICS coding for units on the BR
 Monthly sample of 600 businesses
• Allows small constant surveys conducted by a few
individuals
• Reduces QAS interviewer re-training and ensures survey
consistency from month to month
• Provides timely, relevant measurements
• Gives an indication of the inactive units and coding quality
over time
 Units eligible for selection exclude very large and complex
businesses and recently surveyed units
4
Statistics Canada • Statistique Canada
2016-03-18
Quality Assurance Survey Results
from June 2007 - July 2008
% Death
Rate
% NAICS 2
Digit Error Rate
% Error Rate by
Revenue
% Volatility
Rate
All sectors
18.5
13.2
1.3
2.5
11 Agriculture, Forestry, Fishing & Hunting
14.8
8.2
0.5
5.2
19
23
3.0
6.5
22 Utilities
14.5
7.6
0.0
-
23 Construction
17.4
7.3
0.9
0.1
16
20.2
0.3
2.2
41 Wholesale Trade
21.8
24.2
0.3
5.7
44-45 Retail Trade
16.4
8.3
0.1
2.1
48-49 Transportation & Warehousing
21.6
12.8
4.7
5
51 Information & Cultural Industries
22.7
13. 6
1.5
3.1
52 Finances & Insurance
19.2
22.7
2.8
2.2
53 Real Estate, Rental & Leasing
24.9
16.6
1.9
2.2
54 Professional, Scientific and Technical Services
18.4
10.5
1.0
0.1
23
33.5
8.5
3.8
17.3
29.7
8.9
5.5
16
16.9
1.4
1.6
62 Health Care & Social Assistance
16.1
3.5
6.2
-
71 Arts, Entertainment & Recreation
20.2
12
14.2
2.6
72 Accommodation & Food Services
17.9
3.5
0.0
0.1
81 Other Services except Public Administration
18.8
9.9
4.8
1.7
Sectors
21 Mining, Oil and Gas Extraction
31-33 Manufacturing
55 Management of Companies and Enterprises
56 Administrative & Support
61 Educational Services
5
Statistics Canada • Statistique Canada
2016-03-18
% Death
Rate
% NAICS 2
Digit Error
Rate
% Error
Rate by
Revenue
%
Volatility
Rate
1.3
2.5
Quality
Assurance Survey Results
Sectors
from June 2007 - July 2008
All sectors
18.5
% NAICS 2
Digit Error Rate
% Error Rate by
Revenue
% Volatility
Rate
All sectors
18.5
13.2
1.3
2.5
11 Agriculture, Forestry, Fishing & Hunting
14.8
8.2
0.5
5.2
19
23
3.0
6.5
22 Utilities
14.5
7.6
0.0
-
23 Construction
17.4
7.3
0.9
0.1
16
20.2
0.3
2.2
41 Wholesale Trade
21.8
24.2
0.3
5.7
44-45 Retail Trade
16.4
8.3
0.1
2.1
48-49 Transportation & Warehousing
21.6
12.8
4.7
5
51 Information & Cultural Industries
22.7
13. 6
1.5
3.1
52 Finances & Insurance
19.2
22.7
2.8
2.2
53 Real Estate, Rental & Leasing
24.9
16.6
1.9
2.2
54 Professional, Scientific and Technical Services
18.4
10.5
1.0
0.1
23
33.5
8.5
3.8
17.3
29.7
8.9
5.5
16
16.9
1.4
1.6
62 Health Care & Social Assistance
16.1
3.5
6.2
-
71 Arts, Entertainment & Recreation
20.2
12
14.2
2.6
72 Accommodation & Food Services
17.9
3.5
0.0
0.1
81 Other Services except Public Administration
18.8
9.9
4.8
1.7
Sectors
21 Mining, Oil and Gas Extraction
31-33 Manufacturing
55 Management of Companies and Enterprises
56 Administrative & Support
61 Educational Services
6
13.2
% Death
Rate
Statistics Canada • Statistique Canada
2016-03-18
Interceptor Module
 Intercepts requests for updates received from the data
collection centres
 The following events are intercepted:
• Classification changes
• Legal name changes
• Business Status changes
 Survey program managers have the choice to decide if they
want to use this module
 After 35 days in the interceptor module, requests for updates
are processed automatically to avoid the accumulation of
events
7
Statistics Canada • Statistique Canada
2016-03-18
Number and status of update requests
received in the Interceptor Module
From January 2008 to September 2008
# of
update
requests
Total # of
update requests
8
# of
update
requests
cancelled
% of
update
requests
cancelled
# of
update
requests
accepted
% of
accepted
requests
13 500
4 900
36 %
8 600
64 %
Services Surveys
7 797
1 941
25 %
5 856
75 %
Distributive Trade
Surveys
1 571
899
57 %
Manufacturing
Surveys
1 533
919
60 %
614
40 %
Others
2 599
1 141
44 %
1 458
56 %
Statistics Canada • Statistique Canada
672
43 %
2016-03-18
Survey Frame Assessment Tool
 Indicates changes on the survey frame from one period to the
other
 Easily accesses the details of the changes
 Analysis performed just before the release of the monthly
Generic Survey Universe File
 Identifies the largest contributors to the change from one
period to the another
 Measures the quality of work being performed by the people
making updates to the frame
 Provides constructive feedback and a good indication of
additional training that may be required
9
Statistics Canada • Statistique Canada
2016-03-18
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by
type of updates between the June 1st and July 1st G-SUF, 2008
# of additional
establishments
Type of updates
Total
revenue
(000,000)
New establishments:
1) From profiling activities
349
$ 9 400
2) From assigning a NAICS code
11 800
$ 5 699
3) From obtaining a size measure from
administrative source
11 329
$ 6 480
1 100
$ 5 600
5) From the Quality Assurance Survey
125
$ 10
6) From a new description received from CRA
270
$ 76
24 973
$ 27 265
NAICS Changes to existing establishment:
4) From profiling activities
Total
10
Statistics Canada • Statistique Canada
2016-03-18
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by
type of updates between the June 1st and July 1st G-SUF, 2008
# of additional
establishments
Type of updates
Total
revenue
(000,000)
New establishments:
1) From profiling activities
349
$ 9 400
2) From assigning a NAICS code
11 800
$ 5 699
3) From obtaining a size measure from
administrative source
11 329
$ 6 480
1 100
$ 5 600
5) From the Quality Assurance Survey
125
$ 10
6) From a new description received from CRA
270
$ 76
24 973
$ 27 265
NAICS Changes to existing establishment:
4) From profiling activities
Total
11
Statistics Canada • Statistique Canada
2016-03-18
Survey Frame Assessment Tool
Number of additional establishments and their revenue contribution by
type of updates between the June 1st and July 1st G-SUF, 2008
# of additional
establishments
Type of updates
Total
revenue
(000,000)
New establishments:
1) From profiling activities
349
$ 9 400
2) From assigning a NAICS code
11 800
$ 5 699
3) From obtaining a size measure from
administrative source
11 329
$ 6 480
1 100
$ 5 600
5) From the Quality Assurance Survey
125
$ 10
6) From a new description received from CRA
270
$ 76
24 973
$ 27 265
NAICS Changes to existing establishment:
4) From profiling activities
Total
12
Statistics Canada • Statistique Canada
2016-03-18
Survey Frame Assessment Tool
Example of the detailed regarding each update that contributed to the
changes in the GSUF from June 1st and July 1st G-SUF, 2008
Operating
entity
NAICS
Before
S76921157
Revenue
Size
Changes
User ID
Division
Update
Source
444190
1 679 790 761
Birth – New
NAICS
stlogae
BRD
On-line Profiling
S77093355
413190
413190
1 512 330 738
Birth - New
Establishment
Stat flags
menamar
BRD
On-line Profiling
S75628386
412110
419120
1 277 462 141
NAICS Change
corejoa
IOFD
On-line Profiling
S77021577
336310
336310
689 831 265
Birth – Size
measure
External
BRD
Activation CRA
336310
545 678 900
Birth – NAICS
lemaroc
BRD
On –line NAICS
S66586043
13
NAICS
After
Statistics Canada • Statistique Canada
2016-03-18
Thank you
Questions
14
Statistics Canada • Statistique Canada
2016-03-18