Implementing Selective Editing
and Its Reflection on the Data Quality
(Business Survey 2012)
Authors :
Azza Omer
Mohammed Aleghefli
Sara Al Dhaheri
Copyright © 2013 Statistics Centre - Abu Dhabi
Content
 Introduction to Abu Dhabi & UAE
 SCAD Business Surveys
 AES Editing Process
 Re-designing the Editing Strategy
 Selective Editing – Definition
 Selective Editing used in AES 2012
 Process of Selective Editing
 Edit Reports
 Findings
www.scad.ae
2
Introduction
The Emirate of Abu Dhabi
• Almost 43 years since the formation of the UAE.
• UAE comprises Abu Dhabi plus six other Emirates.
• Abu Dhabi accounts for over 80 percent of UAE land mass.
• Abu Dhabi contains around one third of UAE population.
• Emirati citizens make up around 20 per cent of Abu Dhabi population.
• Abu Dhabi oil represents around 10 per cent of global reserves.
www.scad.ae
3
Introduction
Statistics Centre – Abu Dhabi
•
SCAD is the official source of statistical data for Abu Dhabi.
•
Celebrated five years of operation in April 2014.
•
Operates in accordance with international best practice to provide
range of statistics.
Our Mission:
1. Develop and organize a statistical system in the Emirate.
2. Provide relevant and reliable official statistics serving policy makers,
the business community and the public.
www.scad.ae
4
SCAD Business Surveys - 2013
• SCAD Business Surveys – known as Annual Economic Surveys
(AES) key data source for the compilation of economic statistics .
• The survey results enable the construction of key economic
indicators the GDP.
• The suites of Annual Economic Surveys have been conducted since
2007.
• AES 2012 sample size 2200 Establishments
• Collection period 5 months
• Paper questionnaire - commercial accounting
www.scad.ae
5
AES Editing Process
www.scad.ae
6
Re-designing the Editing Strategy
Questions driving the re-design of the editing strategy
• Do we need to re-validate every edit failure?
• How to leverage the automated editing to improve data quality?
www.scad.ae
7
Selective Editing – Definition
• Procedure which targets only some of the micro data items or
records for review - OECD
• Follow-up of validation failure for only those contributors where likely
size of the correction is expected to make a material difference to
the results – NSI
www.scad.ae
8
Selective Editing used in AES 2012
• First time used in SCAD Business Surveys
• Method adopted was thoroughly reviewing, validating and editing
key variables
• Optimized the sources available for editing data
www.scad.ae
9
Process of Selective Editing
6/1/2013
7/1/2013
8/1/2013
9/1/2013
10/1/2013
5/15/2013
Output Team
Data
Processing
Team
Data Collection Team
AES 2012 – Multi-Process Editing
www.scad.ae
10/10/2013
Editing from paper questionnaire
Field
Researchers
Collect
Paper
Questionnare
Auditors Review
Sign off for Entry
Field
Researchers
Daily
Reports
DataBase
Selective
Edit Checks Expert Edits
Escalation
Reports
10
Edit Report I: 18.09.2013
Count OF TOTAL
Count of RELATED
Count of LOGICAL
Edit Failures
Edit failure
Edit Failure
315
17
6
136
TRADE
141
19
0
4
3
TRANSPORTATION
77
10
1
15
4
TELECOM
38
14
0
10
5
REAL ESTATE
22
5
0
15
6
INDUSTRY
339
75
6
73
7
INSURANCE
38
7
0
22
8
FINANCIAL SERVICES
28
8
1
5
9
SME
638
4
0
59
10
HOLDING
11
3
0
2
11
CONSTRUCTION
208
15
0
9
12
BANKING
27
60
0
1
1882
237
14
351
Obs
SECTOR
Records Count
1
SERVICES
2
Total
www.scad.ae
11
Edit Report: 10.10.2013
SECTOR
Obs
Records Count
Count OF TOTAL
SUM OF RELATED
SUM OF LOGICAL
Edit Failures
FRQUENCY
FRQUENCY
1
SERVICES
329
0
0
4
2
TRADE
150
0
2
3
3
TRANSPORTATION
85
0
0
2
4
TELECOM
45
0
0
2
5
REALESTATE
22
0
0
1
6
INDUSTRY
348
0
0
73
7
INSURANCE
39
0
0
0
8
FINANCIAL SERVICES
30
1
0
1
9
SME
639
0
0
0
10
HOLDING
13
0
0
2
11
CONSTRUCTION
223
0
0
0
12
BANKING
32
0
0
0
1955
1
2
88
Total
www.scad.ae
12
Findings
• Editing done in a timely manner
• Significant decrease of key variables edit failure
• Focused automated editing on a predefined set of records improve
the data quality
• Enhancing the method by implementing the scoring techniques
www.scad.ae
13
Contact
For more information please read the paper.
• Website: www.scad.ae
• Authors:
Azza Omer aaomer@scad.ae
Mohamed Aleghefli: msaleghfeli@scad.ae
Sara Aldhaheri: saaldhaheri@scad.ae
www.scad.ae
14
www.scad.ae
15