Results Generated from the questionnaire
disseminated prior to the workshop
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
The objective of the questionnaire
•
To better understand data processing activities at the country
level
•
To invite country experiences with the goal of providing a
forum for further collaboration on the effective use of
techniques and methods in data processing
•
To support the development and management of the workshop
and future activities
•
To understand what information and technical training is
needed on the use of specific daata processing methods
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Capture:
Methods for census/survey data capture
•
Common methods used for census survey data capture
were:
• manual data entry
• OMR
• ORC/ICR
•
Several countries are interested advancing efficiency
through the use of PDA’s and have expressed that they
plan to use the for the next census round
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Capture:
Scanners and features used by countries:
•
Fujitsu w/Twain software driver (Egypt)
•
OPScan7 & OPScan10: 500 sheets/hr w/validation OR
5000 sheet per/hr no validation used with scantools
software (Kuwait)
•
Kodak ds Digital Science Scanner 3520 40 to 85 ppm
(depending on resolution, orientation, feeding, etc.)
(Morocco)
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Capture: outsourcing of processes
•
With concern to manual data entry, the data capture
process is not outsourced.
•
•
Methods included the use of a database management
system such as Oracle along with CSPro where data
entered, edited and coded in-house.
With concern to OMR & OCR/ICR the data capture process
is often outsourced.
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Capture:
Planned data capture method for next
census round
•
Some countries are undecided as of which method to
choose
•
OMR/OCR/ICR is planned for use and outsourced
(e.g. Iraq)
•
PDA’s are also proposed for use (e.g. Kuwait,
Palestine, Qatar)
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Capture:
Archiving methods and policies used for
storing forms
•
Many countries use electronic means for the storage of
forms. Some countries store forms both electronically
and in hardcopy format.
•
Several countries have laws requiring the storage of
forms for a given time ranging from ~10 to ~60 years.
•
Issues raised in the storage of hardcopy forms are that
they take up space and may be damaged after a given
time period
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Editing:
Coding for Major Classifications of
Occupations
•
All offices use coding for major classifications of
occupations, industry and education.
•
•
•
•
Occupation- most use ISCO with several countries
using nationally specific systems also
Industry- most use ISIC with several countries
using nationally specific systems also
Education- Most countries use ISCED with several
countries using nationally specific systems also
Nationality was also mentioned as a major
classification in which coding is used.
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Editing: Manual or Automated Coding
•
Coding is done manually in most cases with some countries
using both manual and automated methods.
•
•
When automated, the software is developed in house (e.g.
Egypt) or through a commercial produced such as Oracle
(Lebanon) or developed by a private contractor and
configured further by NSO staff (e.g. Morocco)
All countries have an editing system as a part of the
census/survey processing
•
The dominant error detection systems expressed within
the questionnaire were
• validity check
• consistency check
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Data Editing:
•
•
Editing consists of two main stages (Iraq)
•
Stage 1: during data entry sessions validation rules are
used within the data entry program;
•
Stage 2: after data entry process completed using
additional validation rules by different processes like
merging files in order to find out & resolve outliers
In many cases manual methods for imputation are used with
the following software CSPro, IMPS, SPSS, Oracle. Countries
create automated routines using statistical software tools such
as SPSS and STATA and batch editing programs attached with
the data entry program (CSPro batch editing tool).
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Staff and Training
Syria
Process:
Full-time: Part-time:
Data capture
350
25,000
Data coding
200
500
Error detection 20
20
Imputation
200
1,000
Qatar
Process:
Part-time:
Data capture
100
Data coding
120
Error detection 120
Imputation
120
Palestine
Process:
Full-time:
Data capture 100
Data coding
120
Error detection 200
Imputation
10
Lebanon
Process:
Full-time:
Data capture 700-800
Data coding
50
Error detection 50
Imputation
-
Jordan
Process:
Full-time: Part-time: Ad- hoc:
Data capture
350
16,000
700
Data coding
25
155
20
Error detection
10
80
5
Kuwait
Process:
Full-time:
Data capture
Data coding
Error detection 10
Part-time:
100
30
30
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Cont…
•
Training for each step ranges widely across countries
•
Data Capture ~5 days to 2 months
•
Data Coding ~5 days to 2 months
•
Error Detection ~5 days to 2 months
•
Imputation ~ 1 Day
Example: Manual
Data capture - 1 week
Data coding - 3 days
Error detection - 3 days
Imputation - 1 day
Example: OMR
Example: PDA
Data capture - 12 days Data capture - 14 days
(for Buildings & Housing Units
Data coding - 5 days
Census) & 14 days (for
Error detection - 5 days Population Census)
Data coding - 7 days
Error detection - 7days
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
Quality control procedure country examples in
relation to the various steps of data processing:
•
•
•
•
•
Egypt:
•
Training is given to data capture/data entry staff to help better
understand questionnaire and fieldwork approaches
Jordan:
•
Design special data entry programs for error detection; tables of
frequencies for reviewing results, their consistency and accuracy
Lebanon:
•
Specially trained team for verification and performance tests with the
objective of establishing quality standards of a desired level for each
operation (i.e. manual editing, coding, data entry and computer editing)
Palestine:
•
Performing checks which are added to editing applications according to
the feedback of specific subject matter departments
Qatar:
•
Double keying; verification of data keyed from paper are some of the
quality control procedures implemented
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008
END
Thank You
Workshop on International Standards, Contemporary Technologies and Regional Cooperation,
Noumea, New Caledonia, 04–08 February 2008