Results of the questionnaire
on Census Data Processing
(A total of
9 questionnaires were received)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Fathia ABDELFADIL
UN-ESCWA
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Part B:DATA CAPTURE
Question 1: Does your office use any of the
following methods for census/survey data capture?
– Manual data entry
– Optical Mark Recognition (OMR)
– Optical Character Recognition/Intelligent Character
Recognition (OCR/ICR)
– Personal Digital Assistant (PDA)
– Internet
– Other
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
 9 countries replied.
- 2 countries : Manual data entry & other
( -Syria: manual using CSPro
- Iraq: manual + traditional system analysis &
programming )
- 3 countries : Manual data entry and PDA
(- Jordan: manual +PDA
- Qatar: manual (planning to use PDA);
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- Lebanon: manual questionnaire with data entry on
oracle)
-1 country : Manual data entry and OMR
( - Kuwait-OMR was used in census 1995)
- 1 country: ICR
(- Egypt)
- 1 country: Manual data entry and internet
(- Palestine: manual; CSPro 3.3 for census data
entry/editing; - Oracle for buildings & housing units)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- 1 country- Manual data entry, OMR, OCR/ICR &
PDA
( - Morocco)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 2: If your office uses optical data
capture:
i. what are the features of the scanner in use? (e.g. pages perminute, resolution, type),
 3 countries replied
(- Egypt: Fujustu scanner high speed-dropout
color (16 doullex/mim,
- Kuwait: OPScan 7 & OPScan 10; speed 500
sheets/hr with validation; 5000sheets/hr with
no validation )
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
– Morocco: 40 to 85 page per minutes depending on
resolution, feeding orientation and documents size.
Resolution 200 or 300dpi. Documents input: max 66 x
30.5. Feeding capacity: 250).
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
ii. What kind of scanning software does your office use (e.g.
Softrac, Kodak, Scamax,)?
 3 countries replied
( - Egypt: Use TwAin dirver in application prog,
- Kuwait: Scantools ,
- Morocco: Kodak ds Digital Science Scanner 3520.)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 3: Does your office outsource part or all
of the data capture/editing process?
 7 countries – replied No
 2 country – replied Yes (Egypt, Morocco)
i. If yes, what part of the process is outsourced?
- Egypt: Scanning & Auditing Data processing;
- Morocco: 50% of data operators are temporarily
hired.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
ii. If not, what are the methods your office uses
for data capture?
 6 countries replied
(- Jordan & Kuwait, Qatar & Syria: traditional method,
- Palestine- Manual data entry; PDA as pilot,
- Iraq- Using db system like Dbase, Foxpro, Oracle: Data
entry done in COSIT in addition to decentralized data
entry).
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 4:What data capture methods does your
office plan to use for the next census?
 7 countries replied:
(- Jordan: hasn't decided yet,
- Kuwait, Palestine & Qatar : PDA in 2010 census,
- Iraq: Scanning census questionnaires,
- Morocco: OCR/OMR.
- Syria: Paper questionnaire)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 5: Does your office plan to use optical
data capture for the next census/survey?
 8 countries replied:
- 5 countries: No (Kuwait, Lebanon, Palestine,
Qatar & Syria);
- 2 countries: Yes (Iraq & Morocco);
- 1 country: May be (Jordan)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
i. If so, what technology?
 2 countries replied
- (Iraq: ICR/OMR
- Morocco: OCR/OMR)
ii. If so, will you outsource part or all the process?
Which part(s) will be outsourced?
 1 country replied: (Iraq- all the process)
iii. If in-house, how many staff work on data capture?
 1 Country replied: ( Morocco- 90 fulltime & 70 ad-hoc)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
 Question 6:
What kind of concerns does your
office have with data capture?
 8 countries replied
(- Egypt: Archiving according to application needs,
- Jordan: Planning and decision making; to help
achieving sustainable development,
- Kuwait: speed; accuracy and valid data on time for
dissemination,
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- Iraq: Accuracy & Timing,
- Morocco: Processing census questionnaires
required large investment, which is not fully used once
the operation is completed,
- Palestine: Internet; PDA, Data consistency, Data
quality,
- Qatar: Quality and Timing,
- Syria: Quality.)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 7: What are the archiving
methods/policies used for the storage of paper forms
and electronic forms?
 7 countries replied:
(- Jordan : well equipped areas for all hardcopy
forms which usually damaged 10 years after
census;
- Lebanon: Archiving the questionnaires and the data
in the oracle db;
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- Kuwait: well organized storage, wooden boxed,
serially numbered. E forms kept in db & backups;
- Palestine : E. format (oracle db), Papers as per
decree issued by Council of Ministries;
- Iraq: E. Methods;
- Qatar: Conventional papers storage;
- Morocco: Archiving paper questionnaires for a
period not less than 60 years; questionnaires
processed using OCR/ODMR are stored in hard
drives and DVDs.)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Summary points on Data Capture
- Manual data entry method is used by most of the
countries;
-Optical data capture method is used by 2 countries;
- Software, and good scanners are available in the
countries that use optical data capture method .
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Summary points on Data Capture
(cont’d)
- Outsourcing of data capture and data editing is not
applied in most of the countries.
- Optical data capture for the next census/survey has
not yet been decided by most of the countries,
however few countries mentioned that they are
planning to use ICR/OCR/OMR.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Summary points on Data Capture
(cont’d)
- Some concerns were pointed out by some countries
regarding data capture such as accuracy, quality and
timely dissemination of data.
- Archiving methods/policies are used including
storage of paper forms and e. forms.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Part C:DATA EDITING
Question 8: Does your office use coding for major
classifications of occupations, industry and education?
 9 countries replied Yes, as follow:
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Country
Egypt
Jorda Kuwait* Leban Morocco Palesti Qatar
n
on
ne
Syria
Iraq
nationall
y
specific
system
nationall
y
specific
system
Nation
al &
Intern
ational
Nation
al &
Intern
ational
ISCED nationall
y
specific
system
-
Nation
al &
Intern
ational
-
Classifications
Occupation
(e.g., ISCO,
nationally
specific system
Industry (e.g.,
ISIC, NACE,
NAICS,
nationally
specific system
Egyptia
n
coding
ISCO
ISCO
1988
-
Nationall
y specific
system
ISCO
ISCO
Egyptia
n
coding
ISIC
ISIC Rev 3
ISIC
ISIC
ISIC
Education
(e.g., ISCED,
nationally
specific)
If others,
please specify
Egyptia
n
coding
ISCED
ISCED
1997
-
European
Comm.
NACE +
nationally
specific
system
Nationall
y specific
-
Nationa
lity
Manual
s
-
-
* Planning to use.
-
ISCED
-
22
8i. If coding is done, is the method manual or automated or
a combination?.
 9 countries replied.
(- Jordan, Kuwait, Palestine, Qatar, Iraq and Syria:
manual method,
- Egypt: automated,
- Lebanon & Morocco: combination.)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
8ii. If automated, then what software application is used
(whether developed in-house or commercial)?
 3 countries replied
(- Egypt: in house,
-Lebanon: Oracle,
- Morocco: Developed by private contractor.)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Question 9: Does your office have an editing
system as part of the census/survey processing?
 9 countries replied Yes.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
9(a). If yes, what kind (if any) of error detection system is
used? (e.g., validity checks for all items individually,
within record consistency checks, across record
consistency checks, macro tabulation checks)
(- Egypt: validation in db & all records ,
- Jordan & Qatar: All of the above,
- Kuwait: Batch prog., check data for data class,
type, range.. ,then relational validation checks)
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
-Lebanon: Validity checks for all items individually,
with record consistency checks, across consistency
checks , macro tabulation checks.
- Morocco: Validity check, record consistency check
are used while data is being processed by the OCR,
normal video coding and formula video coding.
-Palestine- Application is designed to detect errors
over item level, record level; validity check, consistency
& independency check,
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- Syria: Record consistency check,
- Iraq: 2 stages: i. During data entry sessions using
validation rules; ii. after data entry using additional
validation rules by different processes like merging
files to find/resolve outliers.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
9(b).If yes, what kind (if any) of imputation method is
used? (manual, cold deck, hot deck, method based on
Fellegi-Holt principle, other methods)
 7 countries replied
- Egypt: other methods,
- Jordan: not used,
- Palestine, Qatar & Syria : manual,
- Morocco: Cold deck and hot deck were used in Census 2004,
- Iraq- Creating specific automated routines using stat tools
(SPSS,STATA) & batch editing tools.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
9(c). If there is an editing and imputation system, what
software is used for this purpose? (Developed in-house,
CSPRO, other commercial or free software, ex.)
 5 countries replied
- Lebanon : Oracle/SPSS ,
- Morocco: IMPS, CSPRO,
- Palestine- CSPro 3.3,
- Syria – CSPro ,
- Iraq- CSPro, Foxpro, SPSS, STATA .
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Summary points on Data Editing
- Coding for major classifications such as ISCO for
occupation, ISIC rev3 for industry and ISCED for
education are used by most countries;
- Manual coding method is a common method in most
of the countries ;
- Countries apply various editing systems such as
validity checks, record consistency and detection of
outliers.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Part D: Staff Training
Question 10: How many census staff are involved
in each of the steps of data processing in your office?
Please include both regular NSO staff as well as
contractual staff.
 7 countries replied, as follow:
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Number of census staff involved in data capture
Process
Data
Capture
Duration
Country
Full-time
Part-time
Ad-hoc
Total
Egypt
-
-
-
-
Jordan
350
16,000
700
17050
Kuwait
-
100
-
100
Lebanon
700-800
-
-
700-800
Morocco
16
-
-
16
100
-
-
100
Qatar
-
100
-
100
Syria
350
25,000
-
25350
Iraq
-
-
-
-
(Scanners)
Palestine
Number of census staff involved in data Coding
Process
Data
Coding
Duration Full-time
Part-time
Ad-hoc
Total
Country
Egypt
Jordan
25
155
20
200
Kuwait
-
30
-
30
Lebanon
50
-
-
50
Morocco
60
-
60
120
Palestine
120
-
-
120
Qatar
-
120
-
120
Syria
200
500
-
700
Iraq
-
-
-
-
Number of census staff involved in Error Detection
Process
Duration
Full-time
Part-time
Ad-hoc
Total
Country
Egypt
-
-
-
-
Jordan
10
80
05
95
Kuwait
Error
detection Lebanon
10
30
-
40
50
-
-
50
Morocco
10
-
10
20
Palestine
200
-
-
200
Qatar
-
120
-
120
Syria
20
20
-
40
Iraq
-
-
-
-
Number of census staff involved in Imputation
Process
Imputation
Duration
Full-time
Part-time
Ad-hoc
Total
Countries
Egypt
-
-
-
-
Jordan
-
-
-
-
Kuwait
-
-
-
-
Lebanon
-
-
-
-
Morocco
3
-
-
3
Palestine
10
-
-
10
Qatar
-
120
-
120
Syria
200
1000
-
1200
Iraq
-
-
-
-
Question 11: For how long does the training last
for each of the component steps?
 8 countries replied, as follow:
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
country
Egypt
Jordan
Duratio
n
Data
2
2 weeks
Capture month for
s
building
&
housing
unit
census
and 2
weeks
for pop.
Data
2
1 week
Coding month
s
Error
I
detecti month
on
Imputat
ion
Kuwai Lebano
t
n
Morocco
Palestin
e
Qatar
Syria
Iraq
12
days
2 weeks
5 days
3 days
3 days
1 week
5 days
2 weeks
5 days
7 days
5 days
3 days
1 week
5 days
2 weeks
5 days
1 day
5 days
3 days
Data
capture by
COSIT staff
&
sometimes
getting
support
from
internationa
l org;
Data entry
also COSIT
staff with
outsourcing
-
-
-
-
7 days
3 days
5 days
1 day
-
Question 12:Does your office implement a set of
quality control procedures in relation to the various steps
of data processing?
 9 countries replied
( 8 countries: Yes, one country: No -Kuwait)
- Egypt : Yes
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Jordan- Yes
- Field and office editing for the range, consistency and
coverage;
- On-line verification for the range, consistency and
coverage;
- Designing special data entry programs for error
detection ;
- Designing tables of frequencies for reviewing the results,
their consistency and accuracy ;
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
- Designing duel tables for reviewing accuracy and
consistency of the results and the inter- relations of
questions with each other ;
- Designing the tables of final results , checking the data of
each table and the other relevant tables for consistency;
- Reviewing the results for consistency and accuracy prior
to dissemination;
- Conducting a seminar for discussing the results.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Lebanon- Yes :Special team for verification of performance
: tasks- establish a quality standards of a desired level
for each of the operation, including manual editing,
coding, data entry and computer editing.)
Morocco- Yes : Producing data with a minimum accepted
error rate, systems and methodologies in accordance
with international norm are applied, questionnaires
processed using other methods than OCR/OMR such as
CSPro are subject to double keying.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Palestine- Yes :(Once a district data is captured, and
cleaned, data is distributed to subject matter departments
(i.e. Labor, Education, Demography, … etc) to validate the
data in terms of consistency, comparability. New checks
are added to editing application according to the feed back
of subject matter departments.
Qatar- Yes: Double keying, verification of data keyed from
paper.
Syria- Yes: using SPSS;
Iraq- Yes : Field training.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Summary points on Staff Training
- Staff involved in data capture are more compared
with other data processing steps such as data coding,
error detection and imputation.
- Countries implement quality control procedures for
various data processing steps.
UNSD-ESCWA Regional Workshop on Census Data
Processing in the ESCWA region
Doha, 18-22 May 2008
Thank you.