Evaluation of Content Error
Pres. 10
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Evaluation of Content Error
 Objective of Content Error Evaluation
 Learn consistency in reporting of information between P –
and E- samples on those items that are used for matching
and estimation
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Examples - Tabulations for Content Error Estimates
 Sex as reported in the census and as reported in the PES
for matched persons
 Age as reported in the census and as reported in the PES
for matched persons
 Family relationship as reported in PES for matched
persons
NOTE: PES should check content errors only for those items
that support PES procedures
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Table for content error evaluation
Age in census
0-4
Age
in
PES
5-9
……
Total
55-59
60+
0-4
X 1.
5-9
X 2.
….
…..
55-59
X i.
60+
X s.
Total
X .1
…
…
X .i
X .s
n
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Evaluation of Content Error
Net difference rate (NDR)
 The NDR is the difference between the number of cases in
the census and number of cases in the PES that fall under a
category (e.g. age-group) relative to the total number of
matched persons in all response categories
NDR 
where
xi  xi 
100
m
xi  unweighted census number of cases in the i th category
xi  unweighted PES number of cases in the i th category
m  unweighted number of matched cases
s  total number of response categories for characteri stic x
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Evaluation of Content Error (contd.)
Index of inconsistency
 It is a relative number of cases for which the response varied
between the census and PES. In a n x n table, numbers in
diagonal cells show consistency (agreement). Off diagonal
numbers represent inconsistency. This is calculated for each
response from a category
Iˆ 
 xi  xi  2 xii 
1
xi m  xi   xi m  xi 
m
i  1,2,3,  , s
where xii  number of cases where categoryi
is a responsecategory in both thecensus and thePES
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Other indicators of content errors
Aggregate index of inconsistency :
 Gross difference rate (also proportion of difference
to the diagonal): number of discrepancies between
response at the census and at the PES relative to the
number of matched persons.
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Other indicators of content errors
 Rate of agreement:
> Complement of gross difference rate. A low rate
of agreement means a high degree of variability
and vice versa
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
How to interpret content error measures
Index
Low
Moderate
High
Absolute value of NDR
< 0.01
0.01-0.05
>0.05
Index of Inconsistency
<20
20-50
>50
Aggregate Index of
Inconsistency
<20
20-50
>50
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Table for content error evaluation
Age in census
0-4
Age
in
PES
5-9
……
Total
55-59
60+
0-4
X 1.
5-9
X 2.
….
…..
55-59
X i.
60+
X s.
Total
X .1
…
…
X .i
X .s
n
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010
Thank You!
United Nations Workshop on the 2010 World Programme on Population and Housing Censuses: Census
Evaluation and Post Enumeration Surveys, Amman, Jordan, 21-24 November, 2010