Quality considerations in
Statistical Surveys
Mukesh K Srivastava
FAO Statistics Division
What is Quality?
Concepts: What is Quality?

Means different things to different people






ease in use and comforts (beds sheets, clothes)
reliability (long life)
risk associated with use of product (electrical
goods)
value for money (substitute perfume)
credibility (news paper)
Association of quality with “Brand names”


efforts to establish credibility (advertisements)
Giorgio Armani vs. DOS, MOA
Concepts: Scope of quality

Product
–


Process (computerized eye testing)
Inputs
–
–
–

Mozzarella di Buffala
Vino di Toscana
“Champagne” of specific region of France
Formal statements of quality need a framework


Characteristics
an agreed set of characteristics/variables on which
information is to be provided for comparison
Total Quality Management:

INPUT- THROUGHPUT- OUTPUT
Quality in Official Statistics

National Quality Assessment Frameworks
–


http://unstats.un.org/unsd/dnss/QualityNQAF/nqaf.aspx
A review reveals that the quality does not have the
“same” meaning across the Globe, though there is
broad consensus on its importance and key
characteristics.
Distinguish between:
–
–
–
Quality of Data
Quality of Survey
Quality of the Statistical system of a Country (Country
Assessments of Global Strategy)
Quality of Statistics: Product

How close to Reality?
–

Errors: measured by distance from reality
Sources of Errors:
–
–
Due to sampling and estimation procedure
Non-sampling errors (applicable to both sample
survey and complete enumeration)



Coverage
Measurement
Response
Measures of quality of survey data

Estimates of Sampling errors: a measures of
reliability of the estimate
–

(closeness to the true parameter)
Depends upon
–
–
Sampling design
The estimator (formula) e.g. mean, median, mode as measure of
central tendency
– The values in the specific sample (bad sample)

Statement about efforts to control non-sampling error
–
–
extent of coverage
quality of response (post-enumeration survey)
Process elements of survey quality

Decision to undertake survey
–
–

–
–
–
survey concepts
sample design
development and testing of
measurement instruments

–
–
data sources
training of enumerators
non response
Data capture and data
processing
–
–
data capture
editing and imputation
procedures
–
–

Relevance, accuracy,
timeliness, comparability,
coherence, data analysis,
disclosure control
Documentation and
dissemination
–
Data collection
–

user needs analysis
survey objective definition
Data Analysis and output
quality
Survey design
–


metadata documentation
dissemination strategies
data management
Improvement cycles
–
–
–
adaptability/flexibility
expertise in relevant areas
quality management
How to achieve
Quality in Official Statistics?

UNSD



EUROSTAT



Principles governing international statistical activities
Fundamental Principles of Official Statistics
http://unstats.un.org/unsd/methods/statorg/FP-English.htm
National Methodological Report on survey (model: Ref. doc.)
The European Self Assessment Checklist for Survey
Managers (Ref. doc)
MEDSTAT workshop
‫شكرا جزيال‬
Message:
1.
2.
3.
Statisticians should make an effort to
develop a culture of quality in their domain
by disseminating information on quality
along with data.
Quality information helps in brand
positioning
Quality information increases demand of the
products and thus enhances
revenues/resources.