Conference on Data Quality for International Organizations

advertisement

Conference on Data Quality for International Organizations

Newport, Wales, United Kingdom, 27–28 April 2006

Session 3: Collection, management and dissemination of current best methods, minimum standards, and best practices

“The role of best practices in building a global statistical system”

Presenter: Stefan Schweinfest, UNSD

Outline of presentation

1. The “What”: Standards/Best Methods/Best

Practices

2. The “How”: Collection/Management/

Dissemination

3. The “Why”: Transparency/

Harmonization/Capacity Building

4. Three UNSD examples of dissemination of ‘best practices’

5. Points for discussion.

The “What”: Standards/Best

Methods/ Best Practices

Surprising lack of terminological clarity: ‘standard’,

‘code’, ‘system’, ‘recommendation’, ‘principle’, ‘best methods’, ‘best practices’, ‘good practices’, ‘current practices’….

abstract – concrete (‘standards’ – ‘best practices’)

‘Standard’ = norm / model

Statistical Standards: Approved by legislative body eg. Statistical Commission – question of ‘enforcement’;

Minimum vs. maximum standards

‘Best Practices’ = applied procedures; value judgement

The “How”: Collection/Management/

Dissemination

Collection

 Scope: metadata - complex process descriptions

 Timing: time of data collection – separate

 Mode: questionnaire – assessment mission

Management

 difficult to organize and store (textual information)

 difficult to maintain up-to-date

Dissemination

 whether or not? (sensitive country information?)

 Mode: paper (static) or Internet (dynamic)

The “Why”: Transparency/

Harmonization/Capacity Building

Questions: What is the primary purpose of disseminating “Best Practices”?

 Transparency: Extension of metadata

 Capacity building: sharing of knowledge

 Harmonization: Convergence of practices, towards pre-established norm (‘soft enforcement’).

Three UNSD examples of

Dissemination of Best Practices

Trade Index Numbers

“National Practices in Compilation and Dissemination of

External Trade Index Numbers – A Technical Report” UN,

New York 2005, ST/ESA/STAT/SER.F/86

Poverty Statistics

“Handbook on Poverty Statistics: Concepts, Methods and

Policy Use”, UN, New York, forthcoming,

ST/ESA/STAT/SER.F/99, http://unstats.un.org/unsd/methods/poverty/default.htm

Statistical Organization

“Official Statistics - Good Practices Website” http://unstats.un.org/unsd/goodprac/default.asp

Trade Index Numbers (1)

Prompted by a requested by Statistical Commission

Collection mode: questionnaire with follow-up (1999-

2002)

Response: 76 countries and one customs union;

Relatively simple and straightforward questions on 8 categories;

Comprehensive response from respondents;

One-time paper publication (2005): standard description approx. 1 page per country

Trade Index Numbers (2)

6

7

4

5

3

4

1

2

Category

Index number series produced

Source of information

Index calculation methods

Limitations of indices, problems and other methodological observations

Release dates

Revision policy

Dissemination

Compiling agency and contact information

Response Rate

100

100

100

75

84

75

99

100

Poverty Statistics (1)

Part of work on Handbook poverty statistics (inventory of methods)

Collection mode: paper/electronic questionnaire with follow-up during regional workshops (2005)

Responses used for analysis: 65 countries;

Relatively complex questionnaire with approx. 50 detailed questions; also contained request for actual data;

Uneven responses from respondents;

One-time paper publication (2006): tabular summary presentation of main findings (16 pages)

Poverty Statistics (2)

Category Response Rate

5.

6.

7.

8.

1.

2.

3.

4.

9.

10.

National poverty rate (data)

Poverty rate absolute or relative

95

97

Income vs. expenditure

Level of calories threshold

97

68

Use of equivalent scales 98

Poverty specific CPI used to costing the basket of goods 65

Regional patterns in food habits taken into account

Number of food items in the basket

Length of recall period(s)

Diaries method used to collect consumption data?

[….]

68

69

95

35

Statistical Organization (1)

Part of follow-up activities to Fundamental Principles of

Official Statistics (Information sharing)

Collection mode: Initial call for provision of information

(2000);

Selected information on 86 countries;

No questionnaire; posting of original material; assignation of keywords based on 10 principles and related categories;

Availability of information related to categories very uneven;

 Internet based data base (ad hoc updates by UNSD).

Statistical Organization (2)

Principle Category

Advisory bodies

User consultation

Organizational planning and operation

Presentation of statistics

Dissemination and marketing

Release policies

Dealing with the media

[….]

“Response Rate”

49

81

78

95

55

95

92

Summary observations

Three ‘collections’ of best practices prompted by different circumstances (data/no data; stage of methodological development);

Availability of information for about 30-40% of member countries;

Direct communication crucial to improve response rate and response quality;

Labour intensive exercises; challenge of sustainability

Limited knowledge of use of information.

Points for Discussion

What other examples of collection of ‘best practices’ are there?

What are the experiences with the cost-benefit analysis for the exercise?

Under which circumstances are collection efforts of ‘best practices’ most useful?

Do you agree that the collection of ‘best practices’ is an effective tool to promote ‘convergence’ among countries?

Download