CITIZEN REPORT CARDS
Session1 - A Brief Introduction
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
1
The Citizen Report Card Story
From an IDEA to an INSTITUTION
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
2
BACKDROP: BANGALORE 1993
Inadequate and inefficient public services
Wide corruption and abuse of discretion
Non responsiveness of agencies and political
leadership
Weak response from citizens, civil society &
media
The impetus for change had to come from:
 Political leadership and administration, or
 Active citizens and civil society
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
3
WHY PUBLIC SERVICE DELIVERY IS
WEAK?
Lack of Exit Options
Poor Corrective Actions
Weak Collective Actions
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
4
The Way Out..
STRENGTHENING CITIZEN’S / CONSUMER’S
“VOICE” TO RAISE THE DEMAND FOR CHANGE
& REFORMS
&
STRENGTHENING AGENCY / PROVIDER’S
“RESPONSIVENESS” TO FACILITATE CITIZENS
PARTICIPATION & INVOLVEMENT
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
5
THE BIG CHALLENGE…
Can Citizen’s “VOICE” be a trigger
for change?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
6
CITIZEN REPORT CARDS
THE CONCEPT
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
7
A School Report Card
Name: Gopa; Class: IX
SUBJECT
MAX.
Marks
Highest
Marks
Actual
marks
English
100
75
83
Maths
100
80
70
Science
150
85
76
Social Studies
150
130
95
Total
500
435
324
Rank in class: 16/30
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
8
USE OF SCHOOL REPORT CARDS
 SIMPLE EVALUATION TOOL
 STRUCTURED FOR SIMPLE COMMUNICATION
 PROVIDES SUMMATIVE FEEDBACK ON
PERFORMANCE
 SCOPE FOR COMPARISON WITH OTHER STUDENTS
 STARTING POINT FOR REFLECTION & CORRECTIVE
ACTION
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
9
CRCs: Defining Features
 Pioneered by Public Affairs
independent assessment in 1993
Centre
as
an
 Credible user feedback on public services
 Provide opportunities to communities to demand
more access, responsiveness & accountability from
service providers (from Shouting to Counting)
 Creates an effective diagnostic tool for service
providers and encourage them to introduce citizen
friendly practices
 Create an environment to facilitate
mobilization & `Rights-Based’ strategies
demand
 Report always in PUBLIC DOMAIN
 Not a one-off effort – continued benchmarking
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
10
CONTENT OF CITIZEN REPORT CARDS
 FEEDBACK FROM ACTUAL USERS OF SERVICES REGARDING
 AVAILABILITY, ACCESS & USAGE OF SERVICES
 QUALITY & RELIABILITY
 INCIDENCE OF PROBLEMS & RESPONSIVENESS OF




SERVICE PROVIDERS
HIDDEN COSTS - CORRUPTION & FORCED
INVESTMENTS
SATISFACTION WITH SERVICE
REASONS FOR DISSATISFACTION
SUGGESTIONS FOR IMPROVEMENTS
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
11
LET US NOW SEE SOME EXAMPLES…
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
12
Physical Access to Facilities
 Access to a protected public drinking water source
within 100 mts from residence: 54%
 Access to a medical facility within 3 kms from
residence: 73%
 Access to a Primary School within 1 km from
residence: 66%
 Access to a public bus throughout the year: 54%
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
13
QUALITY / RELIABILITY
Users reporting breakdown of public water taps
(at least once in 2 weeks): 53%
Presence of doctors at public hospitals at the
time of visit: 34%
Proportion of parents reporting regular
attendance of primary school teachers: 12%
 Proportion of users expressing complete
satisfaction with punctuality of public buses: 14%
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
14
Quality of Compliant Redress
(Feedback from households who had a Problem)
Agency
Power
Company
Water Board
Driving
Licence
Urban
Authority
Police
No. of visits to
% satisfied
register compliant with redress
3
51
4
54
2
31
8
15
2
33
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
15
The Bribery Matrix (General Households)
SERVICES /
AGENCIES
Percent Claiming to Average payment
have paid a bribe
per transaction in
Indian Rupees
Survey in Bangalore (1999)
Water Supply Board
11
561
Power Corporation
09
1563
Municipal
Corporation
52
3759
Telephones
26
245
Transport Office
57
637
Development
Authority
35
1653
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
16
Corruption & The Urban Poor
SERVICE /
AGENCY
PERCENT
PAYING
BRIBES
AVERAGE
AMOUNT
(RUPEES)
WATER
SUPPLY
04
200
PERCENT
CLAIMING
BRIBE WAS
DEMANDED
50
MUNICIPAL
SERVICES
ELECTRICITY
34
270
73
10
250
100
RATION
SHOPS
POLICE
11
84
83
46
4193
88
PUBLIC
HOSPITALS
53
640
63
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
17
Eg. RANKING OF SERVICES BY CITIZEN
SATISFACTION
Agency
Power Company
Water Board
Food rations
Telephone Co.
Driving License
Urban Authority
Police
Rank
3
4
2
1
6
7
5
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
18
STAGES IN CITIZEN REPORT CARD
ASSESSING SITUATION & DEFINING SCOPE OF ACTION
PREPARATIONS AND COLLECTING CITIZEN FEEDBACK
RATING OF SERVICES
DIALOGUE AND RESPONSE OF AGENCIES
CITIZEN ENGAGEMENT IN REFORM
PERIODIC BENCHMARKING AND PUBLIC REVIEW
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
19
The CRC Journey
Defining the Scope of CRC
Development of Questionnaire
Sampling Design
Data collection
Data Analysis
Interpretation
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
20
Defining the Purpose of the CRC
Statement of Purpose
 General Vs Specific
What service(s) or sectors do you wish to
cover?
Do you want to focus on a single service
provider or multiple services
Is there a government policy or program
that you wish to assess?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
21
Examples
Generalgetting feedback from people on quality of
public services
getting feedback from patients on quality of
health services
verifying whether the government is
providing the facility it has promised
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
22
Examples
Specific  finding out usage, satisfaction with quality of service,
satisfaction with behaviour of staff, extent of
corruption, problem resolving capabilities of the
agency
 finding out satisfaction with quality of care provided,
level of cleanliness, extent of corruption, behaviour
of doctors and other staff
 comparing what the charter says with ground
realities! eg - free noon meal scheme - is it provided,
how is the quality etc
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
23
CRC Methodology
Session 1 -THE SCIENCE OF SURVEYS (1)
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
24
The CRC Journey
Defining the Scope of CRC
Development of Questionnaire
Sampling Design
Data collection
Data Analysis
Interpretation
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
25
GATHERING COMMUNITY FEEDBACK
Focus Group Discussions
Why FGDs?
Where to conduct?
Who should facilitate?
Who should participate?
What questions to ask?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
26
FGD – The Process





Background Information to be collected prior to the FGDs
(from the key community informant)
Social (communities, settlements) & Institutional mapping
(access & use of key public, social and traditional institutions) –
separately for different communities
Profile of basic services in the village: drinking water sources,
nearest school, health facility, credit (formal and informal),
police station, post office, telephone, bus, power
Presence of non governmental organizations and private
agencies in the delivery of water & sanitary services
Sources of livelihood
Special problems if any encountered by households in the
locality
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
27
FGD – The Process (2)
Getting the introduction right
The Right Group Size
The Questioning Route – “Funnel method”;
Open ended; Avoid too many “why”
Right Moderation – Giraffes & Wolves
Collecting Data
Analysing Data
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
28
FGD – THE PROCESS (3)
KEY THEMES TO FOCUS:
 ISSUES IN RELATION TO THE SERVICE –
AVAILABILITY, ACCESS,USAGE & QUALITY
 INCIDENCE & TYPE OF PROBLEMS
 PATTERN OF RESPONSE TO PROBLEMS
 COSTS – HIDDEN, LEGAL, FORCED, WILLINGNESS
TO PAY FOR BETTER SERVICES
 RECAP OF PROBLEMS AND ISSUES
 SUGGESTIONS FOR SERVICE QUALITY
IMPROVEMENT
 POST DISCUSSION ANALYSIS – RELEVANT ISSUES
FOR SURVEY
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
29
AREA
MEN
WOMEN
DRINKING WATER
URBAN
1. Access not a major issue;
source available within 500
m
2. Issues: single source
(springs) and increase in
number of users and
multiplicity of uses results in
demand over supply;
not willing to share costs
3. Suggestions: increase in
number of sources; replacing
old pipes and reducing initial
connection costs.
1. Access not an issue
2. Issues: irregular
availability; locational
variations and students
spending a lot of time
collecting water;
willing to share costs
3. Suggestions: restoring
storage tanks, which are
out of function for more
than 30 years; increasing
the sources of water.
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
30
Designing the Instrument
Components of a schedule/ questionnaire
Investigator information
Lead-in / Introduction
Filter / qualifiers questions
Demographics
The body of the schedule - sections if
needed - sub- schedules if required
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
31
Sections of the Questionnaire (1)*
IDENTIFICATION : Name & date of survey
LEAD IN QUESTION - Good morning. We are from abc NGO and collecting
data about the work of the new health facility.
FILTER QUESTIONS – Has anybody in your family used the facility during
the last 1 year? (Yes/No) – If No Skip to Q No x (Q36.)
DEMOGRAPHICS – How old are the members of your family
AWARENESS: Do you know that you can get free medicines? (Y/N) (Q43.)
AVAILABILITY: Does your village have the health facility? (Y/N) (Q35.)
EASE OF ACCESS: How far away is the facility from your home? …
USAGE: Which facility do you normally use for major illness? (GRID)
Dist. Hospital/Private doctor/Village health facility
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
32
Sections of the Questionnaire (2)*
RELIABILITY: Was the doctor/paramedic present when you went for
treatment most recently? – Yes/No (Q.167)
QUALITY: How long did you have to wait to meet the doctor/paramedic?
(Q39.) ….
BEHAVIOUR: What do you feel about behaviour of the doctor/paramedic
in treating you? Fully satisfied, somewhat satisfied, not satisfied
SATISFACTION: What do you feel about quality of treatment? Fully
satisfied, somewhat satisfied, not satisfied
REASONS: Why are you dissatisfied? ……………
COSTS: How much did you pay for treatment? …..
HIDDEN COSTS: Did you pay any extra money or bribe for getting
medicines from the health facility? Yes/No (Q174)
SUGGESTIONS
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
33
Basic Tools in Instrument Design
Different types of questions
 Close ended: Yes/No; Scale (Very satisfied, Partly
Satisfied, Dissatisfied); Range (Kms/Hours/Minutes)
Considering Standards
Coding - pre and post coding
Scaling - 5 pt or 2 pt
Skips
Grids
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
34
Thumb Rules for a Schedule
All research objectives to be covered
Shortest length possible (45 minutes)
Language - simple and convenient based on
respondents’ and investigator’s level of
understanding
Clear instructions for the investigator
Avoid too many open-ended questions
Avoid double-barreled questions
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
35
EXAMPLE OF A DOUBLE-BARRELED
QUESTION
"What do you do to cope with poor quality of
water?"
1- Go to a well
2- Ask a neighbor
3- Buy from a shop
4- Other (Please specify_____________)
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
36
THE CORRECT APPROACH!
In the past one year, have there been
instances when the quality of water has
been poor?
1- Yes (ask next question)
2- No (skip next question)
What did you do in these
instances?_____________
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
37
Pre – Testing
Test whether the flow of the Questions are in
Order
Whether the Respondents understand the Questions
Properly
Whether major issues have been left out
Time taken to complete the Questionnaire
Identify Pointers For Refinement
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
38
Post Pre-Test – Group Exercise
 Did the respondents understand the purpose of the
exercise?
 Were all the questions comprehensible to you?
 Did the respondents have any difficulty in
understanding the questions?
 Was the questionnaire too long?
 Were the questions ordered properly?
 What changes would you like to make to improve
the questionnaire?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
39
CRC Methodology
Session 3- The science of surveys (2)
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
40
The Research Journey...
Defining Research Objectives
Development of Questionnaire
Sampling
Data collection
Data Analysis
Interpretation
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
41
The Issues...
Right team
Briefing
Regular reporting
Time schedules
Cost
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
42
The Right Team
How do we decide?
Familiarity with topography, language
Presentability
Able to relate to target audience
Intelligence
Involvement
Available for the entire duration of
fieldwork
And, INTEGRITY
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
43
MOCK INTERVIEWS – SELECTING THE RIGHT INVESTIGATOR
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
44
The Right Team
How many?
f { Productivity, Timelines, Sample size }
Sample size : 2000
Timeline : 20 days
Productivity per man day : 5
Team size = [(2000/20) / 5]
= 20
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
45
The Final Team...
Field Manager
Field
Coordinator
Field
Supervisor
Field
Supervisor
5 Interviewers 5 Interviewers
Field
Supervisor
Field
Supervisor
5 Interviewers 5 Interviewers
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
46
Quality Checks
At all stages
Before launching fieldwork
During fieldwork
After fieldwork
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
47
Quality Checks
Before fieldwork
Pilot exercise
10-15 interviews
Check for
 Length
 Comprehension
 Flow
 Sensitiveness
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
48
Quality Checks
During fieldwork
Sample of each interviewers’ work checked
100% scrutiny
10% back checks
Accompanied interviews
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
49
Quality Checks
After fieldwork
Sample sizes
Quotas
Representation
Check for Inconsistencies
Match with Secondary Data
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
50
Cost Heads in Fieldwork
Function of
 Nature of project
 Type of respondent
 Selection criteria
 Sample size, sampling
Two broad types
 Overheads
 Out of pocket
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
51
Cost Heads in Fieldwork
Out of pocket
 Printing of questionnaires
 Briefing
 Interviewer fees
 Local conveyance
 Outstation travel
 Others : Venue, equipment
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
52
Cost Heads in Fieldwork
Overheads
 Communication
 Courier / mailing
 Research time cost
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
53
CRC Methodology
THE SCIENCE OF SURVEYS (2)
Theory and Practice?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
54
Field Work – Share your Experience
 How do you conduct back checks?
In what way conducting accompanying
interviews helps?
 How can you ensure that necessary
protocols are followed during data collection?
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
55
Practical Problems in Field Work
Step Undertaken
Problem Faced
Replacement of
investigators in between
the survey
New investigators chipping in
without training
Enumerator after true
effort cannot find the
respondent
Waste of time – does the
research agency pay?
There is a team of 5
enumerators under one
supervisor
He is unable to supervise all
the investigators
simultaneously
Supervisor has been
given a sampling plan
which he needs to
adhere
The required sample in a
particular location is
unavailable
Dr. Sita Sekhar, Public Affairs Foundation, Bangalore, 22nd July 2009
Possible
Solution?
56