Ministero della Giustizia
Direzione Generale di Statistica
Italian Ministry of Justice
Organization, Human Resources and
Services Department
Department of Statistics
Customer Satisfaction Surveys
conducted in the pilot courts of Turin
and Catania
Tbilisi, February 10, 2012
Background
The idea of conducting a satisfaction survey in Italy aimed to Court users
takes its stands from the guidelines prepared by the Quality Working Group
of the CEPEJ.
The idea was proposed and enthusiastically accepted by the Heads of the
Courts of Turin and Catania.
3
Objectives
Main objectives of the survey are:

To evaluate the overall satisfaction level of Justice amongst the
final users

To establish Key Performance Indicators (KPIs) to be tracked
throughout time

To pinpoint the areas of improvement and (propose to) take
action accordingly

To analyze the satisfaction amongst specific sub-samples
(e.g. males/ females, age classes, …)

To plot Performance vs Importance diagram
Performance
Maintain
Focus
Importance
Low
priority
Improve
4
Working group
A key success factor towards the success of such
initiative is to involve as much as possible the
stakeholders of the world-justice.
Thus, the working group to design and conduct the
survey was composed by:

Ministry of Justice representatives
(and as members of the CEPEJ)

Statisticians

Heads of the Courts of Appeal
and Tribunals

Judges and Lawyers

Professors of the University
(Law, Politics and Economic Sciences)
5
Media Attention
The initiative was reported on main national
newspapers and local news broadcast
Courts under survey

Last year we conducted the survey in
Turin and Catania

Depending on the goodness of the
results this kind of survey might be
extended to other courts across the
Country
Turin
In Turin the survey was conducted only
with users while in Catania it was done
Catania
also with other two targets: Lawyers
and Court staff
 Members of the CEPEJ were responsible for managing and
coordinating the initiative in both cities

7
Constraints and methodology
Since this Satisfaction Survey is carried out
without a budget, we have decided to:

maximize the use of internal resources and free
external resources

guarantee the quality of the project through
involvement of internal professional
statisticians and university professors

select a group of around 25 students per city
to conduct the interviews
Additional slides are reported in the second part
of this presentation on how to conduct interviews
8
The sample
Interview Citizens visiting the Courts of Turin and Catania (preferably
people who have already got a contact with the service provided) during
February-March 2011.
Preferably a “random” sample of 600 users
per Court
Alternatively a sample composed of at
least 380 users per Court
Statistical note  within an unlimited population:

A sample of 600 users guarantees that the standard error is lower than 4%

A sample of 380 users guarantees that the standard error is lower than 5%
9
The target
The target has been defined according to the
following criteria:
Criteria of inclusion
•
•
•
•
Parties
Witnesses
Interpreters, experts
Relatives of the Parties, of
witnesses, etc.
Criteria of exclusion *
• Lawyers
• Judges, Prosecutors and staff
of the Court
• Policemen, Bailiffs, etc.
(*) In Catania two specific satisfaction surveys aimed at Lawyers and Court staff
have been carried out separately.
10
The questionnaire
The questionnaire has been designed according to the CEPEJ guidelines,
taking into account specific needs of the Courts in Turin and Catania. For
this reason we have used two slightly different questionnaires.
Both questionnaires are divided into 3 different sections:
•
Screening questions
•
Main Questions. Satisfaction of:
•

Court premises, organization, clean environment

Punctuality of hearings, length of proceedings

Judge professionalism, competence, cordiality

Information points within the Court
and web-sites

Court location and ways of transport
Demographic Questions
11
Promoting the initiative
Before the interviews started, many posters
promoting this initiative were stuck to the walls of
the Courts premises.
This simple action had a great contribution on the
respondents’ willingness to take part into this
survey.
Additional publicity has been obtained through:
•
Lawyers in Turin and Catania: they were asked to
talk about this initiative among their clients.
•
Judges: they were all formally informed about this
initiative.
Catania has requested the CEPEJ to provide some
gadgets to be distributed to the people who
participated to the survey
12
Data acquisition and results
The data collection is performed through an on-line application.
At the end of the day all questionnaires are being loaded-up into a central
database by the interviewers using an on-line form.
The results (charts and graphs) are available in real-time to the members
of the working group.
Statistical Note:
In order to ensure good quality standards, a selection of random
paper questionnaires will be double-checked against the online data
to verify that the questionnaires have been transmitted correctly.
13
Project timing
Oct
2010
Dec-Feb
1010
Feb-Mar
2011
Mar-Apr
2011
May
2011
• Questionnaire Design
• Pilot Interviews
The whole project lasted about 8
months, from very beginning to
presentation of results
• Briefing Sections
• On-field Interviews
• Analysis and Report
• Presentation of results
14
The results: Overall Satisfaction

Overall satisfaction with
the court experience
TURIN
Score:
Score:
CATANIA
Note: Scores are calculated applying
a weighted mean. Weights range
from 1 (Very Dissatisfied) to 5 (Very
satisfied).
Satisfaction vs Importance diagram
TURIN
Satisfaction vs Importance diagram
CATANIA
Italian Ministry of Justice
Organization, Human Resources and
Services Department
Department of Statistics
Key suggestions on how to conduct
interviews and collect the data
Tbilisi, February 10, 2012
Interviewers training
Prior to the fieldwork a number of motivational sessions have
been carried out.
Moreover, in order to ensure good quality standards, interviewers
were carefully briefed on the following:

Phases of the interview (approach,
questionnaire, closure)

Behavior and good conduct

What to do and what to avoid

Questionnaire flow (screening, main and
demographic questions)
19
Behavior and good conduct

Kindness, discretion, self-confidence, inspiring trust

Neutral behavior, attention to non-verbal
communication

Read the questions in the exact order they are
reported into the questionnaire

Availability to read questions and all possible
answers even more than once

Take note of all the comments coming from the
interviewed person
20
The questionnaire - Introduction by the interviewer
INTRODUCTION
Good morning / afternoon, my name is ___________________ and I’m
here today because the Ministry of Justice is currently conducting a
survey aimed at court users such as parties, witnesses, experts,
interpreters, etc
The Ministry of Justice is interested in your own opinion and in your
experience of this court in order to improve the overall quality of the
service.
This is a statistical survey and therefore your answers will be handled
with care. At the end of this survey data will be analyzed in the utmost
anonymous way.
Can I ask you a few questions?
Strict confidentiality of your answers is guaranteed. This
survey is anonymous.
The questionnaire - structure
Conducting one interview takes approximately 15 – 18 minutes
(it is important to notice the person about the actual duration of the interview)





Introduction by the interviewer
Screening questions
Core questionnaire
Demographic questions
Closing
SW utilized for data collection – the tool




Data has been collected using free tools on the
web, called Google Docs
Google Docs is a collection of web-based
documents that let users edit the same file at the
same time so you always have the latest version
In particular, an online form was designed in
order to collect the data submitted by the
interviewers
The main advantages of using an online form for
collecting data are:




Quality of data is guaranteed
Answers can be accessed in real-time
Data is ready for analysis at any time
The workgroup can easily track the interviewing
process
TURIN
CATANIA
SW utilized for data collection – the steps
1

Designing the questionnaire
A form (i.e. questionnaire) can be easily
designed using single and multiple choice
questions, scales, grids, text boxes etc.
2

3
4
Collecting the data
Once the form is online, interviewers can load
their questionnaires via web. The data submitted
is stored in an online spreadsheet document.

Summary report
A summary report with charts and graphs is
available in real-time.

Exporting data into Excel for
further analysis.
Data can be exported into excel format
(amongst the others) and eventually into spss
format for further analysis.
Taking care of all details in the organization
The workgroup must consider all the details related to the
organization, in this slide, for example, we report the map shown to
students to make them aware of the courts premises and location.
PIAZZA GIOVANNI VERGA, Palazzo di Giustizia (1)
VIA FRANCESCO CRISPI, ex Pretura (2)
VIA VERONA, Sezione Lavoro (3)
Sede
Uffici
Numerosità
campionaria
Palazzo di Giustizia
Corte d’Appello e
Tribunale
382
Via Crispi (ex Pretura)
Tribunale
164
Via Verona (Sezione Lavoro)
Tribunale
54
3
1
2
Timing and other practical informations




In our experience, in a working morning, e.g. from 9.00 to
14.00, an interviewer can conduct up to 6 interviews
Interviewers were organized into sub-groups of 6-8 people
with an agreed calendar over a period of 6-8 weeks
It is important to recommend the interviewers that the
choice of the person to be interviewed must be absolutely
random
At the end of each day, each interviewer was requested to
report filled-in questionnaires into the on-line application
Ministero della Giustizia
Direzione Generale di Statistica