•
Questionnaires
•
Likert scale: what is it?
•
Different Likert scales
•
Study objective
•
Methods
- Study participants
- Study design
•
Results
• Conclusions
Likert Scale and Its Variables - A Comparative Study
Currently, people’s opinion about an issue or
service is a valuable way for researchers to
acquire information in diverse areas of
knowledge.
Introduction
Objective
• A questionnaire is an essential instrument that
aims to collect this information based on the
inquisition of the population in study;
Methods
• It consists in a certain amount of questions
covering a topic of interest to investigators;
Results
• Is useful in almost every kind of job.
Conclusion
Questionnaries
Likert Scale and Its Variables - A Comparative Study
Having in mind the need to measure and classify this
information, one reaches the conclusion of the pressing
necessity for scales, tools that permits measuring people’s
attitudes and opinions[1].
Introduction
Objective
Methods
Results
Likert scale
Conclusion
1. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple
treatment groups. Stat Med, 2010. 29(24): p. 2486-500.
Likert Scale and Its Variables - A Comparative Study
What is it?
A scale that consists in asking people to
respond to a series of statements about
a topic, in terms of the extent to which
they agree or disagree with them.[2]
Introduction
Objective
Methods
Applications
Results
[3]
Measure pain, quality of life
Conclusion
LIKERT SCALE
2. Zou KH, C.M., Quinn SA, Beta-mapping and beta-regression for changes of ordinal-rating measurements on Likert scales: a comparison of the change scores among multiple
treatment groups. Stat Med, 2010. 29(24): p. 2486-500.
3.Huber JF, H.J., Zumstein MD, Ruflin G, Lüscher M., Visual circle scale (VCS)-a patient-friendly scale to measure pain compared to VAS and Likert scale. Z Orthop Unfall, 2007.
145(6): p. 795-7.
Likert Scale and Its Variables - A Comparative Study
Different scales
produce different
results
Then, different Likert
scales may produce
different results
Introduction
Objective
Methods
[4]
Search: some articles were found
confirming our idea
Results
Conclusion
DIFFERENT LIKERT SCALES
4 Dawes, J., Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International
Journal of Market Research, 2008. 50(1): p. 61-77.
Likert Scale and Its Variables - A Comparative Study
Does a neutral answer
influence the results?
Different Likert
scales produce
different results?
Already studied
[5]
Introduction
Does a "I don't want to
answer" option
influence the results?
Apparently not
studied yet.
Objective
Methods
Does both "I don't want
to answer" and neutral
options influence the
results?
Apparently not
studied yet.
Results
Conclusion
INFLUENCE OF A CENTRAL ANSWER
5. Kulas JT. , S.A., Middle category endorsement in odd-numbered Likert response scales: Associated item characteristics, cognitive demands, and preferred meanings.
Journal of Research in Personality, 2009. 43: p. 489–49
Likert Scale and Its Variables - A Comparative Study
Discover if the addition of a neutral element, a “refusal
to answer” option or both combined has repercussions in
the results obtained.
Introduction
Objective
Methods
Evaluate the reproducibility of the scales used in the
[6]
questionnaire.
Results
Conclusion
OBJECTIVES
6. Hansson LM, G.M., Bergström R, Factors affecting reproducibility of dietary reports using food frequency questionnaires. Eur J Clin Nutr, 2000. 54(8).
Likert Scale and Its Variables - A Comparative Study
Study Design:
Introduction
One questionnaire with
two versions
Questionnaire 1
Number of
questions in each
questionnaire: 12
Objective
Questionnaire 2
Methods
Results
Conclusion
Same questions in both questionnaires
Methods
Likert Scale and Its Variables - A Comparative Study
Our study is included in a Longitudinal Study type.
BECAUSE :
Introduction
-There is a follow-up period of the individuals in study, which can
be longer or shorter;
Objective
- Data is gathered at least in two points in time which allows us to
observe changes in the population.
Methods
Results
Conclusion
Type of study
Likert Scale and Its Variables - A Comparative Study
Study Participants:
Introduction
All students that belong to the first year of the
Integrated Master’s Degree in Medicine of
Faculdade de Medicina da Universidade do Porto,
distributed by classes 1 to 22
Objective
Methods
Results
Students from classes 6 and 7
Methods
Conclusion
Likert Scale and Its Variables - A Comparative Study
Study Participants:
8:30-11
h
Monday
Tuesday
1
4
Wednesday
Thursday
Friday
6
9
Introduction
Objective
Methods
1416:30h
2
x
7
16:3019h
3
5
8
Methods
10
Results
Conclusion
These ten blocks where
randomly divided in two groups
with SPSS.
Questionnaire 1
Questionnaire 2
Likert Scale and Its Variables - A Comparative Study
About the questionnaires:
Introduction
• Theme of the questionnaire:
Evaluation systems and study methods
Objective
Methods
Theme with less opinion
change probability
Methods
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
About the questionnaires:
Introduction
• Half of the participants answer to version
1 and the other half to version 2
Objective
To avoid certain
biases.
• One month later, the first half answer to
version 2 and the other half to version 1
Methods
Methods
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
About the questionnaires:
• The questionnaires ought to be confidential.
Introduction
Objective
Methods
In order to receive from the
participants their truly
opinion
Methods
The participants wrote their
names on the questionnaires and
we ensured confidentiality .
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
About the questionnaires:
Comparison scale:
I totally agree
I agree
I disagree
Introduction
I totally disagree
Objective
Methods
… plus a neutral answer:
… plus refusal to answer:
Neither agree
nor disagree
I don’t want to
answer
Methods
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
About the questionnaires:
Three types of comparison
Introduction
Objective
Comparison scale
Vs
Comparison scale plus
neutral answer
Comparison scale
Vs
Comparison scale plus
refusal to answer option
Methods
Comparison scale
Vs
Comparison scale plus
neutral and refusal to
answer option
Comparison scale:
Likert scale with “I totally agree”, “I
agree”, “I disagree” and “I totally
disagree”
Methods
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
About the questionaires:
To evaluate the reproducibility
Comparison scale plus
refusal to answer option
Vs
Comparison scale plus
refusal to answer option
Comparison scale
Vs
Comparison scale
Methods
Introduction
Comparison scale plus
neutral and refusal to
answer option
Vs
Comparison scale plus
neutral and refusal to
answer option
Comparison scale:
Likert scale with “I totally agree”, “I
agree”, “I disagree” and “I totally
disagree”
Objective
Methods
Results
Conclusion
Questionnaire 1
Questionnaire 2
I
1. Comparison Scale
2. Comparison Scale
3. Comparison Scale + neutral answer
4. Comparison Scale
5. Comparison Scale + refusal to answer
6. Comparison Scale + neutral answer + refusal to
answer
7. Comparison Scale + refusal answer
I
1. Comparison Scale + neutral answer
2. Comparison Scale
3. Comparison Scale
4. Comparison Scale + neutral answer
5. Comparison Scale
6. Comparison Scale + neutral answer + refusal
to answer
7. Comparison Scale
II
1. Comparison Scale + refusal to answer
2. Comparison Scale + neutral answer + refusal to
answer
3. Comparison Scale
4. Comparison Scale + neutral answer + refusal to
answer
5. Comparison Scale
II
1. Comparison Scale + refusal to answer
2. Comparison Scale
3. Comparison Scale + refusal to answer
4. Comparison Scale
5. Comparison Scale + neutral answer + refusal
to answer
Likert Scale and Its Variables - A Comparative Study
Study Participants:
250
Introduction
Objective
234
Methods
Results
158
Results
Conclusion
REPRODUCIBILITY
Valid
(n)
157
156
157
Question
Scale
I.2
O sistema de
avaliação por
exames é justo.
Standard
II.1
O meu estudo rende
mais quando
trabalho em grupo.
Standard+NR
Proportion of
Agreement
[Confidence
interval]
Kappa
[Confidence
interval]
Weighted
Kappa
[Confidence
interval]
0,76
[0,68;0,84]
0,54
[0,42;0,67]
0,60
[0,49;0,71]
Introduction
Objective
Methods
I.6
Standard+NR+Neu
O número de alunos
tral
por turma no 1º ano
da FMUP é
excessivo.
0,64
[0,56;0,72]
0,47
[0,36;0,58]
N.A.
Results
Conclusion
0,47
[0,39;0,55]
0,30
[0,20;0,41]
N.A.
STANDARD SCALE VS. NEUTRAL OPTION SCALE
Standart Scale
Standard Scale +
Neutral Option
Proportion of
Agreement
[Confidence Interval]
I.4. É justo ser necessário ter
Discordo
Totalmente
15 (9,7)
12 (7,7)
0.29 [0,12;0,52]
frequência a uma disciplina
Discordo
37 (23,9)
33 (21,3)
0.37 [0,24;0,52]
-------
33 (21,3)
-------
79 (51,0)
57 (36,8)
0,37 [0,28;0,48]
24 (15,5)
20 (12,9)
0,26 [0,13;0,44]
3 (1,9)
2 (1,3)
0 [0;0,54]
Discordo
25 (16,0)
30 (19,2)
0,41 [0,26; 0,58]
Não Concordo
nem Discordo
-------
31 (19,9)
-------
Concordo
104 (66,7)
73 (46,8)
0,57 [0,47;0,66]
24 (15,4)
20
(12,8)
0,38 [0,22;0,56]
0 (0)
1 (0,6)
0 [0;0,94]
Discordo
7 (4,5)
3 (1,9)
0,43 [0,12;0,80]
Não Concordo
nem Discordo
--------
12 (7,7)
-------
Concordo
46 (29,7)
57 (36,8)
0,43 [0,32;0,55]
Concordo
Totalmente
102 (65,8)
82 (52,9)
0,64 [0,55;0,73]
Valid
n
155
Question
Answers’ Options
(frequentar pelo menos 2/3)
das aulas para ser admitido
a exame teórico.
I.1
O
156
sistema
de
avaliação
contínua é vantajoso para os
alunos.
Não Concordo
nem Discordo
Concordo
Concordo
Totalmente
Discordo
Totalmente
Concordo
Totalmente
Discordo
Totalmente
I.3 Penso que o grau de
155
exigência
da
FMUP
superior ao de outras.
é
n(%)
n(%)
Introduction
Objective
Methods
Results
Conclusion
STANDARD SCALE VS. REFUSAL TO ANSWER OPTION SCALE
Valid
n
158
n(%)
Answers’ Options
Standard Scale
Question
1 (0,6)
0 (0,0)
0 [0;0]
I.5 Os professores do 1º
Discordo
28 (18,1)
30 (19,4)
0,35 [0,21;0,51]
ano
Concordo
113 (72,9)
87 (56,1)
0,54 [0,45;0,63]
Concordo
Totalmente
13 (8,4)
29 (18,7)
0,14 [0,05;0,30]
Não respondo
-------
9 (5,8)
-------
Discordo
Totalmente
0 (0,0)
3 (1,9)
0 [0;0]
Discordo
12 (7,6)
11 (7,0)
0,15 [0,04;0,39]
Concordo
90 (57,3)
70 (44,6)
0,47 [0,37;0,57]
Concordo
Totalmente
55 (35,0)
64 (40,8)
0,49 [0,38;0,60]
Não respondo
-------
9 (5,7)
-------
Discordo
Totalmente
70 (44,3)
71 (44,9)
0,52 [0,41;0,62]
bebidas
Discordo
57 (37,1)
50 (31,6)
0,35 [0,25;0,47]
energéticas
e/ou
Concordo
25 (15,8)
22 (13,9)
0,31 [0,17;0,48]
vitaminas
para
Concordo
Totalmente
6 (3,8)
11 (7,0)
0,31 [0,10;0,61]
Não respondo
-------
4 (2,5)
da
FMUP
estão
sempre disponíveis para
I.7 Quando há avaliação
contínua,
acho
antipedagógico os alunos
não irem tendo ao longo
do semestre feedback do
seu desempenho
II.3 Tenho necessidade de
recorrer
158
Proportion of
Agreement
[Confidence Interval]
Discordo
Totalmente
tirar dúvidas
158
n(%)
Standard Scale +
Refusal to Answer
Option
a
acompanhar o ritmo de
estudo exigido pelo curso.
Introduction
Objective
Methods
Results
Conclusion
STANDARD SCALE VS. BOTH OPTIONS COMBINED SCALE
Valid
n
158
158
Answers’ Options
N(%)
Standard Scale
Discordo Totalmente
0 (0,0)
II.2 O estudo é mais
Discordo
8 (5,1)
4 (2,5)
0,09 [0,01;0,43]
proveitoso quando
trabalho num local
Não Concordo nem
Discordo
-------
11 (7,0)
-------
silencioso.
Concordo
61 (38,6)
45 (28,5)
0,45 [0,34;0,57]
Concordo Totalmente
89 (56,3)
98 (62,0)
0,67 [0,57;0,75]
Não Respondo
-------
0 (0,0)
-------
Discordo Totalmente
0 (0,0)
0 (0,0)
0,00 [0,00;0,00]
13 (8,2)
8 (5,1)
0,31[0,12;0,59]
-------
11 (7,0)
-------
81 (51,3)
56 (65,4)
0.44 [0,34;0,55]
64 (40,5)
73 (46,2)
0,57 [0,46;0,68]
-------
-------
8 (5,1)
2 (1,3)
3 (1,9)
0,10 [0,01;0,46]
29 (18,4)
26 (16,5)
0,25 [0,14;0,41]
-------
22 (13,9)
-------
84 (53,2)
37 (23,4)
-------
58 (36,7)
47 (29,7)
2 (1,3)
0,37 [0,27;0,47]
0,33 [0,22;0,46]
-------
Question
II.4 Frequentemente acho
Discordo
mais rentável estudar
pelos apontamentos de Não Concordo nem
Discordo
colegas mais velhos do
que pela bibliografia
Concordo
recomendada.
Concordo Totalmente
Não Respondo
158
N(%)
Proportion of
Standard Scale + Both
Agreement
Combined Options
[Confidence Interval]
Scale
0 (0,0)
0,00 [0,00;0,00]
Discordo Totalmente
II.5 É difícil conciliar as
actividades extraDiscordo
curriculares (por exemplo
Não Concordo nem
praticar um desporto,
aprender um instrumento
Discordo
musical, etc…) com o
Concordo
trabalho exigido na FMUP Concordo Totalmente
Não Respondo
Introduction
Objective
Methods
Results
Conclusion
Likert Scale and Its Variables - A Comparative Study
Freedom of choice
Introduction
Reproducibility
Objective
Methods
Results
It depends…
Conclusion
Conclusion
Likert Scale and Its Variables - A Comparative Study
Abel Trzan
João Dias
Ana Fonseca
José Carneiro
Ana Filipa Ferreira
Ana Isabel Maduro
Catarina Tinoco
Guilherme Amorim
Joana Antunes
Maria João Bastos
Maria João Costa
Pedro Barros
Pedro Santos
FACULDADE DE MEDICINA DA UNIVERSIDADE DO PORTO
2º SEMESTRE MARÇO DE 2012