APPENDIX 4: VALIDITY AND RELIABILITY TESTS OF VERBAL RATING SCALE INSTRUMENTS (N=24)
Number of
Properties
Criteria
CONTENT
> 10 Items
20
DESIGN
Original
10
Papers
References*
Overall Evidence
(42, 52, 53, 56, 58, 64, 67, 70, 74,
75, 87, 88, 104, 108)
Modifications
16
Changing Scale / Format
3
(29, 76, 78)
Changing Symptom Descriptors
3
(59-61)
Abbreviating Content
4
(55, 77, 90, 91)
Translation
3
(32, 40, 71)
Modifying for Observer Raters
2
(49, 69, 76)
Content (Patient Participation)
1
(69)
Good
Construct Convergent / Divergent
11
(42, 51, 53, 55, 58, 60, 61, 64)
Poor / Good
Factor Analysis
13
(52, 55, 56, 58, 60, 61, 69, 77, 87,
1 / 0.9% Confirmatory
88, 90, 91)
analysis by gender
Adding / Deleting Items and Dimensions
VALIDITY
subgroups
Criterion Concurrent
7
(40, 70, 76-78, 90, 108)
Poor / Good
Criterion Predictive
1
(77)
Good
Known-Group
14
(42)
Moderate / Good
Type of Tests¥
Response to Change
8
(42, 51-53, 64, 71)
>1
19
(40, 42, 52, 53, 55, 56, 58, 60, 61,
Poor / Good
64, 67, 70, 71, 87, 88, 90, 91,
108)
0
2
(29, 49)
Overall Validity
Most types of
 Poor / Good construct
validity equally
 Weak / Good criterion
tested except
 Moderate / Good
response to
known-group
 Moderate response to
change
change
RELIABILITY
Internal Consistency
(Cronbach’s Alpha)
21
(32, 40, 42, 49, 53, 55, 56, 58, 60,
Moderate / Good
61, 64, 69-71, 76, 77, 87, 88, 90,
91, 108)
Inter-Item Correlations
6
(61, 64, 76, 91)
Weak / Good
Other (Split Halves)
2
(53, 87)
Good
Test-Retest
9
(40, 52, 55, 59, 64, 67, 78, 88)
Moderate / Good
Inter-Rater
1
(76)
Poor / Moderate
Agreement (Other than Pearson’s or
2
(29, 52)
Good
Item Total Coefficients
Intra-Rater
Spearman’s Correlation)
Type of Tests
≥1
7
(40, 42, 55, 59, 64, 87, 108)
NA
0
1
(69)
NA
Overall Reliability
Prevailing
internal
consistency
 Moderate / Good
internal consistency
 Moderate / Good testretest
 Poor / Moderate interrater reliability
*Number of references may exceed the number of instruments as there are multiple studies for the same instrument
¥
Face (content) validity tests were not included