vii
TABLE OF CONTENTS
CHAPTER
TITLE
PAGE
Cover
1
Declaration
ii
Dedication
iii
Acknowledgement
iv
Abstract
v
Abstrak
vi
Table of Contents
vii
List of Tables
x
List of Figures
xi
List of Appendixes
xiii
Nomenclature
xiv
RESEARCH FRAMEWORK
1
1.1 Introduction
1
1.1 Statement of the Problem
2
1.2 Research Objectives
3
1.4 Scope of the Study
3
1.3 Significance of the Study
4
1.5 Research Layout
5
viii
2
REVIEW OF GRADING PLAN AND GRADING
7
METHODS
2.1 Introduction
7
2.2 Grading Philosophies
10
2.3 Definition and Designation of Measurement
12
2.3.1
Levels of Measurement
2.3.2
3
Norm-Referenced Versus Criterion-Referenced
Measurement
2.4 Weighting Grading Components
18
GRADING ON CURVES AND BAYESIAN GRADING
24
3.1
Introduction
24
3.2 Grading On Curves
25
20
3.2.1
Linearly Transformation Scores
25
3.2.2
Model Set Up for Grading on Curves
26
3.2.3
Standard Deviation Method
27
3.3 Bayesian Grading
31
3.3.1 Distribution-Gap
32
3.3.2
Why Bayesian Inference?
33
3.3.3
Preliminary View of Bayes’ Theorem
35
3.3.4
Bayes’ Theorem
37
3.3.5 Model Set Up for Bayesian Grading
41
3.3.6
41
Bayesian Methods for Mixtures
3.3.7 Mixture of Normal (Gaussian) Distribution
45
3.3.8 Prior Distribution
48
3.3.9
54
Posterior Distribution
3.4 Interval Estimation
4
16
61
NUMERICAL IMPLEMENTATION OF THE
BAYESIAN GRADING
64
4.1 Introduction to Markov Chain Monte Carlo Methods
65
ix
4.2
Gibbs Sampling
66
4.3
Introduction to WinBUGS Computer Program
69
4.4 Model Description
69
4.5
Setting the Priors and Initial Values
74
4.5.1
Setting the Prior
74
4.5.2
Initial Values
77
4.6
Label Switching in MCMC
77
4.7
Sampling Results
78
4.7.1.1 Case 1: Small Class
79
4.7.1.2
83
4.8
5
Convergence Diagnostics
4.7.2.1 Case 2: Large Class
87
4.7.2.2
93
Convergence Diagnostics
Discussion
96
4.9 Loss Function and Leniency Factor
101
4.10 Performance Measures
105
CONCLUSION AND SUGGESTION
108
5.1
Conclusion
108
5.2
Suggestions
110
REFERENCES
112
Appendix A – F
116-147
x
LIST OF TABLES
TABLE NO.
TITLE
PAGE
2.1
Comparison of Norm-Referenced and Criterion-Referenced
19
2.2
Rubrics for Descriptive Scale
19
3.1
Grading on Curve Scales for the Scores between Which a
30
Certain Letter Grade is Assigned, the Mean is "set" at C+
4.1
Optimal Estimates of Component Means for Case 1
4.2
Minimum and Maximum Score for Each Letter Grade,
Percent of Students and Probability of Raw Score Receiving
81
82
that Grade for GB: Case 1
4.3
Straight Scale and Standard Deviation Methods: Case 1
82
4.4
Optimal Estimates of Component Means for Case 2
89
4.5
Minimum and Maximum Score for Each Letter Grade,
Percent of Students and Probability of Raw Score Receiving
90
that Grade for GB: Case 2
4.6
4.7
Straight Scale and Standard Deviation Methods: Case 2
Posterior for 95% Credible Interval of Component Means
and its Ratio
90
98
4.8
Leniency Factor and Loss Function Constant
104
4.9
Cumulative Probability for GB; Case 1
105
4.10
Performance of GB, Straight Scale and Standard Deviation
Methods: Case 1
107
xi
LIST OF FIGURES
FIGURE NO.
1.1
TITLE
1.2
A Functional Mapping of Letter Grades
A Partition on Letter Grades
3.1
Plot of the Raw Scores and Corresponding Transformed
Scores
3.2
PAGE
14
15
26
Relationship among Different Types of Transformation
Scores in a Normal Distribution; µ = 60, σ = 10
30
3.3
Hierarchical Representation of a Mixture
44
3.4
45
4.1
Normal Mixture Model Outlined on Each Letter Grades
Graphical Model for Bayesian Grading
4.2
Kernel-Density Plots of Posterior Marginal Distribution
of Mean for Grade B+
4.3
Monitoring Plots for Traces Diagnostics of Mean: (a)
Grade D and (b) Grade B+.
4.4
Gelman-Rubin Convergence Diagnostics of Mean; (a)
Grade D and (b) Grade B+
4.5
Quantiles Diagnostics of Mean; (a) Grade D and (b)
Grade B+
4.6
Autocorrelations Diagnostics of Mean; (a) Grade D and
(b) Grade B+
4.7
Kernel-Density Plots of Posterior Marginal Distribution
of Mean for Grade B
73
85
86
86
87
87
92
xii
4.8
4.9
Monitoring Plots for Traces Diagnostics of Mean: (a)
Grade B and (b) Grade A
Gelman-Rubin Convergence Diagnostics of Mean;
(a) Grade B and (b) Grade A
4.10
Quantiles Diagnostics of Mean; (a) Grade B and (b)
Grade A
4.11
Autocorrelations Diagnostics of Mean; (a) Grade B and
(b) Grade A
4.12
Cumulative Distribution Plots for Straight Scale (dotted
line) and GB Method; (a) Case 1 and (b) Case 2
94
94
95
95
99
4.13
Density Plots with Histogram for Case 1
100
4.14
Density Plots and Histogram for Case 2
100
xiii
LIST OF APPENDIXES
APPENDIX
TITLE
PAGE
A1
Normal Distribution Table
116
A2
Grading via Standard Deviation Method for Selected Means
and Standard Deviation
117
B
The Probability of Set Function and Mixture Model
119
C
Weighting Grades Component
123
D
Some Useful Integrals
124
-The Gamma, Inverse Gamma and Related Integrals
E
WinBUGS for Bayesian Grading
125
F
Bayes, Metropolis and David A. Frisbie
147
xiv
NOMENCLATURE
GC
-
Grading on Curves
GB
-
Conditional Bayesian Grading
MCG
-
Multi-Curve Grading
MCMC
-
Markov Chain Monte Carlo
G
-
Grade Sample Space
N
-
Number of Students in a class
ng
-
Number of Students for Grade g
B
-
Burn-In Period
T
-
Number of Iterations
h {θ x}
-
Conditional Probability Density of Prior
L{x θ }
-
Conditional Likelihood Function of Raw Score
p (⋅ x )
-
π (θ )
-
Prior Distribution
m ( x)
-
Marginal Density of Raw Score
p ( xi )
-
The Probability Distribution of Raw Score
πg
-
Component Probability of Component g
θ
-
Parameter of Interest (Conjugate Prior)
Θ
-
Vector of Parameter of Interest
N ( ⋅, ⋅)
-
Normal Distribution
Conditional Distribution of Conjugate Prior or Posterior
Density
0
IG ( ⋅, ⋅)
-
Inverse Gamma Distribution
Di ( ⋅)
-
Dirichlet Distribution
C (⋅)
-
Categorical Distribution
R
-
Ratio in Gelman-Rubin Statistics
R2
-
Coefficient of Determination
C ( yi , yˆi )
-
Loss Function
CC
-
Class Loss
LF
-
Leniency Factor