advertisement

vii TABLE OF CONTENTS CHAPTER TITLE PAGE Cover 1 Declaration ii Dedication iii Acknowledgement iv Abstract v Abstrak vi Table of Contents vii List of Tables x List of Figures xi List of Appendixes xiii Nomenclature xiv RESEARCH FRAMEWORK 1 1.1 Introduction 1 1.1 Statement of the Problem 2 1.2 Research Objectives 3 1.4 Scope of the Study 3 1.3 Significance of the Study 4 1.5 Research Layout 5 viii 2 REVIEW OF GRADING PLAN AND GRADING 7 METHODS 2.1 Introduction 7 2.2 Grading Philosophies 10 2.3 Definition and Designation of Measurement 12 2.3.1 Levels of Measurement 2.3.2 3 Norm-Referenced Versus Criterion-Referenced Measurement 2.4 Weighting Grading Components 18 GRADING ON CURVES AND BAYESIAN GRADING 24 3.1 Introduction 24 3.2 Grading On Curves 25 20 3.2.1 Linearly Transformation Scores 25 3.2.2 Model Set Up for Grading on Curves 26 3.2.3 Standard Deviation Method 27 3.3 Bayesian Grading 31 3.3.1 Distribution-Gap 32 3.3.2 Why Bayesian Inference? 33 3.3.3 Preliminary View of Bayes’ Theorem 35 3.3.4 Bayes’ Theorem 37 3.3.5 Model Set Up for Bayesian Grading 41 3.3.6 41 Bayesian Methods for Mixtures 3.3.7 Mixture of Normal (Gaussian) Distribution 45 3.3.8 Prior Distribution 48 3.3.9 54 Posterior Distribution 3.4 Interval Estimation 4 16 61 NUMERICAL IMPLEMENTATION OF THE BAYESIAN GRADING 64 4.1 Introduction to Markov Chain Monte Carlo Methods 65 ix 4.2 Gibbs Sampling 66 4.3 Introduction to WinBUGS Computer Program 69 4.4 Model Description 69 4.5 Setting the Priors and Initial Values 74 4.5.1 Setting the Prior 74 4.5.2 Initial Values 77 4.6 Label Switching in MCMC 77 4.7 Sampling Results 78 4.7.1.1 Case 1: Small Class 79 4.7.1.2 83 4.8 5 Convergence Diagnostics 4.7.2.1 Case 2: Large Class 87 4.7.2.2 93 Convergence Diagnostics Discussion 96 4.9 Loss Function and Leniency Factor 101 4.10 Performance Measures 105 CONCLUSION AND SUGGESTION 108 5.1 Conclusion 108 5.2 Suggestions 110 REFERENCES 112 Appendix A – F 116-147 x LIST OF TABLES TABLE NO. TITLE PAGE 2.1 Comparison of Norm-Referenced and Criterion-Referenced 19 2.2 Rubrics for Descriptive Scale 19 3.1 Grading on Curve Scales for the Scores between Which a 30 Certain Letter Grade is Assigned, the Mean is "set" at C+ 4.1 Optimal Estimates of Component Means for Case 1 4.2 Minimum and Maximum Score for Each Letter Grade, Percent of Students and Probability of Raw Score Receiving 81 82 that Grade for GB: Case 1 4.3 Straight Scale and Standard Deviation Methods: Case 1 82 4.4 Optimal Estimates of Component Means for Case 2 89 4.5 Minimum and Maximum Score for Each Letter Grade, Percent of Students and Probability of Raw Score Receiving 90 that Grade for GB: Case 2 4.6 4.7 Straight Scale and Standard Deviation Methods: Case 2 Posterior for 95% Credible Interval of Component Means and its Ratio 90 98 4.8 Leniency Factor and Loss Function Constant 104 4.9 Cumulative Probability for GB; Case 1 105 4.10 Performance of GB, Straight Scale and Standard Deviation Methods: Case 1 107 xi LIST OF FIGURES FIGURE NO. 1.1 TITLE 1.2 A Functional Mapping of Letter Grades A Partition on Letter Grades 3.1 Plot of the Raw Scores and Corresponding Transformed Scores 3.2 PAGE 14 15 26 Relationship among Different Types of Transformation Scores in a Normal Distribution; µ = 60, σ = 10 30 3.3 Hierarchical Representation of a Mixture 44 3.4 45 4.1 Normal Mixture Model Outlined on Each Letter Grades Graphical Model for Bayesian Grading 4.2 Kernel-Density Plots of Posterior Marginal Distribution of Mean for Grade B+ 4.3 Monitoring Plots for Traces Diagnostics of Mean: (a) Grade D and (b) Grade B+. 4.4 Gelman-Rubin Convergence Diagnostics of Mean; (a) Grade D and (b) Grade B+ 4.5 Quantiles Diagnostics of Mean; (a) Grade D and (b) Grade B+ 4.6 Autocorrelations Diagnostics of Mean; (a) Grade D and (b) Grade B+ 4.7 Kernel-Density Plots of Posterior Marginal Distribution of Mean for Grade B 73 85 86 86 87 87 92 xii 4.8 4.9 Monitoring Plots for Traces Diagnostics of Mean: (a) Grade B and (b) Grade A Gelman-Rubin Convergence Diagnostics of Mean; (a) Grade B and (b) Grade A 4.10 Quantiles Diagnostics of Mean; (a) Grade B and (b) Grade A 4.11 Autocorrelations Diagnostics of Mean; (a) Grade B and (b) Grade A 4.12 Cumulative Distribution Plots for Straight Scale (dotted line) and GB Method; (a) Case 1 and (b) Case 2 94 94 95 95 99 4.13 Density Plots with Histogram for Case 1 100 4.14 Density Plots and Histogram for Case 2 100 xiii LIST OF APPENDIXES APPENDIX TITLE PAGE A1 Normal Distribution Table 116 A2 Grading via Standard Deviation Method for Selected Means and Standard Deviation 117 B The Probability of Set Function and Mixture Model 119 C Weighting Grades Component 123 D Some Useful Integrals 124 -The Gamma, Inverse Gamma and Related Integrals E WinBUGS for Bayesian Grading 125 F Bayes, Metropolis and David A. Frisbie 147 xiv NOMENCLATURE GC - Grading on Curves GB - Conditional Bayesian Grading MCG - Multi-Curve Grading MCMC - Markov Chain Monte Carlo G - Grade Sample Space N - Number of Students in a class ng - Number of Students for Grade g B - Burn-In Period T - Number of Iterations h {θ x} - Conditional Probability Density of Prior L{x θ } - Conditional Likelihood Function of Raw Score p (⋅ x ) - π (θ ) - Prior Distribution m ( x) - Marginal Density of Raw Score p ( xi ) - The Probability Distribution of Raw Score πg - Component Probability of Component g θ - Parameter of Interest (Conjugate Prior) Θ - Vector of Parameter of Interest N ( ⋅, ⋅) - Normal Distribution Conditional Distribution of Conjugate Prior or Posterior Density 0 IG ( ⋅, ⋅) - Inverse Gamma Distribution Di ( ⋅) - Dirichlet Distribution C (⋅) - Categorical Distribution R - Ratio in Gelman-Rubin Statistics R2 - Coefficient of Determination C ( yi , yˆi ) - Loss Function CC - Class Loss LF - Leniency Factor