Part II – Chapters 6 and
beyond….
Reliability, Validity, & Grading
Reliability

Definition – consistency or repeatability

Objectivity



A special kind of reliability
Interrater reliability
Can be measured by test-retest correlation
Validity


Definition – truthfulness of a test score
Dependent on reliability and relevance (the
degree to which a test pertains to its
objectives)
Development of criterion-referenced
standards




The judgmental approach
The normative approach
The empirical approach
The combination method
Grading

Issues with grading in physical education

Mostly from chapter 8 & 9
All grades have some degree of
subjectivity



But some have more than others
Should we try to reduce the subjectivity of
our grading practices?
How can we do that?
Formative and Summative Evaluations


Which do we use in PE?
Which should we use?
The Process of Grading (page 130)

Grading across the 3 domains of
performance
The process
1.
2.
3.
4.
Determine objectives
Select tests
Administer tests and compare to
standards
Compute final grades
Objectives



Is the objective an important educational
outcome?
Does every student have an equal
opportunity to demonstrate his or her
progress towards reaching the objective?
Can the objective be measured with
reliability and objectivity?
Test Selection


Reliable and valid
Formative or summative
Results compared to standard

Normative or criterion based references?
Final Grades


How will evaluations be combined to
produce a final grade?
What is acceptable and/or unacceptable for
final grade distributions?
Identify Current practices




For better or worse…
Look at Figure 8.2 on page 131
How do these practices compare with your
own experiences?
How do these practices fare compared to
the previous criteria?
What are the special problems facing a PE
teacher when it comes to grading?
What not to grade on







Attendance
Dressing out
Shower taking
Sportsmanship?
Participation?
Team rank?
Improvement?
What to grade on

Team Sports classes




Sports skill tests
Rating scales
Round-robin tournament performance
Individual sports classes

Performance ratings relative to objectives
What to grade on

Fitness classes


Fitness tests
Knowledge tests should be used in all
classes
What to grade on




Grades should reflect the level of achievement in
meeting the course objectives and nothing else
Grades should be based on status and not
improvement
Grades should not be used for rewards or
punishment
Grades should be taken as measurements rather
than evaluations
Grading Mechanics




Determine and weight course objectives
Measure the amount of achievement
toward the course objectives
Combine measurements into a composite
score
Convert the composite score to a final
grade
Combining scores


Standard scores
Weighted percentages
Assigning grades


Cutoffs
Curves
Artifacts from Syllabus
1.
2.
3.
4.
5.
6.
7.
8.
normative versus criterion based assessment
reflection paper on grading in HPER
student developed grading philosophy
student developed assessment model
student developed assessment plan
student developed rubric
grading spreadsheet
sample stat analysis with SPSS (and/or Excel)
Assignments coming up (all artifacts)





Write grading philosophy paper
Develop assessment plan/model
Develop spreadsheet with sample data
After next chapter develop rubric
Reflection paper
Grading Philosophy



Typed and double spaced
About one page in length
Must describe the following issues:



Will you grade in all 3 domains? If so then what
sorts of things will you include?
To what extent will you grade on improvement
versus achievement (explain why)?
What sort of grade distribution will you feel
comfortable with?
Develop assessment plan/model

Basically a course syllabus for your
hypothetical course

Includes


Course description
Course objectives
 Explain how you will measure each of these

Evaluation information
 Explain how you will combine measurements of course
objectives
Assessment Plan cont….



Must align with your philosophy statement
Must include multiple objectives
Rationale should be provided for any
significant deviations from concepts
discussed in class
Spreadsheet



Must be compatible with course syllabus
Must include sample data
Should be as “user-friendly” as possible



That means as automated as possible – should
show variety of functions and helpful formatting
Must automatically compute final grade
Must use some form of standard scores
Spreadsheet cont….

You should turn in a “soft copy” of your
spreadsheet as an e-mail attachment

Send to butlermi@emporia.edu
Chapter 9

Alternative assessment
Recent trends


Use assessment rather than test
Assessment


The process of collecting information
More politically correct and less threatening
Recent trends



Alternative assessments
Authentic assessments
Often terms are used synonymously
Alternative assessments





Portfolios
Role-playing
Interviews
Event tasks
Individual or group
projects




Open-ended questions
Student logs or
journals
Exhibitions
Observation
Alternative assessments

Pros




Popular
May be fairer – multiple intelligence idea
May be more “authentic”
Cons

Subjective?
Authentic assessment


Designed to take place in a real-life setting
rather than a contrived or artificial setting
Frequently involves directly observable
behavior
Recent Trends

“At the heart of the most recent wave of
educational reform is the identification of
learner outcomes, framed in the context of
standards that define what students should
know and be able to do.” – page 155
Criteria for judging performance

Criteria for judging student performance
have been called many things:




Scoring guidelines
Rating scales
Checklists
Rubrics

Rubrics is currently the most popular
Rubric

A fixed scale and a list of characteristics
describing performance for each of the points on
the scale




See Table 9-2 page 162
See table 9-4 page 163
Can be useful for assessments in the affective
domain
Hellison’s model of social responsibility

See Table 9-6 – page 164
Ideas for rubrics


PE Central
Rubistar
Rubric Assignment


Create a rubric that you could use for
grading a motor skill – part of your
evaluation plan
You may use the sample rubrics in the text
as a guide to develop your own but you
should not copy them word for word
QPA



What is QPA?
How does QPA affect physical education
teachers in Kansas?
KSBE website
No Child Left Behind



What is “NCLB”?
How does “NCLB” affect physical education
teachers?
Web Sites



Government site 1
Government site 2
A different view
Reflection Paper

Reflect on knowledge of assessment and
predict how you may use this knowledge in
your future career.


Has your philosophy on assessment changed?
Some things you might discuss include:



Grade inflation
Improvement versus achievement
Current trends in assessment