Assessment and Evaluation
teaching for success
Happy (belated) Halloween
Happy Halloween cont.
Happy Halloween cont.
Try it out.
• Reflect back on what you taught.
• Use the paper provided
– Write out/create an assessment of what you
taught.
Blooms
What were you assessing?
Where?
Standards (benchmarks)
Learning Outcomes
Competencies
The Need is an always has been there
• D&C 107: 99-100
Wherefore, now let every
man learn his duty, and to
act in the office in which he
is appointed, in all
diligence. 100 He that is
slothful shall not be
counted worthy to stand,
and he that learns not his
duty and shows himself
not approved shall not be
counted worthy to stand.
Even so. Amen.
• Abr. 3: 25 25 And we will
prove them herewith, to
see if they will do all things
whatsoever the Lord their
God shall command them;
• D&C 84: 79 Behold, I
send you out to prove the
world, and the laborer is
worthy of his hire.
Teaching for Success
• Student Write:
Compare and contrast student achievement
and student learning.
– What’s the differences?
– Is the difference important?
– How does/or does the difference impact your efforts
as a:
• Student?
• Teacher?
Teaching for Success
• Define
evaluation:
– "Evaluation" is the systematic
determination of merit, worth, and
significance of something or someone.
• Define
assessment:
– Assessment is the process of
documenting, usually in measurable
terms, knowledge, skills, attitudes and
beliefs.
Evaluation or Assessment?
Evaluation or Assessment?
Evaluation or Assessment cont.
• How can you measure what learning
looks like?
– Traditional methods?
• Is it possible to have an exam that
actually measures what it is testing?
what’s this?
why not?!
•
Usually there are more true answers than false on most tests.
•
If there is no guessing penalty, then guess. You have a 50% chance of
getting the right answer.
•
Read through each statement carefully, and pay attention to the qualifiers
and keywords.
•
Qualifiers like "never, always, and every mean that the statement must be
true all of the time. Usually these type of qualifiers lead to a false answer.
•
Qualifiers like "usually, sometimes, and generally" mean that if the
statement can be considered true or false depending on the circumstances.
Usually these type of qualifiers lead to an answer of true.
•
If any part of the question is false, then the entire statement is false but just
because part of a statement is true doesn't necessarily make the entire
statement true.
what’s this?
• All of the following are correct procedures
for putting out a fire in a pan on the stove
except:
– a.Do not move the pan.
– b.Pour water into the pan.
– c.Slide a fitted lid onto the pan.
– d.Turn off the burner controls.
• Which of the following foods are dairy
products?
• a. milk
• b. ice cream
• c. yogurt
• d. cream cheese
• e. all of the above
______ is a country in South America.
•
•
•
•
•
a. Russia
b. Argentina
c. Mexico
d. Japan
e. none of the above
• Was the infantry invasion of Japan a viable
alternative to the use of the atomic bomb to
end World War II? Is so, why? If not, why
not?
– A. Yes; transport ships were available in sufficient
numbers.
– B. Yes; island defenses in Japan were minimal.
– C. No; estimated casualties would have been
much greater.
– D. No; Japan was on the verge of having an
atomic bomb.
– E. No best answer.
• For example, if your test includes a section with only two
multiple-choice items of 4 alternatives each (a b c d), you
can expect 1 out of 16 of your students to correctly
answer both items by guessing blindly.
why not?!
biased objectivity
• Choose the most general answer when other choices are specific.
• Choose the longest answer when others are much shorter.
• Choose the answer with a middle value when other answers are
higher or lower.
• Choose neither of the similar answers.
• Choose one of two opposite answers.
• Choose the answer that agrees grammatically. For ex: a, and an =
singular, are = plural.
• Choose the answer most synonymous with key words in the
question or statement.
• Count the number of blanks in fill-in questions or statements.
• Choose from among familiar answers. Avoid unknown options.
• Choose the most logical answer to you.
• Avoid answers with absolutes in them. Examples are always, never,
every, none, all, only.)
Checklist for Reviewing Multiple-Choice Items
•
•
•
•
•
•
•
•
Has the item been constructed to
assess a single written objective?
Is the item based on a specific problem
stated clearly in the stem?
Does the stem include as much of the
item as possible, without including
irrelevant material?
Is the stem stated in positive form?
Are the alternatives worded clearly and
concisely?
Are the alternatives mutually exclusive?
Are the alternatives homogeneous in
content?
If the item has been administered
before, has its effectiveness been
analyzed?
•
•
•
•
•
•
•
Are the alternatives free from clues as
to which response is correct?
Have the alternatives “all of the above”
and “none of the above” been
avoided? 9 Does the item include as
many functional distracters as are
feasible?
Does the item include one and only one
correct or clearly best answer?
Has the answer been randomly
assigned to one of the alternative
positions?
Is the item laid out in a clear and
consistent manner?
Are the grammar, punctuation, and
spelling correct?
Has unnecessarily difficult vocabulary
been avoided?
So…
• So… are short answer, open-ended,
and/or essay questions any better?
• It depends.
Types of Testing
Recognition
–
–
–
–
–
True-false
Multiple-choice
Multiple-answer
Matching
Ordering
Recall
– Short Answer
– Completion
– Essay
Average Time
Item Type
• True-false
– 30 seconds
• Multiple-choice and Multiple-answer
– 60 – 90 seconds
• Matching and Ordering
– 30 seconds per response
• Short Answer
– 120 seconds
• Completion
– 60 seconds
• Essay
– 10 – 30 minutes
General Rules
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Simple and direct wording
Avoid jargon
Avoid trivia items
Match items to learning outcomes
Each item has an agreed upon
correct answer
Write more questions than you will
need
Avoid the use of negatives
Enough information to answer the
question
Direct questions preferred
Blanks at the end of the stem
Include words repeated in all
responses
Provide constraints: time, etc.
3-5 per item
Avoid “all of the above” and
“none of the above”
•
•
•
•
•
•
•
•
•
•
•
•
•
Grammatically correct with stem
Similar length and structure
Avoid absolute words
Listed in a logical order
Distracters should be plausible,
can use cliché, use partial
answers.
Group items by type
Sort items by increasing difficulty
Add instructions
Review layout and pagination
Write answer key
Wait to grade – and all at once (or
same conditions).
Item Analysis of test questions
Review poorly answers questions
• Do you see a
problem for tech?
– What about
performance
testing?!
• Do you see a
problem for tech?
– What about
performance
testing?!
• Tests… yes… but
what’s more important
is performance, so…
• Assignments/projects
– BUT HOW do you
fairly grade a
website, a woods
project, and
invention?
Teaching for Success cont.
• What is a rubric?
– How is it related to “grades?”
• What are grades?
– What are the different forms of assessment?
Rubric
• Scoring tool or guide that lists the specific
criteria and the ranges for multiple levels
of achievement for a piece of work or
performance.
– A rubric consists of a set of well-defined
factors and criteria describing the dimensions
of an assignment to be assessed or
evaluated.
Teaching for Success cont.
• So…
– Do you need rubrics?
– How do you write solid rubrics?
• Is it possible to have a 100% perfect rubric?
– How do you write solid exams?
– How do you as a teacher ensure and
adequately measure student learning?
Rubrics: benefits
• Communicates the instructor’s
expectations
• Streamlines the process for feedback to
the student
• Facilitates equitable grading
• Standardizes assessment across different
instructors
Rubrics: how to
• Parts:
– Scale (columns)
– Dimensions (rows)
• Components or outcomes of the assignment
– Criteria descriptions (cells)
Assign points
http://www.uen.org/Rubric/browse.cgi
Rubric Activity
• Split into groups of 3-4
• Determine team roles
• Select an assignment that needs a rubric
– Can be a specific assignment, such as a
research paper for ENG 102
– Can be of a more general nature such as a
class presentation
Rubric Activity Outline
• Step 1: Identify Components
– List 5 major objectives/outcomes of the
assignment
• Write these items as the row headers of the
sheet provided
Step 2: Determine a Scale
– Aim for 3-5 levels
– Can use an odd or even number of items
– Write these as column headings on the sheet
provided
• Outstanding | Accomplished | Proficient | Developing | Beginning
• Accomplished | Average | Developing | Beginning
• Excellent | Good | Needs Improvement | Unsatisfactory
Step 3: Add Criteria
• Create descriptions for each level of
performance for each criteria in the cells of
the rubric
– Bullet points
– Paragraphs
• Write these criteria in the cells of the sheet
provided
Step 4: Assign Points
• Assign points for each level of
performance
• Can use either of the following:
– Discrete values (5, 4, 3, 2, 1)
– Ranges (10-9) for each level
• Indicate the point value on the sheet
provided
– Normally placed with the scale
Step 5: Set Component Weights
• Allows for different levels of importance
– Spelling/grammar – more or less important than
content?
• Determine if weights are necessary for your
rubric
• Assign weights accordingly
Step 6: Trade, Evaluate, and Discuss
• With another group, trade, read through,
evaluate, and then discuss strengths and
weaknesses.
Teaching for Success cont.
• What are grades?
– What do they represent?
– Is there such a thing as a fair grade?
– How does one mix alpha (or letter based grades) with
numeric rating/ scale systems?
– What are some of the problems with grades and
grading systems?
– What are some of the grading systems out there?
Grade
• Standardized measurements of varying
levels of comprehension within a subject
area.
Test, Grades, and Rubric Issues
• Reliability
– “the likelihood that a given measurement procedure
will yield the same description of a given phenomena
if the measurement is repeated.”
• Validity (requires reliability, needs to cover all outcomes)
– “the extent to which a specific measurement provides
data that relate to commonly accepted meanings of a
particular concept.”
• Babbie, 1986
Teaching for Success cont.
• What about state testing and standardized
testing?
– What’s my role (certainly as a TTE teacher)?
Teaching for Success cont.
• Objective:
–
–
–
–
–
Do: “Assessment for Learning.”
What’s this mean?
How do we do it?
What are the key attributes?
Ultimately you have to give a grade for this - what will
this “grade” be?
• (*Grades are important, but when mathematically flawed,
unfair, ineffective - unrelated to student achievement/
learning - then there are grading issues.)