Summative Assessment - Biology Leadership Conference

advertisement
Summative Assessment
Clarissa Dirks
The Evergreen State College
Olympia, Washington
PREVIOUS
THINKING
ABOUT
TEACHING
AND
LEARNING
Smith et al 2005
OUTCOME
OUTCOME
CURRENT THINKING ABOUT TEACHING AND LEARNING
Misconceptions
(formative assessments)
Conceptual Frameworks
(backward design; scholarly teaching)
Metacognition
(formative assessments; testing effect)
AND SOMETHING ELSE . . .
Summative Assessments Drive Learning
THE MONTILLATION AND USES OF TRAXOLINE
It is very important to learn about
traxoline. Traxoline is a new form of
zionter. It is montilled in Ceristanna.
The Ceristannians found that they
could gristerlate large amounts of
fervon and then bracter it to quasel
traxoline. This new, more efficient
bracterillation process has the
potential to make traxoline one of
the most useful products within the
molecular family of lukizes snezlaus.
1.
QUIZ:
What is traxoline?
2. Where is it montilled?
3. How is traxoline
quaseled?
4. Why is traxoline
important?
What were the learning outcomes related to this content?
Misalignments
Lower Order Cognitive Skills = LOCS
Higher Order Cognitive Skills = HOCS
Practice
Testing
Outcome for Student
HOC
LOC
Frustration
LOC or none
HOC
Frustration
HOC
HOC
Excitement
and Learning
Bloom’s Taxonomy
Higher order cognitive skills
(HOCS)
Lower order cognitive skills
(LOCS)
Benjamin S. Bloom Taxonomy of educational objectives. Published by Allyn and
Bacon, Boston, MA. Copyright (c) 1984 by Pearson Education.
Cognitive
Level
Bloom Level
A Simple Phrase to Guide Categorization
Verbs Typically Associated with the Category
HOC
Evaluate
“Defend or judge a concept or idea”
Synthesize
“Create something new”
Analyze
“Distinguish parts and make inferences”
LOC/HOC
Apply
“Use information or concepts in new ways”
LOC
Comprehend
“Explain information or concepts”
Know
“Recall information”
Partial Hierarchy of Bloom’s Taxonomy
Analysis
Synthesis
Evaluation
HOCS
Application
Comprehension
Knowledge
LOCS
Practice Using Bloom’s Taxonomy
Work with a neighbor to categorize your first five test
questions as either a LOC or HOC.
What did you have to consider when doing this?
Could you easily categorize someone else’s
questions in biology?
What about in another discipline?
Writing Multiple-Choice Questions
That Test High Order Cognitive Skills
Warning!
These activities may be hazardous to your
colleague’s health as a result of undue stress.
Faculty often get distracted by the content of a question
and miss out on the process of what is being presented.
Let’s focus on the process!
Structures of Multiple-Choice Questions (MCQs)
Standard MCQ
Stem
Which of the following explains the
many unique properties of water?
a. it has many phases
b. it is a universal solvent
c. its abundance on earth
Key
d. dipole moments
Distractors
MCQ Format
Advantages
Disadvantages
Standard
Useful for measuring HOC
skills if a strong distractors
are used
If not well written the structure
can cue a student to the answer
Context
Dependent
Useful for measuring HOC
skills if strong distractors are
used
Not time efficient
Two-Tiered
The second tier may survey
students’ misconceptions
Not time efficient
Answer to one tier can cue the
answer to the other tier.
Complex
(K-Type)
Requires analytical skills of
comparing and contrasting
Not time efficient
Tends to be very misleading
Matching
Time efficient
Typically not useful for measuring
HOC skills
True/False
Time efficient
True/false with less than 5 items
have increased errors associated
with guessing
Alternate
Choice
Useful for measuring HOC
skills
Not time efficient
Less discriminating than standard
format
Take a few minutes and evaluate your exam.
What kinds of questions are you asking?
Free response, multiple-choice, or both?
Do you have questions that help you to identify
student misconceptions?
Do you have any free response questions that
requires a student to explain their reasoning or
two-tiered MCQs.
Using Two-tiered questions:
A MCQ followed by a free response requiring
students to explain why they answered the
question the way they did will be very revealing!
Enchanted Valley is located within the Olympic National park of
Washington state and is home to many black bears. A researcher is
studying the different types of berries that 20 Enchanted Valley black
bears eat.
Which of the following passages describes the researcher’s data that
would be graphed as a histogram?
A) amount of each type of berry the bears ate in a week
B) amount of salmon berries eaten by each bear in a week
C) amount of huckleberries eaten by male and female bears in a week
In the space provided below, please explain your reasoning for selecting
the answer to the above question.
Membrane Permeability to Urea
Review the graph right.
no
cholesterol
25%
cholesterol
50%
cholesterol
0
5
10
15
20
25
Temperature (C)
Which of the following passages describes the graph?
A) membrane permeability to urea is only dependent on the amount of
membrane cholesterol
B) amount of membrane cholesterol is only dependent on membrane
permeability to urea
C) membrane permeability to urea is dependent on temperature and the
amount of membrane cholesterol
D) amount of membrane cholesterol and membrane permeability to urea
are dependent on one another
30
Membrane Permeability to Urea
no
cholesterol
25%
cholesterol
50%
cholesterol
0
5
10
15
20
25
30
Temperature (C)
Which of the following best describes why you selected the answer that
you did?
The graph shows . . .
A) two dependent variables and one independent variable
B) one dependent variable and two independent variables
C) one dependent variable and one independent variable
D) two dependent variables and two independent variables
Q: Why do we care about student misconceptions?
A1: They make us crazy!
A2: We would like to engage in Scholarly Teaching
and/or Discipline-Based Education Research
Concept Inventories
Two ways of thinking about this terminology:
1. An outline of the key concepts that a course or curriculum will
cover.
2. A MCQ test based of a specific content area that uses common
student misconceptions as distractors.
Concept Inventories in Biology
Genetics Concept Assessment (GCA)
Smith, et al. 2008.
Genetics Literacy Assessment Instrument 2 (GLAI-2)
Bowling et al., 2008a;
2008b
Genetics Concept Inventory
Elrod, 2007
Conceptual Inventory of Natural Selection (CINS)
Anderson, 2003
Anderson et al., 2002
Biology literacy (http://bioliteracy.net/)
Klymkowsky and
Garvin-Doxas, 2008
Diagnostic Question Clusters: Biology
Wilson et al., 2006
D’Avanzo, 2008
Host-Pathogen Interactions (HPI)
Marbach-Ad et al., 2009
Introductory Molecular and Cell Biology Assessment
(IMCA)
Shi et al., 2010
You too can create a concept inventory.
If you do, I hope you would publish it!
It would require that the inventory is valid and reliable.
VALIDITY
Validity: Addresses whether the test measures what is
suppose to measure.
Content Validity: The test presents the key concepts and
misconceptions for a given domain as affirmed by experts.
Construct Validity: The item measures what it is intended to measure.
It requires affirmation by experts.
Face Validity: The test has the appearance of measuring what it
intends to measure, thereby motivating students to persevere in doing
their best.
RELIABILITY
Reliability: Addresses the consistency of a set of measures; it is
based on whether the test results are repeatable and not based
on random error.
Inter- and Intra-Rater Reliability: The consistency of scoring
free-response items by different instructors or the same instructor,
respectively. (GREAT FOR RUBRIC DEVELOPMENT!)
Internal Consistency: It measures whether pair-wise items on the same test,
measure the same latent variable.
Test-Re-Test Reliability: Each student should give relatively the same result
when they take the test at two different times.
Alternate Form Reliability: Each student should give relatively the same result
when they take alternate forms of the same test at two different times.
What are we doing for the rest of the day?
Applying the concepts you learned this morning
for creating teaching materials.
Work in groups to do the following in this order:
1. Create a few learning outcomes using Bloom’s Taxonomy
Target misconceptions? HOCS?
2. Write 2 summative assessments that test the learning outcomes
HOCS? MCQS?
3. Design 1 formative assessment that gives students practice at
working with the material at the same cognitive level at which you
are testing them
Help students address their misconceptions, build a
framework and develop metacognition.
POST THESE TO THE WIKI PAGE!
Download
Related flashcards

Dreams in fiction

81 cards

Rayman

15 cards

Consciousness studies

16 cards

Consciousness studies

41 cards

Create Flashcards