Teaching for
Understanding
in Large Classes:
Active Learning
& Assessment
Diane Ebert-May
Department of Plant Biology
Michigan State University
ebertmay@msu.edu
http://first2.org
The trouble with our times is that
the future is not what it used to be.
-Paul Valery, The Art of Poetry
Objectives: you will be
able to.....
Implement a learning cycle instructional design
Organize and use cooperative groups
Create an inquiry-based, student-centered
classroom
Develop goals/assessments
Use concept maps - maybe
Analyze data to improve instruction
Develop rubrics for assessment
Figure out if the rewards are worth “it”
Learning Cycle
Engage
Explore
Explain
Assess
Engage
Question 1
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
Students learn science best
by doing science.
Question 2
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
Science should be taught as it
is practiced.
Question 3
Please respond on a scale if 0-100
in increments of 10:
How important is it to use
multiple kinds of data to
assess student learning?
Question 4
Please respond on a scale of 0 - 100
in increments of 10:
How often do you use data to
make instructional decisions?
Question 5
Please respond on a scale of 1-5:
5=strongly agree; 4=agree; 3=neutral; 2= disagree; 1=strongly
disagree
Large lectures (+50 at UM) are
active learning environments.
Question 6
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
In my department, excellence
in teaching is rewarded at a
level comparable to excellence
in research.
Question 7
Where on the continuum is
the ideal classroom ?
Question 8
Where on the continuum is
your classroom?
Question 1
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
Students learn science best by
doing science.
Question 2
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
Science should be taught as
it is practiced.
Learners doing science...
Question 3
Please respond on a scale if 0-100
in increments of 10:
How important is it to use
multiple kinds of data to
assess student learning?
How important is it to use multiple forms of
data to assess student learning?
%
Relative Importance
n=127
Question 4
Please respond on a scale of 0 - 100
in increments of 10:
How often do you use data to
make instructional decisions?
How often do you use data to make
instructional decisions?
%
Frequency
n=127
Question 5
Please respond on a scale of 1-5:
5=strongly agree; 4=agree; 3=neutral; 2= disagree; 1=strongly
disagree
Large (+50 UM) introductory
courses are active learning
environments.
Question 6
Please respond on a scale of 1-5:
1=strongly agree; 2=agree; 3=neutral; 4= disagree; 5=strongly
disagree
In my department, excellence
in teaching is rewarded at a
level comparable to excellence
in research.
Question 7
Where on the continuum is
the ideal classroom ?
Question 8
Where on the continuum is
your classroom?
Explore
What’s up with Termites?
1. On a sheet of paper, draw two circles near
each other on the center of the page.
2. Release termites onto paper.
3. Keep creatures safe. I shall collect them in
their original habitat.
4. What do you observe about termite
behavior?
5. Develop a question your group could explore
if you had more time.
(15 minutes - select a timekeeper)
1. Develop 3 possible goals/
objectives for this ‘inquiry’
2. Develop an assessment
appropriate for one of the
goals/objectives.
Team Written response. Overhead.
Reporter Recorder Timekeeper - (10 minutes)
Checker -
Explain
What is assessment?
Data collection with the purpose of
answering questions about…
students’ understanding
students’ attitudes
students’ skills
instructional design and implementation
curricular reform (at multiple grainsizes)
Why do assessment?
Improve student learning and development.
Provides students and faculty
substantive feedback about student
understanding.
Challenge to use disciplinary research
strategies to assess learning.
Assessment
Guidelines for thinking
about research...
What did students learn? (assessment data)
Why did students respond a particular way? (research)
What are the working hypotheses or questions?
What has already been done? Literature says...
How and why to select methods? Conduct study...
How to analyze and interpret data?
What do the results mean?
Are findings valid and generalizable?
Research
Data collection
Assessment Gradient
High
Ease of
Assessment
Multiple Choice, T/F
Diagrams, Concept
maps, Quantitative
response
Short answer
Low
Potential for
Assessment of
Learning
Essay, Research
papers/ reports
Low
High
Oral Interview
Identify desired
outcomes
Determine
acceptable evidence
Design learning
experiences and
instruction
Wiggins and McTighe 1998
Objective (outcome):
Students will demonstrate
understanding of evolution by
natural selection.
Problem (evidence)
Write a scenario that explains the
phenotypic changes in the tree and
the animal. Use your understanding
of evolution by natural selection.
Text
How do we develop rubrics?
1. Describe the goal/objective for the
activity, problem, task...
2. Select the assessment tasks
aligned with goals
3. Develop performance standards
4. Differentiate levels of responses
based on clearly described criteria
5. Rate (assign value) the categories
Scoring Rubric for Quizzes and Homework
Level of Achievement
Exemplary
(5 pts)
General Approach
• Addresses the
question.
• States a relevant,
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (no
errors).
Comp rehension
• Demonstrates an accurate and
complete understanding of the
question.
• Backs conclusions with data
and warrants.
• Uses 2 or more ideas,
examples and/or arguments that
support the answer.
Adequate
(3 pts)
• Does not address the
question explicitly,
although does so
tangentially.
• States a relevant and
justifiable answer.
• Presents arguments in
a logical order.
• Uses acceptable style
and grammar (one
error).
• Demonstrates accurate but only
adequate understanding of
question because does not back
conclusions with warrants and
data.
• Uses only one idea to support
the answer.
• Less thorough than above.
Needs Improveme nt
(1 pt)
• Does not address the
question.
• States no relevant
answers
• indicates
misconceptions.
• Is n ot clearly or
logically organized.
• Fails to use acceptable
style and grammar (two
or more errors).
• Does not demonstrate accurate
understanding of the question.
• Does not provide evidence to
support their answer to the
question.
No Answer (0 pts)
Advantages of Scoring Rubrics
Improve the reliability of scoring written
assignments and oral presentations
Convey goals and performance expectations
of students in an unambiguous way
Convey “grading standards” or “point values”
and relate them to performance goals
Engage students in critical evaluation of their
own performance
Save time but spend it well
Limitations of Scoring Rubrics
Problem of criteria
Problem of practice and regular use
Scoring Rubric website
http://www.wcer.wisc.edu/nise/cl1/flag/
Sample Rubrics for Environmental Science
http://www.msu.edu/~ebertmay/isb202/home.html
What Type of Learning?
Bloom (1956)
Cognitive Domain of Educational Objectives
6 categories Knowledge
Comprehension
Application
Analysis
Synthesis
Evaluation
Condense to 4 - easy to work with
Cognitive Levels
Knowledge - remember
Comprehension and Application grasp meaning, use, interpret
Critical Analysis - original thinking,
open-ended answers, whole to parts,
parts to whole, evaluation
Synthesis - make connections, evaluate
Model for
Learning - System
So what is a concept?
1. Take a piece of paper -- one of the ‘dino-tree’
responses will work-2. Fold it in half.
3. On the top half, draw a bicycle.
4. On the bottom half, describe bicycle in writing.
5. Which is a better representation of the concept
‘bicycle’?
6. Would a concept map be “best”?
Concepts are...
Lexical
Concept Maps
are
represent
connected
with
has
has
Structure
Hierarchy
is constructed
with
Assessment
Concepts
Knowledge or
Understanding
Visual Diagrams
Or Models
Used for
display
Linking Words
Organization
Context
promotes
Reflection &
Learning
Prior
Knowledge
New
Information
www.ctools.msu.edu
Robo Grader in Action
Student’sText
Concept Map
For a course you teach .....
1. Select a concept that is critical for your students to
understand.
2. Identify 4 or 5 subconcepts that are important to
understanding that concept
e.g., DNA - Gene- Chromosome - Enzyme
3. Arrange them by rank order - top most general,
bottom most specific
4. Add linking lines to make connections between two
concepts
5. Add linking words that describe the relationship
between two concepts
Pre-Posttest Analysis
Does active, inquiry-based instructional
design influence students’
understanding of evolution and natural
selection?
Alternative Conceptions:
Natural Selection
■ Changes in a population occur through a
gradual change in individual members of a
population.
■ New traits in species are developed in
response to need.
■ All members of a population are genetically
equivalent, variation and fitness are not
considered.
■ Traits acquired during an individual’s lifetime
will be inherited by offspring.
Explain the changes that occurred in
the tree and animal. Use your current
understanding of evolution by natural
selection.
(AAAS 1999)
Misconception: individuals evolve new traits
% of Students
n=80; p<.01
Misconception: evolution is driven by need
% of Students
n=80; p<.01
In guppy populations, what are the primary
changes that occur gradually over time?
a. The traits of each individual guppy within a population
gradually change.
b. The proportions of guppies having different traits within
population change.
c. Successful behaviors learned by certain guppies are
passed on to offspring.
d. Mutations occur to meet the needs of the guppies as
the environment changes.
Anderson et al 2002
Posttest: Student responses to mc
% of Students
n=171
*
Animal/Tree Posttest:
Gain in student understanding of fitness
% of Students
n=80; p<.01
Design Experiment
Quantitative Data
Qualitative Data
Ebert-May et al. 2003 Bioscience
Question
How do assessment questions help us
determine students’ prior understanding
and progressive thinking about the
carbon cycle.
Instructional Design
• Two class meetings on carbon cycle (160 minutes)
• Active, inquiry-based learning
– Cooperative groups
– Questions, group processing, large lecture
sections, small discussion sections, multi-week
laboratory investigation
– Homework problems including web-based
modules
• Different faculty for each course
– One graduate/8-10 undergraduate TAs per
course
Experimental Design
Two introductory courses for majors:
Bio 1 - organismal/population biology (faculty A)
Bio 2 - cell and molecular biology (faculty B)
Three cohorts:
Cohort 1
Bio 1 (n=141)
Cohort 2
Bio1/Bio2 (n=63)
Cohort 3
Other/Bio2 (n=40)
Assessment Design
Multiple iterations/versions of the carbon cycle
problem
Pretest, midterm, final with additional
formative assessments during class
Administered during instruction
Semester 1 - pretest, midterm, final exam
Semester 2 - final exam
Grandma Johnson Problem
Hypothetical scenario: Grandma Johnson had
very sentimental feelings toward Johnson
Canyon, Utah, where she and her late husband
had honeymooned long ago. Her feelings
toward this spot were such that upon her death
she requested to be buried under a creosote
bush overlooking the canyon. Trace the path of
a carbon atom from Grandma Johnson’s
remains to where it could become part of a
coyote. NOTE: the coyote will not dig up
Grandma Johnson and consume any of her
remains.
Analysis of Responses
Used same scoring rubric (coding scheme) for all
three problems - calibrated by adding additional
criteria when necessary, rescoring:
Examined two major concepts:
Concept 1: Decomposers respire CO2
Concept 2: Plants uptake of CO2
Explanations categorized into two groups:
Organisms (trophic levels)
Processes (metabolic)
Coding Scheme
Code
Organisms
Code
P rocesses and pathways
1
Decomposers
IA
Cellular Respiration
IB
Release CO
IIA
P athway of Carbon
2
P rimary
producers
2
IIA _1: through Air
IIA _2 : through Root
IIA _3 : no mention about pathway
3
4
Herbivore
Carnivore
IIB
Make Glucose
II C
P hotosynthesis
III
Respiration
(glycolysis, Kreb cycle)
IV
Respiration
(glycolysis, Kreb cycle)
Correct Student Responses (%)
Cellular Respiration by Decomposers
Bio1/Bio2
Other/Bio2
Friedmans, p<0.01
Correct Student Responses (%)
Pathway of Carbon in Photosynthesis
Bio1/Bio2
Other/Bio2
Friedmans, p<0.05
Question 9
True or False?
Assessing student learning
in science parallels what
scientists do as researchers.
Parallel: ask
questions
Description:
-What is happening?
Cause:
-Does ‘x’ (teaching strategy) affect ‘y’
(understanding)?
Process or mechanism:
-Why or how does ‘x’ cause ‘y’?
Parallel: collect data
We collect data to find out what our
students know.
Data helps us understand student
thinking about concepts and
content.
We use data to guide decisions
about course/curriculum/innovative
instruction
Parallel: analyze data
Quantitative data - statistical
analysis
Qualitative data
break into manageable units and
define coding categories
search for patterns, quantify
interpret and synthesize
Valid and repeatable measures
Parallel: peer review
Ideas and results are peer reviewed formally and/or informally.
Examples to View
Assessment of Workshop
Minute Paper
Q1 What was helpful to you? Why?
Q2 What would have been helpful to
you ? Why?
Q3 What should we focus on in future
workshops?
IRD Team at MSU
Janet Batzli - Plant Biology [U of Wisconsin]
Doug Luckie - Physiology
Scott Harrison - Microbiology (grad student)
Tammy Long - Plant Biology
Deb Linton - Plant Biology (postdoc)
Rett Weber - Plant Biology
Heejun Lim - Chemistry Education
Duncan Sibley - Geology
Rob Pennock - Philosophy
Charles Ofria - Engineering
Rich Lenski - Microbiolgy
*National Science Foundation