Developing Formative Assessments
Sarah Ades
Penn State University
workshop learning objectives (outcomes):
At the end of this workshop scholars will be able to:
• Describe the role of formative assessment in learning.
• Explain why assessments should be aligned to learning
objectives and outcomes.
• Evaluate different types of assessment activities to find
those that are aligned best with the learner’s abilities and
level of understanding to be assessed.
• Develop aligned assessment activities to provide evidence
of learning to both the instructor and learner.
I am a junior majoring in biology. I usually get A’s in my courses; only a few B’s
so far in college. I totally breezed through high school. It was so easy. I’m think that
I might go to graduate school or medical school.
This semester, I enrolled in introductory microbiology. I approach this class like
most others—I attend lecture (have only missed two this semester!), read the
textbook (usually before class, if I have time), and turn in the homework if it’s going
to be graded. Prof. Lopez is great; he’s really well organized and follows the book
closely. The homework has been helpful for learning the terms and information.
The first midterm exam in this course was NOT what I expected. None of the
questions were multiple choice. Instead, we had to write out short (and sometimes
LONG) answers. I barely finished it in the 2-hour exam period. Plus, three of the
questions tested us on things we never learned and skipped stuff we had covered in
class. For example, we learned about a specific example of the lac operon in class
last week. I worked really hard to memorize all the important steps in the process,
and then it wasn’t even on the test. But there was this question about asking us to
“describe a strategy that bacteria use to regulate gene expression and explain why
such a strategy might have been selected for over time.” How am I supposed to
know about that? I got a 72% on that test. What a crock!
What went wrong?
Handelsman, Miller & Pfund (2007) Scientific Teaching
go to: b.socrative.com
enter code: GX2MXSME
link
Instruction without goals….
Ready?
Fire!
Aim.
aligned course
What knowledge/skills do your
students already have?
What misconceptions do
your students have ?
Why are the students
taking the course?
change
throughout
course
What experiences do
your students bring to
the course?
©2010 Michael Palmer
Assessment
What is assessment?
A method to determine whether of not students learned (or accomplished) the
learning objective.
Evaluate and determine if the students are learning what they are supposed to
learn.
Measuring student learning.
Assessment is how we monitor student's ability to learn.
No real idea to be honest.
A method or means of measuring learning.
A way of determining what students actually understand and at what level they
understand.
To find out if what you think you are teaching them is what they are learning.
Assessment is a process of measuring how effectively a course or program
accomplishes its goals.
A tool that is used to measure how well students have met the learning goals the
educator has defined. It can also provide information to the educator about the
effectiveness of their teaching methods.
Tools used to measure student understanding and inform teaching practices.
A way of evaluating effectiveness of teaching and learning.
Just in Time
Student-focused
Teacher-focused
Other
Assessment
What is role of assessment in teaching and learning?
It gives instructors a way to monitor student learning.
Helps us to understand where the problem is and figure out how to improve teaching and learning.
To assure that students are learning what we intend them to be learning in the course.
Assessment is used to plan, implement, and modify courses.
It is how I know that students have learned a concept or idea. Not only how they have learned but at
what level they are learning it.
To gauge student learning for give formative feedback to help students in the learning process or to
determine final learning gains using a summative instrument.
If done correctly it should guide teaching and learning both.
To find out if you are being successful in your teaching, and to show the student where their
misconceptions are.
Teaching strategies can be modified based on the results of assessment to improve learning
outcomes.
A consistent and defined way of improve teaching (iterative process of assessment, adaptation of
practices, repeat).
Assessment should be aligned with learning outcomes and should help us identify whether or not
we've taught the materials effectively. It should include lots of low-stakes practice (formative
assessment) and culminate in summative assessment(s).
In the formative form it is to inform teaching and learning. What do I (my students) know and what
do we need to work on. In the summative form it informs instructors and students about level of
knowledge.
Just in Time
Student-focused
Teacher-focused
Other
Assessment?
 Assessment a way of providing students and teachers with
information regarding students' progress towards achieving
the learning objectives. Evidence.
 The role of assessment in teaching and learning is to provide
feedback to both the teacher and the student. It allows
teachers to tell whether their teaching methods have been
effective or whether they need to re-teach a particular
concept. On the other hand, it allows students to identify
misconceptions and gaps relating to the new content being
learned. It also allows students to identify if they are studying
effectively.
 Lets student interact with the material and practice skills
Why do we assess / question?
•Gather evidence on student learning (evaluation)
•Improve a course
Guide improvements
•Improve our teaching
•Improve society (?)
•Get feedback on student understanding
Guide teaching
•Elicit misconceptions
•Guide your own instructional decisions
•Make expectations clear to students
Guide students
•Provide feedback to students
•Give students an opportunity to gauge their progress
•Help guide student studying and learning behavior
assessments
Learner
learning
objectives
(outcomes)
•
•
•
instructional
activtiies
What evidence can show that students have achieved the desired results?
What assessment tasks and other evidence will anchor our curricular units and
thus guide our instruction?
What should we look for, to determine the extent of student understanding?
Wiggins and McTighe (2013) Understanding by Design
Novice to Expert
• According to Donald Rumsfeld: “There are known knowns. There are
things we know that we know. There are known unknowns. That is
to say, there are things that we now know we don't know. But there
are also unknown unknowns. There are things we do not know we
don't know.”
conscious unknowing:
know that you don’t
know
unconscious unknowing:
don’t know what you
don’t know
conscious knowing:
know how you know
what you know
unconscious knowing:
don’t know how you know
what you know
Learner
Novice
• Content: isolated pieces of information
to be memorized.
• Handed down by an authority.
Unrelated to world.
• Problem solving: pattern matching to
memorized recipes.
Expert
• Content: coherent structure of
concepts.
• Describes nature, established by
experiment.
• Prob. Solving: Systematic conceptbased strategies. Widely applicable.
GOAL: use assessments to guide learning, build skills, and measure
progression from novice to expert
Ambrose 2010 How Learning Works
http://www.cwsei.ubc.ca/resources/tools.htm
formative
• Adjective – of and relating to a
person’s development
• of and relating to a pedevelopment
• Synonyms
– Developmental
– Growing
– Malleable
– Impressible
– Determining
– Influential
• Purpose – “brief, low-stakes activities
that students do in order to give both
themselves and the teacher feedback
about their level of understanding.”
Stephen Chew in Chronicle of Higher Education
summative
• Adjective – of and relating to a
summation
• of and relating to a pedevelopment
• Synonyms
– Additive
– Cumulative
• Purpose – higher-stakes activities
that are “cumulative in nature and is
utilized to determine whether
students have met the goals or
student learning outcomes at the end
of a course, unit, or program.”
what types of activities can you use for
formative assessments?
http://padlet.com/sea10/formative
what types of activities can you use for
formative assessments?
•
•
•
•
•
•
•
•
•
•
•
•
One minute paper
Muddiest point
Clicker questions- ConcepTest
Empty Outlines
One sentence summary
Productive Study-Time Logs
Misconception/Presumption Check
Compare/contrast
Concept maps
Concept sketches
Examine lab notebooks
Course discussion board/blogging
see also engaugements file on wiki
Designing Aligned Assessments
Endpoint: learning objectives and outcomes – summative assessments
what students will know upon completion of this course/unit/class
Roadmap: formative
assessments and learning
activities
how students will reach goal
- What gear is needed?
- What are the intermediate
steps along the way?
- How will instructor and
students know they are
making progress?
Starting point: prior knowledge and misconceptions
what students know
Where’s the start?
Assessing prior knowledge, gaps in knowledge, and misconceptions
Multiple Choice Quiz:
• include common
misconceptions as options
• use both information
retrieval and application
questions
Concept maps:
• get an overview of what
key ideas students know
and how they think
ideas are connected.
?
Brainstorm:
• ask leading question or pose
a scenario, have students
generate answers
Self-assessment:
• How familiar are you with ___?
a) never heard of it.
b) heard of it but don’t
remember much.
c) know what it is and can
explain what it does.
d) know what it is, when and
how to use it.
Remember:
• Recall does not mean students can apply.
• Look for misconceptions as well as concepts.
• What is missing can be as informative as what is reported.
traditional approach to teaching – Prof. Lopez
you’ve read the book
and sat through the
lectures
now you’re
the expert
ready, set, go!!
developing understanding through formative assessments
- add some steps along the way practice, practice, practice (reinforcement)
increase complexity over time (scaffolding)
To develop mastery, students must acquire component skills and
knowledge, practice integrating them, and know when to apply them.
mastery
know when and how to
apply skills and knowledge
practice skills and
using knowledge
acquire component
skills and knowledge
Ambrose 2010 How Learning Works
Jigsaw activity• Formative assessments Dr. Lopez could have
used in his class addressing the following
course level learning objective.
Learning Objective: students will understand
that bacteria are able to sense the environment
around them and integrate that information to
alter gene expression.
Course Snap Shot
Micrb202: Introductory Microbiology Laboratory (IQB)
3 sections, up to 36 students per section
2nd semester freshmen – first bio lab class
taught by a faculty member with 2 learning assistants
Wed/Fri 2 hours
Inquiry-Based Intro Micro Lab
Goal(s)
By the end of this course
• you will have a strong foundation in scientific inquiry and be prepared to apply
these skills in future science courses.
• you will be able to evaluate issues in “real-life” making you a life-long educated
consumer of science
• you will have the technical skills and fundamental concepts to address important
questions confronting microbiologists in a variety of professions from medicine
to environmental microbiology.
contrasting lab approaches
Inquiry Based
basics provided, students seek
additional information as
needed
array of materials provided
students decide what to use
Traditional
background
information
materials
protocols provided
used to address scientific
question
student-directed
students told what
materials to use
protocols provided
techniques
experimental design
driven by scientific question
open-ended
generally all provided
focal point of experiment
teacher-directed, step-bystep instructions
driven by technique
outcomes
known in advance
active role as researchers
role of students
passive role following
directions
help students formulate
experiments
role of teacher
tell students what to do
learning objectives/outcomes
Students will be
able to:
formulate a clear question
develop a testable hypothesis
plan an experiment
evaluate and synthesize data
communicate findings
scientific method
IQB approach
experimental techniques
Utilize:
sterile technique
key microbiological methods
laboratory safety
concepts of microbiology
Discuss:
microbial evolution
environmental influence on microbes
microbial interactions
identify an unknown microbe
Course Design
• seven modules
• lab guides - background, experimental challenge, key
questions, relevant protocols
- online quiz on major concepts taken before each
module starts
• design experiment - hypothesis, predictions, controls
- brainstorming, group problem solving, peer and
instructor review
• perform experiment
• evaluate data and draw conclusions
- discuss with peers and instructors
- compare results with other groups
• keep lab notebook
• make classroom a scientific community
- discuss have periodic class mini-conferences
• communicate findings - written lab report or oral
presentation
Experiment 3: Bacterial Behavior- Motility and Chemotaxis
One of the distinguishing characteristics of animals is their ability to move in response to stimuli that originate from
within their own bodies or from the outside world. Many of us are attracted to the smell of fresh-baked chocolate
chip cookies and repelled by the aroma of a recently antagonized skunk. An animal moving rapidly toward an
object it recognizes as food or fleeing a harmful chemical are examples from the repertoire of responses known as
animal behavior. Even the simplest and smallest animals exhibit behaviors within their own sensory capabilities…
Resources
You can find basic information about flagella and chemotaxis in any microbiology textbook.
Online resource with an overview of bacterial motility, flagella, and taxis:
http://textbookofbacteriology.net/structure_2.html
Key concept
Bacteria behave in response to stimuli.
Challenge
We will use two different assays to determine if your pets are motile and chemotactic . We will also provide you
with a plate of swimming E. coli and an assay to determine if a substance is an attractant or a repellent. Develop a
hypothesis about E. coli’s responses to different compounds- we will provide you with some sugars and amino
acids. You are also welcome to bring in other substances that you would like to test.
Key Questions
• What advantages does chemotaxis provide for a bacterium?
• High concentrations of repellents increase the tumbling time of E. coli. How does this help the bacteria move
away from the repellent?
• Different bacteria chemotax towards different chemicals. Why might this be?
• Some bacteria are not motile. What factors in their environment might allow them to flourish without selfpropelled motility?
• Penicillium notatum is not motile. However, it spreads over the surface of a petri dish and can even spread
from one piece of bread to another. How might either of these behaviors occur in the absence of self-propelled
motility?
course modules:
reinforcement and scaffolding
techniques
motility assays
simple
microscopy
motility assays
bread, mold, and bacterial motility
the environment and chemotaxis
antibiotic
resistance
measurements
antibiotic
resistance
measurements
count bacteria
purification of
bacteria
count viruses
purification of
bacteria/viruses
microscopy
microscopy
pet
microbe
purification of
isolate mutants
bacteria/fungi
microscopy
count viruses
pathogenesis
random vs.
adaptive
mutation
microbial
fuel cell
bioenergy
skin microbiome
environmental
microbial
host-pathogen
microbial
host-pathogen host-pathogen
influences on
interactions
metabolism
interactions
metabolism
interactions
environmental microbial behaviorenvironmental identification of
environmental environmental
evolution
influences on
influences on
microbiology
microbiology
unknown
microbial growth
microbial growth
identification of
identification of
unknown
unknown
concepts
host-pathogen
interactions
evolution
complex
selective and
differential
growth media
Week 1
activity
Objective:
Students will begin to design experiment.
2
in class
Sa/SuMon Tu
Wed
Th
reading:
basic
micro
concepts
out of class
1
P.
acnes/ph
age
Day 1
intro
mini-lecture
-background
micro
-exp to
design
- overview of
procedure
Fri
Pre-class
• skin microbiometextbook
• Ted Talk podcast
• pop. press article
about P.
acnes/phage
3
Mini-Lecture
• skin
microbiome
• sebaceous
gland
• overview of
unit
• start thinking
abt
experiments to
design
• overview of
procedures
Objective:
Students will grasp basic concepts of microbiome.
Students will be familiar with overview of P. acnes and that there are
phage that infect P. acnes
assessment
1
2
3
preclass online quiz
on readings
basic knowledge
think-pair-sharebrainstorm
• list 3 factors that could
influence antibiotic
resistance of P. acnes
Analyze Data Table on host
range of P. acnes & phage
• describe, interpret, apply
• students explain what they
might find
activity
Week 2
Objectives:
Students will understand (how & why) procedures to be
followed.
Students will make connections to previous units.
Students will understand the experimental strategy for
purification of bacteriophage (Day 2).
Pre-class
• lab manual isolation steps
notebook
check
Sa/SuMon Tu
Wed
Th
out of class
reading:
lab manual
reading:
lab manual
1
Pre-class
• lab manual phage
purification
Day 3
phage
purification
in class
2
notebook
check
Fri
1
Starter activityworksheet/flowchart
Lab work
preclass online quiz
• purpose of main steps
• sterile technique
knowledge level
notebook check
Lab work
3
Day 2
Initial
Isolation
assessment
2
3
preclass online quiz
• purpose of each step
in phage purif.
• i.d. of phage- plaque
and capsid morph.
knowledge/apply level
Fill-in worksheet/flowchart
do as teams - go over as
class
• why purify phage?
• 2nd approach to get phage
notebook check
phage
morphology
Plaques?
yes
Purify!
how?
why?
no
Find those phage!
how?
Motivational methods:
• Relate experiments to real world
• Give students ownership of experiments
• Treat students like scientists
Formative assessments:
• Pre-class quizzes (3 tries)
• In class activities - design experiments, analyze data (own and from literature)
• Notebook checks
• Informal questions while circulating in class
• Students can repeat some activities until they master them
Summative assessments:
• Lab reports (graded by rubric), rewriting allowed
• Key questions – apply learning
• No lab practicums – must master methods to identify and keep pet alive
• Oral reports
Does it work?
•
•
•
•
•
informal observations
students get microscopes themselves to examine a sample
students ask for materials for controls
students discuss amongst themselves experimental design and
results
lab notebook checks show learning - improved hypotheses,
controls
self-reporting on SRTEs
- ...involved thought and understanding. It was very helpful having the instructors come around and asking questions to be sure we
-
-
•
understood why and what we were doing not just to make sure we followed a procedure correctly.
This course built on the material we learned in the beginning and continued to build on it, which personally helped me retain the
information more than just about every course I have ever taken.
The student query aspect of the course allowed me to learn that most that I possibly could have from MICRO 202. I really enjoyed
asking my own questions for each experiment and feeling as though I had autonomy to set up the experiment as I saw fit to answer my
questions. I believe that the set-up of this course allowed me to become a more confident science student and that I now have more of a
sense of how to work in lab setting--it is now common sense to me to start with a hypothesis, to figure out what controls to set up to
make sure that I can distinguish true results from inconclusive data...I don''t think I could have done that nearly as well without this
course.
applications from departmental majors for independent research
- 60% of IQB section, 17% of all other 11 sections
Designing Aligned
Assessments
2. Content
Identify components of
concept.
How are concepts
integrated?
How are concepts
applied?
1. Student
How are concepts used
in real world?
prior knowledge
misconceptions
3. Deliverable
Choose learning activity
and concept/skill to be
assessed.
Decide level of learning
to assess.
Ensure assessment is
appropriate for desired
learning outcome.
Ensure assessment will
give desired evidence of
learning.
skills
Consider how
assessment will be
evaluated.
Learning
objectives
Your turn
• Develop formative assessment aligned to your
learning objective/outcome.
• Pair with another scholar to share your assessments
• Give feedback!
Alignment Grid
Learning
Objective
Taxonomy
Level/Categor
y
Formative
Assessments
Summative
Assessment
The most important method of education . . . always has consisted
of that in which the pupil was urged to actual performance.
—Albert Einstein, Ideas and Opinions, 1954/1982, p. 60
https://www.flickr.com/photos/gforsythe/8204963410/sizes/z/in/set-72157626965187420/
References
• Connecting Learning Goals and Assessments Workshop by Stephanie
Chasteen (CU-SEI), 2011
http://www.cwsei.ubc.ca/resources/workshops.htm
• Assessment strategies, Assessments ASM 2013_Obom_6.13.13.pptx
Biology Scholars 2013 Assessment Residency
• Ambrose, S. et al. How Learning Works, 2010, Jossey-Bass a Wiley
Imprint, San Francisco
• Handelsman, J. et al. Scientific Teaching, 2007, WH Freeman and
Company, New York
• Carl Wieman Science Education Initiative at UBC
http://www.cwsei.ubc.ca/resources/index.html
• Wiggins and McTighe, 2013, Understanding by Design Expanded 2nd
Edition, ASCD