Learning Objectives and
Classroom Assessment
Jeff Froyd, Texas A&M University
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Workshop Presenter
• Jeff Froyd, Director of Academic
Development
• Educational Achievement Division, College of
Engineering, Texas A&M University
• Project Director, Foundation Coalition
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Acknowledgement
• Russ Pimmel, NSF Program Officer
• Former Professor of Electrical Engineering at the
University of Alabama
• Assembled much of the material for similar
workshops
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Objectives: Participants will
gain experience in
• Describing the rationale for preparing and using
learning objectives in an individual course
• Preparing specifications for high quality learning
objectives
• Writing learning objectives for a single course
• Preparing specifications for assessment
processes/tools
• Generating alternative assessment
processes/tools for a single course
• Selecting assessment processes/tools for a single
course
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Agenda
• Background for learning objectives
• Specifications for learning objectives
• Writing learning objectives
• Background for classroom assessment
• Specifications for assessment processes/tools
• Background on alternatives for assessment processes/tools
• Generating alternatives for assessment processes/tools
• Selecting assessment processes/tools
• Review workshop activities
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Agenda
• Background for learning objectives
• Specifications for learning objectives
• Requirements for specifications
• Team exercise: develop specifications for learning objectives
• Workshop exercise: improve specifications for learning objectives
• Writing learning objectives
• Individual exercise: write objectives for a course or a portion of a
course
• Team exercise: review individual objectives
• Workshop exercise: develop a list of suggestions for writing
learning objectives
• Individual exercises: revise objectives Team exercise: review
individual objectives
• Workshop exercise: reflection
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Agenda
• Background for classroom assessment
• Specifications for assessment processes/tools
• Team exercise: develop a set of specifications for assessment
processes/tools
• Workshop exercise: improve sets of specifications for assessment
processes/tools
• Generating alternatives for assessment processes/tools
• Team exercise: select some learning objectives and generate alternative
assessment processes/tools
• Background on alternatives for assessment processes/tools
• Selecting assessment processes/tools
• Individual exercise: select a set of learning objectives and generate
alternative assessment processes/tools
• Individual exercise: select one or more assessment processes/tools that
you would use in your course
• Team exercise: share and review choices of assessment processes/tools
• Review workshop activities
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Learning Objectives
Background for Learning Objectives
Session Objective: At the end of the session,
participants will describe themselves as more
confident in their ability to hold productive
conversations with their colleagues regarding the
place and importance of learning objectives in the
teaching-learning process
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Stakeholders for Learning Objectives
• Who are the stakeholders in conversations
about preparing and applying learning
objectives?
•
•
•
•
Faculty
Students
Employers
Accreditation organizations
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
EC Program Outcomes
(a) an ability to apply knowledge of mathematics, science, and engineering
(b) an ability to design and conduct experiments, as well as to analyze and
interpret data
(c) an ability to design a system, component, or process to meet desired
needs
(d) an ability to function on multi-disciplinary teams
(e) an ability to identify, formulate, and solve engineering problems
(f) an understanding of professional and ethical responsibility
(g) an ability to communicate effectively
(h) the broad education necessary to understand the impact of engineering
solutions in a global and societal context
(i) a recognition of the need for, and an ability to engage in life-long learning
(j) a knowledge of contemporary issues
(k) an ability to use the techniques, skills, and modern engineering tools
necessary for engineering practice.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Bloom’s Taxonomy of Cognitive Learning
• Knowledge: defines, recalls, matches,
reproduces
• Comprehension: explains, gives examples
• Application: discovering, assessing,
computing
• Analysis: breaking down, organizing,
inferring
• Synthesis: creating, putting together
• Evaluation: appraising, judging, selecting
Western
WesternMichigan
MichiganUniversity,
University,1825September
October 2002,
2003,Kalamazoo,
Kalamazoo,Michigan
Michigan
Purpose of Learning Objectives
• Communicate expectations for a
course
• Provide a context for what will be
learned
http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Objectives and Students
• Objectives help students
•
•
•
•
Clarify their personal goals
Provide framework for measuring their success.
Reduce their anxiety
Improve their studying effectiveness
• Objectives help instructors
• Guide preparation of classroom material
• Make homework assignments
• Aid in test design
Source - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Strategies For Workshop
Teams
• Be positive, supportive, and cooperative
• Limit critical or negative comments
• At least 5 positive comments for every negative
comment
• Be brief and concise in discussions
• Avoid lengthy comments, stories, or arguments
• Stay focused
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Roles
• Assign team roles & follow through on
responsibilities
• Coordinator -- Coordinates discussion & develops
consensus
• Recorder -- Writes down the ideas & reports them
• Gatekeeper -- Keeps the team on the subject
• Timer -- Makes sure the team stays on schedule
• With smaller teams – combine gatekeeper &
timer
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Workshop Team Roles
• For first exercise
• Coordinator – Individual with largest class last
semester
• Recorder/Reporter – Individual on left of coordinator
• Gatekeeper/timer -- Individual on left of recorder
• Timer -- Individual on left of gatekeeper
• Roles rotate clockwise on subsequent
exercise
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Form teams of four people
• Time: 5 minutes
• Develop at least four (4) advantages and
four (4) disadvantages of preparing
learning objectives for a course.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
•
•
•
•
•
•
•
Advantages
Disadvantages
Communication with
• Could be too narrow or
students expectations and
focused
what they need to learn
• Explicitly stating learning
Offers focus in preparing
objectives takes (too?)
material to accomplish
much time
goals
• Busy work for faculty
Evaluation tool
• Could turn into curriculum
Tie a course to program or
nightmare
curriculum
• Poorly written objectives
Clarifying goals to
don’t help
students seeks buy-in and • Could be cumbersome,
self-assessment
ambiguous
ABET Assessment tool
• Could be more difficult to
Michigan
University, 18 Septembermeasure
2003, Kalamazoo, Michigan
StatingWestern
goals
facilitates
Learning Objectives
Preparing Specifications for Learning
Objectives
Session Objectives
At the end of the session, participants will
 Write specifications for learning objectives
 Describe themselves as more confident in their
ability to describe quality learning objectives.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Form of Learning Objectives
• Write objectives as student outcome
statements
• Objectives should answer the questions
• "What must students do to prove that they have
succeeded?"
• "What should students be able to do as a
consequence of instruction?"
Source - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Elements of an Objective
• Objective must contain three basic
elements:
• Verb describing an observable action
• Conditions of this action
• “When given x you will be able to..."
• Level of acceptable performance
Source - http://wings.buffalo.edu/vpaa/ctlr/events/brazeau/index.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Verbs for Objectives
Verbs for constructing concrete objectives:
analyze
compare
demonstrate
discuss
identify
justify
grasp
state
compute
appreciate
direct
display
infer
list
report
synthesize
classify
contrast
derive
evaluate
integrate
understand
respond
name
collaborate
define
designate
know
interpret
organize
solicit
explain
Modified from http://www.brown.edu/Administration/Sheridan_Center/pubs/syllabus.html#coura
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Learning Objectives Verbs for
Categories in Bloom’s Taxonomy
• Knowledge
• Define, describe, list, reproduce, enumerate
• Comprehension
• Classify, explain, discuss, give example, summarize
• Application
• Determine, develop, compute, chart, utilize
• Analysis
• Correlate, diagram, distinguish, outline, infer
• Evaluation
• Compare & contrast, critique, justify, conclude
• Synthesis
• Adapt, combine, compare, contrast, design, generate
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Complex Versus Simple Objectives
• One complex objective versus several simple ones?
• High-level versus low-level objectives
Example
• One complex objective (4 or 5 weeks of classes)
“Given a verbal description of a digital module, develop an implementation
using any of 7 different logic devices”
• 15 to 20 simpler objectives (1 or 2 per class)
“Given a verbal description, draw the truth table”
“Given a truth table, obtain a minimum-cost equation”
…
“Draw the the NAND-gate implementation for an equation”
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Comparison Of Complex & Simple Objectives
• Multiple simple objectives
• More manageable “chunks” for students
• Explicit objective(s) for each class
• Simple (more manageable) homework problems
and test questions
• Single complex objective
• Student's attention directed to the overall process
• May lead to higher level learning
• Students must deal with complexity
• Students must subdivide problem on their own
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Advantages Of Simple Objectives
• Advantages of simple objectives are more
important in
• Large classes rather than small classes
• Introductory courses rather than advanced courses
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
How To Deal With
“Understand” In Objectives
• How do you write objectives when you
want students to “understand” a complex
concept, system, or process
• Identify specific tasks that indicate
“understanding”
• Specify objectives for each task
• Similar comments apply to “know”,
“appreciate”, “value”
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
How To Deal With “Understand” In
Objectives -- Example
• In our computer architecture course we want
students to “understand” a sample architecture
made up of several modules
• What would students be able to do if they
“understood”
• Objectives –
• Students should be able to identify:
All the modules and interconnecting signals
Modules involved in a given system-level operation
Output values for a given input values for each module
Sub-module changes given a system level change
…
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Writing Objectives –
Piecemeal Approach
• Writing low-level objectives for a whole course may be
overwhelming
• Use a “piecemeal” approach
• Write your lectures and define the homework as usual
• After each class -- write down what you expect the students to be able to
do
• These become a list of objectives
• Give them to the students before each exam
• Use them to write the exam
• As semester progresses -- may become comfortable
writing the objectives before you prepare your lecture
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Evolving Objectives
• In a 3-credit semester course
• Russ Pimmel (UA, now NSF) started with over 100
objectives
• Four offerings later -- down to about 50
• Eliminated peripheral “stuff” that was not
central
• Broadening, informational, perspective material
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Objectives and Homework
Assignments
• Homework assignments should match objectives
• Students need to practice and explore the skills, knowledge,
and attitudes defined in objectives
• Frequently require supplementary homework
problems
• In some of my courses
• 1/3 of homework is from textbook
• Rest are supplementary problems
• With well defined objectives
• Writing homework problems is straightforward
• Same is true for exam questions
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Student’s Use Of Objectives
• Survey in 400-level required course
• Did you find the objectives helpful?
Yes --- 52 %
No ---- 48%
• Did you read the objectives?
Frequently ------ 22 %
Occasionally -- 37 %
Never ------------- 41 %
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise -- Guidelines For
Learning Objectives
• Task
• Write 3 or 5 guidelines for good learning objectives
• What are the common features?
• What should objectives look like?
• Think of guidelines as specifications
• Methodology
•
•
•
•
Brain storm individually -- 2 minutes
Establish consensus as a team -- 5 minutes
Report team results -- 3 minutes
Revise guidelines as a team-- 2 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
•
•
•
•
•
•
•
•
•
•
Must be testable and measurable
Achievable
Clearly and precisely articulated
Appropriate to course and audience
Relate to program objectives
Linked to course outcomes to allow for
assessment
Simple, one sentence, common format
Simple better than complex
Should be specific and unambiguous
Relate to topic coverage
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Learning Objectives
Preparing Learning Objectives
Session Objectives
At the end of the session, participants will
 Write learning objectives for one or more courses
that they teach
 Describe themselves as more confident in their
ability to describe quality learning objectives.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
EC Program Outcomes
(a) an ability to apply knowledge of mathematics, science, and engineering
(b) an ability to design and conduct experiments, as well as to analyze and
interpret data
(c) an ability to design a system, component, or process to meet desired
needs
(d) an ability to function on multi-disciplinary teams
(e) an ability to identify, formulate, and solve engineering problems
(f) an understanding of professional and ethical responsibility
(g) an ability to communicate effectively
(h) the broad education necessary to understand the impact of engineering
solutions in a global and societal context
(i) a recognition of the need for, and an ability to engage in life-long learning
(j) a knowledge of contemporary issues
(k) an ability to use the techniques, skills, and modern engineering tools
necessary for engineering practice.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Bloom’s Taxonomy of Cognitive Learning
• Knowledge: defines, recalls, matches,
reproduces
• Comprehension: explains, gives examples
• Application: discovering, assessing,
computing
• Analysis: breaking down, organizing,
inferring
• Synthesis: creating, putting together
• Evaluation: appraising, judging, selecting
Western
WesternMichigan
MichiganUniversity,
University,1825September
October 2002,
2003,Kalamazoo,
Kalamazoo,Michigan
Michigan
Challenges in Engineering Education
• Challenges
• Challenge of lifelong learning
• Challenge of problem solving
• Challenge of engineering design
• Challenge of transfer
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
ENGR111
(Mean)
CVEN349
(Mean)
Significance*
Information Processing
60.68
60.29
0.930
Test Strategies
64.33
55.18
63.27
59.29
0.794
0.342
Anxiety
60.52
67.12
0.147
Attitude
42.47
63.30
34.56
59.29
0.080
0.397
Concentration
61.31
54.56
0.144
Self-testing
52.47
60.24
55.23
37.92
45.04
47.65
0.006
0.005
0.134
LASSI SCALE
Skill Component
Selecting Main Ideas
Will Component
Motivation
Self-regulation Component
Study Aids
Time Management
Lifelong
Learning at Penn
State
Self-Directed Learning
Readiness Survey (SDLRS),
Guglielmino & Associates,
http://www.guglielmino734.
com/prod01.htm, March
2003. 27
“Although the data suggest a slight upward upward trend, the trend proved not to
be statistically significant based upon an analysis of variance (ANOVA). Thus the
cross-sectional study did not find evidence of an increase in readiness for selfdirected learning, even for students in the later semesters who are taking elective
courses and their capstone courses.”
Litzinger, T., Wise, J., Lee, S., and Bjorklund, S. (2003) Assessing Readiness
for Self-directed Learning, Proceedings, ASEE Annual Conference
Challenge of Problem Solving
“Despite individual professors’ dedication and efforts to develop
problem solving skill, “general problem solving skill” was not
developed in the four years in our undergraduate program.
Students graduated showing the same inability that they had
when they started the program. Some could not create
hypotheses; some misread problem statements. During the fouryear undergraduate engineering program studied, 1974-1978,
the students had worked over 3000 homework problems, they
had observed about 1000 sample solutions being worked on the
board by either the teacher or by peers, and they had worked
many open-ended problems. In other words, they showed no
improvement in problem solving skills despite the best intentions
of their instructors.”
Woods, D. et al (1997) “Developing Problem Solving Skills: The McMaster
Problem Solving Program,” Journal of Engineering Education,
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Problem Solving
• Ineffective approach #1. give the students
open-ended problems to solve; This, we
now see, is ineffective because the
students get little feedback about the
process steps, they tend to reinforce bad
habits, they do not know what processes
they should be using and they resort to
trying to collect sample solutions and
match past memorized sample solutions to
new problem situations.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Problem Solving
• Ineffective approach # 2: Show them how you
solve problems by working many problems on
the board and handing out many sample
solutions
• This, we now see, is ineffective because teachers know too
much. Teachers demonstrate "exercise solving". Teachers
do not make mistakes; they do not struggle to figure out
what the problem really is. They work forwards; not
backwards from the goal. They do not demonstrate the
"problem solving" process; they demonstrate the "exercise
solving" process. If they did demonstrate "problem solving"
with all its mistakes and trials, the students would brand the
teacher as incompetent. We know; we tried!
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Problem Solving
• Ineffective approach #3: Have students
solve problems on the board
• Different students use different approaches to
solving problems; what works for one won't work
for others. When we used this method as a
research tool, the students reported "we learned
nothing to help us solve problems by watching Jim,
Sue and Brad solve those problems!"
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Problem Solving
• Through four research projects we identified why
and how these and other teaching methods failed
to develop process skills and which methods
were successful in developing the skills
•
Woods, D.R., J.D. Wright, T.W. Hoffman, R.K. Swartman and I.D. Doig (1975) "Teaching
Problem Solving Skills," Annals of Engineering Education, 1, 1, 238-243.
•
Woods, D.R. et al. (1979) "Major Challenges to Teaching Problem Solving" Annals of
Engineering Education, 70, No. 3 p. 277 to 284, 1979 and "56 Challenges to Teaching
Problem Solving" CHEM 13 News no. 155 (1985).
•
Woods, D.R. (1993a) "Problem solving - where are we now?" J. College Science
Teaching, 22, 312-314.
•
Woods, D.R. (1993b) "Problem solving - what doesn't seem to work," J. College
Science Teaching, 23, 57-58.
•
Woods, D.R. (1993c) "New Approaches for developing problem solving skills," J.
College Science Teaching, 23, 157-158.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Engineering Design
The literature is filled with positive comments from students,
instructors, and industrial sponsors who have participated in
capstone design courses. The vast majority of participants feel
that the course benefited all involved.
The nature of capstone design courses, however, often leads to
a purely subjective evaluation with little or no “hard
evidence” of actual benefits. Born, for example, does not
attempt to prove the value of senior level design courses. He
simply states that he is convinced from his experiences that such
courses are valuable. Other educators have similar “feelings” as
to the relative costs and benefits of capstone design courses.
Dutson, A.J., Todd, R.H., Magleby, S.P., Sorensen, C.D., (1997) “A Review of Literature
on Teaching Engineering Design Through Project-Oriented Capstone Courses.” Journal of
Engineering Education
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Transfer
Researches posed this problem to people.
"Suppose you are a doctor faced with a patient who has a
malignant tumor in his stomach. It is impossible to operate on
the patient, but unless the tumor is destroyed the patient will die.
There is a kind of ray that can be used to destroy the tumor. If
the rays reach the tumor all at once at a sufficiently high intensity,
the tumor will be destroyed. Unfortunately, at this intensity the
healthy tissue that the rays pass through on the way to the tumor
will also be destroyed. At lower intensities the rays are harmless
to healthy tissue, but they will not affect the tumor either. What
type of procedure might be used to destroy the tumor with the
rays, and at the same time avoid destroying the health tissue?"
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Transfer
Consider the following story
"A small country was ruled from a strong fortress by a dictator. The fortress
was situated in the middle of the country, surrounded by farms and villages.
Many roads led to the fortress through the countryside. A rebel general vowed
to capture the fortress. The general knew that an attack by his entire army
would capture the fortress. He gathered his army at the head of one of the
roads, ready to launch a full-scale direct attack. However, the general then
learned that the dictator had planted mines on each of the roads. The mines
were set so that small bodies of men could pass over them safely, since the
dictator need to move his troops and workers to and from the fortress.
However, any large force would detonate the mines. Not only would this blow
up the road, but it would also destroy many neighboring villages. It therefore
seemed impossible to capture the fortress. However, the general devised a
simple plan. He divided his army into small groups and dispatched each
group to the head of a different road. When all was ready he gave the signal
and each group marched down a different road. Each group continued down
it road to the fortress at the same time. In this way, the general captured the
fortress and overthrew the dictator."
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Challenge of Transfer
• After the subjects read and summarized this story,
they were asked to solve the tumor problem under
the guise of a separate experiment.
• Given the clear analogy, you might think that
performance would be near ceiling. Surprisingly, only
30% of the subjects offered a convergence solution.
• Moreover, when these same subjects were given the
suggestion that they should use the General story,
80% provided a convergence solution.
• This finding demonstrates that half the subjects could
apply the General story to the tumor problem when
they were instructed to but did not do so on their own.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Focusing Activity (8 minutes)
• INDIVIDUALLY – use 3 minutes to write
your description of learning, what it is,
what it looks like, how you might recognize
when it has occurred, etc.
• AS A PAIR – use 5 minutes to discuss
descriptions with someone sitting next to
you. If you have additional time, develop a
consensus description of learning.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Focusing Activity
• ??
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Individual Exercise -- Writing
Learning Objectives
• Individually write a set of objectives for a
topic representing a few classes
• Something that you recently did in class
• Follow your team’s guidelines
• Questions to consider about your objectives
•
•
•
•
Do they define student behavior?
Are they observable, measurable?
Can you write homework & exam problems?
Are they consistent with the instructor’s intent?
• Time: 15 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise -- Reviewing
Learning Objectives
• Review each other’s objectives
• Questions to consider in reviewing objectives
• Do they follow your team’s guidelines
• Do they define student behavior?
• Are they observable, measurable?
• Can you write homework problems & exam questions?
• Are they consistent with the instructor’s intent?
• 15 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Individual Exercise -- Revising
Your Learning Objectives
• Rewrite your learning objectives based on
your team’s review
• Report on biggest improvement
• 10 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Biggest Improvement
• Put quantifiable goals that students would
achieve
• Clarify my objectives
• Classify learning objectives to go from basic to
specific
• Make a different level of objectives from whole
course to chapter to each lecture
• Make few higher level objective to organize
multiple lower level objectives to support
• Identify more complex goals and separate into
simpler components for teaching
communications
Western
Michigan University, 18 September 2003, Kalamazoo, Michigan
• Improve
clarity
Minute Paper
• Write a one-sentence answer to
the following question:
• What is the “muddiest point” about
learning objectives?
• (What is the most confusing point?)
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Muddiest Point
• How to incorporate those that relate to attitude and enthusiasm for
lifelong learning
• Why do we spend time writing things that we do and we do best?
• Answer to my question about the level and number of learning
objectives that are appropriate for publication in my syllabus. Are
there “intermediate” not published objectives?
• How to relate a given objective to Bloom’s Taxonomy level, and how
important is it to do so?
• Ways of measuring each learning objective
• How many objectives should be used – level of detail – then how do
we practically measure them?
• “Being brief” vs. “Being precise and accurate”
• Being able to link objectives to larger program outcomes or goals
• Organizing subject matter to achieve learning objectives
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise -- Reflection on
Learning Objectives
• Assume that you are a debate
team
• Write the single best pro and con
arguments for the statement
• “Using learning objectives improves
student leaning.”
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Feedback on Learning Objectives
• ??
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Agenda
• Background for classroom assessment
• Specifications for assessment processes/tools
• Team exercise: develop a set of specifications for assessment
processes/tools
• Workshop exercise: improve sets of specifications for assessment
processes/tools
• Generating alternatives for assessment processes/tools
• Team exercises: select some learning objectives and generate alternative
assessment processes/tools
• Background on alternatives for assessment processes/tools
• Selecting assessment processes/tools
• Individual exercise: select a set of learning objectives and generate
alternative assessment processes/tools
• Individual exercise: select one or more assessment processes/tools that
you would use in your course
• Team exercise: share and review choices of assessment processes/tools
• Review workshop activities
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment
Background for Classroom Assessment
Session Objective: At the end of the session,
participants will describe themselves as more
confident in their ability to hold productive
conversations with their colleagues regarding the
place and importance of assessment instruments and
processes in the teaching-learning process
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Pre-Assessment
• Exercise
• Classroom Assessment
• Individual
• Write a one-sentence answer to the
following question:
• “What can you do in the last few minutes of class
to determine how well your class learned what you
taught that day?”
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment
• At end of session, participants will be able
to:
• Define several classroom assessment tools
• Discuss the importance of using classroom tools in
the teaching-learning process
• Write assessment tools for their classes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Resources on the Web
• www.hcc.hawaii.edu/intranet/committees/FacDevCo
m/guidebk/teachtip/assess-1.htm
• www.hcc.hawaii.edu/intranet/committees/FacDevCo
m/guidebk/teachtip/assess-2.htm
• www.siue.edu/~deder/assess/catmain.html
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Types Of Assessment
• Assessment
• Used in many contexts
• Used for several different foci
• Classroom
•
•
Teacher focus: concerns your performance
Student focus: concerns their individual performances
• Program
• Alumni (as a group) focus: ABET Criterion 2
• Graduates (as a group) focus: ABET Criterion 3
• Individual graduate focus: Comprehensive exams
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Types Of Assessment
• Classroom, teacher focus
•
•
Question -- How effective was a lecture, assignment, lab?
Tools -- One-minute paper, student survey
• Classroom, student focus
•
•
Question – Did a specific student achieve the learning
objectives?
Tools -- Exams, reports, presentation
• Program, graduates (as a group) focus
•
•
Question -- How well did a group of students achieve a set
of objectives (outcomes) in a program or course?
Tools -- Standardized tests, design project report analysis
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment
• Two fundamental questions:
• How well are learners learning?
• How effectively are teachers teaching?
• Deals with better learning and more
effective teaching
• Provides feedback about effectiveness as teachers
• How students learn
• How they respond to particular teaching approaches.
• Gives students a measure of their progress as
learners
Modified from www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Incorrect Assumptions About
Teaching & Learning
• Instructors assume students learn what they teach
• Tests, concept inventories, and term papers provide
disappointing evidence to the contrary
• Students have not learned as much or as well as
expected
• Gaps between what was taught and what was learned
• Sometimes considerable gaps
• Instructors notice gaps too late to remedy the
problems
• Classroom assessment can uncover gaps earlier
www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment
Getting Started
• Planning
• Select one, and only one, of your classes
• Choose a simple and quick technique
• Implementing
• Make sure the students understand the procedure
• Analyze student’s responses as soon as possible
• Responding -- “Close the feedback loop”
• Tell students what you learned and what you will do
about it - motivates students to become actively
involved www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-1.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Classroom Assessment – Five
Suggestions
• Don’t use any technique that does not appeal to
you
• Don't make it into a self-inflicted chore or burden.
• Try it yourself before you use it with students
• Allow more time than you think you will need
• To carry out the assessment
• To respond to it
• Make sure to "close the loop"
www.hcc.hawaii.edu/intranet/
committees/FacDevCom/guid
ebk/teachtip/assess-1.htm
• Let students know
•
•
What you learned from their feedback
How you and they can use that information to improve learning
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Process
• Brain storm individually -- 1 minutes
• Establish consensus as a team -- 4 minutes
• Task: Write 2 to 4 guidelines for good
classroom assessment tools (Think of
guidelines as specifications)
• What are the common features?
• What should they look like?
• Do all guidelines have to apply to all tools?
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Tool should be efficient use of time for instructor and
class
• Easy for students to use
• Can be used across different classes and situations
• Built into teaching methodology and doesn’t look like an
add-on
• Tied to learning objectives or limited number of topics
• Flexible, not dependent on particular technology
• Done repeatedly throughout semester
• Easy for faculty to administer and evaluate
• Self-documenting, easy to show results to others
• Clear and precise instructions
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Process
• Brain storm individually -- 2 minutes
• Generate different ideas as a team -- 3 minutes
• Task: Generate as many possible good
classroom assessment tools that might
satisfy your team guidelines
• Generate different ideas
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
•
•
•
•
•
•
•
•
Muddiest point
Quizzes on specific objectives
What is clear?/What is unclear?
What did you learn today?
Strengths/Improvements/Insights (SII)
Self or peer assessment
Who did you share X with today?
Weekly feedback on homework instruction – what
was clear? What was unclear?
• Continuous Improvement – What was beneficial?
What would you change?
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Process
• Brain storm individually -- 2 minutes
• Generate different ideas as a team -- 3 minutes
• Task: Generate as many possible good
classroom assessment tools where the
student work product is primarily graphical
and the tools might satisfy your team
guidelines
• Generate different ideas
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
•
•
•
•
•
•
•
•
•
•
List topics with smiling/frowning faces
Flow charts/flow sheets
Process flow of your problem solving process
Road map to get from A to B
Bar graph of confidence related to specific objective
Free body diagram
Web of association to a particular term
Story board of set up or explanation of particular concept
Fit concept into network or tree of concepts
Draw a concept
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Examples of Assessment
Techniques
• Background Knowledge Probe
• Students respond to short-answers or multiplechoice questions
• General information on their level of preparation
• Minute Paper (most widely used)
• Students write brief response to
• "What was the most important thing you learned during
this class?"
• “What important question remains unanswered?“
www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Examples of Assessment
Techniques (Cont.)
• Muddiest Point (simplest technique, remarkably
efficient)
• Students jot down a quick response to one
• "What was the muddiest point in ....... ?”
• A lecture, a discussion, a homework assignment
• One-Sentence Summary
• Students answer the questions "Who does what to whom,
when, where, how, and why?" (WDWWWWHW)
• Synthesize answers into a simple, informative, grammatical
sentence.
www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Examples of Assessment
Techniques (Cont.)
• What's the Principle?
• Students state the principle that best applies to a
few problems
www.hcc.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htm
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Concept Map
• A concept map is a set of nodes that
represent concepts connected by a labeled
links that describe a link between concepts.
Concept A
Describe how concept A and
concept B are related?
Concept B
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
Building a Concept Map
• Start with a subset of the concepts on
the following page and construct a
concept map that shows the concepts
you have selected and how they are
related.
• Exchange concept maps and share
insights
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Feedback
Derivative
Finite Element Analysis
Integral
Linear Momentum
Angular Momentum
Energy
Interest
Mass
Ideal Gas Law
Fick’s First Law
Fick’s Second Law
Vectors: Dot Product
Vectors: Cross Product
Ordinary Differential Equations
Kirchoff’s Voltage Law
Second Law of Thermodynamics
Kirchoff’s Current Law
Modeling
Problem-Solving
Force
Ohm’s Law
Resistance
Complex Numbers
Logarithmic Function
Electric Flux
Decision Theory
Divergence
Indirect Cost
Capacitance
Bending Moment
Feedback
First Law of Thermodynamics
Entropy
Heat
Electric Field
Magnetic Field
Partial Differential Equations
Determinants
Return on Investment
Phasors
Brainstorming
Exponential Function
Conductivity
Chemical Kinetics
Specific Heat
Elasticity
Malleability
Plasticity
Resiliency
Permittivity
Current
Electric Potential
Curl
Presentation Skills
Democracy
Profit
Density
Molecule
Phase
Shear
Rheology
Frequency Response
Eigenvalue, Eigenvector
Sinusoidal Functions
Work
Displacement
Velocity
Acceleration
Resistivity
Leadership
Hess’ Law
Zeroth Law of Thermodynamics
Electric Potential
Magnetic Flux
Design
Maxwell’s Equations
Power
Ductility
Spring Constant
Stress
Strain
Partial Derivative
Permeability
Charge
Magnetic Potential
Gradient
Paragraph
Rate of Return
Frequency
Atom
Root Locus
Torque
Inductance
Torsion
Polymer
Kinetic Theory of Gases
Cowan’s Teaching Examples
• Bridge design
• Design and build two different bridges and grade
on the lower performance design
• Problem-solving script
• Illustrate script for one type of problem, ask
students to develop a script for another type of
problem
Cowan, J. (1998) On Becoming an Innovative
University Teacher: Reflection in Action.
Buckingham: SRHE and Open University
Press.
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Some Personal Observations
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Relating Student Performance
On Exams To Objectives
• Write exam using objectives
• Select objectives for exam questions from list
• Many objectives -- test questions represent a sample
• Exam question may involve more than one objective
• Use some “hard” and some “easy” questions
• Identify questions (& objectives) a high
percentage missed
• Review idea in class -- give additional work
• Modify lecture, reading, or homework for future
• Change the objective
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Using Cooperative Learning
In-class Exercises
• Examine students work during the in-class
exercise
• If all have a good approach -- may be wasting time
• If all are lost -- may need more explanation
• If one-half to two-thirds have a good approach -- level and
pace are right
• Collect and show a few solutions to in-class
exercises
• If all have correct approach -- may be wasting time
• If all are wrong -- may need more explanation
• If one or two are correct -- level and pace are right
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
“One-Minute” Papers
• Common questions
•
•
•
•
What one thing should be changed about ____?
What one thing should not be changed about ____?
What do you think about ____?
What is the “muddiest” point about ____ ?
• Ask about
•
•
•
•
Course or lecture
Text or chapter
Assignment or test
Teaching style or class activity
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Some Colleagues’
Observations
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Informal Assessment
Techniques
• General Guidelines
• Keep them anonymous
• Use then frequently – better feedback
• Close the loop
• Let students know results of the process
David Cordes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Informal Assessment
Techniques - Daily Activities
• One-minute paper
• At the end of the lecture, ask students for:
• The most important topic that we covered today
• The one topic you are still confused about
• Single sheet of paper, no names
• Can read on the way back to the office
• Look for “common problems”
• Look for “did they understand my focus?”
David Cordes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
ME 360 - Plus / Delta
Assessment #1
• On one side of “sticky” pad
• Put a “+” in upper left hand corner
• What is something that worked well or made more sense in
lab this week?
• On other side of “sticky” pad
• Put a “D” in upper left hand corner
• What is something that could have been done better in lab
this week?
• “Stick” on the door on your way out
Joey Parker
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
ME 372 - 1st Day of Class
• What are a valid set of units for a mass
moment of inertia? (Dynamics concept)
• What is the difference between a capacitor
and a resistor? (Circuits concept)
• What is the equation of the straight line that
passes through the points X=2, Y=7 and
X=7, Y=2? (Math concept)
Joey Parker
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Process
•
•
•
•
Brain storm individually -- 2 minutes
Establish consensus as a team -- 5 minutes
Report team results -- 3 minutes
Revise products as a team -- 2 minutes
• Task: Write 3 to 5 guidelines for good classroom
assessment tools (Think of guidelines as
specifications)
• What are the common features?
• What should they look like?
• Do all guidelines have to apply to all tools?
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Simple, quick, easy to administer and evaluate
• Provide immediate feedback to students – they
need to know you’re using the results
• Tied to a specific topic or objective
• Gives usable, valuable information
• Identification of strengths and areas of
improvement
• Give clear and precise instruction to the student
• Using graphical and textual feedback
• Result in action
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Process
•
•
•
•
Brain storm individually -- 2 minutes
Establish consensus as a team -- 5 minutes
Report team results -- 3 minutes
Revise products as a team -- 2 minutes
• Task: Generate as many possible good
classroom assessment tools that might
satisfy your team guidelines
• Generate different ideas that you have generated or
have seen
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Ask students to identify an application based on concept
presented in class
• Person A teaches Person B, and then they write what was
hard to convey and what was hard to learn
• Identify an objective for class and rate their comfort level
with number or bar graph
• Ask student to give example to transfer example in class
to another context
• Students examine 2-3 methods/opinions/processes for
similar purposes, ask groups to analyze similarities and
differences, solicit results, and then invent their own that
combines methods/opinions/processes
• Ask students to identify concept based on a diagram.
• Ask students to write problem that could be solved by
conceptWestern
presented
in class
Michigan University,
18 September 2003, Kalamazoo, Michigan
Individual Exercise
• Individually write a set of assessment tools for a
class or a topic representing a few classes
• Follow your guidelines
• Consider the following questions about your tool
•
•
•
•
•
Can your students understand the task?
Can your students do the task quickly?
Can you analyze the results quickly?
Can you summarize and report the results easily?
Does it assess student learning?
• 5 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Review each other’s objectives and assessment
tools
• Consider the following questions
•
•
•
•
•
•
Does the tool follow your guidelines?
Can your students understand the task?
Can your students do the task quickly?
Can you analyze the results quickly?
Can you summarize and report the results easily?
Does it assess student learning?
• 15 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Individual Exercise
• Rewrite your assessment tools based on
your team’s review
• Identify the major improvement
• 5 minutes
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Minute Paper
• Write a one-sentence answer to the
following question:
• What is the muddiest point about classroom
assessment?
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Individual Exercise
• How can students be more involved in assessing each
other, as well as the self-assessment we have largely
concentrated on, and do this in reasonable time.
• How often is a reasonable amount of assessment? How
much is too much?
• Going from creating an assessment tool to actually
implementing the assessment.
• Incorporation of feedback will not be every time easy and
straight forward
• Will students see the value of classroom assessment?
Yes, with good approaches. Yes, with good feedback.
• Being able to design a clear, concise assessment tool that
is both beneficial to students and professor
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Team Exercise
• Assume that you are a debate
team
• Write the single best pro and con
arguments for the statement
• “Using classroom assessment tools
improves student learning.”
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Workshop Objectives & Action
Items
• Recall objectives
• At end of session, participants will be able to define, discuss,
& write
• Learning objectives for their courses
• Assessment tools for their classes
• Workshop provided a structure for & experience in
writing
• Learning objectives
• Assessment tools
• Your charge – In one of your courses next
semester use
• Learning objectives
• Classroom assessment tools
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Questions?
Western Michigan University, 18 September 2003, Kalamazoo, Michigan
Some Personal Observations
Western Michigan University, 18 September 2003, Kalamazoo, Michigan