1
Put Your Outcomes to Work!
How to Align Student Learning
Outcomes with Your Assignments
ECPY 761
and Assessments
Katie Partin, PhD
Assistant Director, Office of Institutional Effectiveness
Adjunct Faculty, University of Louisville
Adjunct Faculty, Bellarmine University
2016 Celebration of Teaching and Learning
2
By the end of this session, participants will be
able to:
• Explain Bloom’s Taxonomy
• Create a student learning outcome (SLO)
• Support the use of a certain assessment
3
Active Learning
Active Learning requires:
–Experience
–Reflection
–Integration
–Application
Dewey, 1916
4
The Learning Pyramid:
The recall of different teaching strategies.
That National Training Laboratories (US)
Student’s
recall rate
Students are
increasingly active,
and challenged.
Experience is
increasingly
practical and
multi-sensory
Listening
5%
Reading
10%
Audio-Visual
20%
Demonstration
30%
Discussion groups
50%
Practice by doing
75%
Teach others/immediate use of learning
90%
www.geoffpetty.com or in Mujis, D. & Reynolds, D. (2201)
Students Receive
Information i.e.
Conventional
classroom-based
teaching
Students Apply
their Learning i.e.
Out There
Kegan’s Bridge Metaphor
Educators’ goals for what
and how students should
understand
What students
understand and
how they understand It
Blimling, G. S., Whitt, E. J., American College Personnel Association, & National
Association of Student Personnel Administrators. (1999). Good practice in student
affairs: Principles to foster student learning. San Francisco: Jossey-Bass.
3 Principles of Bridge Building
• Situate learning in the students’ experiences
• Validate students as knowers
• Mutually construct meaning
Dewey + Kegan =
What students
understand/how they
understand it
8
(Experience
Reflection
Integration
Application)
Educator’s goals for
what/how students
should understand
Interpretation of Kegan’s Bridge Metaphor integrated with Dewey’s (1916) ideas of active
learning (as cited in Blimling, et al., 1999).
The Learning Pyramid:
The recall of different teaching strategies.
That National Training Laboratories (US)
Student’s
recall rate
Students are
increasingly active,
and challenged.
Experience is
increasingly
practical and
multi-sensory
Listening
5%
Reading
10%
Audio-Visual
20%
Demonstration
30%
Discussion groups
50%
Practice by doing
75%
Teach others/immediate use of learning
90%
www.geoffpetty.com or in Mujis, D. & Reynolds, D. (2201)
Students Receive
Information i.e.
Conventional
classroom-based
teaching
Students Apply
their Learning i.e.
Out There
Student Learning Outcomes
• THINK: What is a student learning outcome?
• PAIR: Summarize in 1 sentence your collective
understanding of a student learning outcome.
• SHARE: Your sentence with the group as a whole.
11
SLOs “are defined in terms of the knowledge,
skills, and abilities that students have
attained as a result of their involvement in a
particular set of educational experiences.”
www.imt.liu.se/edu/Bologna/LO/slo.pdf
12
Outcomes
• Specific statements about the end results that are
expected from a department, program or service
– From the perspective of the student
• If well written, they are:
– Meaningful
– Measurable
– Manageable
• Tied to the mission and objectives
• Focus on issues of importance
• Evidence can be gathered (quantitative or
qualitative)
13
14
Bloom’s Taxonomy
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
15
“Concise measurable statement that specifies what
students will know, be able to do or be able to
demonstrate when they have completed/participated
in a program/course/project or received a service.
Outcomes are usually expressed as knowledge, skills
or attitudes.
A measurable result of a specific, planned educational
experience for students.”
16
www.uwlax.edu/learningoutcomes/edreading/LearningOutcomes.pdf
Bloom’s Taxonomy
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
17
Bloom’s Taxonomy 2.0
Creating
Evaluating
Analyzing
Applying
Understanding
Remembering
18
The Learning Pyramid:
The recall of different teaching strategies.
That National Training Laboratories (US)
Student’s
recall rate
Students are
increasingly active,
and challenged.
Experience is
increasingly
practical and
multi-sensory
Listening
5%
Reading
10%
Audio-Visual
20%
Demonstration
30%
Discussion groups
50%
Practice by doing
75%
Teach others/immediate use of learning
90%
www.geoffpetty.com or in Mujis, D. & Reynolds, D. (2201)
Students Receive
Information i.e.
Conventional
classroom-based
teaching
Students Apply
their Learning i.e.
Out There
Teach others/immediate use of learning
Creating
Practice by doing
Evaluating
Discussion groups
Analyzing
Demonstration
Applying
Audio-Visual
20
Reading
Understanding
Listening
Remembering
Bloom’s Taxonomy 2.0
Creating: Can the student create new product or point of view?
(construct, create, design, develop, formulate)
Evaluating: Can the student justify a stand of decision?
(appraise, argue, defend, judge, select, support, evaluate)
Analyzing: Can the student distinguish between different parts?
(compare, contrast, examine, question, test)
Applying: Can the student use the info in a new way?
(demonstrate, employ, illustrate, interpret, write)
Understanding: Can the student explain ideas/concepts?
(describe, discuss, explain, identify, paraphrase)
Remembering: Can the student recall or remember the information?
(define, duplicate, list, memorize, recall, repeat, reproduce)
21
By the end of this session participants will be
able to:
• Explain Bloom’s Taxonomy
• Create a student learning outcome (SLO)
• Support the use of a certain SLO
assessment
22
Bloom’s Taxonomy 2.0
Creating: Can the student create new product or point of view?
(construct, create, design, develop, formulate)
Evaluating: Can the student justify a stand of decision?
(appraise, argue, defend, judge, select, support, evaluate)
Analyzing: Can the student distinguish between different parts?
(compare, contrast, examine, question, test)
Applying: Can the student use the info in a new way?
(demonstrate, employ, illustrate, interpret, write)
Understanding: Can the student explain ideas/concepts?
(describe, discuss, explain, identify, paraphrase)
Remembering: Can the student recall or remember the information?
(define, duplicate, list, memorize, recall, repeat, reproduce)
23
SLO Template
As a result of participating in
Upon completion of
At the end of
this program
course
unit
semester
lesson
_______________________,
students will/will
compare
support
interpret
develop
create
be able to __________________________
(Action verbs-Bloom’s taxonomy)
___________________________.
(cognitive skill gained)
24
Specificity is Key
• This course will teach students about
statistics.
• This course will teach students about
statistical tests.
• In this course, students will learn how to use
statistical tests.
• In this course, students will learn how to
choose the most appropriate statistical test
for a given problem.
25
Outcome Examples
• Students will demonstrate appropriate interview
skills during videotapes mock interviews.
• Students will revise faulty copy for a news story.
• Students will correctly classify a variety of plants.
• Students will apply oral communication principles
in making a speech.
• Students will compute the area of a room.
• Students will diagram a sentence.
26
We wrote an SLO!
27
Now what?
28
“the systematic collection of information about student
learning, using the time, knowledge, expertise, and
resources available, in order to inform decisions about
how to improve learning.”
Walvoord, 2004, p. 2
29
3 Steps to Assessment:
1. Articulate your goals for student
learning
2. Gather evidence about how well
students are meeting the goals
3. Use the information for improvement
30
Indirect Measures
• “…ask students to reflect on their learning rather
than to demonstrate it.” (Palomba & Banta, 1999)
• “…signs that students are probably learning, but
the evidence of exactly what they are learning is
less clear and less convincing.” (Suskie, 2009)
• Examples: Survey, Interviews, Course grades,
Retention, Graduation
31
Direct Measures
• “… require students to display their knowledge &
skills…” (Palomba & Banta, 1999)
• “…tangible, visible, self-explanatory evidence of
exactly what students have and haven’t learned.”
(Suskie, 1999)
• Examples: Tests, Papers, Presentations, Projects
32
Brainstorm
What type of assessment could
to determine if:
you use
• Student apply oral communication principles
in making a speech?
• Students can use proper editing symbols
and printers’ marks?
• Student can distinguish between primary
and secondary literature?
33
Formative Assessment
The gathering of information about
student learning—during the
progression of a course or program
and usually repeatedly—to improve
the learning of those students.
Leskes, A. (2002). Beyond confusion: An assessment glossary. AAC&U Peer Review, (4) 2/3.
Classroom assessment should be:
• Learner-centered
• Teacher-directed
• Mutually beneficial
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Classroom assessment should be:
• Formative
• Context-specific
• Ongoing
• Rooted in good teaching practice
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Classroom Assessment Techniques (CATs)
https://ak-hdl.buzzfed.com/static/enhanced/webdr03/2013/6/25/10/enhanced-buzz-28719-1372171084-2.jpg
Background Knowledge Probe
• Collects specific, useful feedback on prior
learning
• A short, simple questionnaire used at the
beginning of a course/new unit/lesson
• Prepare 2 or 3 open-ended questions, a handful
of short-answer questions, or 10 to 20 multiplechoice questions to probe students’ existing
knowledge of that concept, subject, or topic
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Minute Paper
• “What was the most important thing you
learned during this class?”
• “What important question remains
unanswered?”
• Ask students to use a scrap piece of paper
• Give them 1-5 minutes to respond
• Spend 5-10 minutes in the next class
reviewing the results with your students
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Muddiest Point
• Scrap piece of paper
• Anonymous
• “What was the muddiest point in the
____________________?”
homework
film
lecture
discussion
play
assignment
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Problem Recognition Task
•Present a few examples of common problem
types (chi-square, one-way ANOVA, etc.)
•Student’s must recognize/identify the
particular type of problem each example
represents
•Choose examples of several different but
related problem types that students find
difficult to distinguish
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Application Cards
•Hand out index cards
•Ask students to write down at least one
possible, real-world application for what they
have just learned
•3-5 minutes is usually enough time
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Classroom Opinion Polls
•Helps faculty discover student opinions about
course-related issues
•Can be used to prepare to discuss a
controversial issue or to assess opinions after
they have studied the material
•Use pre-/post-test to see if opinions have
changed
•Could pose a question and provide a Likerttype scale “Strong disagree to Strongly agree”
Angelo, T. A., & Cross, K. P. (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Second Ed.
Summative Assessment
The gathering of information at the
conclusion of a course, program, or
undergraduate career to improve learning
or to meet accountability demands. When
used for improvement, impacts the next
cohort of students taking the course or
program.
Leskes, A. (2002). Beyond confusion: An assessment glossary. AAC&U Peer Review, (4) 2/3.
Summative Assessment Examples
• Capstone Projects
• End-of-chapter/unit Tests
• Final Projects
• Midterm/Final Exams
• Papers
• Portfolios
• Recital/Performances
The Value of Rubrics
“A rubric is an instrument based on a set of criteria for
evaluating student work. …Rubrics delineate what knowledge,
content, skills, and behaviors are indicative of various levels of
learning or mastery. Ideally, grading rubrics are shared with
students before an exam, presentation, writing project, or
other assessment activity. Conscious awareness of what he
or she is expected to learn helps the student organize his
or her work, encourages self-reflection about what is being
learned and how it is being learned, and allows opportunities for
self-assessment during the learning process” (Middle States Manual, p. 42).
46
Targets: How we know we’re there!
Key Components
• Align to a specific measure
• Expected scores; level of
performance
• Targets: expected number of
students achieving level of
performance
• Express targets in percentages
if student population is large
enough
Findings
Findings
Key Components
• Align to a specific measure
and target
• Show the result of the
direct/indirect assessment
• Actual number of students
achieving the level of
performance
• Express findings in
percentage if student
population is large enough
Closing the Assessment Loop
Develop or
modify SLOs
Develop, modify,
or review course
or program
Determine
refinements
based on
outcomes data
49
Design &
measure SL as a
result of course
or program
Collect, discuss,
and analyze data
Process of Institutional Effectiveness
With permission from David Garrison, and Carol Yin, LaGrange College
51