Paul D Camp Community
College
Course Assessments
Student Learning Outcomes
Jerry Standahl, Ph.D.
Faculty Academy
Workforce Development Center
February 11, 2009
Agenda









Phases of assessment
Assessment definitions
Assessment vs. grades
Purpose of student learning outcomes
Rubrics
Assessment timeline
Steps to course assessment
Bloom’s taxonomy
Writing measureable outcomes
Phases of an Assessment Program
Denial
It’s a fad. If I ignore it, it will go away.
Acceptance
OK, I guess we have to do it.
Resistance
I feel threatened. My course or my program feels
threatened. My campus feels threatened. Can I
subvert it by not participating in the process or in
some other way?
Understanding
Maybe we can learn something useful. Can we use
what we’ve already been doing?
Campaign
We have a plan. Maybe it’s not perfect, but let’s
get moving!
Collaboration
We have a plan with long-range objectives that
are clearly defined, and, based on our experience
with assessment, we believe it works.
Institutionalization We can’t imagine working without assessment.
It’s a permanent part of our institutional culture.
Definitions
 Assessment
 The process of gathering evidence to
make inferences about how students
are progressing toward specific goals
(Pennington, 2001)
Definitions
 Goal
 Goals are broad, generalized statements
about what is to be learned or
accomplished.
 Example: Students will understand the General
Education requirements at PDCCC
Definitions
 Objectives
 Objectives are specific, measurable, attainable,
realistic, and timely (S.M.A.R.T)

Example: Upon completion of PHI 115, students will be able to list
all of the General Education areas
 Objectives are the foundation upon which you can
build; that you can prove meet the overall goal
 Think of objectives as tools you use to reach your
goals (how am I going to get there).
 The purpose is to ensure the outcomes are focused
clearly enough so that all involved know what is
going on, and so outcomes can be objectively
measured
Definitions
 Objectives must be Learner Centered (A
statement of what will the student be able to
do as a result of a learning activity)
 Focus on student performance rather than
teacher performance
 Are brief, clear statements that describe
desired outcomes
 Outcomes are used as Benchmarks for
assuring course effectiveness
Common Mistakes when Creating
Objectives
 Vague Behavior
 Example: Having a thorough understanding
of history.
 Gibberish
 Example: Have a deep awareness and
thorough humanizing grasp on …
 Not Student-Focused
 Example: Train students on how and where
to find information.
Definitions
 Target Performance Level
 Specific, measurable statements identifying
the performance(s) required to meet the
outcome; Confirmable through evidence
(i.e., Students/graduates will be able to list
75% of…).
 Since you want 100% of students to be able to
do something, SACS considers it is better to
say, ―Students will be able to list 75% of …‖
than to say ―75% of students will be able to list
…‖
Definitions
 Student Learning Outcomes (SLO)
 A specific, measurable competency (Knowledge,
Skills, Values, or Attitudes) that your students should
be able to demonstrate as a result of participation in
a learning activity.
 SLOs can be expressed and measured at the class,
program or institutional level.
 SLOs are not grades, retention rates, graduation
rates, enrollments, FTES, or completion rates.
 SLOs reflect a shift from a focus from ―What am I
teaching?‖ to ―What are my students learning?‖
Definitions
 Direct measures
 Requires students to display their knowledge and
skills as they respond to the instrument itself (i.e.
certifications, standardized exams, pre-post test,
performance appraisals, portfolios, use of rubrics,
oral exams)
 Indirect Measures
 Subjective or perceived measures by students which
display their knowledge and skills (i.e.
questionnaires, surveys, focus groups, archival
records)
Definitions
 Rubric
 A rubric is a set of categories which define
and describe the important components of
the work being completed, critiqued, or
assessed.
 Each category contains a gradation of levels
of completion or competence with a score
assigned to each level and a clear
description of what performance needs to be
met to attain the score at each level.
Why use a Rubric?



Saves faculty time in assessment which provides timely feedback
Provides meaningful detailed feedback that the student/instructor/college
can act on
 Because the rubric is discussed with students at the beginning, they
have a better understanding of the instructor’s expectations
Encourages critical thinking



Facilitates communication with others




Helps new faculty to not only be consistent in teaching assignments identified by the course
syllabus, but also the expectations for student performance
Allows faculty teaching the same course to share rubrics which promotes grade consistency
Helps student support services assist students with specific learning problems identified by
the rubric
Helps us refine our teaching methods


Rubrics encourage self-assessment and self-improvement by encouraging students to think,
reason, and make judgments based on data
Rubrics speed up grading time enormously, thus allowing assignment of more complex
tasks instead of focusing on rote memory skills using multiple choice and short answers
questions
Rubrics will allow faculty to close the loop by identifying problem areas so that
improvement to their course/program can be made.
Levels the playing field for diverse populations

Rubrics act as a translation device in our new diverse environment by spelling out in detail
what is expected
Work Effectively in Teams
<Student Name>
Sample Rubric
1Unsatisfactory
2Developing
3Satisfactory
4Exemplary
Research &
Gather
Information
Does not collect any
information that
relates to the topic.
Collects very
little
information –
some relates to
the topic.
Collects some
basic information
– most relates to
the topic.
Collects a great
deal of
information – all
relates to the
topic.
Fulfill Team
Role’s
Duties
Does not perform
any duties of
assigned team role.
Performs very
little duties.
Performs nearly
all duties.
Performs all
duties of assigned
team role.
Share
Equally
Always relies on
others to do the
work.
Rarely does the
assigned work –
often needs
reminding.
Usually does the
assigned workrarely needs
reminding.
Always does the
assigned work
without having to
be reminded.
Listen to
Other
Teammates
Is always talking–
never allows anyone
else to speak.
Usually doing
most of the
talking – rarely
allows others to
speak.
Listens, but
sometimes talks
too much.
Listens and
speaks a fair
amount.
Average
Score
Grades vs. Assessment



Grades give a global evaluation but do not provide sufficiently
detailed information about which course outcomes students are
mastering well and which are giving them trouble
Course grades alone don’t stimulate faculty discussions about
how to improve student learning of particular course outcomes
Grades sometimes are based on more than mastery of course
content;



Example: Participation, attendance, bonus points
Grades standards often vary widely among different instructors
and do not indicate the same degree of mastery of course
outcomes
Grade inflation (easy tests, generous grading, extra-credit
bonuses) sometimes presents a misleading indicator of student
mastery of course outcomes
Assessment vs. Grades
Assessment
Grades
Formative
Summative
Diagnostic
Final
Non-Judgmental
Evaluative
Private
Administrative
Often Anonymous
Identified
Partial
Integrative
Specific
Holistic
Mainly Subtext
Mostly Text
Suggestive
Rigorous
Usually Goal-Directed
Usually Content-Driven
Purpose for Student Learning
Outcomes (SLO)
 To Communicate to Ourselves
 With greater clarity the intent of your course
 If participants gained appropriate skills,
attitudes, and knowledge
 How successful a learning activity has been
by measuring it against a benchmark
Purpose for Student Learning
Outcomes (SLO)
 To Communicate to Participants
 What we intend for them to learn in the
course
 If students know what is expected of them,
they tend to perform better and rise to
that level
 For self-selection purposes
Purpose for Student Learning
Outcomes (SLO)
 To Communicate to Other Interested
Stakeholders
 Required information for Administration,
Accrediting Bodies (SACS, SCHEV,
IPEDS, VCCS, etc.)
 The purpose and degree of success of
our activities
Timeline for Assessing Student Learning
Outcomes (SLO)


Beginning of term
 Assessment begin with your course syllabus objectives
being written using a student learning outcomes format
 Your assessment template uses your course (SLO)
objectives along with your target performance level for
success
 You decide how to measure each objective (direct and/or
indirect; Rubric, test items, skill performance)
 Send objectives and evaluation methods to your academic
Dean
Before the end of term
 Record for each objective what your findings/results are.
 How will you close the loop (action) to improve on student
learning outcomes in your course next semester?

An action plan is NOT to send students to tutoring or
counseling. It is how you are going to change your
teaching methods and activities next time.
 Send results and action plan to your academic dean with
copy to Director Assessment & IR.
Example Revision of Course Syllabus
Old Course Objectives
• To know PDCCC and VCCS policies and
procedures
•To inform students of support services
available to them
•To teach students about the skills
necessary for successful college
adjustment
• To show students the Learning
Resources Center web page and be
familiar with information literacy
•To teach effective student skills and
memory techniques
•To be aware of the college transfer
process
•To teach interpersonal communication
skills
•To be acquainted with essential facts
concerning AIDS, alcohol and substance
abuse
•To show students how to access the
PDCCC’s Web page
New Course SLO Objectives
Students will …
• Apply knowledge of PDCCC’s policies,
procedures, and resources
• List necessary survival skills for college
success (critical thinking, financial
planning, study skills, and time
management)
• Demonstrate ability to use a computer to
access the Internet, the college website,
and blackboard
•Critique communication skills (oral and
written)
•Assess knowledge of personal
development areas, such as, facts and
risk-level pertaining to AIDS, alcohol, and
substance abuse
Tips in Writing Measurable
Outcomes











Normally limit outcomes to a maximum of 5
Keep things short and simple (KISS)
Make them specific, measurable, attainable, realistic, and
timely (S.M.A.R.T)
For writing objectives use the Audience-Behavior-ConditionDegree Model (A-B-C-D Model)
Establish a target performance level for success (i.e. 70% will
…)
Keep the assessment process manageable and meaningful
(M&M)
Assess the same outcome at lease twice to ensure
documenting continuous improvement
For every outcome define one or more simple measures (direct
and indirect; essay, list, multiple-choice, rubric)
Use Bloom’s Taxonomy and action verbs
Write in the positive instead of the negative
Make sure each outcome is high enough
Objectives Drive Assessment
Approach
 The action verb in the objective should indicate
the appropriate means of measuring that
objective
 Example: ―recognize‖: may indicate
matching, multiple choice, etc.—lower-level
thinking
 Example: ―demonstrate‖: may indicate a
performance assessment or higher-level
demands
 Entry level courses focus more on lower-level
thinking and more advanced courses focus
more on higher-level demands
Bloom’s Taxonomy & Associated Action Verbs to Create
Student Learning Outcomes Objectives
Level I: Knowledge
Involves recalling information without necessarily
understanding it (Describe, Identify, List, Label,
Tabulate)
Level II:
Comprehension
Involves understanding learned material (Contract,
Describe, Summarize, Classify)
Level III: Application
Puts ideas & concepts to work in solving problems
(Apply, Interpret, Chart, Illustrate)
Level IV: Analysis
Breaking information into parts to see
interrelationships & ideas (Analyze, Contrast,
Prioritize, Compare)
Level V: Synthesis
Puts parts together to form something original
(Reorganize, Integrate, Design, Collaborate, Develop)
Level VI: Evaluate
Judging value of evidence based on criteria (Appraise,
Justify, Predict, Rank, Verify, Assess)
Bad Words
XXX
Avoid verbs that are difficult to measure objectively
Bad Words-XXX
Appreciate
Comprehend
Grasp
significance of
Realize
Be aware
Cover
Have faith in
Recognize
Be comfortable
with
Enjoy
Internalize
Study
Be acquainted
with
Familiarize
Know
Understand
Believe
Gain knowledge
of
Learn
Value
Example SLO
Using Bloom’s Taxonomy
st
PDCCC’s 1TheTime
studentCurricular
will …
Students
Retention
Rates
--Describe
the
basic
components
of empirical research
Knowledge
--Give examples of major themes in music and art
--Recognize in complex text local and rhetorical patterns
Comprehension --Correctly classify a variety of plant specimens
--Explain the scientific method of inquiry
--Summarize important cultural traditions in music
Application
--Demonstrate working knowledge of lab safety
procedures
--Apply oral communication principles in making a
speech
--Compute the area of a room
Analysis
--Distinguish between primary and secondary literature
--Diagram a sentence
Synthesis
--Revise faculty copy for a news story
--Formulate hypothesis to guide a research study
--Create a poem, painting, design for a building
Evaluation
--Compare art forms of two diverse cultures
--Critically assess an oral presentation
S.M.A.R.T Goals/Objectives
To Generate Outcomes
Specific
• A specific goal/objective has a much greater chance of being accomplished than a
general goal (Who, What, Where, When, Which and Why)
• General Goal –This year I am going to get into shape.
• Specific Goal/Objective –This year I am going to join a health club and workout 3
days a week.
Measurable
• Establish concrete criteria for measuring progress toward the attainment of each
goal/objective you set
• Stay on track, reach target dates and experience achievement
• How much? How many? How will I know when it is accomplished?
Attainable
• When you identify goals/objectives that are most important to you, you begin to
figure out ways you can make them come true.
• You develop attitudes, abilities, skills, and financial capacity to reach them.
• You can attain most any goal you set when you plan your steps WISELY and
establish a time frame that allows you to carry out those steps.
Realistic
• To be realistic, a goal/objective must represent an objective towards which you
are both WILLING and ABLE to work.
• Your goal is probably realistic if you truly BELIEVE that it can be accomplished.
Timely
• A goal/objective should be grounded within a timeframe. With no timeframe tied
to it there’s no sense of urgency.
• When you set a timeframe, then you have set your unconscious mind into motion
to begin working on the goal.
Writing Effective
and Measurable Objectives:
st
PDCCC’s 1TheTime
Curricular
A-B-C-D
Model
Students
Retention
RatesExample
Element
Description
A = Audience
Who is performing the action? Learning
objectives are always stated in terms of
student outcomes.
Following completion of the Algebra
II class, the student should be able
to plot a quadratic equation using a
graphing calculator in two minutes or
less.
B = Behavior
What will the student be able to do? Use
Bloom’s Taxonomy and action verb that
describe an accomplishment that is
measurable.
Following completion of the Algebra
II class, the student should be able
to plot a quadratic equation using
a graphing calculator in two minutes
or less.
C = Condition
Give the conditions under which the
performance will occur. Be specific.
Following completion of the
Algebra II class, the student should
be able to plot a quadratic equation
using a graphing calculator in two
minutes or less.
D = Degree
Describe the minimum criteria for
acceptable student performance.
Following completion of the Algebra
II class, the student should be able
to plot a quadratic equation using a
graphing calculator in two minutes
or less.
The A-B-C-D Model
 Writing objectives isn’t creative
writing: Just follow a formula!
 Given [Conditions] the [Audience] will
[Behavior] by [Degree].
 [Audience] will [Behavior] to [Standard]
when provided [Conditions].
Exercise: Diagnosing Objectives
(Audience – Behavior – Condition – Degree)
 Participants will be able to name
two reasons why objectives are
important without handouts or
notes.
 Participants will be able to apply
the ABCD Model to their course
objectives and syllabus prior to
next semester with 100%
accuracy.
Course Outcome Assessment Plan
For
<Course being Assessed for Term You Are Teaching>
Goal/Objective
Evaluation method
(With Target
Performance level
for Success (i.e.
70% will …)
(To Evaluate
Expected Outcomes
(Direct and/or
Indirect)
Course Objective 1
Course Objective 2
Course Objective 3
Course Objective 4
Course Objective 5
Findings/Results
Action to be
taken
(Closing the
loop)
Internet Resources for Assessment
http://www.pdc.edu/assessment-and-research/assessment-toolkit.php
General Principles of Assessment:
http://www.tcc.edu/welcome/collegeadmin/OIE/SOA/principles.htm
Writing Measurable Learning Outcomes: http://www.adprima.com/objectives.htm
Evaluation Methods to Measure Outcomes by Programs:
http://www.unf.edu/acadaffairs/IE/alc/
Types of Measures: http://www.provost.wisc.edu/assessment/manual/manual2.html
Action Strategies to Closing the Loop: http://www.siue.edu/~deder/assess/catmain.html
Rubric Creations: http://rubistar.4teachers.org/index.php
http://pareonline.net/getvn.asp?v=7&n=3
http://pareonline.net/getvn.asp?v=7&n=25
Towson Assessment Resources:
http://pages.towson.edu/assessment/office_of_assessment.htm
NC State Assessment Resources: http://www2.acs.ncsu.edu/UPA/assmt/resource.htm
College of Du Page Resources: http://www.cod.edu/outcomes
Assessment Peer Review Electronic Journal: http://PAREonline.net
Virginia Assessment Group (VAG): http://virginiaassessment.org/RPAJournal.php
National Council on Measurement in Education (NCME): http://ncme.org
American Educational Research Association: http://aera.net