Creating SLO Assessment Plans

advertisement
Academy for Planning,
Assessment, and Research
Creating SLO Assessment
Plans
Presented by
ORIE Team
Summer 2013
Table of Contents
I.
II.
Defining Student Learning Outcomes
Creating PLO or CLO Statements
A. Bloom’s Taxonomy
B. Habits of Mind
III. Selecting Measures
A. Direct vs. Indirect Measures
B. Program Level Measures vs. Course Level Measures
IV. Selecting Evaluation Methods
A. Score Weighting Guide
B. Rubrics
VI. Setting Performance Criteria for Individual Students
VII. Specifying Target Outcome for Group Success in meeting the
SLO
Defining Student Learning
Outcomes
The student learning outcome is a statement that explains what
the student is learning, including the accumulated and demonstrated
knowledge, skills, abilities, behaviors, and habits of mind, as a result
of actively participating in the course or program of study.
Program Learning Outcomes answer the question:
What will students know or be able to do when they exit the
program?
Course Learning Outcomes answer the question:
What will students know or be able to do when they complete the
course?
Creating Student Learning
Outcome Statements
Student Learning Outcome Statements
Should align with the purpose of the program or the course;
Should be specific and
measurable, using action
verbs.
Bloom’s Taxonomy-Three Learning Domains:
Cognitive
Affective
Psychomotor
Bloom’s Taxonomy
Knowledge: Recall data or
information
SLO Examples:
Students will be able to
cite, count, define, describe, draw,
identify, know, label, list, match,
name, outline, point, quote, read,
recall, recite, record, recognize,
repeat, reproduce, select, state,
tabulate, tell, trace, underline.
Describe history, purpose, and
scope of physical therapy
Identify the social, political,
economic and cultural
influences and differences that
affect the development process
of the individual.
Bloom’s Taxonomy
Comprehension: Understand
the meaning, translation,
interpolation, and
interpretation of instructions
and problems. State a problem
in one's own words.
comprehend, compute, convert,
defend, differentiate, discuss,
distinguish, estimate, explain,
express, extrapolate, extend,
generalize, give examples, infer,
interpret, paraphrase, predict,
report, restate, review, rewrite,
summarize, translate.
Examples of SLO:
Students will be able to
Distinguish important aspects
of the western moral theories,
from the virtue theory of
ancient philosophers to the
modern theories of act ethics.
Explain georeferencing of
photos or images to maps.
Summarize the principles of
magnetism.
Bloom’s Taxonomy
Application: Use a concept
in a new situation or applies
what was learned in the
classroom into novel
situations in the work place.
apply, calculate, change,
compute, construct,
demonstrate, determine,
discover, dramatize, employ,
examine, illustrate, interpret,
locate, manipulate, modify,
operate, predict, prepare,
produce, relate, show, solve,
use.
SLO Examples
Students will be able to
Implement plan of care for patients
and families within the legal, ethical,
and regulatory parameters.
Apply appropriate organizational
skills for selected types of speeches
Conduct basic laboratory
experiments involving classical
mechanics.
Bloom’s Taxonomy
Analysis: Separates material
or concepts into component
parts so that its
organizational structure may
be understood.
Distinguishes between facts
and inferences.
analyze, appraise, break down,
compare, contrast, diagram,
deconstruct, differentiate,
discriminate, distinguish,
identify, illustrate, infer, outline,
relate, select, separate.
SLO Examples:
Students will be able to
Analyze and design
electrical and electronic
circuits and systems, using
Knowledge of mathematics
and basic sciences.
Analyze self-practice
in relation to the roles of the
professional nurse.
Bloom’s Taxonomy
Synthesis: Builds a structure
or pattern from diverse
elements. Put parts together
to form a whole, with
emphasis on creating a new
meaning or structure.
categorize, combine, compile,
compose, create, devise,
design, explain, generate,
integrate, modify, organize,
plan, rearrange, reconstruct,
relate, reorganize, revise,
rewrite, summarize, tell, write
SLO Examples:
Students will be able to
Integrate the
pathophysiological
assessment findings to
formulate a field impression.
Coordinate human
information and material
resources in providing care
for patients and their families.
Bloom’s Taxonomy
Evaluation: Make judgments
about the value of ideas or
materials.
SLO Examples:
appraise, compare, conclude,
contrast, criticize, critique,
defend, describe, discriminate,
evaluate, explain, interpret,
justify, relate, summarize,
support
Use critical thinking and a
systematic problem-solving
process for providing
comprehensive care.
Students will be able to
Analyze and evaluate
economic events that apply to
the preparation of financial
statements.
Habits of Mind
Arthur Costa and Bena Kallick
(2000)’s Habits of mind:
Persisting, thinking flexibly, and
striving for accuracy
Examples of Habits of Mind:
 Develop and use effective
time management skills
 Follow directions correctly
 Appreciate negatives and
failures as opportunities to
learn and discover choices
SLO Examples:
Students will demonstrate time
management skills, adhering to
all deadlines for assignments,
tests, and projects.
Students will demonstrate
effective therapeutic patient
communication/relationship.
Selecting Measures
A. Direct vs. Indirect Measures
To measure student learning outcomes, use the direct
measures primarily, but also use indirect measures for
triangulation of data
Direct Measures: Refer to the type of data that can be
used to directly measure students’ knowledge and skills
Examples: Pre-test, post-test, comprehensive subject
matter exam, licensure exam, portfolio, thesis, writing
assignment, internship evaluation, capstone assignment,
and performance demonstration (recital, art exhibit, or
science project)
Selecting Measures
Indirect Measures: Refer to the type of data that can be
used to infer student learning or achievement
Examples: Surveys, Interviews, Focus Group Studies,
Document Analyses, or Audit Measures (course grades,
retention/graduation/job placement rates)
Selecting Measures
B. Program Level Measures vs. Course Level Measures
PLO Measures:
CLO Measures:
Exit exam (standardized,
licensure/professional
exams)
Thesis, Dissertation
Capstone Course Paper
or Project
Portfolios
Exams, Tests
Rubric-Based Assignments:
Projects
Essays
Portfolios
Presentations
Performance Demo.
Selecting Measures
C. How to Select Measures?
Measuring Knowledge,
understanding
Tests, Exams, Essays,
Portfolios, Presentations
Measuring Skills
Performance Demonstrations,
Projects, Presentations,
Internship Evaluation
Measuring Habits of Mind
Surveys, Interviews, Behavior
Observations, Focus Groups,
Document Audit
Examples of Measures
Aligned with SLOs
SLOs
Measures (Data Related to
Student Work /Performance
/Opinion)
Students will be able to describe
history, purpose, and scope of
physical therapy
(Knowledge)
Exam on the history and purpose of
Physical Therapy
Essay describing the scope of
physical therapy
Students will be able to distinguish
important aspects of the western
moral theories, from the virtue
theory of ancient philosophers to the
modern theories of act ethics.
(Understanding)
Exam or Essay Questions on the
Exam summarizing the principles of
ethical theories
Essay comparing the similarities
and differences between the ancient
and modern theories
Examples of Measures
Aligned with SLOs
SLOs
Measures (Data Related to
Student Work /Performance
/Opinion)
Students will be able to apply
appropriate organizational skills for
selected types of speeches.
(Skills)
Demonstration of public speaking
Speech in writing
Students will demonstrate time
management skills, adhering to all
deadlines for assignments, tests,
and projects. (Habit of Mind)
Grade Book Audit
Survey on student time
management skills
Selecting Evaluation Methods
Aligned with Measures
Measures (Data Related to
Student Work /Performance
/Opinion)
Evaluation Methods (How do
instructors assess students’ work
or performance?)
Exam on the history and purpose of
Physical Therapy
Essay describing the scope of
physical therapy
Score Weighting Guide with
scores assigned to each exam
question
Grading Rubric specifying levels of
quality in identified areas
Essay Questions on the Exam
summarizing the principles of ethical
theories
Essay comparing the similarities
and differences between the ancient
and modern theories
Grading Rubric specifying levels of
quality in identified areas
Grading Rubric specifying levels of
quality in identified areas
Selecting Evaluation Methods
Aligned with Measures
Measures (Data Related to
Student Work /Performance
/Opinion)
Evaluation Methods (How do
instructors assess students’ work
or performance?)
Demonstration of public speaking
Grading Rubric
Speech in writing
Grading Rubric
Grade Book Audit
Grade Book Record
Survey on student time
management skills
Survey Instrument
Creating a Rubric
A rubric is a systematic scoring guide to evaluate
student performance.
Three Essential Parts of a Rubric:
1) A Set of Indicators of Learning
2) A Scale
3) A Set of Descriptors
Creating a Rubric
Indicators for a rubric-based assignment (essay, project, speech,
performance) should be aligned with the instructional objectives. If an
instructor’s objectives of teaching an essay are related to helping
students compose an essay that is focused, coherent, organized, etc.,
then these features should serve as indictors of learning.
Example of Indicators for an Essay for an English Class:
 Focus
 Coherence
 Organization
 Sentence Structure
 Word Choice
Creating a Rubric
A scale indicates points to be assigned in scoring a piece of work on a
continuum of quality. High numbers are typically assigned to the best
work.
Scale examples:
Needs Improvement (1)... Satisfactory (2)... Good (3)... Exemplary (4)
Beginning (1)... Developing (2)... Accomplished (3)... Exemplary (4)
Needs work (1)... Good (2)... Excellent (3)
Novice (1)... Apprentice (2)... Proficient (3)... Distinguished (4)
Creating a Rubric
Descriptors are used to describe signs of performance at each level; the
description needs to include both strengths and weaknesses (Weaknesses
should be described particularly in lower levels of performance).
Examples of Descriptors:
Word Choice
4-- Vocabulary reflects a thorough grasp of the language appropriate to the audience.
Word choice is precise, creating a vivid image. Metaphors and other such devices
may be used to create nuanced meaning.
3-- Vocabulary reflects a strong grasp of the language appropriate to the audience.
Word choice is accurate, but may be inappropriate in a couple of places.
2-- Vocabulary reflects an inconsistent grasp of the language and may be inaccurate
or inappropriate to the audience.
1-- Vocabulary is typically inaccurate and inappropriate to the audience. Word choice
may include vague, non-descriptive, and/or trite expressions.
Example of a Rubric
(A Snapshot)
Indicators
Explanation of Issues
Evidence
Critical Thinking VALUE Rubric (AACC)
Capstone
Milestones
4
3
2
Issue/problem to be Issue/problem to be Issue/problem to be
considered critically is considered critically considered critically is
stated clearly and
is
stated but description
described
stated, described,
leaves some terms
comprehensively,
and clarified so that undefined, ambiguities
delivering all relevant understanding is not unexplored,
information necessary seriously impeded by boundaries
for full understanding. omissions.
undetermined, and/or
backgrounds unknown.
Information is taken
from source(s) with
enough
Interpretation/
evaluation to develop
a comprehensive
analysis or synthesis.
Information is taken
from source(s) with
enough
interpretation
/evaluation to
develop
a coherent analysis
or synthesis.
Information is taken from
source(s) with
some interpretation/
evaluation, but not
enough to develop a
coherent analysis or
synthesis.
Benchmark
1
Issue/problem to
be considered
critically is stated
without
clarification or
description.
Information is
taken from
source(s) without
any interpretation/evaluation.
Setting Criteria for Meeting
SLO
To determine whether students have successfully met the learning
outcomes, we need to specify what score or rating an individual student
needs to get in order to be considered as meeting the learning outcome.
Examples:
Measure: Exam on the history and purpose of Physical Therapy
Evaluation Method: Score Weighting Guide with scores assigned to
each exam question
Criterion for Meeting the SLO: A score of 70 or above on the exam
Measure: Essay describing the scope of physical therapy
Evaluation Method: Grading Rubric specifying levels of quality in
identified areas
Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4
Specifying Target
Outcomes
To determine whether students as a group have successfully met the
learning outcomes, we need to specify what % from the group should
meet the learning outcome.
Example I:
Measure: Exam on the history and purpose of Physical Therapy
Evaluation Method: Score Weighting Guide with scores assigned to
each exam question
Criterion for Meeting the SLO: A score of 70 or above on the exam
Target Outcome: 85% of the students will score 70 or above on the
exam.
Specifying Target
Outcomes
To determine whether students as a group have successfully met the
learning outcomes, we need to specify what % from the group should
meet the learning outcome.
Example II:
Measure: Essay describing the scope of physical therapy
Evaluation Method: Grading Rubric specifying levels of quality in
identified areas
Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4
Target Outcome: 70% of the students will score a 3 or above on their
essays
Recap: SLO Assessment
Plan
Student Learning Outcome Assessment Plan
Mission:
PLO or CLO
Statement
Measure (Student Teacher's
Work/Performanc Evaluation
e)
Method
Criterion
(What cut-off
score is
considered as
meeting PLO or
CLO for each
student?)
Target Outcome
(Group Success:
% of the group
meeting the PLO
or CLO)
PLO Assessment Plan:
(Example)
Associate Degree in Political Science Program Learning Outcomes
Mission: The Political Science Department is committed to producing students who are knowledgeable about the operation of
Texas and national government. Students should also have the ability to think critically about important political topics that will
prepare them for transfer to higher institutions of learning and provide a framework for lifetime learning skills.
PLO Statement
Measure (Student
Work/Performance)
Teacher's Evaluation
Method
Criterion
(What cut-off score is
considered as meeting
PLO for each student?)
Target Outcome
(Group Success: % of
the group meeting the
PLO)
Students will understand
the American politics
subfield of political
science.
GOVT2304 (Intro to
Political Science) Essay
Question on American
Politics
Essay Assessment Rubric A score of 3 or above 80% of the students
on essay (on a scale of answering the essay
1-5)
question will get a
score 3 or above.
Students will understand GOVT2304 Essay
the political theory
Question on Political
subfield of political
Theory
science.
Essay Assessment Rubric A score of 3 or above 80% of the students
on essay (on a scale of answering the essay
1-5)
question will get a
score 3 or above.
Students will demonstrate Internship Project
a level of civic
engagement
Comments on strengths 70% of the comments
and weaknesses of the in the evaluation letter
students’ internship
are positive.
performance in an
evaluation letter
95% of the students
will get a positive
evaluation letter from
their internship
supervisor.
CLO Assessment Plan:
(Example)
CLO Statement
Engl 1302 Rhetoric and Composition Course Learning Outcome Assessment Plan
Measure (Student
Teacher's Evaluation
Criterion
Target Outcome (Group
Work/Performance)
Method
(What cut-off score is Success: % of the group
considered as meeting meeting the CLO)
CLO for each student?)
Students will be able to
Persuasive Essay
present a valid argument
using convincing evidence.
Essay Grading Rubric
A score of 3 or above 80% of the students
(1-5 scale)
taking the course will
receive a score of 3 or
above.
Students will be able to
demonstrate appropriate
use of standard English.
Grammar Quiz
Writing Portfolio
a. Quiz grading standard a. A score of 70 on the
b. Portfolio Grading
quiz
Rubric
b. A score of 3 or
above (1-5 scale)
75% of the students
taking the Engl1302
course will receive a
score of 70 for the quiz
and a score of 3 or
above for the portfolios.
Student will apply
researched information in
writing a research paper.
Summary of Readings
Research Paper
a. Summary Grading
Rubric
b. Research Paper
Grading Rubric
75% of the students
taking the Engl1302
course will receive a
score of 3 or above for
the summary
assignment and for the
research papers.
a. A score of 3 or
above (1-5 scale)
b. A score of 3 or
above (1-5 scale)
Do’s and Don’ts
When Creating an SLO
Assessment Plan
Do’s
1. When creating PLO or CLO statements, DO include both lower
level and higher level of cognitive skills: not only knowledge,
comprehension, but also application, analysis, evaluation,
and synthesis.
2. When selecting assessment measures for PLOs, DO include
both course-embedded assignments and program level
measures (thesis, exit exams, or national exams, or graduate
portfolios, or internship project, or capstone course project).
3. Encourage faculty participation in the design of the Assessment
Plan.
4. Try to use common assessment assignments or tools, but allow
for flexibility.
Do’s and Don’ts
When Creating an SLO
Assessment Plan
Don’ts
Don’t be overly ambitious—Create only 5 to 7 PLOs
or CLOs, and use only 7-10 pieces of data!
Academy for Planning,
Assessment, and Research
Download