Assessment 101:
The Core Curriculum
Susan Hatfield - Winona State
UNC Wilmington
November 2005
SHatfield@winona.edu
Common Reactions to
Assessment Initiatives
•
•
•
•
•
Ignoring it
Bribing someone else to do it
Complaining about it
Losing sleep over it
Sitting down and writing it
Assessment is
-- first and foremost -about student learning.
A Little Quiz
Quiz
All faculty need to be actively engaged
in assessment for a department or
program to really be “doing
assessment.”
Quiz
All faculty need to be deeply
committed to assessment for a
department or program to really be
“doing assessment.”
Attitudes toward Assessment
70%
15%
Hostile
15%
Accepting
Enthusiastic
Level of Commitment
Quiz
The best way to build a campus culture
of assessment is for top
administration to be prescriptive in
student learning outcomes,
assessment measures and methods.
Quiz
All departments and programs should
be moving ahead on assessment at the
same pace, meeting specific goals
within specified time frames.
Quiz
Effective assessment programs have
eliminated indirect measures and
measures of departmental
effectiveness (satisfaction, etc.).
Quiz
Assessment is a time-intensive add-on
that will be a huge burden to faculty
who are already overburdened.
Quiz
Effective programmatic assessment
plans have every faculty member
assess every outcome in every course
every semester.
Quiz
Accrediting organizations expect to see
fully realized assessment plans during
site visits.
Quiz
The most effective assessment
programs are ones in which the
students are not aware they are
being assessed.
Uncertainty
• Uncertainty related to questions of
•
•
•
•
•
•
How to do it
Why it needs to be done
What to do with the data
How the data will be used
How to find the time to implement it
What support is available
Reducing Uncertainty
• Clear
•
•
•
•
•
•
Understanding
Definitions
Processes
Rules
Resources
Understanding of pitfalls
The Assessment
Core Curriculum:
10 things you should
know about assessment
Core Curriculum
1. Assessment terminology
2. Evolution of assessment initiatives
3. Difference between direct and
indirect measures of learning
4. Assessment requires exertion and
intention
5. Writing student learning outcomes
Language of Assessment
• A. General skill or knowledge category
GOAL
• B. Specific accomplishments to be achieved
OUTCOME
• C. Activities and Assignments to help
students learn LEARNING EVENTS
• D. The key elements related to the
accomplishment of the outcome
COMPONENTS
Language of Assessment
• E. The objects of analysis OBJECTS
• F. Data indicating degree of
achievement CHARACTERISTICS
• G. Combination of data indicating
relative degree of achievement of the
learning outcome INDICATORS
Goals
Goals
Organizing Principle
Category or Topic Area
Subjects
Goals
Composition
PE
Humanities
Fine Arts
Natural Sciences and Mathematical Sciences
Social and Behavioral Sciences
Interdisciplinary Perspectives
Learning Outcomes
Student Learning Outcomes
Communication
Writing
Relating Speaking Listening Teaming
Learning Events
Learning Events
•
•
•
•
•
•
•
Assignments (in class and out of class)
Feedback on practice
Self evaluation
Peer evaluation
Role Play
Pre Tests
Simulation
Learning Objects
Student Learning Outcomes
Communication
Writing
Sales
Relating Speaking Listening Teaming
Components
Student Learning Outcomes
Goal
Outcome Outcome Outcome Outcome Outcome
Learning events
Object
Student Learning Outcomes
Communication
Writing
Relating Speaking Listening Teaming
Demonstration
Verbal
Nonverbal Evaluative elements
Organization
Performance Characteristics
Student Learning Outcomes
Communication
Writing
Relating Speaking Listening Teaming
Demonstration
Verbal
Nonverbal
Organization
Indicators
Student Learning Outcomes
GOAL
Outcome Outcome Outcome Outcome Outcome
Degree to which outcome
is achieved
Object
component
component
component
component
indicator
Core Curriculum
1. Assessment terminology
2. Evolution of assessment initiatives
3. Difference between direct and
indirect measures of learning
4. Assessment requires exertion and
intention
5. Writing student learning outcomes
Maturing Assessment
BEGINNING
PROGRESS
MATURING
INSTITUTIONAL
RESPONSIBILITY
DEPARTMENT
RESPONSIBILITY
Maturing Assessment
BEGINNING
PROGRESS
MATURING
INDIRECT
MEASURES
DIRECT
MEASURES
Maturing Assessment
BEGINNING
PROGRESS
MATURING
PROCESS
MEASURES
OUTCOME
MEASURES
Maturing Assessment
BEGINNING
PROGRESS
MATURING
CLASSROOM
ASSESSMENT
PROGRAM
ASSESSMENT
Maturing Assessment
BEGINNING
PROGRESS
MATURING
INSTITUTIONAL
EFFECTIVENESS
STUDENT
LEARNING
Evolutionary Trajectories
98 99 00
MATURING
MAKING
PROGRESS
BEGINNING
01
02 03 04 05
Core Curriculum
1. Assessment terminology
2. Evolution of assessment initiatives
3. Difference between direct and
indirect measures of learning
4. Assessment requires exertion and
intention
5. Writing student learning outcomes
Direct Measures of Learning
• Capstone experience
• Standardized tests
• Performance on national licensure certification or
professional exams
• Locally developed tests
• Essay questions blind scored by faculty
• Juried review of senior projects
• Externally reviewed exhibitions performances
• Evaluation of internships based upon program
learning outcomes
Indirect Measures of
Learning
• Alumni, employer, and student surveys
(including satisfaction surveys)
• Exit interviews of graduates and focus
groups graduate follow up studies
• Retention and transfer studies
• Length of time to degree
• ACT scores
• Graduation and transfer rates
• Job placement rates
Non-Measures of Student
Learning
Curriculum review reports
Program review reports from external evaluators
Faculty publications and recognition
Course enrollments and course profiles
Faculty / student ratios, percentage of students
who study abroad
• Enrollment trends
• 5 year graduation rates
• Diversity of the student body
•
•
•
•
•
Core Curriculum
1. Assessment terminology
2. Evolution of assessment initiatives
3. Difference between direct and
indirect measures of learning
4. Assessment requires exertion and
intention
5. Writing student learning outcomes
Exertion without Intention
Intention without Exertion
Intention and Exertion
Core Curriculum
1. Assessment terminology
2. Evolution of assessment initiatives
3. Difference between direct and
indirect measures of learning
4. Assessment requires exertion and
intention
5. Writing student learning outcomes
Learning Outcome Format
– Students should be able to
<<action verb>> <<something>>
COMPREHENSION
EVALUATION
APPLICATION ANALYSIS SYNTHESIS
KNOWLEDGE
Cite
Count
Define
Draw
Identify
List
Name
Point
Quote
Read
Recite
Record
Repeat
Select
State
Tabulate
Tell
Trace
Underline
Associate
Classify
Compare
Compute
Contrast
Differentiate
Discuss
Distinguish
Estimate
Explain
Express
Extrapolate
Interpolate
Locate
Predict
Report
Restate
Review
Tell
Translate
Apply
Calculate
Classify
Demonstrate
Determine
Dramatize
Employ
Examine
Illustrate
Interpret
Locate
Operate
Order
Practice
Report
Restructure
Schedule
Sketch
Solve
Translate
Use
Write
Analyze
Appraise
Calculate
Categorize
Classify
Compare
Debate
Diagram
Differentiate
Distinguish
Examine
Experiment
Identify
Inspect
Inventory
Question
Separate
Summarize
Test
Arrange
Assemble
Collect
Compose
Construct
Create
Design
Formulate
Integrate
Manage
Organize
Plan
Prepare
Prescribe
Produce
Propose
Specify
Synthesize
Write
Appraise
Assess
Choose
Compare
Criticize
Determine
Estimate
Evaluate
Grade
Judge
Measure
Rank
Rate
Recommend
Revise
Score
Select
Standardize
Test
Validate
Lower division course
outcomes
COMPREHENSION
EVALUATION
APPLICATION ANALYSIS SYNTHESIS
KNOWLEDGE
Cite
Count
Define
Draw
Identify
List
Name
Point
Quote
Read
Recite
Record
Repeat
Select
State
Tabulate
Tell
Trace
Underline
Associate
Classify
Compare
Compute
Contrast
Differentiate
Discuss
Distinguish
Estimate
Explain
Express
Extrapolate
Interpolate
Locate
Predict
Report
Restate
Review
Tell
Translate
Apply
Calculate
Classify
Demonstrate
Determine
Dramatize
Employ
Examine
Illustrate
Interpret
Locate
Operate
Order
Practice
Report
Restructure
Schedule
Sketch
Solve
Translate
Use
Write
Upper division
Course / Program
outcomes
Analyze
Appraise
Calculate
Categorize
Classify
Compare
Debate
Diagram
Differentiate
Distinguish
Examine
Experiment
Identify
Inspect
Inventory
Question
Separate
Summarize
Test
Arrange
Assemble
Collect
Compose
Construct
Create
Design
Formulate
Integrate
Manage
Organize
Plan
Prepare
Prescribe
Produce
Propose
Specify
Synthesize
Write
Appraise
Assess
Choose
Compare
Criticize
Determine
Estimate
Evaluate
Grade
Judge
Measure
Rank
Rate
Recommend
Revise
Score
Select
Standardize
Test
Validate
Learning Outcome Rules
– Only one action verb
– Identify single accomplishments
– Focus on students, not faculty or
curriculum
Example #1
Gather factual information and apply it
to a given problem in a manner that is
relevant, clear, comprehensive, and
conscious of possible bias in the
information selected
of Bias
Example #2
Imagine and seek out a variety of
possible goals, assumptions,
interpretations, or perspectives
which can give alternative meanings
or solutions to given situations or
problems
Example #3
Formulate and test hypotheses by performing
laboratory, simulation, or field experiments in at
least two of the natural science disciplines (one of
these experimental components should develop, in
greater depth, students’ laboratory experience in
the collection of data, its statistical and graphical
analysis, and an appreciation of its sources of error
and uncertainty)
Core Curriculum
6. What an assessment plan looks like
Write
Relate
Speak
Listen Participate
Component
Component Component Component Component
Component
Component Component Component Component
Component
Component
Component
Component Component
Core Curriculum
6. What an assessment plan looks like
7. How assessment works
How Assessment Works
O
U
T
C
O
M
E
Cycle 1
Cycle 2
Cycle 3
LEvent 1
LEvent 2
LEvent 3
New / Revised
LEvent 1
New / Revised
LEvent 2
New / Revised
LEvent 3
New / Revised
LEvent 1
New / Revised
LEvent 2
New / Revised
LEvent 3
component
component
component
component
component
component
component
component
component
component
component
component
BASELINE
Process
Reflection;
Compare
results
against
Benchmarks,
Standards,
Targets,
Past
Performance
Core Curriculum
6. What an assessment plan looks like
7. How assessment works
8. Why you need to define your
outcomes
teacher4 teacher2 teacher1 teacher3 teacher5
Speaking
eye contact
style
appearance
gestures
rate
evidence
volume
poise
conclusion
sources
transitions
examples verbal variety
organization attention getter
Can our students deliver
an effective Public Speech?
eye contact
style
appearance
gestures
rate
evidence
volume
poise
conclusion
sources
transitions
examples verbal variety
organization attention getter
Core Curriculum
6. What an assessment plan looks like
7. How assessment works
8. Why you need to define your
outcomes
9. How to implement the plan
Student
Learning
Outcomes
Phase Four
Course
1
Course
2
X
Course
3
X
X
X
X
X
X
X
X
X
X
X
X
X
Course
5
X
X
X
X
Course
4
X
X
X
Core Curriculum
6. What an assessment plan looks like
7. How assessment works
8. Why you need to define your
outcomes
9. How to implement the plan
10. What to do with the data
Patterns of Evidence
DP1
DP2
DP3
Interpreting Data
• Consistency - over time
• Consensus - different populations
• Distinctiveness - different
situations/ variables / items
Consistency
• Examines the same practice of and
individual or group over time
• Key question:
» Has this person or group acted,
felt, or performed this way in the
past / over time?
Consistency
How well are students performing on the
departmental learning outcome measures?
High
performance
Low
performance
00
01
02
03
04
05
Consensus
• Comparison to or among groups of students
» Variation between disciplines, gender, other
demographic variables
• Key questions:
– What is the general feeling, outcome, attitude,
behavior?
» Do other groups of people act, perform or feel
this way?
Consensus
How well are students performing on the
departmental learning outcome measure?
High
performance
Low
performance
Females
Males
Transfers
OTA
Distinctiveness
• Examines individual or cohort perspectives
across different outcomes
• Key Question:
» Does a person or group perform equally as
well on different outcomes?
Distinctiveness
How well are our students performing on
the learning outcomes?
High
Performance A
N
A
L
Y
S
Low
I
Performance S
R
E
S
E
A
R
C
H
W
R
I
T
I
N
G
T
H
I
N
K
I
N
G
E
T
H
I
C
S
S
P
E
A
K
I
N
G
Fundamental Question #1
• Do we have enough data from
which to really draw conclusions?
Fundamental Question #2
• Does the data represent an
identifiable trend in the level of
activity / achievement /
accomplishment?
Fundamental Question #3
• Does the data represent an
acceptable level of activity /
accomplishment / achievement
given our mission and values?
Fundamental Question #4
• Are the differences in the subpopulations acceptable?
Fundamental Question #5
• What can we do about it?
Really Big Mistakes
Big Mistakes in Assessment
• Assuming that it will go away
• Allowing assessment planning to become
gaseous
• Assuming you got it right -- or expecting to get
it right -- the first time
• Not considering implementation issues when
creating plans
Big Mistakes in Assessment
• Borrowing plans and methods without
acculturation
• Setting the bar too low
• Assuming that you’re done and everything’s OK,
or rushing to “Close the Loop”
• Doing it for accreditation instead of
improvement
Big Mistakes in Assessment
•
Confusing program effectiveness with student
learning
• Making assessment the responsibility of one
individual
• Assuming collecting data is Doing Assessment
Assessment 101:
The Core Curriculum
Susan Hatfield - Winona State
UNC Wilmington
November 2005
SHatfield@winona.edu