Module 2
Norming
Expectations and
Assessments for
Student Learning
2.15.05
Choose
Improvement
Take
Stock
Build
Institutional
Capacity
Cycle of
Assessment
and
Improvement
Make
Improvements
and
Evaluate
NORM
STUDENT
LEARNING
OUTCOMES
Norm
Assessments
Identify
Improvement
Strategies
Choose
Improvement
Take
Stock
Norm
Student
Learning
Outcomes
Build
Institutional
Capacity
Cycle of
Assessment
Improvement
Make
Improvements
and
Evaluate
NORM
ASSESSMENTS
Identify
Improvement
Strategies
NB —— note well
Separating
-student learning outcomes
from
-student learning outcomes assessment
-wastes time,
-confuses the coherence of programs, and
-looses opportunities for
collaboration and collective norming
Step by Step:
Improving
Student Learning
Step 1: Dialogue
• Dialogue about a department’s
philosophy and assumptions of
teaching and learning should guide
the choice of student learning
outcomes:
– A developmental philosophy would set
increments of increasingly complex
SLOs
– A problem-based learning philosophy
would implement complex, “messy”
learning outcomes in every course
ACTIVITY: DIALOGUE
1
1. What are the philosophies or principles about teaching
and learning on which we’ve built our curriculum,
instructional design, pedagogy, or use of educational
tools?
2. What methods of teaching or educational experiences
develop the knowledge, understanding, habits of the
mind, ways of knowing, and problem solving that this
discipline, profession, trade, program or institution
values?
3. How do we impart those ways of thinking, knowing and
problem solving to students?
4. Which students benefit from which teaching strategies,
educational processes, or educational experiences?
1Maki,
P.L. (2004). Assessing for Learning. American Association for Higher Education,
Sterling, VA: Stylus.
On poster paper
• Write at least one response to
each bulleted question.
Step 2: Norm Expectations
• Faculty discuss and debate what
students should know and be able to
demonstrate upon completion of one
course or the program.
• Expectations are collaboratively
authored and collectively accepted.
Full- and adjunct/ part-time faculty
who teach a course come to
consensus.
1Maki,
P.L. (2004). Assessing for Learning. American Association for Higher
Education, Sterling, VA: Stylus.
ACTIVITY: DIALOGUE ABOUT
EXPECTATIONS
• What are the critical minimum
learning/ performance outcomes for
students in this course?
• What do students need to progress to
the next course in this sequence? (Do
students usually take more courses in
this sequence?)
• What outcomes do students need to
meet their own goals (vocational
certificate, AAS, transfer?)
• What outcomes has institution
defined?
On poster paper
• Write at least one response to
each bulleted question.
Collectively Set Expectations for a Course
EXPECTATIONS
• use active verbs that identify student
performance: create, apply, construct,
formulate
• align with desired outcomes for program or
degree
• align with disciplinary philosophy of learning
• are collaboratively authored and collectively
accepted
• incorporate professional, licensure, or
standardized outcomes
• can be assessed qualitatively or
quantitatively
Step 3: Norm and Implement
Assessment(s)
• Align assessment procedures with
programmatic philosophy/
assumptions about learningWhat are the values of program?
• Collectively identify evidence/
criteria/ performance indicators/
primary traits
• Collectively set rubric for scoring
Assessment should fit the nature of a
program’s goals using methods which
reflect the type of learning that is
valued, rather than methods which
are most easily constructed or scored.
For example, if courses emphasize
open-ended problems, the
assessment procedure should
emphasize the same objective. Facing
students with a multiple-choice test
to measure open-ended problem
solving would be inappropriate.
<www.maa.org/past/ql/ql_part4.html>
Using Criteria/ Standards/ Traits
• Share with students
• Consistently grade each student’s work
according to criteria and rubric
• Guide students in self-use of criteria
• Collect and analyze outcomes
• Use outcomes to improve teaching
• Share learning activities, outcomes,
assessments, criteria, rubrics with
other faculty who teach this lesson
1Adapted
from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA:
Modesto Junior College.
BUILDING A RUBRIC1
Criteria/
Standard/
Primary Trait
Excellent
Superior
Satisfactory
Poor
Unsat
START
HERE
1Adapted
from Scroggins, B. (2004). Targeting Student Learning. Modesto ,CA:
Modesto Junior College
ACTIVITY: MORE DIALOGUE1
What do faculty do now?
• How do you determine an A from a B from a C;
acceptable from not acceptable? (“I know it when
you see it” versus written criteria for grading
every assignment?
• Are your criteria easy enough to use that you use
them every time? (consistent use)
• Do you share criteria and rubrics with students
before assignments? Do you encourage students
to self-assess their work using the criteria and
rubrics?
• How do you use the results of assessment, of
criteria to improve your assessment methods,
criteria, and/or rubrics?
1Adapted
from Scroggins, B. (2004). Targeting Student Learning. Modesto,CA:
Modesto Junior College
REVIEW: Thinking through SLOs
• What outcomes do you want for your
students?
• From whom do you want these
outcomes? (targeted ~ all)
• When do you want these outcomes?
(entry — middle — exit)
• What evidence will be credible and
agreed upon by faculty? (assessment
method)
• What criteria is acceptable? (criteria)
ACTIVITY: NORM SLOs
• DRAFT SLOs components
(columns), using one of the
formats offered or making
one to meet your local
needs.
Step 4: Compile and share results
• Gather results/ data across
sections of a course and/or across
courses within a program and/or
across courses in a sequence
• Share and discuss results; identify
gaps in student learning; discuss
effectiveness of current teaching
and assessment methods
• Use accessible terminology and
formats for reporting data
Analyzing Results
• Plotting students’ achievement along
agreed-upon criteria/ standards/
traits focuses attention on patterns of
strengths and weaknesses
• Visually presented using comparative
tables focuses attention on why some
groups do well while others do not
• Focus attention on WHY?
ACTIVITY: COMPILING RESULTS
SLO
Statements
Assessment
Methods/
Criteria
Results of
Assessments
Identifying
Improvement
Strategies
Choose
Improvement
Take
Stock
Build
Institutional
Capacity
Make
Improvements
and
Evaluate
Cycle of
Assessment
and
Improvement
Norm
Student
Learning
Outcomes
Norm
Assessments
IDENTIFY
IMPROVEMENT
STRATEGIES
Collecting data
≠
Improvement in
student learning
outcomes
Step 5: Collectively set
improvement plan
• Collectively identify patterns of
weakness in student learning
• Allow time to reach consensus
about “Why” and about changes/
improvements to address gaps
• Collectively distribute specific tasks
and locations for piloting
improvements
Potential improvement areas
• Improvements in instruction
• Improvements in curriculum/
content
• Improvements in student support
• Changes in motivation and
engagement of students???
Instructional
Improvement
• More active forms of learning,
including projects, applications,
simulations
• Decreased use of lecturing
• Alignment between values/ desired
goals of program and pedagogy/
assessment procedures
• • may also improve motivation and
engagement of students
ACTIVITY: IMPROVING INSTRUCTION
• How does your institution focus on
instructional skills of faculty now?
• Are there opportunities for faculty to
observe one another, team teach, video and
review their own teaching, or “practice”
effective pedagogy in other ways?
• What is your institutional culture
surrounding teaching? (I never hear chalk
anymore; I hear lots of chairs moving; I
don’t see instructors in front of the room,
etc)
• What role does pedagogy play in hiring and
personnel evaluation at your campus?
On poster paper
• Write at least one response for
each question.
Curricular Improvement
• Interdisciplinary connections
(increases relevance)
• Learning communities of linked
courses (increases “community”)
• Infusion of applications into liberal
arts or infusion of
math/communication skills into
career-technical courses
• • interdisciplinary connections may
improve student motivation and
engagement
Appropriate improvements
• Students don’t connect what they learn in
one course with content/ concepts of
another course: link courses
• Developmental students don’t connect
learning activities with their own goals:
contextual courses
• Even students who are placed into math
or English courses can’t do the required
work: re-evaluate placement instrument
• Technical students don’t have necessary
basic skills: contextual courses; infusion
of academic into technical content
ACTIVITY: CURRICULAR IMPROVEMENT
• Describe previous curriculum reform
endeavors on your campus. What gaps
in student learning were these
changes to address? What data did
you collect and analyze? How
effective was/ were those curriculum
reform(s)?
• Which curriculum improvements seem
promising to meet the learning gaps
you’ve identified?
On poster paper
• Write at least one response for
each question.
Student services
improvement
• Student services decentralized:
connected to disciplines,
departments
• Student services centralized: one
stop shop
• Student services faculty engaged in
teaching
• How does student services
contribute to student learning
outcomes?
ACTIVITY: IMPROVING LEARNING
VIA STUDENT SERVICES
• Describe previous reforms in
student services on your campus.
• What gaps in student learning were
these changes to address? What
data did you collect and analyze?
How effective was/ were those
student services reform(s)?
• Which student services revisions
seem promising?
On poster paper
• Write at least one response for
each question.
Improvement in
Student Motivation and
Engagement
• Difficult to address directly
• Usually results from improvements in
instruction, curriculum or student
services
• • Many efforts to “fix” students
contradict the CCC mission
Moving from data to improvement
• Where will your campus target its
initial efforts?
– target population
– content area
– program
• Which strategy(ies) are aligned
with those improvement goals?
• Which strategy(ies) appear to offer
the best cost/benefit?
Reporting Out
• Prioritize your efforts.
• Where will you put your first
efforts to improve student
learning: improvements in
instruction, curriculum,
student support, or??
• Why?
Evaluating
Improvement
Strategies
Choose
Improvement
Take
Stock
Build
Institutional
Capacity
Cycle of
Assessment
and
Improvement
Norm
Student
Learning
Outcomes
Norm
Assessments
MAK E
IMPROVEMENTS
AND
EVALUATE
Identify
Improvement
Strategies
Making and Evaluating
Improvement Strategies
• Improvement strategies are
aligned with student learning gaps
• Baseline data provides comparison
• Collaboratively author and
collectively accept new outcomes,
learning activities, curriculum/
student services reform
• Collect data on effectiveness of
improvement strategies
Step 6: Continue
SLOA Cycle
• Reassess how well students
improve based on implemented
changes
• Periodically update assessment
plans that build in a cycle of
inquiry
Mapping Backwards from
Outcomes to Learning Activities
• Establish desired outcome
• Then credible assessment
• Then criteria/ primary traits
• Then rubric for scoring
• Then content/ learning
activities that align with
outcome and assessment
Confusion: Where to Start??
• Some colleges start with institutional
outcomes: Gen Ed
• Some colleges start with Perkins/
VTEA funded occupational programs
• WHERE is not nearly as important as
STARTING because this is continuous
professional learning to improve
student learning
• Ready, Fire, Aim as you gain
experience
Homework Report
• How does Program Review
compare to SLOACs?
• What benefits/ disadvantages
would there be for revising
Program Review to incorporate
some/all SLOAC data-driven
decision making?
Norena Norton Badway, Ph.D.
Principal
Phone 209-951-7477 home office
209-946-2168 University office
209-601-7121
Email badway@aol.com
nbadway@pacific.edu