Welcome
Cañada College
Flex Day
March 9, 2012
A Review of Best Practices
Based on a review of the literature and evidence from the
site visit teams connected with four in-depth research
investigations: Completion by Design, Achieving the
Dream, the Aspen Prize research and the California BRIC
Initiative.
The 50+ Community Colleges examined include both large
and small colleges; those in urban, suburban & rural
locations; those serving high percentages of both
traditional and non-traditional students, those experiencing
budget growth and budget decline.
Effective basic skills
educationSmart investments in
Evidence
informed
High degree
of teacherStrong linkages
between
A focus
onclassroom
student
completion
technology
decision
making
at
learner
contact
planning
&
budgeting
Frequent
dialogue
across
silos
Highly
focused
set
of
institutional
Mandatory
orientation & advising
every levelOngoing
professional
priorities
Strong peer support
systems
development
A focus
on
equity
&
closing
the
Bringing
successful
pilot
Emphasis on Pathways
Strong
ties
to
community
achievement
programs
to
scale
Entrepreneurial
ingap
finding
Frequent & in-depth dialogue
Routine
tracking of student
resources/funding
with employers
performance
Review of site visit documentation led to the creation of a set of
categories that each college was scored against
Identifying
the recurring
themes
The following categories represent the recurring
themes across all the evidence reviewed.
1. Use of Evidence
a)
b)
c)
d)
Providing access to quality, actionable data
Ensuring multiple groups review & act on data
Use Data to guide planning, budgeting & Program design
Collect evidence for cost-benefit analysis
2. Coherent, Structured & Systematic Approach
a)
b)
c)
d)
Understanding student pathways
Focusing classroom, support, structures & prof dev on student outcomes
Providing sustainable, tailored support to students
Integration of efforts across divisions
3. Equity in Student Access & Achievement
a)
b)
c)
d)
Access for low-income & underreresented students to quality programs
Leadership’s advocacy for equitable outcomes
Review of student data by race & ethnicity for action planning
Using experiences of underrepresented students for planning
4. Scalability & Sustainability of Approaches
a)
b)
c)
d)
Professional development & succession planning for sustainability
Institutionalizing successful approaches without relying on soft money
Systematic consideration of whether all populations are being supported
Using cost benefit analysis to asses programs
5. Leadership
a)
b)
c)
d)
Personal investment of institutional leaders in student success
Cultivating shared ownership of student success across the institution
Rewarding actionable inquiry & experimentation
Intentional prioritization of concrete, sustainable initiatives
But a review of the characteristics displayed by the highest
performing colleges reveals a short list of capabilities
associated with sustained success
1. Rewarding actionable inquiry &
experimentation
2. Intentional prioritization of initiatives
3. Ensuring multiple groups review & act on
data
4. Clear pathways & coherent structures
5. Evidence-rich conversations with employers
and the community
A few comments
on the use of
evidence
Review of Best Practices is most meaningful when it
informs decision making at the proper place of practice
1,000 ft
Perspective
Best Practices to
improve the graduation
rate for degree or
certificate seeking
students
100 ft
Perspective
Best Practices for
helping students
completing course
sequences within in
programs
On the
Ground
Institutional
Strategies
Program
Improvements
Best Practices to support
the development of new
course methodology to
improve student success
Classroom
Innovation
Experimentation does not always yield
shortest-path solutions
Desired Outcome
Starting Point
Acknowledge the limits of data
and the importance of experience
Data rarely enter a vacuum and suddenly
illuminate a hidden truth that triggers action.
Data join the narrative that experts have developed
through years of observation & practice.
The narrative explains and reacts to the data and
the two coevolve to shed new light on the issues
we study.
And what do we do when the
evidence is ambiguous?
What to do when you reach the limits
of your research and yet still face
multiple
choices?
The
process
of inquiry is
Domain of possible
solutions
not a search for an absolute
truth
Trust your
experience, your
intuition & choose !
Gregory M Stoup, Cañada College
We answer the questions that
eliminate dead end solutions
Program Learning Outcomes
Programs, Student Goals & Learning Outcomes
Student Goals
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
The idea: Institutional & Program Level
Assessment helps determine whether
students can integrate learning from
individual courses into a coherent whole*
Transfer to 4-year
GE Learning Outcomes
Transfer to 4-year
Earn AA/AS Degree
Earn AA/AS Degree
Program Learning Outcomes
Earn Certificate
Earn Certificate
Take various courses for a promotion
Take various courses for a promotion
Take a single course to upgrade a job skill
Addressed through
Take a single course to upgrade a job skill
Course Level
University student course takers
Personal enrichment
Assessment
Personal enrichment
Others?
Others?
* Source: Palomba, C.A. & Banta T.W. 2009. Assessment Essentials. San Francisco: Jossey-Bass
Cañada College Programs
• Engineering/CIS
• Mathematics
• Physical Sciences (physics,
chemistry, astronomy, earth
science)
• Biological Sciences (biology,
health sciences)
• Social Sciences (Anthropology,
Political Science, History,
Economics, Philosophy,
Psychology, Sociology, Geography)
• Language Arts (English, Literature)
• Fine & Performing Arts (Art, Art
History, Music, Theater)
• ESL
• Basic Skills
• Foreign Language
• Communication Studies
• Design, Technology &
Creative Arts (Interior
Design, Fashion, MART,
Architecture)
• Kinesiology & Athletics (PE,
Fitness, Athletics, Dance)
• Business & Accounting
• Medical Assisting
• CBOT
• Early Childhood Education /
Child Development
• Paralegal
• Human Services
• Honors Transfer
How should we assess our Program
Learning Outcomes?
Program Portfolios – Once students enter a program
they are instructed to retain course work from
multiple courses in that program. The collective body
of work should related to the PLOs. That body of
work is then reviewed by faculty and assessed
against specific PLOs.
Options to
Consider
Alternatively, faculty can assemble samples of
student work from various classes and use the
collective work sample to assess specific program
learning outcomes.
How should we assess our Program
Learning Outcomes?
Student Presentations: Observations of any behavior
such as performance, project presentations, oral
exams, recitals, etc. are evaluating using a locally
develop rubric.
Scoring Rubrics: Rubrics can be used to score any product or
performance. The rubric delineates the criteria used to
discriminate among levels of achievement. Generally two
raters are used to review each student product and a third
rater is used to resolve discrepancies.
How should we assess our Program
Learning Outcomes?
Embedded Questions on Assignments or Exams:
Questions related to program learning outcomes can
be embedded within course assignments or exams.
For example, a final exam in a higher level course in a
program could include a question or set of questions
relating to your program SLOs. Faculty grade the exams
as usual and then separate exam questions that are
linked to the program SLOs for analysis. The findings are
reported as an aggregate.
How should we assess our Program
Learning Outcomes?
Locally Developed Final Exams: Faculty create an
objective exam for graduating students that is
aligned with the program SLOs.
How should we assess our Program
Learning Outcomes?
Other options:
Locally Developed Final Exams
Pre-Test/Post Test Evaluations
Student Exit Interviews/Survey/Focus Groups*
* While this is an indirect method of assessment, this might
be a good option for programs having a small number of
graduates in a given assessment cycle.
Stage in the process
AfterWhat
lookingthis
at the
evidence
means
toyou
us
make a determination that:
Program Learning Outcomes
have been created
a) The PLO didn’t really get to the
They
areofpublished
& the
students
heart
what you had
in mind
and
areyou
aware
of to
them
decide
modify your PLO
Develop or identify a tool for
assessing them
b) The results were a little “iffy” and
This
could
belike
anytoone
or more
you
would
collect
more of
theevidence
PLOs onbefore
our list
drawing
conclusions
Collect some evidence
Review the evidence
Respond to the evidence with
some action
have
collected something
c)You
Your
expectations
were met orin
theexceeded.
way of output/evidence
In discussion withfrom
students
thatyou
aredecide
tied totoyour
PLOs
colleagues
continue
to examine the same PLO next year
possibly
develop
another
PLO
Youorhave
looked
at how
students
on theirweren’t
PLOs (possibly
d)performed
Your expectations
met and
against
a rubric)
and documented
you have
conversations
with your
colleagues
on how to address it.
that
information
Course Alignment within Programs
Also subject to review by accrediting teams
Generated by our TrakDat software
A supplement to direct methods of Program
Learning Outcomes assessment (by itself not
sufficient as evidence of PLO performance)
Institutional Learning Outcomes: General Educational
Communication
Skills
Critical & Creative
Thinking
Understanding
Society & Culture
Scientific &
Quantitative
Reasoning
Program Learning Outcomes: Art history
Writing Skills: Skills for
writing critically and
persuasively, with specific
application to art history.
World Cultures: Knowledge
about cultures remote from
students’ own, attained
through study of works of art
produced by those cultures.
Visual Analysis: Ability
to recognize, distinguish,
and characterize styles
of objects, artists,
movements, media
Course Learning Outcomes: Art 101 (Intro to Visual Arts)
Identify how an appreciation of the visual arts’ influences the quality of life.
Define major historical and contemporary movements in art and discuss how art reflects
its time and culture.
Explain how the elements of form and principles of design work together with the
creative process to produce a work of art.
Wrapping up
Any questions?