SLO Assessment Review Committee
2011 Annual Report
Respectfully Submitted by:
Marcy Alancraig, Dennis Bailey-Fournier, Alicia Beard, Tama Bolton, Jean GallagherHeil, Paul Harvell, Craig Hayward, Renee Kilmer, Brian King, Susan Nerton, Veronica
Lundquist, Margery Regalado, Georg Romero, Chris Steele (student representative)
Introduction and Background
In response to the change in accreditation standards in 2002, the Cabrillo Faculty Senate,
working closely with the Student Learning Outcomes Coordinator and with
administrators, designed a comprehensive SLO assessment plan: assessment of student
learning outcomes occurs in all sectors of the college as part of on-going Program
Planning (departmental review) processes. Departments in each assessment sector of the
college—Transfer and Basic Skills Instruction, Career Technical Education, Library,
Student Services, and Administration— measure their contributions to students' mastery
of the college’s core competencies. Each sector of the college creates its own method to
assess student success. See the SLO website for a detailed description of the methods
used in each area (http://pro.cabrillo.edu/slos/index.html).
Programs and services undergo Program Planning on a rotating basis; only a few
departments complete the process each year. Because of the number of programs within
its purview, the Instructional component began by phasing in SLO assessment. Now that
this phase is complete, the full scope of assessment, called The Revolving Wheel, has
been implemented. All Instructional departments are now in the process of assessing
students’ mastery of course, certificate, and degree SLOs within the on-going Program
Planning process. See the SLO website for a detailed description of each stage of the
Revolving Wheel of Assessment (http://pro.cabrillo.edu/slos/index.html).
Student Services has also been phasing in SLO assessment, first by writing and then
revising their departmental SLOs, and developing assessments for them. A grant
received by the college, the Bridging Research Information and Culture Technical
Assistance Project, (sponsored by the Research and Planning Group and funded by the
Hewlett Foundation) provided needed training in Student Services assessment methods
during Spring 2011. Prior to the training, a few departments had piloted some assessment
measures, but most began this activity in earnest after last spring’s sessions. Student
Health Services, Student Employment, Student Affairs/Student Activities and
Government and Vice President of Student Services are almost done with writing their
Program Plans that include the assessment of one SLO. All other departments expect to
have each assessed at least one departmental SLO as well as completed a Program Plan
by June 2012.
1
Administration (composed of departments or administrative offices in the President’s
component, Administrative Services, Student Services and Instruction) has spent the last
four years discussing how their departments contribute to student mastery of the college
core competencies and how to measure it. Because they provide a wide range of services
that enable teaching and learning to occur, but are not directly involved in the formal
learning process, their role in assessing SLOs has been difficult to define. At a fall 2011
accreditation training held by the Accrediting Commission for Community and Junior
Colleges, Dr. Jack Pond revealed that the Commission expects Administration to write
and assess Administrative Unit Outcomes (AUOs) or goals as part of an on-going
Program Planning Process. This shift away from SLOs is new, but fits with the tenor of
Cabrillo’s discussions. The extensive dialog has provided an opportunity for
Administration to create a more sustainable, data-defined Program Planning process,
though actual AUOs and assessment measures have yet to be developed. A Program Plan
written by the Instructional Office in 2010 has been adopted as a template for all of
Administration. The President’s Office, The Vice President of Student Services’ office
and the Vice President of Administrative Services’ office expect to have a Program Plan
completed by January 2012. In addition, Duplications and the Bookstore are completing
Program Plans within Administrative Services.
No matter the assessment sector, all college departments that write a Program Plan by
June in a given year forward their assessment reports to the SLO Assessment Review
Committee. This committee is chaired by the Student Learning Outcomes Assessment
Coordinator and is designed to include representatives from the Student Senate, Faculty
Senate, CCEU, CCFT, and a manager along with representatives from Administration,
Student Services, Library, and Instruction (both Transfer & Basic Skills and CTE). The
Campus Researcher and Accreditation Liaison Officer serve as ex officio members of the
committee.
The function of the SLO Assessment Review Committee (ARC) is to read and analyze
the assessment reports submitted, looking for student needs and issues that may be
occurring across the campus and also scanning for possible issues for campus-wide
dialog. In addition to analyzing the collective contents of the assessments submitted each
year, ARC critically analyzes and evaluates its own function and all assessment processes
on campus. ARC writes a report about its analysis, submitting it to campus governing
bodies authorized to act upon ARC’s recommendations, including the Governing Board,
the Master Planning Committee, the College Planning Council, the Faculty and Student
Senates and both unions, CCFT and CCEU. For more detailed information on ARC’s
charge,
membership
and
duties,
please
see
the
SLO
website
(http://pro.cabrillo.edu/slos/index.html).
This report reflects ARC’s review of the assessment results for those departments that
completed Program Planning in the 2010-2011 academic year.
2
Assessment Process: Facts and Figures
Participating in this year’s assessment were twelve instructional departments, eight
serving Transfer and Basic skills and four in CTE.
Assessment Sector
Area
Transfer and Basic Skills
All 8 scheduled departments submitted
Program Plans
All 4 scheduled departments submitted
Program Plans
Report not due until 2012
Work is in progress and will be submitted
in 2012
Work is in progress and will be submitted
in 2012
Career Technical Education Programs
Library
Student Services
Administration
Participation
The charts below capture the participation of Cabrillo faculty in assessment activities.
Since this assessment took place over a number of years, an average rate was calculated.
Basic Skills/Transfer
Department
% of full time
presenting
assessment
results
Adaptive PE
50%
Art Studio
90%
Dance
100%
English
63% *
Health Science 100%
History
100%
Learning Skills 100%
Reading
91%
*Several sabbatical leaves affected
assessment results.
% of adjunct
% of full time % of adjunct
presenting
discussing
discussing
assessment
results
results
results
0
100%
0
50%
100%
81%
50%
100%
50%
22%
83%
42%
60%
100%
60%
57%
100%
57%
66%
100%
33%
42%
91%
88%
the overall percentage of full time faculty presenting
3
Figure 1. Basic skills/Transfer Assessment:
Full time and part time faculty assessment presentation rates
*Several sabbatical leaves affected the overall percentage of full time faculty presenting
assessment results in the English Department.
Figure 2. Basic skills/Transfer Assessment:
Full time and part time faculty assessment discussion rates
4
CTE Departments
Department
% of full time
presenting
assessment
results
Accounting and 100%
Finance
Digital Media
75%
CIS
100%
CEM
0
% of adjunct
presenting
assessment
results
0
% of full time % of adjunct
discussing
discussing
results
results
100%
0
40%
67%
0
100%
67%
0
50%
50%
0
Figure 3. CTE Assessment: Full time and part time faculty assessment presentation
rates
5
Craig Hayward 12/5/11 4:35 PM
Comment: I noticed you took the Basic
Skills/Transfer table out. It is “good form” to
present both a table and a figure, as some people
prefer one or the other. But I’ll leave that to your
discretion, as format concerns are sometimes more
important.
Figure 4. CTE: Full time and part time faculty assessment discussion rates
Like last year, ARC notes with concern the uneven participation of adjunct faculty. As
the charts demonstrate, more adjuncts participated in the discussion of assessment than
those who undertook the assessment itself. In part, this is because the discussions occur
during Flex week, when adjuncts have more opportunity to participate and collaborate
with members of their department. The actual assessment and its analysis takes place
during the course of the regular semester. But since so many of our courses are taught by
adjuncts, ARC continues to be concerned by this lack of full participation, particularly in
some smaller departments.
6
Assessment Progress
•
Transfer and Basic Skills and Instructional Support
Department
Adaptive PE
All Core 4 Assessed
3 out of 4 assessed

Art Studio
Dance

Health Science

History

Learning Skills
Reading
None
None
2 out of 4 assessed
English
Course SLOs Assessed
SLOs for 11 courses assessed
 All course SLOs assessed
None
 All course SLOs assessed
1 out of 4 assessed

2 course SLOs assessed
According to the Revolving Wheel of Assessment, all transfer and basic skills
departments were to have assessed each of the Core 4 and all of their course SLOs.
ARC is heartened that more course level assessment occurred this year compared to
last, but notes with concern that less assessment of the Core 4 was completed. One of
the departments that did not complete assessment of the Core 4 has only two full-time
faculty and many adjuncts; dates on the assessment analysis forms indicate that
assessment occurred before and after the chair’s extended medical leave. Learning
Skills assessed one of the Core 4 and was then moved to Student Services where that
assessment is not required.
7
•
Career Technical Education
Dept
Wrote
Assessment
Plan
Accounting and

Finance
Certificate SLOs
Assessed
None
Digital Media

None
Computer
Information
Systems

Construction
Energy
Management

One certificate was
assessed though
analysis forms were
not submitted to ARC
and CIP
None
Course
SLOs
Assessed
1. Assess the effect of
business decisions on
the income statement,
balance sheet and
statement of cash flow
1.Analyze and evaluate
DM project content to
communicate ideas
visually.
2.Investigate DM
trends to anticipate
clients’ needs
All SLOs for CIS 103,
90, 192,
According to the Revolving Wheel of assessment, CTE programs were to write an
assessment plan, and begin to assess some course and certificate SLOs. ARC is
pleased that all CTE departments wrote an assessment plan or implemented the one
created from their last round of Program Planning, Several assessed some course
SLOs and CIS assessed the SLOs for one certificate although the Assessment
Analysis forms for this was not submitted with their program plan. ARC notes with
concern that two of the departments that did not complete any assessment are small,
with only one or two full-time faculty.
ARC’s analysis of all of the Instructional assessment results recognizes that SLO
assessment has moved from being a new and piloted idea to an on-going institutional
process, carried out under the direction of the Program Chairs and Deans. There have
been many personnel changes in both groups. Though they are invited to work with
the SLO coordinator individually, ARC recommends that more group trainings occur,
tailored to the issues these groups face in helping faculty to assess SLOs.
8
• Student Services
Currently, departments are using the training they received from the Bridging
Research Information and Culture Technical Assistance Project to assess newly
revised departmental SLOs. Along with accomplishing this important work, they are
also undergoing program planning. Student Health Services, Student Employment,
Student Affairs/Student Activities and Government and Vice President of Student
Services have assessed one SLO and will complete their program plans in Fall 2011.
June 2012 is the date when all other departments in Student Services will have
finished their plans along with assessing at least one departmental SLO. • Library
The library completed its program planning cycle in 2006-2007 and since then has
assessed its services according to its assessment plan. This last year the Library
assessed the SLOs for its Library 10 course and also completed its annual survey with
questions about two of the college core competencies: Communication and Personal
Responsibility and Professional Development. The results of those assessments,
along with all others completed since 2006-2007, will be submitted with the Library’s
Program Plan to CIP in spring 2012.
.
• Administration
The President’s office, the Vice President of Student Services’ office and the Vice
President of Administrative Services’ office expect to have completed a program plan
using the newly adopted template from the Instruction office by January 2012. The
college Bookstore and Duplications are in the process of completing program plans
within Administrative Services.
Assessment Results: Emerging Needs and Issues
The departments who assessed the college Core 4 competencies and individual course
SLOs identified the following key student needs and issues:
• A need to improve student reading and writing skills.
• A need to continue to experiment with and develop better instructional
practices and pedagogy.
• A need for faculty to work on improving rubrics.
Those departments proposed the following strategies to help meet these needs:
• More faculty training and development.
• More departmental meeting time devoted to a discussion of pedagogical
issues, not just departmental business.
• Redesign of course delivery methods.
• Rubric writing training.
9
• More training on contextualizing the teaching of basic skills (reading,
writing and math) in CTE courses.
• Closer work with the writing center.
• The History department changed the recommended preparation for their
courses from eligibility for English 100 to eligibility to English 1A,
making clear their expectation that students need a higher level of writing
and reading skills to succeed in their classes.
Analysis of Cabrillo’s SLO Assessment Process
ARC’s analysis of this year’s SLO assessment process noted issues that arise from the
institutionalization of our assessment cycle. While the SLO Coordinator serves as a
necessary resource for the entire college, the directors of our assessment efforts
within instructional departments are now Program Chairs and Deans, with the
Council of Instructional Planning serving in an evaluative capacity. Yet, many
Program Chairs are new to the position, and sometimes the college. In addition, the
HASS and VAPA divisions have new deans who have come from other schools.
Training is clearly needed, focused on how to organize and keep track of the
Revolving Wheel of Assessment for individual departments and how to encourage
adjunct participation. ARC recommends the following to help deal with these issues:
• Provide increased flex training designed especially for Program Chairs,
prospective Program Chairs, and Deans.
• Convene a special meeting for the program chairs of smaller departments
(those with only one or two full-time faculty and many adjuncts) to brainstorm
organizational strategies for SLO assessment.
• Create a web tool that will provide each Program Chair with an individualized
schedule of its department’s Revolving Wheel of Assessment, a semester by
semester calendar of what assessment should occur at any particular time.
The SLO Coordinator will work with the PRO office to create this tool and
post it on the SLO web site.
• Post examples of a full assessment cycle on the SLO web site for transfer/
basic skills and for CTE, taking the assessment process from the classroom to
the department level, with examples of assignments, rubrics and required
forms. This will help new faculty and program chairs have a concrete
example of what is expected in our SLO assessment process.
ARC noted that while important discussion is occurring within transfer and basic
skills departments about student mastery of the college core competencies, it might
increase student success if we could dialogue across disciplines. ARC recommends
that Fall Flex 2012 include interdisciplinary discussions about each of the four core
competencies, providing faculty and staff with a chance to discuss how students are
faring across the college.
10
As noted earlier, adjunct participation in SLO assessment is still a major concern.
One of ARC’s recommendations for last year was to develop a survey to discover the
extent of adjunct participation in SLO assessment and to illuminate the roadblocks
they may face. As the committee created the survey last spring, ARC decided it
would be worthwhile to solicit this information from everyone on campus, to take the
“SLO temperature” of the entire college. In order not to compete with the Campus
Climate survey and to avoid “survey burnout” as Cabrillo develops others to prepare
for writing our accreditation self-evaluation, the Planning and Research Office will
distribute the survey during the week after Thanksgiving this fall semester (2011). In
the spring, ARC will analyze the results and convene any follow up focus groups, if
necessary, to gain more information.
Another recommendation from last year’s report was for ARC to explore finding an
electronic reporting mechanism for SLO assessment results. Since Instruction is
beyond the phase-in stage and all departments are expected to be assessing SLOs, the
committee had a concern about how to report our results in a more ecologically
sustainable and organized format. We worked with the BRIC (Bridging Research
Information and Culture), Technical Assistance Team, who were made available to us
as part of a grant from the Research and Planning Group, to discover our options. We
discussed the merits of some pre-packaged programs (eLumin, TrakDat,
CurricUNet’s program planning module, for example), but could not settle on a
satisfactory instrument, in part due to budget constraints. In December 2011,
program chairs in Instruction will be required, for the first time, to submit this year’s
assessment results along with their annual reports. Though the reports will use an
electronic format, housing them in an easily accessible and useful way is still an
issue. ARC recommends that the PRO office and the SLO Coordinator work together
to create a solution to this problem.
Last year’s report directed ARC to explore options for adding a quantitative
component to our SLO assessment reporting process. The BRIC Technical
Assistance Team provided us with an outside expert’s view on this issue. The BRIC
team praised Cabrillo for its great emphasis on dialogue and our reporting
mechanism’s ability to capture it. However, they felt it looked as if we were avoiding
sharing numerical data in our reports simply because such data was not a required
element. ARC recommended to the Faculty Senate that we embark on a pilot process
to include numerical data. The Senate endorsed the proposal, with a caution to
carefully observe whether dialogue diminishes within departments as a result. There
was also some concern that this will require more work for program chairs who will
be required to “crunch” the numbers reported by their faculty in order to report a
departmental average. The following departments have volunteered to take part in
the pilot: Athletics, CABOT, Early Childhood Education, Economics, English,
Digital Media, Speech Communication, and the Stroke Center.
The discussion of the pilot project also resulted in the Faculty Senate approving a new
measure to assess SLOs: pre and post testing. This adds to the two other course
embedded methods (using a major assignment or analyzing test questions) that
11
faculty may choose from to measure a course, or core competency SLOs. The SLO
web site must be updated to include this new method.
Student Services has made great progress in their work with SLO assessment this
year. The committee commends them for making such good use of the training
offered by the BRIC Technical Assistance Team and moving forward with both SLO
assessment and program planning. ARC looks forward to seeing the results of this
effort next year. In addition, ARC recommends that the SLO Coordinator hold more
workshops and trainings especially for Student Services staff (such as the Admissions
and Records and Financial Aid retreat that she will participate in during January
2012). ARC notes that these workshops should not be scheduled during Flex Week
(when workshops are held for faculty), because many Student Services staff are so
busy working with students that they are not free to attend.
Administration continues to make progress on its assessment efforts, with some
program plans expected to be completed by January 2012, despite a necessary focus
on the on-going budget crisis. As noted in last year’s ARC annual report, the
Program Planning template being used for those plans is an excellent one and
promises assessment, though none has occurred so far. The program plan template
noted above does include "outcomes" section; however, an assessment instrument
that can be used by all departments in administration still needs to be developed. In
keeping with new suggestions from the Accrediting Commission for Community and
Junior Colleges (see page 2), ARC recommends that each department or office within
Administration write Administrative Unit Outcomes or goals which can then be
measured. ARC also notes that there has been discussion but no progress in creating
a venue for review of all Administration Program Plans. Currently, Administration
Program Plans are reviewed at the component level.. While it is up to Administration
to decide the composition of that reviewing body, ARC encourages consideration of a
broad participation model, like the one used at CIP. CIP has found it very beneficial
to have input from those in other campus areas outside Instruction, and
Administration may feel the same way if campus representatives are included in their
oversight group.
ARC will continue to help the college meet the ACCJC’s 2012 deadline for
Proficiency on their SLO rubric and to prepare for our 2013 Accreditation, serving as
readers for the chapter written about our SLO efforts. We will also continue to
support all of the offices and functions in the college to sustain ongoing and
creditable planning cycles that tie student learning to budget decisions.
12
Commendations
ARC salutes the Computer Information Systems department for assessing
all of its courses and one certificate in such a thoughtful and thorough
manner.
ARC commends the English Department for using the resulting dialogue
to better align sections of the same course, including working to create a
departmental rubric for the research paper in English 1A, and for
assessing all course SLOs and the Core 4.
ARC lauds the History Department for the design of its course SLOS,
providing consistency across the department and for assessing each of
them along with the Core 4.
Recommendations
New Recommendations for Teaching and Learning
The Faculty Senate has primary responsibility for providing leadership in teaching
and learning, particularly in areas related to curriculum and pedagogy. The
recommendations below will be put into affect by Faculty Senate and college shared
governance committees.
Recommendation
Responsible Party
Time Line
Provide sustained faculty
development for the improvement of
pedagogy and the sharing of best
practices. Urge departments to use
flex hours, though not necessarily
during Flex Week, to do this.
Staff Development committee
and Program Chairs
Spring and
Fall 2012 flex
13
New recommendations for SLO Assessment Processes
Recommendation
Responsible Party
Time Line
Provide training about organizing and
facilitating departmental SLO
assessment to Program Chairs and
Deans.
SLO Coordinator
Spring 2012 Flex
Convene a meeting of Program Chairs
of smaller departments to brainstorm
organizational strategies for SLO
assessment
Create a web tool that lists the calendar
for every Instructional department’s
SLO assessment schedule.
Post examples of a full assessment
cycle on the SLO web site for transfer/
basic skills, CTE and Student Services
Facilitate interdisciplinary discussions
about student mastery of each of the
four core competencies
Write Administrative Unit Outcomes
for each department in Administration
SLO Coordinator
Spring 2012
SLO Coordinator and
PRO office
Spring 2012
SLO Coordinator and
PRO office
Spring 2012
ARC members and
SLO Coordinator
Fall 2012 Flex
Administration
Develop an Assessment Instrument and
reporting format for Administration
Program Planning that can be used by
all the departments in this area
Provide on-going training and
workshops in SLO assessment to
Student Services staff
Serve as readers for the chapter on the
college’s SLO efforts that will be
included in the 2012 Accreditation
Self-Evaluation.
Revise and update SLO web site
Administration &
PRO
Incorporate this
activity into the
rotating Program
Planning Process
beginning in Fall
2012
Fall 2012
SLO Coordinator
Spring 2012
ARC
Fall 2012
SLO Coordinator with Fall 2012
the help of the PRO
office
14
Completed Recommendations from the 2010 ARC Report
Past Recommendation
Action Taken
Any Next Steps?
Develop recommendations
for making the SLO
reporting process electronic
Exploration of options;
project tabled due to
budget constraints
Explore adding a
quantitative component to
SLO reports
Pilot created; several
departments volunteer
Work with PRO office to
make electronic copies of
current forms easily
accessible
Undertake pilot; SLO
Coordinator revises
reporting form and works
with program chairs to
report results
Recommendations in Process from the 2010 ARC Report
Past Recommendation
Actions Undertaken
Next Steps
Survey adjunct faculty to
assess their awareness of
Cabrillo’s SLO process
Survey created and
expanded to include
everyone on campus;
administered November
2011
Survey not yet
administered so no action
taken
Analyze survey results
Inform potential hires of
Cabrillo’s SLO process and
participation expectations in
new faculty trainings,
mentorships and in the
Faculty Handbook
No action taken
Convene meeting with
Human Resources; write a
section for the Faculty
Handbook
Create a venue or reporting
mechanism for
Administration’s Program
Plans
Discussions within
Cabinet
Continue discussion
Based on survey results,
brainstorm ways to include
more adjuncts in SLO
assessment; create a plan for
sharing those ideas with the
college
15
Develop follow up if
necessary after survey
results are analyzed
Past Recommendations that are now Institutional Practices
Date of
Recommendation
2008
Past Recommendation
Actions Taken
Offer an intensive SLO
Assessment workshop for all
faculty in instructional
departments two years in advance
of Program Planning
Support ongoing, sustained staff
development in the assessment of
student learning, including rubric
development.
Annual Spring Flex Workshop
2009
Share effective practices and
methods for modeling strategies
for assignments
Ongoing Flex Workshops
2009
Provide support for faculty as they On-going Flex workshops;
confront challenges to academic
creation of Student Honor Code
ethics, such as plagiarism and
other forms of cheating
Provide sustained faculty
On-going Flex workshops
development for addressing
student learning needs in reading,
research and documentation, and
writing
2009
2009
2009
Communicate to the college at
large the importance of
maintaining and documenting a
college-wide planning process
that systematically considers
student learning, including noninstructional areas.
16
Ongoing Flex Workshops
“Breakfast with Brian” flex
workshops; development of the
Faculty Inquiry Network;
Bridging Research Information
and Culture Technical
Assistance Project; discussions
about Student Success Task
Force Recommendations;
campus wide focus on Student
Success
Emerging Trends
2007
2008
2009
2010
2011
Emerging Needs and
Issues
Recommendations for
Teaching and Learning
Recommendations for SLO
Assessment Processes
Students need stronger
skills in writing, reading,
and college readiness;
The longer a student is
enrolled at Cabrillo, the
more positive their
association with the
Library
Increase emphasis on
class discussions and
student collaboration;
More tutorial assistance for
students
Encourage greater adjunct
involvement; Continue to
educate the Cabrillo community
about the paradigm shift
Teachers want more frequent Encourage greater adjunct
collegial exchange; Improved involvement. SLO workshop
facilities/equipment needed. for programs two years in
advance of Instructional
Planning and for non
instructional programs;
Develop system of succession
and dissemination of expertise
in SLOAC across campus.
Students need more
Provide ongoing, sustained Encourage greater adjunct
instruction in reading,
faculty development; share involvement. Communicate to
research and
effective practices and
the college the importance of
documentation, and
strategies for modeling
maintaining and documenting a
writing; Concerns about assignments.
planning process that
plagiarism
systematically considers student
learning.
Some students need more Provide faculty
Encourage greater adjunct
instruction in basic
training in new pedagogies, involvement. Embed SLO
academic skills and
technology, and
assessment expectations in
college survival skills.
contextualized instruction.
faculty hiring, new hire training
Support the teaching of
and mentoring practices.
college survival skills across Develop an electronic means
the curriculum.
for SLO assessment result
reporting. Explore adding a
quantitative component.
Students need to improve Provide faculty development Provide training to Deans and
their reading and writing to improve pedagogy and
Program chairs on organizing
skills.
sharing of best practices. Use SLO tasks. Revise web site and
flex hours, but not
add web tools to assist
necessarily during flex week, organizing the process. Hold
to do this.
campus-wide discussions on
Core 4 assessment results.
Undertake pilot for numerical
reporting.
17