STEM Pathways Crafton Hills College Project Year 2 (2012-­2013) External Evaluation Report Prepared by
Young-Ji Lee
External Evaluator
September 2013
Table of Contents
Executive Summary ..................................................................................................3
Year One Review ......................................................................................................5
Evaluation Method ....................................................................................................6
Project Status.............................................................................................................9
Recommendations and Suggestions ........................................................................14
Conclusion...............................................................................................................15
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
2
EXECUTIVE SUMMARY
Crafton Hills College (CHC) located in Yucaipa, California is a comprehensive, public two-year
institution in the San Bernardino Community College District. In 2011, CHC was awarded a
five-year Title III, STEM (Science, Technology, Engineering, and Mathematics) grant from the
U.S. Department of Education. The goal of the STEM Pathways project is to significantly
increase STEM degree attainment and transfer rate of Hispanic and low-income students. Project
activities include early exploration in STEM fields, curriculum development, transfer support
services, supplemental instruction, equipment upgrades, and professional development. STEM
Pathways completed its second year of the project from October 1, 2012 to September 30, 2013.
Major startup tasks were successfully completed in Year 1 including personnel hires, steering
committee meetings, and initial activity implementation. Year 2 progress showed integration of
project activities and institutionalization taking shape. The project team was also solidified with
the hiring of Transfer Specialist. An overview of Year 2 achievement is presented in the chart
below.
PROJECT ACTIVITY
Early Exploration in
STEM
YEAR 2 ACCOMPLISHMENTS
Number of early exploration events/activities: 17
Number of elementary school students served: 400
Number of middle school students served: 1,385
Number of high school students served: 725
Number of CHC students served: 70
New STEM courses: CHEM 150H and 151H
ENVS 101 and 101H (in progress)
Course Re-design/
Program Design
Curriculum Review
Transfer Support
Services
Revised/Approved STEM courses:
GEOG 110/110H
GEOG 100/100H
CHEM 123
CIS 114 and 116
MATH 250, 251, 252, 265, and 266
Number of students advised for STEM transfer: 55
Number of students with a transfer plan: 71
Number of transfer related workshops offered: 18
Number of career development events offered: 2
Supplemental Instruction Number of students served thru SI: 151
Number of STEM learning communities formed: 7
(SI)/Learning
Number of SI leaders trained: 5
Communities
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
3
STEM Course
Articulation
Lab Equipment
Upgrades
Number of STEM courses articulated (ASSIST): 31
BIO 130, 130H, and 131
CHEM 150, 150H, 151, 151H, 212, and 213
CIS 104, 111, 113, 114, and 116
CSCI 110, 120, 200, 230, 240
ENGR 101, GEOL 100/100H
MATH 250, 251, 252, 265, and 266
PHYSIC 250, 251, and 252
Number of disciplines with updated equipment: 7
Anatomy and Physiology, Cellular Biology,
Engineering, GIS/Math, Geology, Micro Biology,
and Physics
Number of STEM faculty trained: 14 faculty participated in one of
43 professional development offerings on campus
Faculty/Staff Training
Learning Computer Science through the Lens of Culture and
Society: Computer Science workshop for 20 high school teachers
Southern CA STEM Service Learning Institute: STEM teaching
event for 85 high school teachers and after school professionals
STEM Pathways continued to engage the campus community. The establishment of STEM
Academy and STEM Trek offered programming on campus while partnerships with K-12
schools increased STEM awareness off campus. Faculty involvement was further visible through
professional development and equipment upgrades. A strong commitment from the college
executive staff showed progress towards institutionalization of the project.
Overall, CHC was successful in implementing Year 2 tasks. The output numbers above indicate
quality academic and student support services. Data related to project activities were readily
available through the Office of Research and Planning. Evidence-based decision making was
clear. Interviews with faculty, staff and students confirmed active engagement across the college.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
4
YEAR ONE REVIEW
Several recommendations were made in the first evaluation report to strengthen the STEM
Pathways project. The status on the proposed recommendations from Year 1 is outlined below.
Year 1 Recommendations
1. Continue to collect Time and Effort Form
from all personnel including those not
being funded from the grant
2. Develop the STEM Procedures Manual
Current Status
 Time and effort forms now reflect funding
sources (for those partially funded by the
college) to document institutional
commitment.
●
In progress
3. Develop regular progress reports
 Monthly report in place
4. Develop a clear spending plan for the
carryover
 Spending plan communicated with the
Program Officer. Budget expenditure at
appropriate level.
5. Continue to examine transfer data
 Working closely with the Office of
Research and Planning to look at data.
STEM cohort established.
6. Develop an assessment tool for
faculty/staff training
●
In progress. The Office of Research and
Planning to develop a strategy.
At the end of Year 2, most of the recommendations were completed. Samples of time and effort
form and monthly report indicated timeliness of the reporting procedure. For those positions that
are paid partially paid by the college, the funding source was detailed on the time and effort
form. The project staff continued to work closely with the Office of Research and Planning to
examine transfer data. Data from the state and STEM cohort will be used to measure the
objective related to transfer in the future years. Lastly, the budget expenditure was at appropriate
level. The Annual Performance Report submitted in December 2012 stated that $868,635 (99.8%
of Year 1 budget) was encumbered and expended.
Two recommended items--the STEM Procedures Manual and assessment tool for professional
development--were work in progress. During this evaluation, Project Director discussed plans to
review and update components of the manual to share with staff. The project team also discussed
ways to work with the Office of Research and Planning Office to improve the overall evaluation.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
5
EVALUATION METHOD
The external evaluation for the STEM Pathways project consisted of two site visits on August 13
and September 6, 2013. The first visit included discussion and interviews with project personnel,
CHC president, and a STEM student. A follow up visit allowed for a discussion on the
preliminary outcomes of the evaluation and strategies for improvement. Additional
interviews/meetings included a STEM student, Transfer Center Director, and STEM Steering
Committee.
SITE VISIT SCHEDULE
August 13, 2013
Topic
Names of Attendees
Year 2 Overview
STEM Academy and STEM TREK
Rick Hogrefe, Project Director
Patricia Menchaca, STEM Pathways
Coordinator
STEM Student
Cheryl Marshall, President
Karen Childers, Director of Resource
Development & Grants
Ernesto Rivera, STEM Transfer Services
Coordinator
Benjamin Gamboa and KeithWurtz,
Institutional Research and Planning
Robert Brown, Alternative Learning Strategies
Coordinator
Rick Hogrefe, Project Director
STEM Experience
Institutionalization
Project Management
Transfer Services
Evaluation
Supplemental Instruction and
Professional Development
Wrap Up
September 6, 2013
STEM Experience
Discussion
Transfer Center (Title V Project)
Project Activities
STEM Student
STEM Team
Transfer Center Director
STEM Steering Committee
A few broad questions were posed during the evaluation to assist project personnel to think about
the successes and challenges faced this year.
1. Describe what changes were made in Year 2 after the first evaluation. What were the
successes and challenges associated with implementing this year's tasks?
2. How has STEM Pathways impacted/benefitted the institution and students?
3. What are the goals and objectives for activities not articulated in the grant?
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
6
Measurable Objectives
The STEM Pathways project has nine measurable objectives. This evaluation assessed the status
of these objectives based on interviews, completed tasks, tracked data, internal evaluation,
institutional research, and progress reports. The project goals and Year 2 target measures are
detailed below.
STEM GOALS
MEASURABLE OBJECTIVES
YEAR 2 TARGET
Goal (1) CHC must increase the
number of Hispanic students
receiving degrees in STEM
programs.
Objective 1: CHC will increase by 25% (5% per year) the
numbers of Hispanic students graduating with an AA, AS or
Transfer Certificate from the 2009-2010 baseline of 18
degrees/certificates to 54 by 2016, a 200% increase.
Goal (2) CHC must increase
capacity to transfer more
Hispanic students to four-year
STEM programs.
Baseline 2010: 18
Year 2: 32
Objective 2: By Fall 2016, CHC will increase the number of
Hispanic students transferring to UC or CSU systems
declaring majors in STEM from nine students in 2009-2010
to 27 students in 2016, a 200% increase.
Goal (3) Increase the use of
evidence-based decision
making.
Increase the number of Hispanic
students entering STEM programs
at CHC
Increase the number of students
successfully completing
coursework and obtaining a STEM
degree and transfer readiness
Baseline 2010: 9
Year 2: 17
Objective 3: By 2016, CHC will increase the number of
STEM programs that utilize data to inform decision-making
from one program (CHEM) in the 2009-2010 academic year
to eight programs(CHEM, ANAT, BIOL, MICRO, PHYSIC,
GEOL, CS, GIS), an overall increase of 700%.
Baseline 2010: 1
Year 2: 5
Objective 4: By 2016, 15% of incoming CHC students who
participated in a STEM related activity while in high school
will register for at least one STEM course the Fall semester
following their high school graduation.
Baseline 2010: 0%
Year 2: TBD
Objective 5: By 2016, CHC will increase the number of all
students successfully improving from Intermediate Algebra
(MATH-095) to Calculus (MATH-250) from 5 to 50, a 900%
increase (180% per year).
Baseline 2010: 5
Year 2: 23
Objective 6: By 2016, CHC will increase the percent of all
students successfully completing STEM courses with a ―C
grade or better from 61.9% to 70%, an increase of 8.1%
(1.6% increase in the rate each year).
Baseline 2010: 66.5%
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
Year 2: 68.2%
7
Increase the number of articulated
STEM courses between CHC and
local four-year institutions
Improve the currency of laboratory
equipment and experiences
Provide faculty development to
meet the needs of Hispanic
learners in STEM courses
Objective 7: By 2016, ten (10) new STEM courses, two new
STEM programs and 30 revised courses will have completed
approval processes, curriculum development, and
articulation.
Baseline 2010: 0/0/0 Year 2: 8/2/15
Objective 8: By 2016, 100% of CHC STEM courses will
have technologically current equipment and instrumentation.
Baseline 2010: 0% Year 2: 40%
Objective 9: By 2016, 75% of STEM faculty will have taken
part in faculty development/training AND 50% will include
at least one new classroom technique proven to work with
Hispanic learners.
Baseline 2010: 0%/0% Year 2: 30%/20%
The documents reviewed during this evaluation are listed below. The materials were uploaded
and shared online. Initial and follow-up communication was done through emails. Review of
most project materials were done prior to the site visits. Research briefs and update on STEM
Academy and TREK were provided during the visits.
○
○
○
○
○
○
○
Staff time and effort forms
Monthly progress reports
Summary of outputs
Budget report
Equipment inventory
Travel requests
Steering committee meeting minutes
○
○
○
○
○
Examples of materials developed for
various project activities
Evaluation instruments
e-newsletter (STEMinar) and website
Submitted Year 1 Annual Performance
Report (APR)
Research Briefs
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
8
PROJECT STATUS YEAR 2 EVALUATION
Project Management
Project management procedures showed clear examples of monitoring project progress.
The timeliness of the reporting procedure and percentage of effort reported for key personnel
were verified. The monthly progress reports and internal tracking demonstrated evidencebased approach in project implementation. The Office of Research and Planning also
published several research briefs that validated project efforts throughout the year. At the
time of the evaluation, CHC expended approximately $910K. The remaining funds
(approximately $125K) will be expended/encumbered by September 30th.
Project Objectives
The implementation timeline for Year 2 showed successful completion of grant tasks.
Objective progress is detailed in the next few pages. Note, some of the annual target
measures have been corrected from Year 1.
Objective 1: CHC will increase the numbers of Hispanic students graduating with an AA, AS
or Transfer Certificate from the 2009-2010 baseline of 18 degrees/certificates to 54 by 2016,
a 200% increase.
Year 2 Target Measure: 32 Hispanic
Year 2 Progress: 44 Hispanic students
students graduating with an AA, AS or
graduating with an AA, AS or Transfer
Transfer Certificate
Certificate
Objective 2: By Fall 2016, CHC will increase the number of Hispanic students transferring
to UC or CSU systems declaring majors in STEM from nine students in 2009-2010 to 27
students in 2016, a 200% increase.
Year 2 Target Measure: 17 Hispanic
Year 2 Progress: TBD
students transferring with STEM majors
CHC overwhelmingly met the Year 2 target goal for degree attainment. There were 44
Hispanic students graduating with an AA, AS or Transfer Certificate (144% increase over the
baseline). The transfer numbers will be verified in the coming year as the data is currently
being collected by the Office of Research and Planning.
During this year, two programs were established to support students: STEM Academy and
STEM Trek. STEM Academy assists new STEM majors to explore STEM fields and
increase academic readiness. STEM Trek offers social and academic support for those
students who completed the STEM Academy. Participating students are to be tracked
longitudinally for retention, degree attainment, and transfer. One incentive for students to
participate in the programs is priority registration. This has been a proven strategy to attract
and retain students. In Spring 13, 17 students enrolled in STEM Trek.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
9
Some of the challenges identified during the first
semester of STEM Trek were: conflicting
schedules, lack of formal documentation of student
responsibilities, lack of student community and
under-preparedness of students. Changes made in
the following semester addressed these issues by
establishing STEM Academy, focusing on
community building, creating an activity calendar,
requiring formal documents, and tracking students
using ACCESS database. As a result, Fall 13 data
showed 34 students in STEM Trek and 35 students
in STEM Academy. Of the total 69 students, 26
had priority registration. Initial success of the
programs showed 5 transfers to the following
universities:
Crafton's STEM Pathways
program has benefited my
academic career by making the
transfer path to a four year
university clearer through
workshops designed specifically
to meet the needs of STEM
students, providing priority
registration so that I can get the
classes I need to transfer in a
timely manner, and allowing for
the opportunity to meet with an
academic counselor so that I
know I am on track to transfer as
1. UC San Diego: Electrical Engineering
planned. Through the STEM
2. UC San Diego: Environmental Engineering
program at Crafton, I have been
3. Cal Poly Pomona: Aerospace Engineering
able to meet many fellow STEM
4. Cal State East Bay: Chemistry
students, visit other university
5. UC Riverside: Neuroscience
campuses, and listen to guest
speakers discuss their careers as
The list of support services discussed during the student interviews was
reflective of the
scientists.
services that are provided and tracked for STEM Academy and STEM Trek students. The
STEM Transfer Services Coordinator further reported that 71 students developed a transfer
STEM Student, Biology Major
plan, 18 transfer related workshops were offered, and 2 career development events were
provided this past year.
Objective 3: By 2016, CHC will increase the number of STEM programs that utilize data to
inform decision-making from one program (CHEM) in the 2009-2010 academic year to
eight programs(CHEM, ANAT, BIOL, MICRO, PHYSIC, GEOL, CS, GIS), an overall
increase of 700%.
Year 2 Target Measure: 5 STEM
Year 2 Progress: Chemistry, Anatomy,
programs
Biology, Physiology, and Micro
CHC continued to increase the number of STEM programs that utilize data to inform
decision-making. With Chemistry as the baseline, CHC met the annual target measure by
including Anatomy, Biology, Physiology and Micro (an overall increase of 400%) programs
this year.
Objective 4: By 2016, 15% of incoming CHC students who participated in a STEM related
activity while in high school will register for at least one STEM course the Fall semester
following their high school graduation.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
10
Year 2 Target Measure: TBD
Year 2 Progress: Outreach to 725 high school
students at various events
The baseline number and annual target measure for this objective were under review at the
time of the evaluation. During this evaluation process, the STEM Team and Office of
Research and Planning discussed strategies to improve the tracking method for high school
students who participate in outreach activities.
Collaborative effort between the project staff and college outreach team (SOAR) continued
to yield positive results, reaching over 700 high school students. Most significantly, the first
SciFri event for high school students was held this year. The event evaluation showed that
100% of the respondents were satisfied with the event and that they learned more about the
science programs at CHC. Early exploration went beyond the high schools to include
elementary and middle schools. A total of 12 outreach events served additional 400
elementary and 1,385 middle school students in Year 2.
Objective 5: By 2016, CHC will increase the number of all students successfully improving
from Intermediate Algebra (MATH-095) to Calculus (MATH-250) from 5 to 50, a 900%
increase (180% per year).
Year 2 Target Measure: 23 students
Year 2 Progress: 25 students
CHC met the annual goal by increasing the number of students successfully improving from
Math 95 to Math 250 from 5 to 25 (400% increase over the baseline) students.
Objective 6: By 2016, CHC will increase the percent of all students successfully completing
STEM courses with a ―C grade or better from 61.9% to 70%, an increase of 8.1% (1.6%
increase in the rate each year).
Year 2 Target Measure: 68.2%
Year 2 Progress: 63.1%
Supplemental instruction (SI) was provided for 7 STEM courses. A total of 151 (70 students
in Fall 12 and 81students in Spring 13) students were served. Internal study on the success
rate of SI showed the true benefits of SI. Research briefs indicated the following results:
29% increase in success for students who attended one or more SI sessions
31% increase in success for students who attended two or more SI sessions
Hispanic students who attended two or more SI sessions were substantially more
likely to complete the course (70%) than Hispanic students in the same section who
did not attend any SI sessions.
● 100% of the respondents agreed that SI leader provided helpful learning strategies
and created a supportive environment.
●
●
●
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
11
In addition, 7 STEM learning communities were formed in Year 2. Changes in Year 3 may
include SI for Biology and Chemistry. SI and learning communities will be further reviewed
and improved to meet the annual measure in the coming year. Based on the student
interviews, it was clear that academic support services continued to be a vital component of
the project.
The STEM Pathways Program at Crafton Hills College has been vital to
the success of the participants in that the program staff are available at
every turn to advocate for STEM students. That takes the form of
ensuring availability of the classes we need, keeping us informed of
every opportunity that could be of interest (symposia, lectures, tours,
etc.) and providing advisement and critical 'insider' knowledge of the
best strategies for transfer and academic success. Without this support
from the STEM Pathways Program, the overwhelming obstacles that
STEM students face would greatly diminish most of our outcomes.
STEM Student, Biology Major
Objective 7: By 2016, ten new STEM courses, two new STEM programs and 30 revised
courses will have completed approval processes, curriculum development, and articulation.
Year 2 Target Measure
STEM Courses: 4
STEM Programs: 2
Revised Courses: 12
Year 2 Progress
STEM Courses: 5 courses in CS (Year 1)
CHEM 150H and 151H
ENVS 101 and 101H (in progress)
STEM Programs: Computer Science (Year 1)
GIS (in progress)
Revised/Approved STEM courses:
GEOG 100/100H and 110/110H
CHEM 123, CIS 114 and 116
MATH 250, 251, 252, 265, and 266
CHC continued to make progress on this objective and met the target measure for this year.
Course development in Chemistry and Environmental Science were ongoing while 12
courses in Geology, Chemistry, CIS and Math were revised.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
12
Objective 8: By 2016, 100% of CHC STEM courses will have technologically current
equipment and instrumentation.
Year 2 Target Measure:
Year 2 Progress:
40% of STEM courses (19 STEM
$315,873 expended in equipment and supplies
courses)
to upgrade 7 STEM disciplines
Baseline: 47 STEM courses
CHC purchased $287,873 ($200 or more per item) in equipment and approximately $28,000
($200 or less) in supplies. Internal procedure identified equipment as being items with a
value of $200 or more. Seven STEM disciplines benefitted from the equipment upgrade in
Year 2. The disciplines include Anatomy/Physiology, Cellular Biology, Engineering,
GIS/Math, Geology, Micro Biology, and Physics.
Objective 9: By 2016, 75% of STEM faculty will have taken part in faculty
development/training AND 50% will include at least one new classroom technique proven to
work with Hispanic learners.
Year 2 Target Measure:
Year 2 Progress: 14 STEM faculty
30% (15 faculty) participating in
participated in one of 43 professional
professional development and 20% (10
development activities on campus (Spring 13
faculty) will include classroom technique
records)
Baseline: 53 STEM faculty
Two new resources were created to further professional development. STEMinar Newsletter
includes STEM related articles on best teaching practices, links to online resources, and
upcoming professional development events. Secondly, STEM Pathways Blog serves as an
online repository of latest research on Science Education, sample course material, and past
STEMinar Newsletters. There were 14 STEM faculty who participated in one of 43
professional development activities in Spring 13.
During the first site visit, several strategies were discussed to improve STEM faculty
development in Year 3. One strategy is to review the different types of professional
development and mode of delivery. The Alternative Learning Strategies Coordinator
identified three ways in which faculty obtain professional development: 1) individual effort,
2) campus offerings, and 3) digital/online. The review may provide insight to faculty interest.
Another approach is to conduct a needs assessment. The outcome may lead to new training
ideas. Plans to work closely with the Office of Research and Planning were noted.
Furthermore, CHC trained local K-12 educators at two events. The STEM team collaborated
with the San Bernardino Community College District on Southern California STEM Service
Learning Institute. A total of 85 high school teachers and afterschool professionals discussed
STEM teaching strategies and participated in hands-on activities to be applied in the
classroom. Continuing from last year's success, 20 high school teachers participated in the
Learning Computer Science through the Lens of Culture and Society workshop to
incorporate computer science in the classroom.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
13
RECOMMENDATIONS and SUGGESTIONS
Recommendations and suggestions offered in this section are to strengthen the STEM
Pathways project. It is the responsibility of CHC to follow the compliance requirements of
the U.S. Department of Education.

Complete the project procedures manual and share with project staff. The manual
should include templates for standard forms.

Complete travel report after conference attendance. A report on how the travel relates
to the project objectives to be described.

Document the progress on institutionalization. CHC is to institutionalize three staff
members; tracking the progress may be included in the monthly report.

Consider organizing the monthly report to reflect reports from each of the activity
lead. Each lead could write the progress of his/her component and sign next to their
section.

Develop assessment tools for professional development activities. Student satisfaction
should also be assessed for STEM Trek and Academy students.

Review the processes for identification of prospective STEM students at the high
schools. Continue to work with the Office of Research and Planning to set up the
tracking method.

Consider creating a brief overview of the project activities and objectives to distribute
to all project staff. Based on the interview with the Alternative Strategies
Coordinator, the overview may be distributed during SI leader and tutor training to
increase awareness of the project objectives.

Based on the discussion during the Steering Committee Meeting, research state and
federal government agencies for possible internship opportunities for students.

Consider presenting at conferences. CHC has already shown evidence on student
success. Sharing best practice may help other institutions as well as connect with
other resources that may lead to partnership opportunities.
CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
14
CONCLUSION
The STEM Pathways project remains vital to Crafton's effort in creating a transfer
culture. An important project component initiated during the second year was the STEM
Trek and Academy. Incentives such as priority registration and dedicated STEM transfer
advisor were attractive features for students. A sense of community was also commonly
discussed during student interviews. Project staff expressed that these relationships are
critical in helping students succeed. For Year 3, the STEM team set a goal to serve 100
STEM students through these programs.
Further partnership with the campus outreach team and Transfer Center strengthened ties
with the feeder high schools and four-year institutions. Moreover, outreach events for
elementary, middle and high schools reached over 2,500 students this year. Strategies to
improve project awareness and professional development were ongoing. The STEM team
agreed that continued dialogue with the local feeder schools and outreach to the four-year
institutions are necessary to create a continuum of transfer services.
Overall, Crafton completed a successful second year. The STEM team along with the
Office of Research Planning continued to rely on data to make evidence-based decisions.
Crafton met most of the Year 2 target numbers. Most notably, CHC increased the number
of Hispanic students obtaining an AA or AS degree. Several research briefs showed
initial successes of the project in supplemental instruction and outreach components.
Development of assessment tools for various components will be evaluated next year.
Year 3 evaluation will continue to track the implementation timeline to assess goal
attainment. CHC STEM Pathways Project Evaluation Report
Year 2: 2012-2013
15