A “Realistic Appraisal” of Student Educational Planning Student Learning Outcomes Revisited (2008‐2009) Esau Tovar
Counseling Department Santa Monica College February 2010 (Recommendations section revised May 2010) ESAU TOVAR Santa Monica College 2009‐2010 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
2 EXECUTIVE SUMMARY This report describes in detail the assessment of two student learning outcomes adopted by the
Counseling Department at Santa Monica College. These outcomes pertain to the appraisal of
students’ educational status in relation to their overall goals (i.e., educational planning) and to the
proper identification and sequencing of course prerequisites. Using a multilayered approach
assessing behavioral, cognitive, and attitudinal domains, the report details outcomes associated with
educational planning competencies demonstrated by students and evaluated by professional
counselors using a common rubric, an educational planning quiz, and an educational planning
survey.
Data were collected during two assessment cycles in 2008 and 2009 and comparisons were drawn
between cohorts of students to determine how well the intended competencies were met, and how
changes implemented between cohorts impacted competencies of second year cohort students.
Accordingly, it was found that 67% of students in the fall 2008 cohort demonstrated competency in
educational planning as evidenced in an educational plan they produced. The corresponding
percentage for the fall 2009 cohort increased to 79%., an overall proficiency increase of 18%.
Moreover, 69% of students expressed confidence in their ability to develop and adjust an
educational plan on their own based on their individual goals. The report provides additional
evidence on specific tasks students were required to complete as part of the assessment exercises in
which they engaged as part of the Student Success Seminar (Counseling 20) in which they enrolled.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
3 Table of Contents EXECUTIVE SUMMARY ........................................................................................................................................ 2 INTRODUCTION ................................................................................................................................................. 4 PURPOSE OF REPORT ......................................................................................................................................... 5 OPERATIONALIZING SLOS ................................................................................................................................... 5 ASSESSMENT METHODOLOGY .............................................................................................................................. 5 INSTRUMENTATION ...................................................................................................................................... 5 SAMPLE SELECTION ...................................................................................................................................... 6 SLOS ASSESSMENT RESULTS ................................................................................................................................ 7 EDUCATIONAL PLANNING COMPETENCIES DEMONSTRATED BY STUDENTS ......................................................... 7 STUDENT APPLICATION OF EDUCATIONAL PLANNING COMPETENCIES—QUIZ .................................................. 10 EVALUATION OF EDUCATIONAL PLANNING AND TRANSFER PRESENTATIONS ..................................................... 12 RECOMMENDATIONS FOR IMPROVEMENT ............................................................................................................ 19 APPENDIX A: PROGRAM LEVEL STUDENT LEARNING OUTCOMES ............................................................................. 21 APPENDIX B: RUBRIC, QUIZ, AND SURVEY FORMS ..................................................................................... 26 EDUCATIONAL PLAN COMPETENCIES RUBRIC ................................................................................................ 26 EDUCATIONAL PLANNING QUIZ ................................................................................................................... 27 EDUCATIONAL PLANNING SURVEY ................................................................................................................ 28 Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
4 A “Realistic Appraisal” of Student Educational Planning Student Learning Outcome Revisited INTRODUCTION The Counseling Department has established a number of student learning outcomes (SLOs) for a variety of
key services offered to students at the college. SLOs are assessed on a rotating basis and consistent with SMC
practice, SLOs are mapped to relevant institutional learning outcomes (ILOs) as noted below. Appendix A
discusses in some detail the general plan followed by the Counseling Department to engage in assessing our
SLOs, discussions held to review past findings, and changes implemented following these discussions.
ILO 1:
Through their experiences at SMC, students will acquire the self-confidence and
self-discipline to pursue their intellectual curiosities with integrity in both their
personal and professional lives.

SLO 1:
Student will formulate a realistic self-appraisal of their educational status and its
relationship to their overall goals.
SLO 2:
Students will identify their math and English course sequences and chart their
individual math and English course sequences to achieve their educational goals
as a result of the educational planning presentation in the Counseling 20 classes
and subsequent counseling session(s).
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
5 PURPOSE OF REPORT This report delineates the outcomes for the second evaluation of SLO 1 and SLO 2 above that was
conducted in Fall 2009. The first evaluation was carried in Fall 2008 and results were examined by counseling
faculty (counselors and instructors) to determine if we met the intent of the SLOs. Following this
examination, a series of changes were implemented in counseling presentations conducted in Counseling 20
classes, which addressed some shortcomings found in our fall 2008 evaluation. Given these changes, this
report draws comparisons between fall 2008 and fall 2009 outcomes.
OPERATIONALIZING SLOS SLOs 1 and 2 are primarily concerned with the competencies that students exhibit as a result of direct
interaction with a counselor in order to develop an educational plan pertinent to and conducive to fulfilling
their stated educational goals. We are cognizant that some students at SMC may have multiple counseling
contacts or, in some cases, zero contacts, but educational planning or an extended conversation on their goals
may not always take place in individual student appointments. Given this, the Counseling Department has
implemented an outreach effort to students enrolled in Counseling 201 (Student Success Seminar) classes by
assigning a counselor to one or two class meetings to conduct presentations on educational planning and to
show students step-by-step how to prepare a realistic educational plan pertinent to their educational goals.
Once the presentation is conducted, students are asked to prepare their own “ed plan” using a variety of
resource materials prepared by the Counseling 20 instructional team. They are then required to meet with a
counselor in-person to review the plan to ascertain its accuracy and determine if based on what we know of
the student (including his/her academic record, placement results, and background characteristics), the ed
plan can realistically be accomplished. This one-on-one meeting allows counselors to discuss in greater depth
students’ educational goals and provide them with guidance or additional resources to facilitate their success.
During the counseling session, counselors evaluate the ed plan on five criteria as elaborated below. The
evaluation is guided by an Educational Plan Competencies Rubric developed by the Counseling Department.
ASSESSMENT METHODOLOGY INSTRUMENTATION Three assessment instruments were developed to assess SLOs 1 and 2 as described below:
1. Educational Plan Competencies Rubric: Instrument used by counselors to assess how well students
prepared their ed plans prior to meeting with counselors and to determine how “realistic” these were
based on students’ personal and academic backgrounds and goals. The four major elements assessed
were: identification of general education requirements, identification of major requirements and
prerequisites, English/ESL/Math course prerequisite sequencing, and educational plan realistic
appraisal. [Administered initially in Fall 2008 with 249 ed plans evaluated; administered again in Fall
2009 with 436 ed plans evaluated].
Counseling 20 is one of the “top” courses taught at SMC. Enrollment in this class reached 5,235 in 2007-08 according to the
Successful Course Completion Rates 2004-05 to 2007-08 report issued by the Institutional Research Office (11/11/2008).
1
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
6 2. Educational Planning Quiz: Quiz administered to students after a counselor presentation and the
completion of individual ed plans to assess their understanding of the educational planning process
and requirements for course completion and transfer. The quiz integrated a case scenario from which
students then answered a series of questions. [Administered in Fall 2008 with 251 quizzes analyzed;
administered again in Fall 2009 with 301 quizzes analyzed].
3. Educational Planning Survey: Instrument used primarily to assess the in-class educational planning
presentation and a separate transfer presentation. The survey was administered after a counselor
presentation and the completion of individual ed plans. Students were asked to indicate educational
planning elements where additional time/discussion would have helped them understand the
educational planning process better. [Administered in Fall 2008 with 253 quizzes analyzed;
administered again in Fall 2009 with 332 surveys evaluated].
SAMPLE SELECTION With respect to the sample size for the Educational Plan Competencies Rubric, a total of 249 student
educational plans were reviewed by a counselor in fall 2008 and 436 in fall 2009. A general email was sent by
the Counseling Department chairperson to counselors requesting that as students came to meet with them to
use the rubric and record their responses on a rubric sheet. They were then asked to forward completed
rubrics for data input and analysis. As evident in the number of students enrolled in Counseling 20 for the
corresponding semesters, it appears that the vast majority of ed plans evaluated were not recorded on the
rubric sheet. Additionally, it is possible that some counselors simply never forwarded the rubrics or did not
use the rubric when meeting with students.
Given some logistical challenges in surveying every Counseling 20 class, we opted to administer the
Educational Planning Quiz and the Educational Planning Survey to approximately 20% of the 65 classes
taught in Fall 2008 (13 classes selected) and to 40% of the 63 classes taught in fall 2009 (25 classes selected).
A stratified equal probability random sample selection method was employed to select participating classes.
Stratification occurred in the following order with the resulting samples described in Table 1:



Type of schedule: traditional M-Th classes that met twice per week vs. alternative schedules such as 3
hour block classes, Fridays, Saturdays, etc.
By instructor teaching experience: experienced vs. new.
By instructor employment status: full-time vs. part-time.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
7 Table 1. Stratified Random Sample Selected Number Not selected Number Selected Total Eligible Stratification Variable Type of Schedule 2008 2009 2008 2009 2008 2009 Percent Selected 2008 2009 Traditional 38 27 10 19 48 46 76.90% 76.00% Alternative 14 10 3 6 17 16 23.10% 24.00% Seasoned 32 36 8 24 40 60 61.50% 96.00% New 20 1 5 1 25 2 38.50% 4.00% FT 17 11 4 8 21 19 30.80% 32.00% PT 35 26 9 17 44 43 69.20% 68.00% Total 52 37 13 25 65 65 20.00% 40.30% Instructor Experience FT Status SLOS ASSESSMENT RESULTS Based on the analysis of the educational planning competencies rubric, the educational planning quiz, and the
educational planning survey administered, it appears that the vast majority of students participating in this
assessment benefited substantially from the educational planning presentation conducted by a counselor in
the Counseling 20 class. Specifically, 79% of students in the fall 2009 cohort demonstrated competency in
educational planning as evidenced in the ed plan they produced. The corresponding percentage for the fall
2008 cohort was 67%. Hence we see a statistically significant increase in student mastery (p < .001).
Moreover, 69% of students expressed confidence in their ability to develop and adjust an educational plan on
their own based on their individual goals. The sections that follow discuss in greater detail findings pertaining
to the three methods of assessment we employed for SLOs 1 and 2.
EDUCATIONAL PLANNING COMPETENCIES DEMONSTRATED BY STUDENTS The Counseling Department continues to assess educational planning competencies demonstrated by
students enrolled in Counseling 20 classes. This assessment follows one or two educational planning
presentations conducted by counselors or Counseling 20 instructors to educate students on the nuances of
major, general education, transfer, prerequisite identification and requirements that will facilitate the
fulfillment of students’ educational goals. Given these presentations and self-directed research, students are
expected to “Formulate a realistic self-appraisal of their educational status and its relationship to their overall goals” (SLO 1)
and craft a comprehensive and thoughtful educational plan reflective of this exercise.
Data to assess educational planning competencies was first collected in fall 2008 and again in fall 2009. This
section draws comparisons between these two periods.
As noted in Figure 1 and detailed in Table 2, statistically significant differences were found between fall 2008
and fall 2009 students across all competencies assessed. Specifically, students in fall 2009 composed better
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
8 quality educational plans and counselors reviewing these plans rated them significantly higher (M=3.34) in
comparison to those in fall 2008 (M=2.94). Mean differences are significant at p < .001. Overall,
approximately 84% of fall 2009 students demonstrated proficiency in the competencies assessed, as
compared to 67% of fall 2008 students. This represents a 17 percentage point differential, or a net
improvement rate of 25%. According to the analysis:





76% of fall 2009 students (compared to 67% of fall 2008 students) were generally proficient in
identifying all applicable general education courses, units, and prerequisites for their identified goal.
77% of fall 2009 students (compared to 63% of fall 2008 students) were generally proficient in
identifying all applicable courses, units, and prerequisites for their chosen major.
79% of fall 2009 students (compared to 67% of fall 2008 students) were generally proficient in
identifying all relevant prerequisite course sequences for math, English/ESL, and other subjects as
needed.
79% of fall 2009 students (compared to 68% of fall 2008 students) were generally proficient in
developing a three semester/session educational plan and classifying courses as meeting GE, major,
prerequisites, or elective requirements.
84% of fall 2009 educational plans (compared to 67% of fall 2008 educational plans) reviewed were
deemed by counselors to be generally to very realistic in enabling timely progress toward students’
educational goal (given individual student circumstances).
Figure 1. Student educational plan mean differences by cohort year (all differences significant at p < .001) 4
3.5
3
2.8
3.25
3.16
3.12
2.76
3.34
3.25
2.88
2.85
2.94
2.5
2
Fall 2008
1.5
Fall 2009
1
0.5
0
General Education Major Course Requirements Prerequisite & Prerequisites Sequence
Ed Planning & Classification
Realistic Appraisal
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
9 Table 2. Student Educational Plan Competencies Demonstrated by Cohort Fall 2008 Competency Demonstrated N General Education Pct. Mean SD Fall 2009 N Pct. Mean SD P* (1) Mostly not noted, most items missing 52 20.9% 45 10.3% (2) Seldom noted, multiple items missing 31 12.4% 60 13.8% (3) Usually noted, few items missing 82 32.9% 129 29.6% (4) Always noted, no items missing 84 33.7% 202 46.3% 436 Total 249 100.0% Major Requirements & Prerequisites 2.8 1.12 100.0% 3.12 1.00 < .001 (1) Mostly not noted, most items missing 65 26.1% 54 12.4% (2) Seldom noted, multiple items missing 26 10.4% 44 10.1% (3) Usually noted, few items missing 63 25.3% 117 26.8% (4) Always noted, no items missing 95 38.2% 221 50.7% 436 Total 249 100.0% Course Prerequisite Sequence Score 1.04 < .001 23.3% 49 11.2% (2) Seldom noted, multiple items missing 25 10.0% 41 9.4% (3) Usually noted, few items missing 62 24.9% 97 22.2% (4) Always noted, no items missing 104 41.8% 249 Total 249 100.0% 436 3.16 58 2.85 1.20 100.0% (1) Mostly not noted, most items missing Ed Planning & Classification 2.76 1.22 57.1% 100.0% 3.25 1.03 < .001 (1) Mostly not noted, most items missing 56 22.5% 52 11.9% (2) Seldom noted, multiple items missing 25 10.0% 41 9.4% (3) Usually noted, few items missing 62 24.9% 90 20.6% (4) Always noted, no items missing 106 42.6% 253 58.0% Total 249 100.0% 436 Realistic Appraisal 2.88 1.19 100.0% 3.25 1.05 < .001 (1) Plan is not very realistic 48 19.3% 39 9.0% (2) Plan is somewhat realistic 35 14.1% 29 6.7% (3) Plan is generally realistic 50 20.1% 112 25.8% (4) Plan is very realistic 116 46.6% 254 58.5% Total 249 100.0% 434 2.94 1.17 *Mean differences between Fall 2008 and Fall 2009 cohorts. Santa Monica College Counseling Department 100.0% 3.34 0.95 < .001 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
10 STUDENT APPLICATION OF EDUCATIONAL PLANNING COMPETENCIES—QUIZ In conjunction with assessing the educational planning
competencies above, we sought to determine if students could
accurately demonstrate their understanding of the educational
planning process and requirements for course completion and
transfer a few weeks after they completed their educational plan.
This was accomplished by administering an in-class quiz which
integrated a case scenario of a recent high school graduate
interested in transferring to CSU Fullerton after attending SMC.
According to the scenario, the fictitious student was: 1) pursuing
a major in sociology; 2) taking Math 81 (Basic Arithmetic) and
English 21A (English Fundamentals 1); and lastly, would be
completing Math 52 (Elementary Statistics) and her general
education requirements prior to transfer.
The seven-item quiz administered to Counseling 20 students
indicated that most students could correctly identify the relevant requirements to enable the fictitious student
to transfer to CSU Fullerton. Additionally, they demonstrated their understanding of the number of units
typically required for transfer and what constitutes an “elective” course. Relatively few changes from fall 2008
to fall 2009 were found as evidenced in Figure 2 and Table 3. In some cases, the percentage of students
correctly identifying a response declined by a few percentage points. Most notably, the percentage of students
correctly indicating the course prerequisite sequence for Math 52 and the availability of transfer information
on ASSIST increased by 7 and 8 points, respectively. Using a different methodology, the quiz was scored.
Out of a possible 9 points, students in fall 2008 obtained an average of 6.46 (N = 251) points, whereas those
in fall 2009 obtained 6.54 (N = 301) points (not statistically significant).
What is clear from this evaluation is the fact that the majority of students do not recognize the fact that a
potential CSU Fullerton transfer student may use not only IGETC for CSU, but also the CSU GE breadth
pattern. This is likely due to counseling presentations emphasizing IGETC as the plan of choice for students
who may be contemplating multiple universities across segments.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
11 Figure 2. Percentage of Students Correctly Identifying Educational Planning Quiz Responses 30%
40%
50%
60%
70%
80%
90%
100%
Use IGETC for CSU to Transfer to CSU Fullerton
Use CSU GE to to Transfer to CSU Fullerton
Sociology 1 Meets GE & Major Requirements
Prerequisite Sequence for Math 52
Fall 2008
Fall 2009
Units Needed to Transfer
English 1 Prerequisite
Transfer Information Availability: TCC
Transfer Information Availability: ASSIST
Transfer Elective Definition
Table 3. Percentage of Students Correctly Identifying Educational Planning Quiz Responses Fall 2008 Fall 2009 N Pct. N Pct. N Pct. Use IGETC for CSU to Transfer to CSU Fullerton 194 77.3 221 73.4 415 75.2 Use CSU GE to to Transfer to CSU Fullerton 114 45.4 124 41.2 238 43.1 Sociology 1 Meets GE & Major Requirements 152 61.5 168 58.3 320 59.8 Prerequisite Sequence for Math 52 168 68.6 221 75.4 389 72.3 Units Needed to Transfer 235 97.9 280 96.2 515 97.0 English 1 Prerequisite 184 77.3 210 73.9 394 75.5 Transfer Information Availability: TCC 211 84.1 227 75.4 438 79.3 Transfer Information Availability: ASSIST 180 71.7 242 80.4 422 76.4 Transfer Elective Definition 237 95.2 276 93.2 513 94.1 Santa Monica College Counseling Department Total [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
12 EVALUATION OF EDUCATIONAL PLANNING AND TRANSFER PRESENTATIONS While the results presented thus far point to the fact that the vast majority of students have acquired the
knowledge and skills to craft a realistic educational plan, we also sought to evaluate the quality of the
educational planning and transfer presentations conducted in the classroom. The results presented in Figures
3-6 and Tables 4-6 generally indicate students’ satisfaction with the presentations. However, notable
differences were observed when comparing responses from fall 2008 and fall 2009. Specifically, it seems that
favorable responses decreased to a statistically significant degree (p < .001) in most aspects of the
presentations. For example, the percentage of students indicating that completing the educational planning
assignment lead to increased confidence in formulating or revising a plan of their own, decreased by 14
percentage points from 80% in fall 2008 to 69% in fall 2009. Given these decreases, the Counseling
Department ought to investigate potential causes for decreased scores and enact changes to the educational
planning assignment or the presentations.
As a first step, note should be taken that a significant portion of students indicated their need for counselors
to spend additional time in class to help them address some concerns:





Explaining the Ed Plan assignment—18%
Exploring individual educational goals—36%
Explaining general education requirements—23%
Explaining major requirements—36%
Explaining the sequence for Math and English courses—14%
Furthermore, a significant proportion of students indicated a preference for meeting individually with a
counselor more than once to address the following specific concerns:




Choosing a major and/or college to transfer to—50%
Gathering information on other colleges/universities or majors—62%
Understand how to complete the Ed Plan—27%
Understanding how to make corrections the Ed Plan—29%
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
Figure 3. Mean Educational Planning Presentation Rating by Cohort Educational Planning Presentation
5
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
4.37
4.25
3.83
4.22
3.93
3.86
3.73
2.65 2.7
Fall 2008
Fall 2009
Clear and easy to understand
Helped me Helped me Answered most Was understand identify courses of my questions overwhelming, how to choose I need to take
with too much a university
information
Santa Monica College Counseling Department 4.03
13 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
Table 4. Percentage Agreement of Educational Planning Presentation Rating by Cohort Fall 2008 N Fall 2009 Total Pct. N Pct. N Pct. Clear and easy to understand Strongly Disagree 1 0.40% 25 7.70% 26 4.70% Disagree 8 3.40% 35 10.80% 43 7.70% Neutral 27 11.60% 34 10.50% 61 11.00% Agree 93 39.90% 107 33.00% 200 35.90% Strongly Agree 104 44.60% 123 38.00% 227 40.80% Helped me understand how to choose a university Strongly Disagree 1 0.40% 30 9.40% 31 5.70% Disagree 5 2.20% 27 8.50% 32 5.90% Neutral 18 7.90% 25 7.90% 43 7.90% Agree 87 38.30% 88 27.70% 175 32.10% Strongly Agree 116 51.10% 148 46.50% 264 48.40% Helped me identify courses I need to take Strongly Disagree 1 0.40% 28 8.80% 29 5.40% Disagree 7 3.10% 35 11.00% 42 7.70% Neutral 28 12.40% 35 11.00% 63 11.60% Agree 94 41.80% 73 23.00% 167 30.80% Strongly Agree 95 42.20% 146 46.10% 241 44.50% 0.40% 18 5.70% 19 3.50% Answered most of my questions Strongly Disagree 1 Disagree 12 5.30% 34 10.80% 46 8.50% Neutral 38 16.70% 57 18.20% 95 17.50% Agree 106 46.50% 110 35.00% 216 39.90% Strongly Agree 71 31.10% 95 30.30% 166 30.60% Was overwhelming, with too much information Strongly Disagree 35 15.60% 56 17.80% 91 16.90% Disagree 80 35.60% 86 27.40% 166 30.80% Neutral 59 26.20% 99 31.50% 158 29.30% Agree 30 13.30% 42 13.40% 72 13.40% Strongly Agree 21 9.30% 31 9.90% 52 9.60% Santa Monica College Counseling Department 14 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
Figure 4. Mean Transfer Presentation Rating by Cohort Transfer Presentation
5
4.5
4
4.37
3.94
4.2
4.16
3.77
3.74
3.5
3
2.59 2.64
2.5
2
Fall 2008
1.5
Fall 2009
1
0.5
0
Clear and easy to Helped me Answered most of Was overwhelming, understand
understand how to my questions
with too much choose a university
information
Santa Monica College Counseling Department 15 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
Table 5. Percentage Agreement of Transfer Presentation Rating by Cohort Fall 2008 N Pct. Fall 2009 N Pct. Total N Pct. Clear and easy to understand 2 1.00% 20 6.20% 22 4.10% Disagree 2 1.00% 33 10.20% 35 6.60% Neutral 17 8.10% 34 10.50% 51 9.60% Agree 83 39.70% 97 29.80% 180 33.70% 105 50.20% 141 43.40% 246 46.10% Strongly Disagree Strongly Agree Helped me understand how to choose a university Strongly Disagree 1 0.50% 21 6.60% 22 4.20% Disagree 4 1.90% 28 8.80% 32 6.10% Neutral 37 17.90% 65 20.30% 102 19.40% Agree 83 40.10% 95 29.70% 178 33.80% Strongly Agree 82 39.60% 111 34.70% 193 36.60% Answered most of my questions Strongly Disagree 0 0.00% 24 7.50% 24 4.60% Disagree 4 2.00% 29 9.10% 33 6.40% Neutral 32 16.10% 52 16.30% 84 16.20% Agree 83 41.70% 116 36.40% 199 38.40% Strongly Agree 80 40.20% 98 30.70% 178 34.40% Was overwhelming, with too much information Strongly Disagree 32 15.90% 57 18.00% 89 17.20% Disagree 74 36.80% 103 32.50% 177 34.20% Neutral 55 27.40% 85 26.80% 140 27.00% Agree 25 12.40% 40 12.60% 65 12.50% Strongly Agree 15 7.50% 32 10.10% 47 9.10% Santa Monica College Counseling Department 16 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
Figure 5. Mean Rating for Presentation Outcomes by Cohort Presentation Outcomes
4.4
4.28
4.3
4.2
4.12
4.1
4
3.9
3.9
Fall 2008
3.81
3.8
Fall 2009
3.7
3.6
3.5
Found entire process very helpful
Feels more confident in ability to develop and adjust an ed plan based on individual goals
Table 6. Percentage Distribution of Presentation Outcomes by Cohort Fall 2008 N Pct. Fall 2009 N Pct. Total N Pct. Found entire process very helpful
Strongly Disagree 1 0.40% 32 9.80% 33 5.80% Disagree 4 1.60% 28 8.60% 32 5.60% Neutral 32 13.00% 28 8.60% 60 10.50% 99 40.10% 90 27.60% 189 33.00% 111 44.90% 148 45.40% 259 45.20% Agree Strongly Agree Feels more confident in ability to develop and adjust an ed plan based on individual Strongly Disagree 0 0.00% 30 9.30% 30 5.20% Disagree 5 2.00% 20 6.20% 25 4.40% Neutral Agree Strongly Agree 44 17.70% 51 15.70% 95 16.60% 116 46.60% 104 32.10% 220 38.40% 84 33.70% 119 36.70% 203 35.40% Santa Monica College Counseling Department 17 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
18 Figure 6. Percentage of Students Needing Additional Assistance with Ed Plan Assignment Additional Assitance Needed
0%
10%
20%
30%
40%
50%
60%
70%
18%
18%
Explaining the Ed Plan assignment
Exploring my own educational goals
36%
44%
23%
23%
Explaining general education requirements
34%
36%
Explaining major requirements
8%
Explaining the sequence for Math and English courses
14%
Choose a major and/or college to transfer to
50%
58%
55%
Gather information on other colleges/universities or majors
62%
31%
27%
Understand how to complete the Ed Plan
34%
29%
Understand how to make corrections to my Ed Plan
Fall 2008
Fall 2009
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
19 RECOMMENDATIONS FOR IMPROVEMENT Based on the outcomes of this study, it is clear that participants are developing or enhancing their knowledge
in the educational planning process, and most importantly, they generally appear to be cognizant of what
constitutes a realistic educational plan. Overall results indicate that approximately 84% of students in the fall
2009 cohort demonstrated desired educational planning competencies as evidenced in the educational plan
they produced. This represents a 25% increase as compared to students in the fall 2008 cohort. Moreover,
69% expressed confidence in their ability to develop and adjust an educational plan on their own based on
their individual goals.
If the exercise of crafting an educational plan following an in-class counselor presentation is to reap its full
potential, additional counseling time may afford students the additional time they require to address individual
concerns in the context of the classroom. This is not only evidenced in Table 2, but in student responses to
how the ed plan and the transfer presentations could be improved to facilitate their learning and goal
completion (Figure 6).
To address some of these shortcomings, the Counseling Department should consider spending additional
time discussing issues most problematic to students, including how to gather information on other
colleges/universities’ majors, how to choose a college to which to transfer, exploring educational goals, and
explaining major requirements. Perhaps additional assignments in these areas should be integrated into the
Counseling 20 curriculum. These assignments could then be compiled as part of a portfolio.
Additional Considerations—Future Directions Based on March and May 2010 Counseling Department discussions, the following recommendations were
offered by counselors and instructors to address some of the findings in this report:
Revising the Counseling Educational Planning assignment, rubric, and quiz:



The ed plan worksheets students complete will incorporate a section where counselors will record
each score assigned to the draft submitted by the student for review. This will in turn inform the
instructor who grades the assignment, of the accuracy, comprehensiveness, and effort the student
made to complete the assignment.
The ed planning quiz will be lengthened and questions revised to incorporate a general knowledge
section related to transfer and educational planning, and will expand the case scenario. Questions
related to the case scenario will be substantially more complex as the quiz will be administered soon
after the assignment has been completed, rather than at the end of the semester.
The quiz will be objective and it will be graded by every instructor who will collect item analysis
information using scanners already available at the college. In this manner, instructors will have
immediate feedback on how students performed on the quiz and will address any areas where
students may have been unclear about the transfer or ed planning process.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
20 Recommendations for reinforcing “How to effectively gather the necessary information on other
colleges/universities and/or majors” in Counseling Sessions:












Provide Counseling In-Services on Bridges and Choices programs
Create a you-tube video showing counselors working with students (from various programs)
explaining basics about majors and colleges
Create a “How To” hand-out with respect to finding information about majors and colleges.
Use a questionnaire about school criteria/size, location, etc to generate topics for students to
consider.
Encourage students to attend college fairs and specialized fairs (Art fair, Engineering Fair)
Generate a document that provides useful websites and can be handed out to students
Walk students through certain informational sites
Use workshop outlines (about private schools/elites) give this handout to students
Show occupational handbook on the web
Make frequent use of transfer/articulation website and show students how to use it.
Use a questionnaire about school criteria/size, location, etc to generate topics for students to
consider.
Instructors need “smart” classrooms so they can demonstrate websites in the C.20 classes – they can
even demo how to navigate among some of the websites (like Assist.Org)
Recommendations for reinforcing “How to make corrections to the student’s educational plan (as
opposed to the counselor just making corrections directly for the student) in Counseling Sessions:




After finishing the educational planning assignment with a student, take 5 minutes to allow student to
“explain back” their plan
For those who schedule a second visit because they need more time, try to “prep” student before
follow-up visit on what he/she should focus on in advance
Emphasize to students to read course descriptions and prerequisites and reinforce content of
courses.
Need to “package” topics addressed on the transfer website onto the Counseling home page and
make them user friendly for the counselor and the student (ex. “how to choose a major, school, etc)
Recommendations for reinforcing “How to Select a Major” in Counseling Sessions:










Find out what student really likes, ask pointed questions before providing direction
Provide reality check for students
Separate pre-professional requirements from major requirements
Show pathways for each
Explain/demonstrate Bridges.com and other career resources
Use “undecided major contract” (EOP&S has this document)
Explain that each student must undergo a process of self-exploration
Consider an internship in your area of interest
Talk to professors and other people in your field of interest
Check out college websites (ex, 4 yr. institutions , assist.org, career websites. Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
21 APPENDIX A: PROGRAM LEVEL STUDENT LEARNING OUTCOMES Date: March 23, 2009
Program Name: Counseling Department and Transfer Services
Santa Monica College Institutional Learning Outcomes
Through their experiences at SMC, students
will:
Personal Attitudes and
Behaviors
Analytic & Communication
Skills
Applied Social Knowledge
and Values
Applied Knowledge and
Valuation Of The Physical
World
Acquire the self-confidence and self-discipline to
pursue their intellectual curiosities with integrity in
both their personal and professional lives.
Obtain the knowledge and academic skills
necessary to access, evaluate, and interpret ideas,
images, and information critically in order to
communicate effectively, reach conclusions, and
solve problems.
Respect the inter-relatedness of the global human
environment, engage with diverse peoples, and
acknowledge the significance of their daily actions
relative to broader issues and events.
Take responsibility for their own impact on the
earth by living a sustainable and ethical life style.
Student Learning Outcomes: What can students do, think, or know as a result of receiving services from
your program, center or office? Please identify at least two.
1.
Counseling Services:
As a result of the transfer and counselor presentations on educational planning in the Counseling 20 course
and the subsequent individual counseling session:
A.
Students will identify their math and English course sequences and chart their individual
math and English course sequences to achieve their educational and transfer goals.
B.
Students will increase self-confidence by formulating a realistic self-appraisal of their
educational status and its relationship to their stated educational and transfer goals(s).
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
22 2.
Counseling Instruction:
As a result of taking a Counseling course, students will:
A.
Develop self-discipline for student success by being in class on-time and turning in
assignments on time.
What plan will be used to assess these outcomes? Please identify the following components:
1. Frequency (how often per academic year):
Counseling Services:
Each semester, counselors will rate the ability of Counseling 20 students to accurately chart their
individual math and English course sequences, identify all necessary courses and conduct an adequate
self-appraisal of whether they can realistically meet their goals, as a result of the educational planning
sessions provided by counselors in the C.20 classes.
Each semester, an educational planning quiz and student surveys will also be administered and
distributed to C.20 students after the educational planning process has concluded in class. The
quizzes will measure the students’ ability to “apply” what they learned in the educational planning
process about charting.
English and math course sequences, planning the correct coursework to achieve academic goals and
conducting a realistic appraisal of a student’s ability to meet planned goals. The surveys will solicit
feedback directly from the students about how the educational planning process can be improved.
Counseling Instruction:
Each semester, all counseling instructors will calculate their class averages for on-time attendance and
on-time submission of assignments, using a standardized departmental instructional SLO report
form. As part of this assessment, instructors will continually evaluate and, if needed, improve the
clarity of course expectations included in the instructor’s syllabus and the instructor’s individual
communication with students.
2. Method(s) used to assess each SLO (survey, focus group, evaluation of transcripts, etc.):
Counseling Services:
Counselors will use the “Educational Plan Competencies Rubric” for scoring each C.20 student that
they see in session.
Student educational planning quizzes and surveys will also be administered and distributed in the
Counseling 20 classes after the educational planning process has concluded.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
23 Counseling Instruction:
Instructors will use the standardized Counseling department instructional SLO report form.
Please attach a copy of measurement tools you have used or plan to use.
See attached.
Please describe any assessment results you have obtained.
Counseling Services:
Refer to preceding report.
Counseling Instruction:
Results for the Counseling Department Instructional SLO have now been tabulated and analyzed.
With respect to missing classes, reports from 27 class sections indicated that 84% of the students met
the attendance expectations noted by the instructor. A smaller percentage of DSPS students missed
more classes.
With respect to students arriving persistently late to class, reports from 22 class sections revealed that
on average, 16% of students arrived late. This was particularly the case in one “Study
Skills/Orientation to Higher Education” class and two career classes. Note should be taken that two
of the six career class sections reported persistent late arrivals approaching 50%.
Lastly, the great majority of students submitted their assignments on time. The average rate was 77%.
As with persistent lateness, it appears that students in career classes had greater difficulty meeting
deadlines. (See Table #2 on the next Page).
Overall, the percentage of students in counseling courses who regularly attend class, arrive on time
and submit required assignments on time is fairly is fairly high – 84%. For the first two items and
77% for the third item. Instructors in the Counseling Department discussed these findings and ideas
for increasing the percentages of students who regularly attend class, practice punctuality and submit
assignments by the required deadlines for Spring 2009 at the institutional day meeting on Thursday,
May 19.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
24 Please describe the program changes you have made as a result of your SLO assessments.
Counseling Services:
Ideas generated for improving the educational planning assignment at the departmental flex day on
Mar.6, 2009, include the following:
For Counselors






Consider spending additional time discussing issues most problematic to students (e.g., major
prerequisites, selection and application of major courses also applicable to general education)
When students are missing an entire element (or entire elements) of the assignment, send them
to the mentoring table to rework the assignment and then return to the counselor
Arrange more mentoring services and advertise mentoring more to help C.20 students get
connected with the mentors
Incorporate Educational Planning workshops in the Student Services Workshop offerings.
Set up Educational Planning labs (like the UC/CSU application labs)
Do an “Ed Plan” Study Hall
For the Classroom:










Pair up students according to major to work in Ed planning groups in C.20
Utilize small group process as much as possible
Require visit to peer mentors before seeing a counselor
Provide C.20 students with several sample plans – use sample plan when introducing the task
Break down task and assign point values to each step/segment of the ed.planning assignment.
Provide additional blank copies of the educational plan so plans can be redone when necessary
Consider spending additional time discussing issues most problematic to students (e.g., major
prerequisites, selection and application of major courses also applicable to general education)
Encourage/require students to make a Counseling appt. to start talking to a counselor to discuss
major & campus options and find out what classes they would need for one or more
majors/campuses
Find ways to have students apply their newly gained knowledge immediately during the
presentation.
Ex. Utilize a problem-based learning approach during counselor presentation by placing students
into small groups and ask them to design a sample educational plan that integrates all elements.
The opportunity for in-class discussion and debate may allow them to clarify “muddy” issues and
educate one-another. Having crafted an in-class ed plan—reviewed by the counselor—students
would then be best suited to craft their “homework” ed plan on their own.
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
25 Counseling Instruction:






As a result of the departmental meeting on Mar.19, 2009, Counseling Instructors agreed to the
following for Spring 2009 and Fall 2009:
All instructors need to collect attendance and late arrival data effective Spring 09
All Instructors will use a revised SLO tracking form in Spring 2009 and Fall 2009, in order to report
outcomes in a quantitative fashion, in addition to the written reflections provided.
Attach absences to grading (ie., participation)
Spend time early in class discussing issues that create tardiness (raise awareness)
Counseling Instructors should ensure that all course syllabi contain clear language that details rules
and consequences regarding late arrivals, absenteeism and failure to complete assignments on time.
The additional optional recommendations were discussed:




Constant reinforcement – consider using E Companion so students can be responsible for tracking
their attendance, assignment completion and punctuality.
Utilize Early Alert
Counseling Instructors could make a concentrated effort in the future to have at least one
conversation with students who demonstrate habitual tardiness or absenteeism and/or failure to
submit assignments by the required deadline.
Have students complete reflection statements regarding their efforts immediately following
completion of an assignment.
Edited LSG 03-23-09
Santa Monica College Counseling Department [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
APPENDIX B: RUBRIC, QUIZ, AND SURVEY FORMS
EDUCATIONAL PLAN COMPETENCIES RUBRIC Santa Monica College Counseling Department 26 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
EDUCATIONAL PLANNING Q UIZ Santa Monica College Counseling Department 27 [A “REALISTIC APPRAISAL” OF STUDENT EDUCATIONAL PLANNING]
EDUCATIONAL PLANNING SURVEY Santa Monica College Counseling Department 28