College of Education Program Planning Committee Report First program planning self-study – 2002-2003 Submitted to Provost Sigler March 30, 2009 The Program Planning Committee thanks the college and each department for making progress toward meeting University and WASC requirements. All departments submitted their first self-studies and assessment reports in fall 2008, except for Secondary Education, which was exempted because they don’t offer a degree, and Child and Adolescent Development which has been participating in university program planning for some time. Child and Adolescent Development is on a different program planning schedule and is not included in this report. We understand that both the state of California and the federal government place heavy burdens on the departments in order for the college to be certified for teacher preparation. We don’t wish to increase the burden and wish to work with the college to coordinate program planning and assessment with existing credentialing requirements. In the next self-study we expect each department to use data currently available from the Office of Institutional Research to analyze trends over the previous five years and to help to plan for the next five. We understand that the college is trying to implement a better method for following progress of students through the programs. We applaud and encourage this effort, but having that in place is not a prerequisite for using other data available. Sutee Sujitparapitaya, AVP for Institutional Research, and the office of Undergraduate Studies are anxious to work with you to make existing data more usable, if necessary. All reports begin with a description of the College and a college committee review that is identical for all programs. The college should take care to identify the relative strengths and opportunities for improvement of each department. While all the programs may face some common challenges to the education community, each department should also be evaluated on its own merits. All departments have a list of student learning objectives (SLOs) posted on the Assessment website. Only one of the departments used these in the January 2009 assessment report. Almost without exception the programs are reporting process rather than data and results. Departments might consider having SLOs be the same as the professional standards required by the California Commission on Teacher Credentialing (CCTC), if that would simplify the work. Most departments seem to be confusing program assessment with student evaluation. The Program Planning Committee encourages the College to invite the university Director of Assessment, Jackie Snell, the college assessment facilitators, Toni Campbell and Mary McVey, to professional development events to help change their practices to better align with the new CCTC requirements as well as WASC and University requirements. Dan Apple of Pacific Crest is another resource that may be appropriate for faculty development. All departments need to be encouraged to make progress on collecting student performance data directly on specific student learning objectives, to make curricular and pedagogical changes directly linked to the student learning data, and to follow-up by collecting additional student performance data to measure the change in learning associated with the curricular or pedagogical changes. We believe this is consistent with the new CCTC guidelines and encourage you to work with the university resources listed above to integrate and streamline the process as much as possible. Normally accredited programs include the accreditation site visit report as part of the program planning binder. An exception was made for these first submissions, but the site reports should be included in the future. Page 1 of 9 An assessment report from ALL programs conforming to the attached spring 2009 through spring 2010 is due to Undergraduate Studies March 1, 2010. All programs on campus are being asked to submit a report on that date in order for the University to prepare an interim report for WASC which is due in the fall of 2010. This appears to be well-aligned with accreditation requirements for the programs in Education. Since the next accreditation site visit is planned for spring 2010, the next self-study for all departments except Child and Adolescent Behavior, and including Secondary Education, is due the office of Undergraduate Studies by December 1, 2010. Associate Dean Elaine Chin and AVP for Institutional Research, Sutee Sujitparapitaya affirm that accuracy in coding for students in credential programs should be vastly improved starting with in fall 2008. Therefore self-studies should include a discussion of all the required data elements provided by the Office of Institutional Research as well as reports of the visiting accreditation teams. 2008 – 2009 Program Planning Committee Debra Caires Mary Calegari Peter Chua Elaine Collins Bob Cooper M. E. Fayad Beverly Grindstaff Xiaolu Hu Thuy Le Quynh Lu Bill Nance Dan Perales Annabel Prins Lori Rodriguez Jackie Snell Pam Stacks Gary Stebbins Patricia Stroh Sutee Sujitparapitaya Shailaja Venkatsubramanyan Ashwini Wagle CC: Chairs: Susan Meyers, Dean, College of Education Elaine Chin, Associate Dean, College of Education Beth Von Till, Chair, Curriculum and Research Bob Cooper, AVP Undergraduate Studies Pam Stacks, AVP Graduate Studies and Research Bill Nance, Vice-Provost Page 2 of 9 Appendix: Summary of College of Education Self-Studies (including comments to the departments) All credential programs in the College of Education are accredited by the state of California every seven years. A national accrediting body accredits the College on the same schedule. In addition the State requires assessment of student learning reports in alternate years. The next site visit for both state and national accreditation will occur in spring 2010. When students are enrolled in both credential programs and master degree programs (for example secondary education credential and math MA), the Office of Institutional Research tracks them as math MA students but does not track them as credential students. This creates some problems for the programs and the College of Education is working on tracking credential students. In particular, for our purposes, departments have a hard time participating in University Program Planning because the OIR data is incomplete. So far the university has only asked that degree programs (not credential programs) participate. There has been considerable debate over the years, and especially recently in the Program Planning Committee, but generally the feeling seems to be that credential programs, as much as any, need to plan for enrollments. Communicative Disorders Summary of the Self-Study This department offers two degrees, a B.A.in Communicative Disorders and Sciences and a M.A in Education with a concentration in Speech-Language Pathology. Students are required to complete two internships, one with a school and one in a medical setting The California Commission on Teacher Credentialing may approve a new teaching credential in the near future that would require some courses that can only be taught in this department. If this comes to pass it would greatly strain their resources. Planned curriculum changes: Eliminate an undergraduate course on communication and ageing while increasing the coverage of those topics in other courses Develop an upper division course on Fluency Disorders Reactivate a graduate course on swallowing disorders Develop a new graduate course on augmentative and alternative communication Blend four courses into two Phase out the professional issues seminar and combine content with the research methods seminar For the last two years the program has about 175 FTES. The department reports banner recruitment in those years, resulting in increasing the quality of the students in the program. They would like to increase the number of students who choose to write a thesis. This program is doing somewhat better at recruiting students from underrepresented minorities, compared to the national average, but is somewhat behind in recruiting male students. Thank you to the department for discussing this topic and for including the Required Data Elements produced by the Office of Institutional Research. Summary of the Assessment Report Students are assessed by faculty on key assignments. They are strong in ‘synthesizing critical information regarding assessment and treatment of individuals with acquired brain-damage.” Students are routinely rated as ‘exceeding expectations’ on being prepared to begin student teaching or the externship (offcampus practicum). Students excel at certain aspects of clinical performance, but the faculty has determined that they need a better rubric for assessing performance on other aspects. They have implemented the new rubric for the on-campus practicum and will begin implementing it in the off-campus “externship” in AY 2009-2010. They have changed questions on an exam in order to give students more practice in the types of questions asked on the ETS exam (see the next paragraph). Page 3 of 9 Students are required to take an ETS exam in Speech-Language Pathology. Average scores in 7 categories are reported. This exam tests knowledge and the application of knowledge. The department interprets the data as “showing program completers have the required breadth and depth of knowledge necessary for successful practice.” SJSU students scored below the national average on an area covered in their undergraduate education, so the department is looking for ways to incorporate more foundational material in the graduate courses and to emphasize the connection between foundational knowledge and advanced study. Students also scored slightly below the national average on clinical management. They report being committed to improving student knowledge in this area, but do not report a specific plan. The department appears to be collecting student performance data directly on specific student learning objectives, and attempting to make curricular and pedagogical changes directly linked to the student learning data. The Program Planning committee would like to encourage collecting student performance data to measure the change in learning associated with the curricular or pedagogical changes. Counselor Education Summary of the Self-Study The department offers a M.A. in Education with a Concentration in Counseling and Student Personnel, and a Credential in Pupil Personnel Services. They plan to develop a new course on the topic of student services in higher education, expand study abroad and attract more international students, and develop a new course in career counseling for undergraduate students. They don’t report having an undergraduate program, so this may be a GE course. They also plan to improve the MA project guidelines and improve student advising. Both the MA and the Personnel credential experienced a large increase in enrollment in fall 2007 when the state began requiring more counselors placed in schools. Both the student body and faculty are ethnically diverse. The student body is heavily weighted toward females, the faculty somewhat less so. Thank you to the department for discussing the data provided by the Office of Institutional Research. The department has made impressive efforts toward service learning and collaboration with school districts. During the five-year review period 2002-2007 they played a leadership role in the following projects Counselor Education Transformation Grant, with the Eastside Union High School District. Study Abroad in Honduras, Zambia and China Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) – two projects in conjunction with five local school districts. School Counselor Leadership conference The department requests many increases in resources, but also reports that institutional support for personnel, space, equipment, materials, and supplies is adequate. Indeed, ten faculty members received .2 release-time to supervise 105 students in spring 2008. The department plans to hire two new faculty members over the next two years. At least six classes are offered each semester through the TV network available at the Instructional Resources Center (IRC). A broadcasting assistant is available in the IRC site and coordinator is available at each remote site to assist with technical issues and class management. Summary of the Assessment Report Counselor Education is perhaps ahead of most departments in the college by having student learning objectives that are mapped onto professional standards. The department has attached some grading rubrics, one of which may be linked to professional standards and includes some data for spring 2008 (the Field Supervisor’s Evaluation). The report does not discuss the meaning of the data. We encourage the department to make progress on collecting student performance data directly on specific student learning objectives, to make curricular and pedagogical changes directly linked to the student learning data, and to follow-up by collecting additional student performance data to measure the change in learning associated with the curricular or pedagogical changes. Page 4 of 9 Educational Leadership Summary of the Self-Study The department offers an M.A. in Education Administration and Supervision and an MA in Administration and Higher Education. It offers a Preliminary and Professional Administrative Services credential. Between fall 2003 and spring 2008 FTES usually varied between 370 and 400 (04-05 and 06-07 were lower). Headcount is only slightly higher, which would mean most students are taking a full 12 units, while the department reports most students are part-time. The department should work with the Office of Institutional Research to check on the accuracy of these numbers. The department reports “continued growth” though the FTES table shows an increase only in the most recent year (2007-8). It appears that headcount in classes has increased over the years of the review (indicating, perhaps, that students are taking more classes). If the number of students graduating each year shows any pattern, it is that most students graduate in alternate years. The department should look into this. If it is a pattern, might there be course scheduling problems? The department finds it hard to recruit faculty because faculty salaries are lower than public school administration salaries During the last five years the department has begun offering classes in East San Jose and the South Valley in order to serve students where they live and work. The department reports an objective to “align all courses, assignments, signature assignments, readings, and instructional strategies” in 2007-8. Since this was submitted in summer 2008, the work may have already begun or even finished. Even so the committee recommends making focused changes based on student learning results rather than trying to “realign” everything. All multi-section courses have coordinators who convene at least two instructor meetings per semester where student work is reviewed. Some raw data are reported in a table, but there is no interpretation of the data. The department reports having modified the “unit development assignment” and enhanced the assignment on analyzing a school or district budget, without reporting why or what the resulting effects on student learning were. An attached table shows 99% of students meeting the budget analysis standard on the first attempt. Probably this is after the modification of the assignment, but the table is not discussed in the report. The department reports needing additional faculty advising for the student Action Research Project, but do not suggest a plan for achieving that. An attached table lists assignments and scoring mechanism for five Student Learning Objectives. The first three learning objectives have student performance data attached. Apparently 100% of students are achieving acceptable performance on these five assignments, at least by the 2nd rewrite. The department needs to be made aware that WASC does not accept student self-assessment as a direct measure. Please separate student self-assessment data from the faculty assessment data. The department requests additional administrative staff and technical and instructional support for distance learning and electronic portfolios. They report that support for CMS and PeopleSoft has improved. The department submitted a Substantive Change proposal to WASC in December 2006. Approval of this proposal would make official the off-campus programs in Salinas, King City, Soledad, Santa Cruz Hollister, Watsonville, Seaside, and Monterey. They request help from the CSU in gaining approval. Currently students in these remote locations are required to take more than half of their classes in San Jose. Approval of the proposal would allow SJSU to offer more off-campus courses. The department currently has just 3 tenured/tenure track faculty members and approximately 82% of classes are taught by part-time lecturers. They have requested three new tenure track positions. Page 5 of 9 Summary of the assessment report Report is organized around the key assignments and reports overall pass rates. Apparently the department has developed or is developing some assessment rubrics to analyze specific skills and knowledge. Student learning objectives are not listed in the report, but probably the questions asked on the employer survey are directly aligned with the learning objectives. One course was moved to one semester earlier in the course sequence in response to student essays on the program. The Program Planning Committee encourages the department to make progress on collecting student performance data directly on specific student learning objectives, to make curricular and pedagogical changes directly linked to the student learning data, and to follow-up by collecting additional student performance data to measure the change in learning associated with the curricular or pedagogical changes. Elementary Education Summary of the Self-Study The department offers preparation for a multiple-subject credential. They also offer an M.A. in Education with a concentration in Curriculum and Instruction. Most students in the program are reentry students who are employed full-time. The department offers a number of options for completing the program that differ primarily in the length of time needed to finish. There is no analysis of students enrolled in the program by FTES, ethnic group, nor any other criteria specified for the self-study. A few charts are offered in appendix A, but not discussed in the self-study (four actually, on one page, which are then repeated several times in the next few pages.) The charts are inadequately labeled and there is no reporting of where the charts came from. There is no data from the Office of Institutional Research discussed or reported in the self-study. The self-study section on assessment discusses the department’s involvement with PACT (Performance Assessment for California Teaching, a consortium of 30 teacher preparation institutions across California), and a bit about the process, but does not list student learning outcomes for the program(s) nor does it discuss student learning. Full participation in PACT is not funded. Summary of the Assessment Report This report does give a table of the number students in each option over the last two years (without a total.) Two years is the cycle for California Commission on Teacher Credentialing accrediting, for whom this report was prepared. Program Planning guidelines require five years of data, which would better indicate whether there is a trend. The middle-level emphasis began in 2004. No other changes to curriculum or pedagogy are reported. They report key assignments and percent of students who: Exceed standards Meet standards Emerging competency Do not meet standards All candidates undergo a summative evaluation of their teaching performance near the end of the program. Table 5 summarizes student success, and there may be a somewhat smaller percentage of students passing this performance test in the second year than in the first of the two years reported. The grading rubric includes 35 areas of performance. Areas identified as student strengths are: professional/legal responsibilities, professional growth, social environments, and student engagement and making content accessible. Page 6 of 9 Areas identified as having lower rates of acceptable student performance than others are: monitoring student learning, instruction of English learners, instructional time, interpreting and using assessments, and learning about students. Student performance on the PACT (Performance Assessment for California Teaching) Teaching Event is also summarized in Table 5. Apparently the teaching event is designed to evaluate students rather than to train professors to assess. A little more context regarding what PACT is and how it is used would be very helpful to administrators and faculty outside the department in interpreting the report. In this evaluation a somewhat higher percentage passed in the second year over the first. The first year reported was the pilot year, and the improvement in the second year “suggests more appropriate alignment of coursework and assessment throughout the program”. Strengths identified by the PACT rubric were for planning and instruction (intellectual engagement). Areas in which students scored less well were: monitoring student learning while teaching, and assessment (analysis of student work and using assessment results to plan), and academic language (addressing language demands). The department reports “Analysis of candidate Teaching Event work samples and ongoing faculty discussion has led to the development of embedded signature assessments that will allow candidates to develop and fine-tune areas of both strength and need” without specifying what the changes are. A student survey is conducted and summarized by the Chancellor’s office. Students were most satisfied with subject matter instruction, especially for math, with English a close second and science and social studies not far behind. Areas of weakness, according to the students are: supporting English learners, health education, mainstreaming, using technology, and curriculum areas of dance, music, art and physical education. The CSU also administers an employer survey. Response rates are not reported here, but it should be noted that each candidate serves a partly paid teaching internship as part of the program. Employers are required to evaluate the student teachers, so response rates should be very high. The department summarizes the results as “on virtually every dimension assessed and for each composite group, our candidates were rated as less well prepared than the CSU system on average; and these ratings seem to have declined across the past six years.” This would seem to be cause for serious concern and a plan for remedying this should be discussed with Pam Stacks, AVP, Grad Studies & Research, well before the self-study is due to NCATE In combining the above reports the department reports a notable improvement in student knowledge about creating positive social environments over previous reviews due to adding a course on classroom management and positive learning environments. They also report quality field placements as a strength. Four areas for improving the program are identified: Addressing the needs of English Learners Monitoring student learning Interpreting and using assessment Learning about students The department has initiated faculty and supervisor development on the topic of English learners. The department has particularly embraced this topic for improvement. Over the past 18 months monthly field supervisor meetings have focused on features of English, language development and academic language, and analysis of instruction that supports academic language development. The PACT rubric for addressing English learners has provided concrete examples and a common language for dialog on this topic. They have developed a new assignment on this topic in one course. There is no single course largely responsible for teaching assessment, but the department plans to increase the opportunities for practice and feedback on assessment. They have a plan in place for reviewing other weaknesses at a later date. Realization that a subset of low scores were caused by supervisors misinterpreting the scale has led to increased training and calibration across supervisors. Page 7 of 9 The increasing use of Waypoint for assessment will increase the amount of data collected and ease analysis and reporting. They are working with the Special Education department to strengthen the mainstreaming course taken in their program. They are beginning to look at other CSUs for ways to strengthen the arts in the SJSU program. In response to the low ratings by employers, the department is planning focus groups with local principals and Human Resources directors. We encourage the department to make progress on collecting student performance data directly on specific student learning objectives, to make curricular and pedagogical changes directly linked to the student learning data, and to follow-up by collecting additional student performance data to measure the change in learning associated with the curricular or pedagogical changes. Special Education Summary of the Self-Study The department reports four credential programs with two levels of credentials for each. Credential, Deaf and Hard of Hearing Credential, Early Childhood Special Education Credential, Mild/Moderate Disabilities Credential, Moderate/Severe Disabilities Each credential offers two levels: Level I is a non-renewable 5 year credential. Level II is a full credential that requires an additional year of supervised teaching. The department also offers a Master of Arts in Education with a concentration in special education and a minor in Special Education for SJSU undergraduates. The catalog lists three minors. The department should resolve this discrepancy. The catalog also seems to have the certificate listed as a credential. Minor, Atypical Child Studies Minor, Deaf Education Minor, Special Education Certificate, Early Childhood Specialist We urge the department to clarify the programs it offers and make sure the catalog accurately reflects the programs. The number of students enrolled appears to have declined a bit, while FTES has remained steady. The Deaf and Hard of Hearing program is small and has had to cancel some classes due to low enrollment, but they want to start two new programs: A certificate of authorization for physical and health impairment and deaf-blind added to the Moderate/Severe credential (they should explain this in a way that someone outside the department can understand). Education Specialist Communication Development credential: They will implement a process to include Autism Spectrum Disorder preparation and authorization in the Education Specialist credential and design a certificate for Autism Spectrum Disorders expertise. The program is improving advising and faculty hope this will fix the class cancellation problem. The CCTC approved a plan to offer English Learner Authorization with each of the eight credentials. This presents the challenge of embedding new information in all (or most) classes. They report a plan to eliminate the part-time Records Specialist, who monitors each student’s progress toward graduation, and emphasize the importance of the role. The report reiterates a variety of data from OIR but does not interpret the meaning of this for the department. Page 8 of 9 From the report it appears the department may not understand that assessment is not the same as evaluation of students nor that course grades and student surveys are not sufficient for assessment. Summary from the Special Education assessment report: In the Mild to Moderate credential program standards 18 and 25 were singled-out for improvement. The report doesn’t specify what standard 18 is, but standard 25 is “Characteristics and Needs of Individuals with Mild to Moderate Disabilities.” This would appear to be the core of the program and should be addressed. In the Moderate to Severe Disabilities program the department identified communication and collaborative partnerships, and assessment and evaluation as areas of student performance that could improve. The department reports that students are struggling with the new English Learner standards. For the Early Childhood Credential program the department identified strengths but no areas for improvement. While only nine students are included in the data, a glance at the table of results suggests there is a problem with the assignments, the rubrics, or the standards, for results to be so similar across 10 standards. Perhaps they are using course or project grades rather than assessing the individual standards. The Deaf & Hard of Hearing program had even fewer students included in the data, but the report identifies standards 15 and 17 as possible areas for improvement: managing learning environments and Assessment, Curriculum, and Instruction. The program planning committee would like to suggest that the department should relegate most of the charts and tables to appendices and devote more of the report to interpreting the data and, especially, to describing a plan for improving student learning. In particular, a report organized either around programs or standards would be more useful to the department and more understandable by outside readers than a report organized around course assignments. The Program Planning Committee encourages the department to make progress on collecting student performance data directly on specific student learning objectives, to make curricular and pedagogical changes directly linked to the student learning data, and to follow-up by collecting additional student performance data to measure the change in learning associated with the curricular or pedagogical changes. Summary of External Review No external review was required of the departments for this first submission. Page 9 of 9