Riverside Community College District Instructional Program Reading Discipline Program Review June 2002 Draft # 2 Prepared by: Katie Smith Diane Deickmeyer Lori Ogata-Keeler Mitzi Sloniger 1 Executive Summary Topic I. Where are we now? The Reading Discipline at Riverside Community College provides a wide scope of instruction designed to meet the needs of a diverse student population. Remediation is a critical focus of the discipline as we strive to repair and restore what is deficient in the basic skills of our under-prepared students, such as understanding vocabulary in context, locating the main idea and supporting details, and utilizing other comprehension skills. Beyond remediation, however, we seek to enhance critical thinking skills such as analyzing an argument, detecting bias, drawing inferences and conclusions, and synthesizing information. Despite recent efforts at the K-12 level to offer programs and set standards to help the youth in our communities become better readers, reading continues to be a major area where high school graduates fail to do college level work. The community college must, then, continue to provide programs and services that are broad in scope yet able to address specific needs of individual students. RCC, like most community colleges, extends access to large numbers of minorities, the disadvantaged, and the nontraditional student. “These groups are often less academically prepared than their peers (Gardiner, 1994)” . As the student population continues to increase and become more diverse, the need for the college to provide access to reading courses and programs for basic skill development increases. Universities continue to defer to the community college system for basic skill development as well. The goal of the reading discipline is to assist RCC in the quest to meet its mission in the area of reading development. The Reading curriculum offers a sequence of three pre-collegiate classes that starts at an upper elementary level of reading instruction and extends to college-level reading instruction. Specialized classes, such as Rapid Reading, Reading for Academic Success, and a Reading Practicum are also offered. As stated in the Reading Discipline Handbook, reading “is not simply a basic skill that will allow students to succeed academically and professionally. Reading is a higher-level skill that will encourage students to think critically and thus prepare themselves to assume their roles as citizens.” Reading is a “skill that will not only inform and entertain but also enrich peoples lives.” In short, the discipline supports the academic goals of our students, as well as provides enrichment for their professional and personal lives. 1 Grimes, S. K., & David, K. C. (1999). Underprepared community college students: Implications…..differences. Community College Review, 27 (2), 73. 2 A. Purpose, goals and relation to institution: The discipline of Reading is directly related to the following District Mission, Goals, and Vision and Value statements: Preparation of students for “critical and independent thought and self-reliance.” (Mission Statement) Provision for “pre-college, tutorial, and supplemental instruction for underprepared students.” (Mission Statement) Improvement of “student retention and success…by establishing new programs and course sequences that lead students to opportunities for transfer education and career preparation.” (Goals) Utilization of “advances in information technology to improve effectiveness of instruction” (Goals) “Commitment to students…excellence in teaching…responsive to the communities we serve.” (Vision & Values) B. History: We are currently undertaking our first autonomous program review effort under the new discipline structure. The last program review (1999-2000) was undertaken with the English and Speech Communication Department (English, Reading, Speech Communication, and ESL). A draft was prepared; however, the process was not completed due to an inability to access and/or correctly interpret appropriate statistical data. No recommendations were made. Reading became a discipline in Fall 2000 as part of a district-wide restructuring effort. As part of the Program Review Pilot Project, the discipline has had the opportunity to self-assess and share dialogue with seven full-time faculty members’ input. This process has provided a venue for new faculty to learn historical information about the discipline and for members to address discipline issues as a strong, unified body. In response to the need for accurate data for self-assessment, we have asked for specific data that answers questions more pertinent to our discipline. Institutional Effectiveness has also been better able to supply appropriate statistical data in this review process than last. I) Issues relating to history: 1. Growth - The Reading discipline has increased the number of full-time faculty with the hiring of three full-time faculty members in Spring 2000. We currently have seven full-time tenure-track faculty positions and one full-time temporary position, which will become a full-time tenure track position in 2002-2003. There are four adjunct positions in the district. The increase in staffing was necessary to accommodate the increase in enrollment in the discipline. 3 2. Staffing - The number of full-time reading instructors in the district has allowed for placement of one full-time, tenure track reading instructor each at Moreno Valley and Norco Campuses, resulting in the subsequent increase in course offerings at the sites and spaces used for classrooms and laboratories. A paraprofessional divides a full-time position between the two campuses. Additional instructors and aides, full- and part-time, will be needed to meet future demands. 3. Institutional Issues - The Curriculum Committee has addressed the process by which the college measures student performance to meet the Reading graduation requirement. The committee reviewed whether reading courses should be used as prerequisites for English courses, or as a requirement for graduation. As a result, reading instructors are seeking to substantiate their discipline practices with appropriate assessment data. 4. Placement - Accuplacer, a computerized test, was selected as the college’s placement instrument for reading in the spring of 2000. Preliminary results from the validation research that is currently being conducted indicate that the new placement procedure is working very well. There is a high correlation (r= .69) between the recommended placement and instructors' rating of student preparation levels. Additionally, no disproportionate impacts in placement have been detected by ethnicity, gender, or disability status. Possible age differences in the effectiveness of placement scheme are still being investigated. The process of validating placement scores continues as well. 5. Strategies - In an effort to provide access to a larger segment of our student population, there has been an increase in the use of technology across all campuses. The most prominent examples of such use are the online Reading 83 and Reading 3 courses, in addition to the Hybrid Reading 82 course. Other examples of technology include the use of internet-based lab assignments geared for the interactive involvement of students in a classroom environment and the use of networked instructional programs. Reading students have access to Internet sites through the Writing and Reading Center Laboratories. An updated version of the Weaver Reading Efficiency Program (a skills based software tutorial program) was installed in the fall of 2000. A pilot program called Success was planned and implemented for one summer session in 1999 in an effort to meet the needs of students who were unable to directly enter a fouryear institution because of inadequate reading skills. The program was abandoned in deference to the onset of The First Year Experience Program the following year, which had similar objectives and was more comprehensive. The First year Experience Program does not, however, have a reading component. II.) Issues relating to goals: An examination of goals set last year (2000-2001) revealed that while they are still worthy goals, perhaps there is a need to prioritize them and set additional ones. 4 Goals for the Promotion of Student Learning: One identified goal was to develop paired courses for multiple-section content area courses by Spring 2001. The Reading discipline is continuing to consider paired courses for content areas. We realize that if we develop these paired courses, we may have the ability to draw comparisons helpful in answering the question of whether or not Reading classes should be required or used as a prerequisite for other courses. A second goal was to develop learning (reading) modules for single-section content area courses. Reading workshops on single skills have been offered to all students enrolled in the English, speech, ESL, and reading courses. The number of workshops and skills taught should be increased to include discipline specific content. In addition, students from other disciplines, not just the English and Speech Communication Department, should be made aware that the workshops are offered. Goals for the discipline’s contribution to the broader college community: A third goal was to develop faculty workshops on basic organizational strategies that could be implemented in any existing curriculum for faculty to help students understand how to read required textbooks – by Spring 2002. Tim Brown and Cornelia Wylldestar developed Faculty Development modules for 4faculty.org, which provided information on content area reading instruction. In addition, the Fund for Student Success Grant speaks to many of the instructional issues that concern adjunct faculty as well as fulltime faculty across content areas. Specific FLEX Day workshops, however, have not yet been offered due to time constraints and obligations of faculty members. A new goal identified this year was to design a strategy by which the discipline can assess our effectiveness and guide our decision-making. This would include getting feedback from Institutional Effectiveness, evaluating student performance in the classroom, and seeking other available resources. C. Programs & Curriculum: The discipline has courses that range from remedial to college level. The lower levels (Reading 81 and 82) have laboratory requirements for the practice of skills taught in classroom format. Courses in each subsection below are sequenced from high to low. Discipline Courses: Pre-Collegiate Core Sequence: Reading 83, Reading Level III, 3 units, non-degree Prerequisite: qualifying placement score or successful completion to Reading 82 Reading 82, Reading Level II, 3.5 units, non-degree Prerequisite: qualifying placement score or successful completion to Reading 81 5 Reading 81, Reading Level I, 3.5 units, non-degree Prerequisite: none Specialized Courses: Reading 3, Reading for Academic Success, 3 units, transferable Prerequisite: none, recommended for college level readers Reading 2, Rapid Reading, 2 units, transferable Prerequisite: none, recommended for college level readers Reading 66AB, Specialized Training for Reading Skills Development Aides, 2 units, non-degree Prerequisite: none. Designed for competent readers Reading 86, Corrective Spelling, 3 units, non-degree Prerequisite: none Reading 95, Specialized Topics in Reading, .5 units, non-degree Prerequisite: none. Designed for Workforce Preparation Reading Practicum Courses: Reading 88, Reading Laboratory, 1 unit, non-degree Prerequisite: none Reading 87, Reading Tutorial, .5 units, non-degree Prerequisite: none Students who currently enroll in a reading practicum register for English 96 (.5 units) or English 97 (1 unit). This has occurred as a result of combining the physical laboratories for English, reading and ESL student under one Writing and Reading Center director, who oversees practicum registrations. The discipline will need to discuss whether the course numbers, Reading 87- 88, should be retained. Curriculum Changes: Several curriculum changes have occurred during recent years. Course outline revision for Reading 2 was completed in November 2000. Course approval for Special Topics in Reading was received from the curriculum committee May 23, 2000. Reading 3: Reading for Academic Success and Reading 86:Corrective Spelling were approved by the curriculum committee as online courses March 13, 2001. The format for delivering Reading 83 and Reading 82 has been broadened to include the distance education format 6 and were first offered in this format during the academic year of 1999-2000. Reading 83 became one of the first reading courses statewide to be offered in an online format. Use of this format by the department has increased the number of course sections by six per year. All course outlines are attached in Appendix A. Students served: The discipline serves a broad cross section of students from the college and community. According to output data from MIS files (see Appendix B) the number of students enrolled in reading classes has increased steadily from 645 in 1998 to 1077 in 2001. Of those, 63.8 % are females; the majority are under age 19 years old (52.9), predominantly Hispanic (39 %), and predominantly English speakers (82.5%). Many of the students (38.9 %) want to earn an A.A. degree leading to a B.A. degree. Many students were identified as continuing (44.2 %) or first-time students (33.6 %). Pre- or co-requisites: Reading 82 requires a qualifying placement test score or successful completion of Reading 81. Reading 83 requires a qualifying test score or successful completion of Reading 82 or ESL 73. All other classes have no pre- or co-requisites. Identifying the successful completion of Reading 83 as one of multiple ways to demonstrate reading competency required for graduation was a major accomplishment. Students must now (as stated in the 2000-2001 College Catalog) demonstrate reading competency by obtaining: 1. a satisfactory score on RCC's placement test equivalent to placement in college level reading; OR 2. completion of Reading 83 with a "C" or higher; OR 3. satisfactory reading score on a standardized reading test approved by the English department. Students who do not meet Riverside Community College's reading competency requirement should enroll in a reading class within their first 18 units undertaken at the college. 4. Students who have completed an associate's or higher degree at an accredited institution are exempt from the reading competency requirement. Waivers may also be granted on a case-by-case basis. 7 Learning objectives and competencies: Learning objectives have been identified for all courses and are reflected in the course outlines, establishing clarity for expected competencies for classes that have pre- or corequisites and to delineating appropriate sequencing. D. Student Outcomes Assessment: The development of an effective assessment plan must be an immediate discipline goal. Assessing student outcomes will be part of that plan. The growth of the discipline and the district, plus the focus on student learning outcomes makes strategic assessment planning imperative. Means of assessing student success The Reading Discipline review team has divided assessment into three categories: (1) student success within a course, (2) student success in the Reading program (placement in, and completion of recommended Reading coursework) and (3) success in other disciplines. Student success within a course is currently being assessed by indirect measures: Passing grades (A, B, C or CR) percentage of students retained Passing grades matched with course repetition Student success within the Reading Program is currently being assessed by: Passing grades matched with recommended placement The program review team recognizes that the discipline is lacking in available information to adequately measure the success of the program, further indicating the need for a workable assessment plan. Standardized tests (Stanford Diagnostic Reading Test and Nelson-Denny Test) provide additional information on the skill development of students; however, they are currently used only to diagnose skill deficiencies and to plan remediation, not to assess program success. It would be beneficial to calculate the average growth of successful reading students in the district by percentile ranking (a national ranking) and to compare their growth, for example, with students who 1) were not assessed 2) did not follow assessment recommendations 3) followed assessment recommendations or 4) completed the course unsuccessfully. This would require all instructors to administer in-class assessment uniformly and to submit results to Institutional Effectiveness for calculations. 8 Faculty Involvement in Placement Assessment The Reading Discipline has worked to set new assessment standards for entry into the different course levels (Reading 81, 82, 83, and college level). All Discipline members participated in the validation of a new placement instrument (Accuplacer) by evaluating the placement of students in their courses. This information was compared to Accuplacer placement recommendations cut scores. Faculty members continue to participate in discussions regarding the Reading Graduation Requirement and ways to strengthen efforts to provide early intervention for students. In academic year 2000-2001, two faculty members studied the tutoring services of the Reading Center and designed new approaches for tutor/student accountability and tracking, as well as technological approaches. One approach was to survey students being tutored after each tutoring session. Another was to use computer software to obtain information about students being tutored. Current Status of Assessment Plan Discipline members are currently asking questions that will generate additional data that informs specifically about the effectiveness of our curriculum, our instruction, and student success. We recently asked for data that answered, for example, whether students who complete their recommended Reading coursework fare better in academic classes than students who place into but fail to take suggested coursework. E. Collaborations with other units: In an effort to collaborate with other units at RCCD and reach beyond the classroom walls, the discipline has articulated with the departments at each campus that include the discipline instructors, as well as with the Curriculum Committee, Matriculation, and the Academic Senate regarding the Reading 83 graduation requirement. We have offered reading-focused workshops to students through the Writing and Reading Center. We have offered reading and study skills workshops to faculty through Faculty and Staff Development. Instructors collaborated closely with Open Campus to establish online and hybrid course offerings. Reading instructors dialogued with the developers of the Middle College High School Program at Moreno Valley, which included reading courses in the program structure. We also continue discussions with administration, faculty, and students on the subject of remediation by serving on committees and task forces dealing with that issue. In addition, we established a CSU, UC transfer agreement for Reading 3. As we increase staffing at each campus, we envision being able to increase our outreach which currently includes the Middle College High School Program, the First Year Experience Program, the CAL-Works curriculum design, as well as the L. A. Times and Press Enterprise Newspapers in Education Programs. We will continue consultation with programs and services at other local institutions during the decision-making process for change in the district related to reading. 9 F. The discipline is constrained in our ability to offer courses and services to students by limited classroom space, laboratory space, personnel, technology and software. At the Riverside campus, classroom space has been maximized. Growth will require additional space. The space shared with the English and ESL disciplines is inadequate, limiting the reading student’s access to technology. A planned relocation to a remodeled Martin Luther King, Jr. Library does not relieve this problem. At the Moreno Valley campus, a computer lab equipped to service 30 students in a permanent space is needed, in addition to a full-time reading paraprofessional to staff the labs. Presently, Reading 81 and 82 classes rove among the computer labs, contingent upon space availability. At the Norco campus, an open Writing and Reading Center, which contains approximately 36 computers with Internet access and two pay-for-print printers, is available for shared usage to students in English, Reading, and ESL classes. In addition, a shared CIS/Math Lab classroom with approximately 36 computers with Internet access has been made available to reading classes at specifically designated class times. There are currently no classrooms specifically designated for the reading discipline only. The Weaver Reading Efficiency System is installed and accessible only in the Writing/Reading Center. The shared CIS/Math Lab is equipped with Altirus software, as well as an LCD projector and a non pay-for-print printer. Recently, additional reading skill software and audio text equipment was purchased for use with Reading 81 students. This affects student access to computer-aided instruction. For personnel assistance, a part-time reading paraprofessional is shared with the Moreno Valley Campus. There are no part-time instructors available during the fall, spring, or winter sessions, and only one part-time instructor is available during the summer session. G. Overall assessment of the discipline performance: In the course of our discussion, the discipline agreed that meaningful self-assessment could not be done without first agreeing on the critical areas of assessment that should be our focus. Also, we realized that we needed to determine some sort of point system or set of standards by which to measure. In order to accomplish this, we first examined the Mission Statements, Visions, and Values of RCC to determine which of these we felt were most critically linked to our discipline. We agreed on the following: Three Primary Areas of Focus Preparing students for success in college courses Pre-college remediation Community partnership 10 We then further discussed and agreed upon a five-point scale to reflect our selfassessment. That point system was as follows: Five-Point Scale 1= Commendable 2= Strong 3= Satisfactory 4= Inconsistent 5= Unsatisfactory The faculty discussed the activities, growth, accomplishments, and direction of the reading discipline, as we perceived them to be over the recent three-year period. We then voted, by a show of hands, on our opinion of the Discipline’s standing in each of the three areas noted above. The results were as follows: Results of Self-Assessment Student Preparation Pre-college Remediation Community Partnership =2 =2 =4 We agreed that a particular area of continued focus and determined improvement should be community partnership. Topic II. Where do we want to be? A. Findings - Environmental Scan The reading discipline desires to serve 100 % of the students whose placement scores indicate a need for remediation or developmental learning, as well as to provide resources to assist the entire student population in becoming proficient in college level reading. The latest assessment/placement cumulative course distribution survey provided by Institutional Research for the period from 7/1/01 – 5/18/02 shows that 15,342 students (excluding high schools) took the placement test. Of those, 19.9% scored at the college level, 17.5% scored at Reading 83, 19.8% at Reading 82, and 42.8% at Reading 81. We are clearly serving a population of under-prepared students at the college. The students with the lowest skill level make up the largest category. Because the challenge to succeed is so great and progress can be reflected in various ways, we wish to define success as rate of progress indicated by an increase in student achievement as demonstrated by course completion with a grade of C or better; increases in grade equivalency, percentile ranking, or stanine scoring on standardized tests; persistence in course enrollment; and successful completion of academic courses. 11 To further describe our discipline population, the analysis of Reading Students Characteristics data from Fall-1998 – Fall-2001 describes the typical reading student as 24 or younger (78.7%). A large percentage (63.6) is female. The group is largely Hispanic (39%), English speaking (82.35), desiring to transfer with an A. A. (38.9%) or undecided (30.8). Many are continuing students (44.2%) or first-time students (33.6%). A majority of students (65.8%) earn a C or better overall at RCC. See Appendix C. The Enrollment Simulation and Planning Summary section of the Environmental Scan document (McIntyre, p.5) depicts continued growth of the student who is served by our institution and the need to tailor services to student needs. To ensure that we are able to meet students’ needs in the classroom, we propose to remain vigilant of class size. National recommendations for federally funded reading programs (typically K-12) favor low teacher/students ratios for developmental or at risk students. The Discipline recommends maintaining a cap of 30 for its pre-collegiate courses. A fifteen to 30 range in the number of students provides enough students for collaborative interaction to occur as well as for instructor management. Management and learning are facilitated by adequate seating per class, which allows students to form groups, instructors to move about between rows and desks, and students who have disabilities or interpreters to access classrooms. Laboratory space should accommodate classes without infringement from other students. Space allocation should be adequate at all three campuses. The Discipline desires a seamless curriculum for development of skills. This can be in the format of classes, computer assisted instruction, labs, projects, or workshops. Students should be able to have their learning competencies identified and begin work at their level of need. They should have resources available to expediently move them from one level of achievement to another. Students need strong guidance to follow placement recommendations for enrollment. Research data from Institutional Effectiveness shows that when students (fall, 1998 cohort) followed placement recommendations, or registered for a course lower than was recommended, they succeeded at a higher rate than those who failed to follow the placement recommendation or who enrolled in a course higher than was recommended. Additionally, more than half of the students who enrolled in the recommended course level experienced success. The discipline must still question whether this percentage (58.9%) can be increased. A concern of the Reading Discipline is that there is no designated classroom space in the district for reading classes. All classroom space is shared with other disciplines or departments. Laboratory space in the district is also shared. This limits our ability to increase course offerings. Hybrid and Internet courses provide additional non-physical space; however, but as the student population grows, more physical space will be needed. An additional concern is the California State Universities’ (CSU) plan to reduce its remedial instruction to minimal levels by 2007 (McIntyre, p. 29.) RCC will most likely need to anticipate growth in the Reading Discipline as a result of this action. In the interim, RCC should collaborate with CSU to address remediation issues. 12 Finally, the community needs services provided by RCC to assist students who seek post high school education, remediation, workforce preparation, and life-long skills development. We must strengthen our resources and compel students to take reading classes early in their academic careers. B. Internal Review: In order to meet the need of preparing the students from the community with precollegiate remediation and preparation for the competitive nature of our society, we must make these determinations: Immediate Goals: 1. Work with Institutional Effectiveness to implement a process to use standardized test results and other qualitative measures to evaluate student success 2. Explore strategies for serving the expanding student population: faculty, space, and technology Faculty: How will we supply the staff needed to offer a full curriculum at the Moreno Valley and Norco campuses? Space: How will we secure space to accommodate enrollment increases? Technology: What is needed to support curriculum offerings? 3. Strengthen the reading graduation requirement to ensure that identified students select reading courses early in their academic planning 4. Determine if we should validate Reading 83 as a prerequisite to English 50A to compel students to take reading courses early 5. Seek opportunities to partner with the community and other units of the college to support student needs Long Range Goals: 1. Format our curriculum to increase service for a growing population of students 2. Determine the specific data we need to adequately assess the discipline 13 C. Revised Vision – restatement of what the discipline desires to be We view reading as a basic skill that allows students to succeed academically, professionally, and socially. Decision-making and critical thinking are essential to student success in the communities in which they live. Through our curriculum and services, we provide a foundation upon which students can build varying degrees of competencies for living, a fundamental right of all citizens. As a result of quality instruction and strategic interactions, we should positively impact the ability of students, particularly those who are at risk, to gain knowledge from print media and to process ideas logically. The use of these competencies can improve various aspects of their lives. D. Gaps between the current and desired state The gap between the current state of the Discipline and its vision is primarily one of impact and reliability. The Discipline needs to systematically assess its impact on student learning and preparation for independent living. The Discipline now has the problem of not being able to effectively articulate to what degree we are impacting the development of students whose needs are the greatest. When we design an assessment plan that gives feedback on student learning outcomes, persistence in courses, and retention at the college for students who take reading courses, we will be better able to determine our needs relating to the growth of the Discipline. Topic III. What we need to do to get there: The discipline must meet the following needs in order to be effective: Identify an assessment strategy that provides district-wide information. Clearly, the Reading discipline needs to continue its pursuit of hard data that examines our inherent belief that early onset reading instruction for remedial students is absolutely necessary in order to truly fulfill our commitment to give students a meaningful higher education experience. Utilize assessment results (SDRT/Nelson Denny) from classes in addition to institutional research data to indicate/support student progress and to make decisions relating to curriculum, instruction, growth, policies and procedures. Increase course offerings at Moreno Valley and Norco to achieve a full range of courses, and to increase college-specific assessment data. Increase faculty at Moreno Valley and Norco to accommodate increased/expanded curriculum offerings. Secure more space (Riverside); better utilize space (Moreno Valley, Norco). Encourage students to take a reading practicum. 14 Increase access to technology. Resources needed to support initiatives, strategies, goals, and activities: Seek ways to continue to meet with students and increase student service with little increase in designated physical space. Increase curriculum offerings to include more hybrid courses, computer-aided instruction, online courses and web-enhanced instruction at the Riverside, Moreno Valley and Norco Campuses. Increase faculty by the equivalent of two full-time positions (one at Moreno Valley and one at Norco). Gain support from Institutional Effectiveness to design and implement the assessment plan and to gather and analyze data needed for ongoing evaluation. Adhere to instructional plans and departmental procedures. Acquire appropriate technology (hardware and software). Topic IV. Evidence needed to track our progress: Documents/evidence needed to demonstrate success (to be further outlined in an assessment plan) Documents/evidence of student placement scores Documents/evidence of student scores on standardized tests (Nelson Denny or SDRT). Evidence of links between success in program and retention in the district. Information regarding reasons students discontinue enrollment in classes or leave the college to clarify some loss of student enrollment. Utilization of a system to track student progress through and beyond the college district. Information to help us understand student objectives. Assessment Plan Status The Reading Discipline is currently asking more meaningful questions as to how to accurately assess the effectiveness of its curriculum in order to better meet the needs of 15 students and the community. Plans are being made to consult with Institutional Effectiveness to outline procedures for continual analysis of data. A rubric for self-evaluation is now established as a result of the cooperative efforts of all the Reading Discipline members. There remains a need for statistical data regarding the effects of reading instruction as it pertains to student success in content area classes. Meaningful ways for utilizing results from SDRT and Nelson-Denny tests are being considered. Topic V. Improvement of the discipline self-study process: The discipline Self–Study process has been an effective tool for providing input from discipline faculty members. The Program Review Resource Guide, when finalized, should provide clarity of the process by, specifically, outlining target dates, resources, and explanations of each section of the guide. Disciplines should be given suggestions on how to develop self-evaluation rubrics and a routine timeline for regular self-study. Discipline facilitators should take primary responsibility for the on-going collection of data and should assume leadership in the self-study process. 16