Evaluation Plan ELFH 606 Rita Cron, Diana French, Amber Jaggers, and Julie Scoskie Introduction: General Description of the Project In Kentucky, state and community leaders were keenly aware something had to be done to increase retention and educational attainment across the board in order to remain competitive in the 21st century economy. The state called for a seamless, integrated system of adult education and services that would result in greater numbers of adults receiving a General Educational Development (GEDs), entering, and persisting in postsecondary education. There were several very compelling reasons for change, including Kentucky’s low national ranking in educational attainment, the high percentage of students entering community college needing remediation, poor retention of lower-level students, and limited resources. The K-12 adult education providers were in “competition” with the community college. The funding for adult education is provided via a competitive process and these respective institutions were frequently competing for the same grant funding. Leaders at Jefferson County Public Schools Adult Education and at Jefferson Community and Technical (JCTC) collaborated in developing a program to address the challenges faced by students. The mantra—Increasing Educational Attainment from GEDs to Ph.D.’s—was adopted by key stakeholders in the community. Joint marketing efforts included the slogan, “First your GED, then JCTC.” Educational Enrichment Services (EES – pronounced “ease”) is a Jefferson County Public Schools Transitions Program funded by Kentucky Adult Education in partnership with Jefferson Community and Technical College (JCTC). EES offers free courses in reading, writing, and math for those students entering college whose entrance exam scores fall below the minimum. The goal of EES is to improve retention and academic success for remedial college students. The program is designed to assist students whose college entrance exam scores fall below a designated level. The partnership has resulted in an annual savings to the students of approximately $400,000 in tuition and higher student retention rates at the college for students beginning their college experience as EES participants. Classes are located at two JCTC campuses in Louisville, Kentucky. In addition to a review of fundamentals, EES instructors embed college success navigation tools (study strategies, campus resources, financial aid, and email) in the curricula. Students who complete EES have demonstrated they are better prepared academically and personally for the transition into developmental or credit courses at JCTC. Students eligible for EES classes score below 28 on COMPASS math, below 21 on COMPASS writing, and below 51 on COMPASS reading. COMPASS is an untimed, computerized test that helps evaluate an individual’s skills and place them into appropriate courses. The hypothesis of this program was that retention of students enrolled in the EES program would be equal to JCTC’s one-year retention percentage. The program results exceeded the expectations stated. In order to analyze discrepancies, a concordance table was used to convert COMPASS to grade level scores. When a discrepancy was uncovered between the TABE and COMPASS scores of more than one grade level, an interview of the student was conducted by the teacher. If the teacher discovered that the student did not put forth their “best effort” on COMPASS, the student was referred back to JCTC for placement in a higher level course. Detailed description of the EES program A. Purpose of the evaluation The purpose of the evaluation was to determine if a partnership between JCPS Adult Education and JCTC improved retention of basic skills deficient incoming college freshmen. Retention would be tracked for course completion and retention at the college one year after EES enrollment. B. Audiences for the evaluation included: C. Administrative leaders at JCTC and JCPS Adult Education Instructors at JCTC and JCPS Adult Education. Policy-makers at the state level including Kentucky Adult Education, the Council on Postsecondary Education, and elected officials. The Workforce Investment Board whose mission includes increasing education attainment across the board from GED’s to Ph.D.’s. Generic description of program components It is a program partnership between JCTC and JCPS Adult Education providing a system of student identification, referral and instructional delivery for incoming JCTC freshmen in the areas of reading, writing, and math. D. Evaluation model that serves as a framework The management-oriented evaluation model designed by Daniel Stufflebeam guided the program evaluation. This model was selected due to the decisions that need to be made to impact long-term sustainable program improvements. The model’s core components are context, input, process, and product evaluation (CIPP) and applied to the EES program as follows: Contractual Agreements The purpose of the program evaluation determined if student retention improved as a result of EES program and if students made academic gains as measured by pre- and post- TABE scores. Identified procedures checked the accuracy of the scores by requiring both the COMPASS and the TABE test. If discrepancies emerged, trained staff evaluated the individual cases. Quantitative analyses assessed the program. A formal Memorandum of Mutual Understanding assured the necessary information needed to perform the evaluation would be available to the internal evaluators. The audience for the formative reports was instructional leaders and support personnel. Audiences for the summative reports at the conclusion of one year were instructional leaders and policy makers. Frequent informal meetings and formal scheduled monthly meetings with an agenda and meeting minutes kept communication open and fluid. Context Evaluation The goal was to evaluate student retention and academic gains. Trained data collection clerks at JCPS adult education and JCTC monitored and collected the demographic and assessment results. Modified PeopleSoft registration software at JCTC placed an identifier in the database for EES students to track students as long as they attended JCTC. Retention and test scores were reported to instructional leaders at the end of each semester. Input Evaluation The input evaluation would consists of: 1) the number of people referred to EES and their course completion 2) the difference in the preand post-test results 3) how many EES students were retained one year after enrolling in EES 4) JCTC data on entering freshmen retention not referred to EES one year after enrolling. It was determined there were not existing programs to serve as a model. Existing staff were utilized for the program implementation and evaluation. The evaluation report was shared with instructional staff and leaders at each respective institution for feedback and approval prior to releasing the results to the public. Process Evaluation The process of conducting the evaluation was agreed upon including how the attendance and scores would be monitored and recorded. There were two lead evaluators (JCPS Coordinator and JCTC Remedial Chairperson) responsible for collecting, organizing, and reporting the data. Progress was reported monthly to instructional leaders and adjustments were made based on the reports to guide continuous program improvement. Impact Evaluation The program evaluation had to measure the impact of EES on JCPS adult education, JCTC, and the students enrolled in the program. Input was solicited from community and business leaders to learn their perspectives on the program’s importance and the impact on our community. Effectiveness Evaluation The quality, cost-effectiveness, and significance of the program outcomes were documented. The EES program retention outperformed the college in a very costeffective manner. The funds used to provide the instruction were reallocated from the JCPS adult education budget. Retention outcomes of EES students were higher than average for adult education programs. The tuition saved by students needing remediation assisted with access and affordability issues faced by students. Similar programs were not available to use as a comparison. The unanticipated consequence was the retention results were stronger than the college’s retention rate even though EES participants were the lowestfunctioning, most at-risk students. Sustainability Evaluation Due to the reallocation of existing resources, the program’s prospects for long-term sustainability were good. The effectiveness of the program and the results indicated the students were the real beneficiaries of the program. The program assisted JCPS adult education in qualifying for incentive dollars which mitigated any cost of operating and expanding the program. The sustainability evaluation report calls for a follow-up study to assess the long-term implementation results. Transportability Evaluation The EES program is transportable due to the program relevance and need for to increase educational attainment. EES documentation and the alignment of institutional missions facilitate the feasibility of program replication. The program would require adaptation for smaller programs with too few students to from a separate class; however, program curricula and strategies are appropriate regardless of the size of the program Metaevaluation The EES program should be evaluated by an independent evaluator and the findings used to strengthen the program and evaluation. Final reports were prepared based on the intended audience: instructional leaders, elected officials, and business/community leaders. The program reports included details on how the program was planned, conducted, funded, staffed, and how it could be replicated. It included obstacles faced in aligning curricula and personnel issues and well as tables and graphs to illustrate comparisons in demographic and retention data between EES and JCTC. The Final Synthesis Report . The CIPP evaluation questions relative to the EES program evaluation are as follows: QUESTION 1) What needs to be done? 2) How should it be done? 3) Is it being done? 4) Did it succeed? AREAS OF CONCERN Improved freshman student retention at JCTC was the primary focus. JCTC’s retention rate at the end of the first semester was only 53%. Partner with adult education to combine resources to meet the educational needs of the lowest-functioning, most at-risk students entering JCTC. The course completion increased for the EES students from 53% to 70%. Student retention increased one year after EES enrollment from 45% to 56%. Leaders and staff at JCTC and JCPS Adult Education judged the extent to which the students served stayed in the program and made educational gains based on pre- and posttest scores on the national Test of Adult Basic Education (TABE). Enabling eighty-eight percent of students served to bypass one or more developmental education course saving those students both time and money. The management-oriented evaluation model was used to guide leaders to partnership stakeholders to data driven, informed decisions. PROCESS EVALUATION: Implementation Questions/Objectives/ HOW IS IT BEING DONE? A. Description of the project's implementation objectives 1. What you planned to do (services/interventions/training; duration and intensity of service/intervention/training) The EES Program serves adults functioning at or below the 8th grade-level equivalency by concurrently enrolling them in JCPS Adult Education and JCTC classes. This program serves as a model which could be replicated in other counties and states. JCPS Adult Education rebranded the name of the program from Adult Basic Education/GED to Educational Enrichment Services. The majority of the target audience holds a high school diploma; however, they function below the 8th grade level. The program mirrors the classes offered by JCTC and conducted twice a week, for 12 weeks, lasting for 75 minutes in both fall and spring semesters. EES requirements include: a course syllabus, required attendance, and homework. PeopleSoft registration enrolls students and it is entered into the Adult Education Aerin database. The major challenge faced was securing the necessary resources to adequately fund the creation and sustainability of the program. Money was needed to align curriculum, produce materials for students, equip seven college classrooms, purchase computers and software, and hire instructors to teach intensive courses to diverse learners. 2. Whom you planned to administer program (staffing arrangements and qualifications/characteristics of staff) JCTC and JCPS Adult Education jointly taught the first semester. All staff had a bachelor’s degree or higher. Some JCTC staff taught both for adult education and EES classes when possible. JCPS assigned a staff member to coordinate the program. Key leadership met at least monthly to informally evaluate the program. Staff at each respective institution aligned curricula to ensure a smooth, successful transition of students to enter the next level at JCTC. 3. Target population (intended characteristics and number of members of the target population to be reached by each service/intervention/training effort and how you planned to recruit participants) The target population for the EES Program was 800 incoming JCTC freshman who scored below a mandated score in reading and/or math on the COMPASS college entrance exam. Students are referred by JCTC based on COMPASS scores. The cut scores are non-negotiable and help JCTC staff remain objective and the referrals are easily understood. The referral cut score, determined by JCTC, and negotiated with JCPS Adult Education prior to each semester, is based on the changing needs of the students, partners, and policies. A. Statement of evaluation questions (Were program implementation objectives attained? If not, why not? What were the barriers/facilitators of attaining objectives?) How many students enroll in the EES classes? How many students complete the EES classes? What is the pre-and post-test gain? How many students enroll the next semester at JCTC? How many students persist and are enrolled one year after taking EES? EES objectives were attained. There were no barriers; however, the major challenge was funding. B. Description of data collection methods and data collected for each evaluation question 1. Description of sampling procedures Demographic information is collected from the student on the student including name, address, contact information, referral source, race, age, and educational attainment on hand-written enrollment forms. Every student enrolled in EES was included in the program evaluation. COMPASS college entrance scores are converted to grade-level equivalency scores as reported on the TABE C. Students are entered and enrolled in the JCTC and JCPS registration and data management systems. Students are given the TABE test and scores are analyzed and recorded. Students are post-tested at the end of every semester and information is entered in the data management system. Semester data including enrollment, course completion, and gains are collected by the EES coordinator and reported at the leadership meetings regularly and to other stakeholder group annually Description of data collected (instrumentation) and methodology of data collection COMPASS and TABE pre- and post- test data is collected using a standard adult education enrollment and separation forms. The testing procedures outlined in the testing manuals are followed to ensure valid and reliable scores. Staff members administering the test are trained by JCPS resource teachers in proper test administration procedures. Students receive two different assessments; COMPASS and TABE. The COMPASS is the college entrance exam. ACT has developed a concordance table that provides a TABE grade level based on the COMPASS score. The COMPASS score is the assessment that generates the referral to EES. Further, the TABE test is required by adult education and administered to the student the first day of EES classes. If there is a discrepancy between the test results, further discussion with the student provides referral guidance to the best class placement. The COMPASS (ACT, 1997) is an adaptive computer-based assessment comprised of reading, writing, mathematics, and English as a second language. The COMPASS is normed on the entire population taking this assessment. The COMPASS validity is based on the ability to place students in the most appropriate level of instruction and to assess the effectiveness of the remedial education for revised placement. Equivalent form reliability for the COMPASS ranges between .73-.90. The test/retest reliability data were not reported. ACT offers a predictive validity system to measure a student’s predicted probability of success in a standard course. Median correlations between COMPASS and course grades appear adequate. Standard error measurements for each individual’s test score provide colleges with confidence intervals for determining their localized cut scores for minimum student performance. The COMPASS score determines the referral to the EES program. Each EES student takes the Test of Adult Basic Education (TABE) before receiving instruction. Every EES student enrolled in the fall semester (263 students) were included in the EES program evaluation. The TABE design (CTB/McGraw-Hill, 2004) provides valid competency levels and measures a student’s progress in the areas of reading, mathematics computation, applied mathematics, and language. The test combines norm-referenced and criterion-referenced tests. Examinees are ranked relative to a norm group. In addition, criterion-referenced information provides instructional needs of examinees. The TABE is normed on a sample of examinees enrolled in Adult Basic Education programs. The TABE’s internal consistency reliability is rated as acceptable (.88-.95). The test manual does not include reliability on test/retest or inter-rater reliability. The EES program evaluation compared COMPASS scores with TABE scores using a crosswalk developed by ACT. JCPS and JCTC have a formal information sharing agreement outlined in the attached Memorandum of Collaboration (MOC). The MOC facilitates the access to data providing longitudinal information the student’s success in postsecondary education beyond EES. PRODUCT EVALUATION: This answers the question, “Did it succeed?” Participant Outcome Objectives: A. B. C. Description of participant outcome objectives: 1. What changes were participants expected to exhibit as a result of participation in each service/intervention/training component? 2. What changes were participants expected to exhibit as a result of participation in the project in general? Statement of evaluation questions, evaluation design, and method for assessing change for each question: 1. Did retention rate increase for freshmen? 2. Did intervention prove cost effective for students/JCTC? 3. Did quantitative approach yield valid and reliable results for decision to continue or discontinue intervention? 4. Did using COMPASS and TABE provide valid and reliable data for retention information? Discussion of data collection methods (for each question) 1. Description of sampling procedures: Incoming freshman students selected. 2. Description of data collected: COMPASS and TABE test scores collected from incoming freshman students. 3. Description of methodology of data collection: COMPASS and TABE embedded with student schedule. Student performance data collected using PeopleSoft. EES data collection and research was paramount. JCPSACE and JCTC worked together to help Kentucky Community and Technical College System (KCTCS) create a system in KCTCS’s database, PeopleSoft, to track EES students. This collaborative data collection had many benefits. For the student, the EES class followed their class schedule along with their other courses allowing it to blend with other coursework. Secondly, it allowed “flagging” EES students tracking progress as they moved through college. Flexible writing allowed any adult education provider in the state to use the system. It contained fields to store information about college scores and activity, adult education TABE, Official GED Practice Test, and GED scores as well. For data collection, performance measures were used which “assess an individual’s ability to perform on an achievement test, intelligence test, aptitude test, interest inventory, or personality assessment inventory” (Creswell 161). The COMPASS and TABE as described earlier determined program needs for each individual. The program methodology used a mixed methods approach by examining test scores upon entrance (quantitative), monthly meetings discussing program effectiveness, strengths, and weaknesses (quantitative and qualitative), student reflections upon EES exit (qualitative), and student tracking through program and college (quantitative). Program Evaluation Standards The standard was Met M The standard was Partially Met P The standard was Not Met N The standard was Not Applicable NA Utility Standards Standards Rating Comments U1 Stakeholder Identification M School and college leaders, instructional staff, students, policymakers U2 Evaluator Credibility M Internal evaluators, who also served as EES advocates, used formative evaluations. The evaluators were knowledgeable about the history and the structure of the EES program model. U3 Information and Scope M The purpose of the evaluation was to determine if a partnership between JCPS Adult Education and JCTC improved retention of basic skills deficient incoming college freshmen. U4 Values Identification M The value would be tracked and determined based on course completion and college retention. U5 Report Clarity M Oral feedback, technical reports documented student retention, Aerin database, PeopleSoft U6 Report Timeliness and Dissemination M At conclusion of fall and spring 2003 and monthly formative assessments guided decisions U7 Evaluation Report M Program was continued and expanded Feasibility Standards Standards Rating Comments F1 Practical Procedures M Economical and practical due to field setting and utilized pre-collected data F2 Political Viability M Contractual agreement between JCPS and JCTC, developmental education faculty F3 Cost Effectiveness M Conducted by internal evaluators and process was mitigated by increased performance resulting in additional funding Propriety Standards Standards Rating Comments P1 Service Orientation M Designed to meet the needs of struggling incoming college freshmen P2 Formal Agreements M Mutual understanding of collaboration (MUC) was signed by JCPS and JCTC P3 Rights of Human Subjects M Sharing and confidentiality to protect students’ rights to privacy was stated in the MUC; aggregate reports protected student identities in accordance with FERPA P4 Human Interactions M Evaluation activities did not interrupt student work and confidentiality was maintained P5 Complete and Fair Assessment M Aggregate reporting and MUC; strengths were retention, data-driven decisions, student reflections, and class completion; weaknesses were initial lack stakeholder accountability and initial lack of performance outcomes P6 Disclosure of Findings M Results given to all stakeholders except students P7 Conflict of Interest M Internal evaluations could hold bias P8 Fiscal Responsibility M Internal evaluators utilized reflecting sound accountability procedures which prudent and ethically responsible; staff time Utility Standards Standards Rating Comments U1 Stakeholder Identification M School and college leaders, instructional staff, students, policymakers U2 Evaluator Credibility M Internal evaluators, who also served as EES advocates, used formative evaluations. The evaluators were knowledgeable about the history and the structure of the EES program model. U3 Information and Scope M The purpose of the evaluation was to determine if a partnership between JCPS Adult Education and JCTC improved retention of basic skills deficient incoming college freshmen. U4 Values Identification M The value would be tracked and determined based on course completion and college retention. U5 Report Clarity M Oral feedback, technical reports documented student retention, Aerin database, PeopleSoft U6 Report Timeliness and Dissemination M At conclusion of fall and spring 2003 and monthly formative assessments guided decisions U7 Evaluation Report M Program was continued and expanded Accuracy Standards Standards Rating Comments A1 Program Documentation M EES offers free courses in reading, writing, and math for those students entering college whose entrance exam scores fall below the minimum. A2 Context Analysis M Remedial incoming freshmen need transition classes to function successfully in society. A3 Described Purposes and Procedures M The purpose of the evaluation was to determine if a partnership between JCPS Adult Education and JCTC improved retention of basic skills deficient incoming college freshmen monitored by management-oriented CIPP model. A4 Defensible Information Sources M TABE and COMPASS administered to every EES student which nationally normed and recognized. A5 Valid Information M TABE and COMPASS test scores compared using the ACT crosswalk. Any discrepancies were analyzed on individual bases by trained staff. A6 Reliable Information M TABE and COMPASS administration followed testing protocol. JCPS insured accuracy of data input. A7 Systemic Information M TABE and COMPASS ACT crosswalk insured accurate and reliable test results and minimized error. A8 Analysis of Quantitative Information M Enrollment data compared with course completion data then referenced to college completion. EES averaged 70% course completion rate and 56% college retention rate. A9 Analysis of Qualitative Information M Students with discrepancies of TABE and COMPASS results were interviewed to determine causative factors. A10 Justified Conclusions M EES matriculates more students into postsecondary education. EES averaged a 70% course completion rate (compared with JCTC’s rate of 53%). EES maintains an average of a 56% retention rate (compared with JCTC’s rate of 45%). Over half of the students are academically prepared to bypass one or more developmental courses saving both time and money. A11 Impartial Reporting M Internal evaluators had vested interest in reporting accurate data. A12 Metaevaluation M Program evaluation standards for Educational Programs by the Joint Committee Standards for Educational Evaluation were used to measure each component. Appendix A- Mutual Understanding of Collaboration Between Jefferson County Public Schools Adult and Continuing Education and the Jefferson Community and Technical College District This Memorandum of Understanding is entered into as of the 1st day of September, 2009 by and between Jefferson County Public Schools Adult and Continuing Education (JCPSAE), 3332 Newburg Road, Louisville, Kentucky 40218, and the Jefferson Community and Technical College District (JCTC), 109 East Broadway, Louisville, Kentucky 40202. The Jefferson County Public Schools are a public and political subdivision of the Commonwealth of Kentucky. The Jefferson Community and Technical College District is a constituent part of the Kentucky Community and Technical College System, 2760 Research Park Drive, Post Office Box 4092, Lexington, Kentucky 40512. Witnesseth: Whereas, JCPS Adult and Continuing Education and the Jefferson Community and Technical College District both have statutory missions which include the delivery of educational services to adults in Metro Louisville, and Whereas, both parties are committed to the enrollment and educational attainment goals adopted by the Workforce Investment Board (KentuckianaWorks), the Department for Adult Education and Literacy of the Workforce Development Cabinet, the Kentucky Community and Technical College System (KCTCS), and the Council for Post-Secondary Education; and Whereas, both parties, working in partnership, have complementary strengths and resources in adult and developmental education and are committed to maximizing the use of these strengths and resources for the good of the community; and Whereas, both parties are committed to working together to meet the workforce needs of current and future business and industry in the Metro Louisville area; and Whereas, both parties have already established a working relationship which includes cooperative efforts at Ahrens Adult Education Center, and JCTC locations; Now therefore, in consideration of the premises and the mutual covenants and conditions contained herein, the parties hereby enter into the following basic agreement: 1. To continue with established concurrent enrollment opportunities for JCTC and JCPSAE students. 2. To continue to transition GED and English Language Learning (ELL) students into post-secondary education. 3. To continue the comprehensive “interagency referral system” which includes but not be limited to an agreement on the referral and placement of incoming students who are assessed in reading, mathematics and writing using TABE and/or COMPASS. ELL students will be referred using the ESL COMPASS (See Appendix A for details.) 4. To continue with GED graduate recruitment initiatives that will include contacts with GED graduates, identification of scholarship funds for selected GED graduates and a presence at GED graduation ceremonies. 5. To review and evaluate the assessment, referral, and placement procedures utilized jointly by JCTC and JCPSAE. 6. To promote the seamless transition into college by holding EES classes on the college campus. 7. To promote effective placement into EES classes. JCTC staff will advise and schedule students into EES classes. JCPSAE will provide EES ESL orientations and an ESL staff person during the late registration week with hours not to exceed 19.75 per registration period. 8. To identify key research markers that will provide sufficient data to fully evaluate student and program success. Internal and external data will be used. 9. To utilize the results of the above-mentioned research to improve the policies, procedures and instructional methodologies of this joint venture. 10. To evaluate and strengthen the services provided by the GED Testing Center located at JCTC. 11. To continue with the joint steering committee and hold at least three meetings per academic semester to evaluate the status of the relationship and determine any necessary changes and improvements. The joint steering committee will be cochaired by the Director of the JCPSAE (or his/her designee) and the President and CEO of JCTC (or his/her designee).The Chancellor of the KCTCS (or his/her designee) will be an ex-officio member. In addition the committee will also include but not be limited to the following people: JCTC Provost, JCTC Academic Deans, JCTC Student Dean, JCTC Division Chair of Reading and Academic Success, JCTC Department Head of ESL, JCTC Department Head of Developmental Math, JCTC Department Head of Developmental English; JCTC Coordinator of Developmental Education, JCPSAE Specialist and JCPSAE Special Programs Coordinators for Adult Basic Education/ESL. 12. To continue to explore ways to generate monies for GED test fees and college tuition for GED graduates. 13. To develop joint marketing strategies that will advance and enhance this initiative. All marketing and promotion materials will be jointly approved. 14. JCTC will provide data to JCPSAE on students enrolled and attending JCTC with the following exceptions: Student’s under the age of 18 Student’s enrolled at a correctional facility Any student requesting privacy under FERPA. Specific data provided on each student will include but not be limited to: First, Middle, Last Name of Student Date of Birth Sex Race College Status (first time freshman, transfer student from 2 year or 4 year institution, reverse transfer from U of L Full-time/Part-time (number of credit hours/semester) Major GPA The College will work with JCPS staff to facilitate the coordination of data transfer between both entities. ` JCPSAE agrees to the following: A. JCPSAE shall not access or otherwise use JCTC student data except for the purposes directly connected with the assessment and operation of the EES Partnership and in accordance with JCTC statutes and regulations. B. JCPSAE shall maintain all JCTC student information in a confidential manner in accordance with all applicable JCTC regulations and state and federal law, including, but not limited to the privacy and security regulations pursuant to the Family Education Rights and Privacy Act (FERPA). C. JCPSAE shall from time to time share reports on the effectiveness of the EES Partnership with JCTC. 15. To share professional expertise by allowing both JCTC and JCPSAE faculty and staff to attend both JCTC and JCPSAE professional development opportunities. 16. This basic agreement is effective from September 1, 2009 until September 1, 2011, and is renewable on an annual basis thereafter. The basic agreement will be evaluated jointly as necessary. Therefore, any necessary changes and improvements in the basic agreement will also be agreed to by the steering committee. The details of the relationship, described in the Appendix A, may be modified at any time upon the authority of the co-chairs of the Joint Steering Committee as long as these modifications are consistent with the basic agreement described in this document. Witness the agreement of the parties hereto, as of the date first written above: Jefferson Community and Technical College Jefferson County Public Schools Adult and Continuing Education By: By: _________________ ____________________ Name: _________________ Name: ____________________ Title: _________________ Title: ____________________ _________________ ____________________ Date: __________________ Date: ____________________ Revised 6.18.10 References Creswell, J. (2008) Educational research: Planning, conducting, and evaluating quantitative and Qualitative research. Upper Saddle River, N. J. Pearson. Fitzpatrick, J. L., Sanders, J.R., Worthen, B.R. (2004) Program evaluation: Alternative approaches and Practical guidelines. Boston, MA. Pearson. Patterson, M.B., Zhang, J., Song, W., Guison-Dowdy, A. (2010) Crossing the bridge: GED credentials and postsecondary educational outcomes. Washington, DC. GED Testing Service. Sanders, J.R. (1994) The program evaluation standards 2nd edition: How to assess evaluations of Educational programs. Thousand Oaks, CA. Sage. Stufflebeam, D.L. (2010) Evaluation theory, models, and applications. San Francisco, CA. Jossey-Bass.