North Central Missouri College 2009-2010 Assessment Plan and Guidebook Including forms Highlights I-E-O Model and Related Assessments Tools Course Assessment Process Process for Academic and NonAcademic Program Review General Education Assessments 1 Table of Contents Mission Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Culture of Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Philosophy of Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Four Dimensions of Student Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment Team. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment Plan. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Workforce Readiness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Transfer Readiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Content Knowledge and Discipline Specific Knowledge & Skills. . . . . . . . . . . Soft Skills. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Testing Reference Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Course Assessment Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Classroom Assessment Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Student Opinion Questionnaires. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Academic and Non-Academic Program Review. . . . . . . . . . . . . . . . . . . . . . . . . . General Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduled Assessment Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reported Changes/Impacts of Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Assessment Activities Under the Workforce Investment Act . . . . . . . . . . . . . . . . . . . Works Cited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 3 3 4 5 5 6 6 7 8 8 9 10 10 10 10 11 12 12 13 14 14 14 15 15 2 The mission of NCMC is to assist individuals in our educational/business community to attain their goals through open admission, reasonable costs, progressive curriculum and services-- delivered by a caring, competent staff in a safe, technology-rich learning environment (For details see North Central Missouri College’s Strategic Plan). select "Strategic Plan" in THE COLLEGE column 08-11 Strategic Plan_highlights.docx INTRODUCTION North Central Missouri College strives to comply with Missouri Department of Higher Education (MDHE) expectations by adopting, where appropriate, its philosophy, purpose, and recommended assessment practices as outlined by the state while personalizing these to address the unique needs of the college’s students, faculty, staff, accreditation bodies, federal agencies, constituents, and other stakeholders. (For details see Missouri Department of Higher Education Website) CULTURE OF EVIDENCE (RATIONALE) The college’s “Culture of Evidence” (Dwyer et al, 2006) assures that graduates are equipped with the skills and knowledge needed to be productive members of the workforce and/or prepared for transfer to 4-year institutions. NCMC collects student learning assessment data using multiple modes of assessment including quantitative and qualitative, direct and indirect measures. Longitudinal reports provide valuable information used to make positive changes. The use of the Input-Environment-Output (I-E-O) Model makes the college’s assessment plan more cohesive by connecting reports about incoming and outgoing students, as well as the assessment processes in place that measure all student experiences while enrolled in the college. Summative information regarding student learning can best be explored across four dimensions: Workforce Readiness, Transfer Readiness, Content Knowledge and Discipline-Specific Knowledge & Skills, and Non-Cognitive or “Soft” Skills. PHILOSOPHY OF ASSESSMENT Assessment at North Central Missouri College should be guided by clearly stated, externally validated student learning processes and outcomes that flow from and support the institutional mission. Assessment, as the culture of evidence, should be fully integrated into the college’s entire operational system, both academic and non-academic. NCMC concurs with the purposes of assessment as outlined by Learning Assessment in Missouri Postsecondary Education (LAMP): 1. Improve student learning 2. Improve program of instruction 3. Improve educational effectiveness of the institution 3 4. Document student learning, program improvement, and educational effectiveness to outside stakeholders (accountability) Assessment is an opportunity for continuous improvement and accountability that occurs at different levels and in different contexts through a “multidimensional approach that can be used to assess student learning for the purposes of monitoring and improving institutional effectiveness in the teaching and learning domains” (Dwyer et al, 2006). The college shall assume responsibility for conducting assessment by gathering meaningful information, analyzing and interpreting the collected data, using the information to identify strengths and barriers, thereby making improvements. Assessment should be transformative in nature, viewed as a process not a product. It is the responsibility of NCMC to ensure that its graduates meet appropriate standards of performance; although in joint ventures, this shall be done cooperatively with other institutions. The college has an obligation to disseminate assessment results for accountability and improvement through appropriate channels in a professional and ethical manner. ASSESSMENT MODEL North Central Missouri College’s Assessment Plan is based on an I-E-O Model. Input (I) is what incoming students bring with them. Environment (E) is the measure of change including all student experiences. Output (O) is the measure of outgoing students’ skills and knowledge. The College Learning Environment Student Inputs Student Outputs Source: Austin (1993) According to the authors of A Culture of Evidence, inputs should be considered first. “There are two types of input measures: measures of simple quantity, such as the number of applicants or number of students admitted; and measures of quality or academic preparation (i.e. SAT/ACT, high school GPA, student’s advanced standing)… At the other end of the educational experience, there are two typical classes of output 4 measures for educational institutions. These can be broken into quantity and quality measures.” Between a student’s first class and graduation or transfer, data is gathered concerning the college learning environment. These records are used by the institution for the purpose of improving teaching and learning. This includes information regarding student engagement which “…is an index of the nature and extent of the student’s active participation in the learning process… an indicator of motivation and habits that carry over into other current and future settings” (Dwyer et al, 2006). FOUR DIMENSIONS OF STUDENT LEARNING MDHE has identified common themes that encompass workplace readiness and general education. These are reflected in three dimensions of student learning: Workforce Readiness, Transfer Readiness, Content Knowledge and Discipline-Specific Knowledge & Skills. Workforce readiness refers to a set of skills and abilities identified by academic and business leaders as important for success in any working environment. These include verbal reasoning, quantitative reasoning, critical thinking, and communication. Transfer Readiness refers to prerequisite skills and abilities students must demonstrate to successfully transition to a four-year institution. Content Knowledge and Discipline-Specific Knowledge and Skills are those that students “must acquire in order to be considered competent within that domain.” Data may be assimilated across discipline-specific course clusters and the resulting compilation used to obtain a deeper understanding of student learning. This may also include professional certification examinations or degrees awarded. A fourth dimension, soft skills (non-cognitive skills), is also important to student success and should be monitored. “In today’s knowledge economy, it is not sufficient for a worker to possess adequate basic cognitive skills and disciple-specific competencies. The nature of work requires that the person be able to work in teams, be a creative problem solver, and communicate with a diverse set of colleagues. The measurement of skills and traits such as creativity, teamwork and persistence has become a major focus” (Dwyer et al, 2006). ASSESSMENT TEAM NCMC has a designated team whose primary role is to assist personnel in the assessment process. The team is overseen by the Dean of Instruction and consists of an assessment consultant from each of the four divisions (one of whom serves as lead consultant), the testing coordinator, coordinator of federal programs, and an editor. This team is also responsible for orienting new faculty to the assessment process, maintaining records, leading the annual review of assessment practices, and providing leadership in planning assessment in-services for faculty. Job Descriptions for the Assessment Team.docx 5 ASSESSMENT PLAN This plan reflects North Central Missouri College’s overall belief system, its “culture of evidence.” Though the components of the I-E-O model may be addressed separately, it is important to note that each section is fully integrated with the other two. For example, results of input data inform and influence both the environment and output data. The same is true for environment, which is partially measured by output data and intimately controlled by the needs of incoming students. Similarly, output information can be compared to input data as a means of measuring institutional effectiveness, which in turn effects positive changes in the learning environment including the four dimensions of student learning. The evolution of this multi-dimensional approach is built on a faculty-driven foundation represented by its Assessment Team whose members represent all elements of the college community. Finally and most importantly, the needs of students are at the heart of assessment at North Central Missouri College. Its Mission Statement includes references to tailoring curriculum and services to help individuals meet the challenges of an ever-changing world (Objective 3), expanding alternative services for students (Strategy 1), strengthening learning initiatives across the curriculum in core areas (Strategy 2), and developing innovative programs that address economic needs (Strategy 3). INPUT NCMC collects the following data as it relates to incoming students: ACT Test scores—an annual, longitudinal report of attending NCMC students’ ACT score including composite scores, and sub-test scores (English, Math, Reading, and Science) is compiled. It also includes a comparison to national and state data. A detailed report is available with the Director of Testing. Placement into DS courses using ASSET & COMPASS—an annual report is compiled including the number of students tested and number/percent of students placed into developmental or college-level English, reading and math. The report includes annual data, including averages, dating back to 2001, but no formal longitudinal report is compiled. Program enrollment trends—an internal report is compiled each year indicating the number of student enrolled and the student’s declared majors. A three-year trend is reported annually to the Board of Trustees. End of Course (EOC) assessments of Course-Level Expectations (CLE’s)— accessibility is currently being explored Trend analysis based on demographic information available such as students origin, age, sex, cultural diversity, economic diversity, traditional vs. nontraditional (example-male nurse), ACT scores as a predictor of success Trend analysis in the areas of dual credit, outreach, in-district, and out-ofdistrict Inquiry cards—follow-up is provided to interested inquirers after college fairs and high school visits Trends from DESE report—Elementary and middle school data is used to predict future enrollment trends Economy and unemployment rates 6 State and Federal allocations available for student scholarships/workstudy ENVIRONMENT NCMC collects and utilizes the following formative data related to current student academic & non-academic practices and services: Course assessment (Access Database)—see later section General Education--three general education goals are being integrated and will be assessed across the curriculum: communications, higher order thinking (critical thinking), and managing information. The remaining five goals (mathematics, life & physical sciences, social & behavioral sciences, humanities & fine arts, and valuing) are assessed within primary and secondary courses and assessment data shared to improve curriculum and instruction. For additional information about state guidelines: o Curriculum Alignment Initiative and MDHE Benchmarking Missouri Department of Higher Education Website (MDHE) National Community College Benchmarking Project o NCCBP Home Academic Program Review (see the later section) Non-Academic Program Review (see the later section) Student Engagement o Community College Survey of Student Engagement (CCSSE) The CCSSE is designed to provide information on student engagement, learning, and retention for community and technical colleges nationwide. The results are useful in providing data about student experiences and to better understand how effectively to engage our students and identify areas for improvement. The CCSSE National Report provides benchmarks comparing NCMC students to other Missouri students. The benchmark areas include: Active and Collaborative Learning, Student Effort, Academic Challenge, StudentFaculty Interaction and Support for Learners. o Survey of Entering New Student Engagement (SENSE) The SENSE is designed to help understand entering students’ earliest experiences, behaviors, and perceptions from the first moments of their interactions with the college. SENSE results are meant to assist in gathering data needed to improve course completion rates, retain more entering students, and improve student persistence and success. The SENSE National Report compares NCMC student data to national percentages of other colleges administering the survey o Cooperative Institutional Research Program (CIRP) The CIRP is a survey given to incoming freshmen during orientation. Its focus is college readiness, student values/beliefs about diversity, and student expectations & civic engagement. o Your First College Year (YFCY) This is a survey designed as a followup survey to CIRP. YFCY results assess the academic and personal development of students over the first year of college, help colleges 7 identify items that encourage student learning, involvement, student satisfaction, retention, and success. Conducted March through June, this looks at freshmen at the end of their first year. OUTPUT NCMC collects the following data addressing the four dimensions of student learning as it relates to outgoing students and utilizes the summative information provided: How to access Annual Assessment Reports My Computer – J Drive Doc Sharing Assessment Folder Annual Testing- Student Surveys Marie Moulin Workforce Readiness o Direct NCLEX-RN and NCLEX-PN are recognized state board examinations for licensure developed by the National Council of State Board of Nursing NLN—pre-nursing and exit examinations are administered to entering and exiting PN and ADN students. The results are available from the Dean of Allied Health Sciences. CTE Exit tool—In compliance with DESE CTE will be adopting new state approved exit assessments. Awards--students are evaluated and awarded by industry leaders i.e. agriculture business writers, animal science and soils judge, pass entrepreneur competitions. Professional Certifications--Students who have completed the curriculum test or are assessed for certification in Human Resources Professional, Microsoft Office Professionals, Managing Personnel, Marketing Professionals, Certified Nursing Assistant (CNA), Licensed practical Nurse with Intravascular Certification, and Registered Nurse. WorkKeys-used for Career and Technical Education students when graduating with an A.A.S. or Certificate, excluding nursing students who use other exit tests. Pre & Post-test Annual Report since inception in fall 2002. Post-test Annual Performance Results by skill level o Indirect Graduation rate-Cohort information on first-time, full-time students graduating in 1.5 times the “normal” time is recorded and reported through IPEDs. The current initiative includes summaries by degree and programs. Graduate Follow-Up Survey (180-day) The Division of Career and Technical Education of the Missouri Department of Elementary and Secondary Education requires a 180-day follow-up survey to be conducted on all career program graduates each year. This is for recipients of both Associate of Applied Science Degrees and 8 Certificates of Specialization. The intent of this report is the presentation of employment and continuing educational status of these graduates GRADUATE FOLLOW-UP QUESTIONAIRRE for CTE.doc Employer Follow-Up Survey This survey is sent to the employers of last year’s graduates. The list of employers is obtained by the Graduate Follow up Survey. This survey provides information about how the employers would rate their NCMC employee in the education received, and overall preparation for employment. The survey also measures skills in communication, analytical thinking, technical abilities, and interpersonal skills. The employees’ soft skills are evaluated on work ethics, motivation/initiative, teamwork, and leadership abilities. Employer Follow-Up Questionnaire 2009.doc Transfer Readiness o Direct MAPP- Assessment used for General Education for AA, AAT, and AGS students. The annual report scores with explanation of what they mean with comparison to national data (prior to 2008, CAAP was used) CBASE-This assessment measures general education and is used for eligibility into teacher education programs. NCMC teacher education majors must take and pass the C-Base test (score of 235 on each section) before they will be awarded their diploma (AAT graduates). Students write an essay, take English, math, science, and social studies tests. The C-Base annual report and results are listed. A comparison table for the state of Missouri and other twoyear institutions is included. Skill scores are charted as high(H), medium(M), and low(L). Skill scores show the number of students testing into each area. o Indirect Graduation rate AA Transfer follow-up survey- will validate, through student opinion, the graduates’ general education knowledge was improved through their learning experience while attending NCMC. This survey is completed on the even years for all AA and AAT transfer students. GRADUATE FOLLOW-UP QUESTIONAIRRE for AA-AAT Transfer.doc 9 Content Knowledge and Discipline-Specific Knowledge & Skills o Direct Access Report from the course assessments in “discipline-specific course cluster” (i.e. Elementary Accounting 1 and 2, Cost Accounting, Payroll Accounting, Computer Accounting, etc.) o How many graduates per program? o How many of the graduates transferred? o How successful? o Indirect Communication between NCMC faculty/staff and faculty at the receiving institution Soft Skills (non-cognitive skills) o Indirect Employer Follow-up-following the Graduate Follow-Up Report for CTE graduates (formerly called the 180-day report), employers are surveyed. The survey questions relate to academic and skill abilities needed in the workplace, student success, and soft skills Note: May be considered as a future focus for the assessment team TESTING REFERENCE TABLE Assessment tools associated with degrees and certificates Testing Reference Table COURSE ASSESSMENT PROCESS Course assessment is the mechanism for providing instructors with data to guide and motivate students to be actively involved in their own learning, assessing that learning, and improving teaching methods. For faculty to actively guide what students learn and how well they learn it, course competencies are determined, articulated, and written down. Determining competencies is first and most important in course development. Once competencies are stated, curriculum, instruction, and assessment can be connected to them. Insuring that assessment techniques can measure stated competencies is vital. Valuable assessment of intricate student learning depends upon collecting various measures of learning from a multitude of sources (Baker & Holm, 2004). Therefore, appropriate assessment tools must also be chosen. A variety of these assessments can be incorporated into any type of class; this will improve both teaching and learning (Sanders, 2001). Decisions are still made about the curriculum, instructional methods, and assessment techniques employed based on established course competencies. In terms of 10 curriculum, decisions are made about which topics to cover and how they connect with previous and forthcoming topics. Instructional methods are chosen to deliver the curriculum, whether through various techniques such as lectures, group activities, readings, and homework assignments. Similarly, various assessment techniques are used. The three primary components of any course are the curriculum, the instructional methods used to deliver the curriculum, and the assessment techniques with which success in attaining course competencies is evaluated. These three components are linked and bound together by the competencies set for the course. Course Assessment is feedback for both students and instructors. Classroom Assessment Process: The Feedback Loop Plan Adapt Teach Respond Assess Analyze For the instructor, assessment can help answer the following questions: To what extent are students achieving the stated course competencies? Do minimum standards need realigning? Is course sequencing effective in promoting student learning? Are the prerequisites and entry levels set at an appropriate level? Can topics be introduced in a more effective way to enable student learning? What parts of this course are students finding most valuable? (year to year trends) How can this course be changed for next time? What grades do the students receive? For students, classroom assessment answers a different set of questions: Does the student know what is most important? Does the student know when they are mastering the course content? How can the student improve study techniques for this course? Does the student know what grade is being earned? Answering these questions and others can inform and improve the quality of student learning in the course. Introduction to Course Assessment 11 Introduction to Course Assessment.docx Course Assessment Form to be completed by the faculty member prior to input in the access database Course Assessment Checklist to be used during the follow-up meeting between faculty member and assessment consultant Course Assessment Check-revised_04-16-09.docx STUDENT OPINION QUESTIONNAIRES These questionnaires will be administered in two courses per semester during the entire period of pre-tenure review and in designated after earning tenure. The first semester of employment, the courses in which questionnaires will be distributed will be chosen by the Division Chairs and appropriate Dean. The questionnaires will be completed online using Course Eval. The Dean/Division Chair and the pre-tenured faculty member will meet to discuss concerns identified. student opinion student opinion questionnaire 12 07.doc questionnaire online 12 07.doc ACADEMIC AND NON-ACADEMIC PROGRAM REVIEW The process of assessment, program/degree reviews, non-academic program reviews, and the consideration of available resources should drive the strategic planning for learning to be optimized. Review of all programs and degrees effects positive change; this proactive thinking is critical to the improvement of quality. The Academic and Non-Academic Program Review has been designed as a component of this process, which encompasses the elements needed to provide the information necessary to promote learning and strategic planning. Program/Degree Review Schedule Program Evaluation Rotation Schedule f09.docx 12 This link may be found on the intranet in the Vice President of Instruction and Student Services section under Program Review Materials. Every program is reviewed on a five- year rotating schedule. Academic Program Review Academic Program-Degree Review Non-Academic Program Review Non-Academic_Progr am_Review GENERAL EDUCATION or Accountability for Achievement of Educational Goals. The eight statewide general education goals constitute the foundation for general education design and implementation. The 42-credit hour general education matrix provides basic information about how institutions fulfill their obligation to meet statewide expectations for accountability. Documentation should include relevant information about how assessments are administered, how student performance is measured (the criteria or rubrics used), how results are used for curriculum improvement, and how the assessment processes are reviewed and validated. Selected results are reported to the CBHE through the Performance Indicator Report, Mission Review, Program Review processes, North Central Association Higher Learning Commission expectations, the national report card, and other sources. There is no single instrument or method for universal measurement of general education student outcomes. Methods for measurement may include: o o o o o Documentation of students attaining or surpassing defined competencies Pre/post testing demonstrating gains in student learning Improvement trends over time Comparative data against appropriate peers Combinations of the above GenEd Assessment Comm Plan (2).docx Math GenEd Assess Plan Mar 09.docx Other plans will be inserted following the fall revision process Also needed, integrated cross discipline approach of 3 GenEd goals 13 SCHEDULED ASSESSMENT MEASURES To assure that assessment is ongoing, NCMC maintains the following schedule which includes the assessment tool, position responsible, and a link to the longitudinal report. Assessment_Annual_ Student_Learning_Assessment_Reports_7-09[1].doc The assessment Plan and Guidebook is reviewed annually by the assessment team for updates and revisions. REPORTED CHANGES/IMPACTS OF ASSESSMENT strategic_plan_nov_ 2006[1].ppt Changes from faculty.xlsx Changes from Changes from administration.xlsx student services.xlsx RESOURCES MDHE LINKS RELATED TO ASSESSMENT Missouri Assessment Consortium--Guiding Principles on Assessment http://www.dhe.mo.gov/macprinciples.shtml Curriculum Alignment Initiative http://www.dhe.mo.gov/casinitiative.shtml Imperative for Change-MDHE objective 1.C http://www.dhe.mo.gov/files/CoordinatedPlan.pdf Learning Assessment in Missouri Postsecondary Education (LAMP) http://www.dhe.mo.gov/lamp.shtml ADDITIONAL RESOURCES Culture of Evidence, http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/3c/f6/5c.pdf 14 ASSESSMENT ACTIVITIES UNDER THE WORKFORCE INVESTMENT ACT INITIAL ASSESSMENT A preliminary evaluation of the applicants’ skill levels, aptitudes, abilities and supportive service needs. COMPREHENSIVE ASSESSMENT Specialized evaluation of the skill levels and service needs of customers, which may include diagnostic testing, use of other assessment tools, in-depth interviewing, and evaluation to identify employment barriers and appropriate employment goals. FULL DEVELOPMENT OF AN INDIVIDUAL EMPLOYMENT PLAN Creation of a documented strategy, which uses information gathered through self-assessment, initial assessment and comprehensive assessment to identify the employment goals, appropriate achievement objectives, and appropriate combination of services for the customers to achieve employment goals. OBJECTIVE ASSESSMENT (typically reserved for job-seeking customers with limited employment/ educational history) an examination of the capabilities, needs, and vocational potential of a participant. Objective Assessment is used to develop a service strategy and education and employment goals. Such assessment is to be clientcentered and diagnostic, an evaluation of a participant’s educational and employment barriers taking into account the participant’s family situation, work history, education, occupational skills, interests, aptitudes, attitudes toward work, motivation, behavior patterns affecting employment potential, financial resources and needs, supportive service needs, and personal employment information as it relates to the local labor market. Works Cited Dwyer, Carol A. et al. “A Culture of Evidence.” Education Testing Service. 9 March 2009. <http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/ 0000019b/80/3c/f6/5c.pdf> MDHE. 2009. Missouri Department of Higher Education. 16 July 2009. <http://www.dhe.mo.gov/academicindex.shtml> 15