Monitoring Report to the Middle States Commission on Higher Education From QUEENSBOROUGH COMMUNITY COLLEGE Bayside, New York 11364 Diane B. Call Interim President Arthur Corradetti Accreditation Liaison Officer October 1, 2010 Subject of the Follow-up Report: To reaffirm accreditation and to request a monitoring report, due October 1, 2010, documenting (1) implementation of comprehensive, integrated, and sustained processes to assess institutional effectiveness and the achievement of institutional mission and goals (Standard 7); (2) implementation of comprehensive, integrated and sustained processes to assess the achievement of student learning outcomes at the course, program, and institutional levels (Standard 14); (3) evidence of adequate institutional support for and faculty leadership in the assessment of student learning (Standard 14); and (4) steps taken to promote a culture in which assessment is understood and valued and in which efforts to assess student learning are recognized and rewarded (Standard 14). To direct a prompt staff visit to discuss the Commission's expectations for reporting. The Periodic Review Report is due June 1, 2014. Date of the Evaluation Team’s Visit: March 15-18, 2009 Introduction Subsequent to the evaluation team visit in March 2009 and the Middle States Commission action in June three months later, Queensborough Community College has made a concerted and campus-wide effort to document, realign, and refine the implementation of comprehensive, integrated, and sustained assessment processes. In some cases, these efforts have served to make more explicit processes that have been in place for years but have been less public and transparent than they could have been. In other cases, new or improved assessment efforts have been adopted. The Middle States directives have had the salutary effect of reinvigorating the institution and galvanizing it into a strategic and systematic reappraisal and refinement of assessment processes and a reaffirmation of the integral place of assessment in the educational enterprise. Queensborough Community College, part of the City University of New York, is an urban campus of approximately 14,500 students characterized by extraordinary diversity. Covering 37 acres and celebrating its 50-year anniversary this past spring, the college has 12 transfer programs, including three dual/joint programs; 17 career programs; and nine certificate programs. A hallmark of the current institution is the Freshman Academy model. Organized according to curricular clusters, the six academies—Business, Education, Health-related Sciences, Liberal Arts, STEM (Science, Technology, Engineering, and Mathematics), and Visual and Performing Arts—offer all first-time, full-time students an educational experience that nurtures the growth of the individual student in a supportive environment through their first two semesters, the period studies identify as the time frame when students are most at risk. From the beginning, the Freshman Academies were conceptualized in terms of assessment. Developed by faculty and staff in concert with an outside consultant, DVP Praxis Inc., an Academy Assessment Protocol established benchmarks and student success indicators by which the academies would be assessed. Community colleges—and community college pedagogy—have finally become the focus of higher education that they warrant, and Queensborough has positioned itself to contribute to the literature of innovative and best practices in the field. Early on in the review of the Middle States Commission’s directives, it was determined that, rather than a culture of assessment or evidence, what is needed for implementing comprehensive, integrated, and sustained assessment processes is a culture of collaboration in which assessment is institutionally recognized as an integral component. To this end, Queensborough adopted a wide array of collaborative efforts, namely: 1) establishing two representative task forces—faculty and administrative—charged with developing a single, comprehensive assessment handbook for campus-wide use; 2) institutionalizing investment in assessment efforts by establishing an Academic Senate standing committee on assessment and institutional effectiveness; 3) aligning the criteria for faculty appointment and reappointment and for tenure and promotion with considerations of assessment efforts as college service; 4) refocusing the new faculty orientation to include an assessment component and building in a semester-long new faculty institute in which discussion of assessment efforts and development of learning outcomes rubrics were featured; and 5) appointing experienced faculty to lead assessment workshops for new faculty. The result has been clarification and refinement of past assessment processes; increased investment by faculty and administrators in the implementation of comprehensive, integrated, and sustained assessment processes and clear institutional support; and the emergence of a culture in which the collaboration required for assessment is better understood and valued and a recognized consideration in tenure and promotion. Since the self-study and evaluation team visit, Queensborough has undergone some institutional changes not specific to assessment but indicative of the broader framework of the institution and with corresponding impact on assessment efforts. The former president of the college, Eduardo Martí, has become vice chancellor of community colleges in CUNY. That the university chancellor tapped Dr. Martí in July 2010 to serve in this capacity is indicative of the new priority CUNY itself places on this important sector of higher education, as well as Queensborough’s progress during Dr. Martí’s tenure as president. The immediate past provost and senior vice president, Diane Call, has become interim president. As provost, Dr. Call was instrumental in establishing a much more enhanced new faculty orientation that emphasizes important college-wide collaborative efforts like assessment and is committed to a culture of collaboration in which assessment is one valued and recognized component. In addition to these changes at the highest level, other changes have occurred in the senior administration. The former dean for academic affairs, Karen Steele, is now interim vice president for academic affairs. Dr. Steele has worked for over two decades in this office and has had extensive experience in assessment and strategic planning. The division of finance and administration is now overseen by a vice president, Sherri Newcomb, formerly dean of this division. Business and financial services, led by Associate Dean William Faulkner, now reports to this vice president. An audit compliance officer has also been added to this division. As a result of these changes to finance and administration, the resource planning and allocation process is considerably strengthened, further refining what was already a very strong strategic planning process. Finally, the associate dean for accreditation, assessment, and institutional effectiveness, Arthur Corradetti, is now charged with facilitating the strategic planning process, previously led by the dean for academic affairs. This organizational change ensures a more seamless link between assessment and planning at the institutional level. The monitoring report, which was prepared by the associate dean for accreditation, assessment, and institutional effectiveness, has had input from the college’s executive team, consisting of the president, vice presidents, and deans and from the College Advisory Planning Committee, consisting of wide representation from the college’s leadership. In addition, an open hearing was held in September 2010 to elicit comment and suggestions from the entire campus community. In the report that follows, and in the supporting appendices, concrete steps toward implementing comprehensive, integrated, and sustained assessment processes are described and documented. 2 Progress to Date and Current Status Acknowledging the interrelationships among the four follow-up items identified by the Middle States Commission—and considering that the steps implemented by the college often address more than a single item—an integrated summary of progress to date has been provided. Following this integrated summary is a list of specific actions discussed in the summary that are relevant to each of the four items. In response to the evaluation team’s report and the Commission’s action, the institution has refocused its efforts to develop and implement comprehensive, integrated, and sustained assessment processes. These efforts may be summarized in five areas: 1. 2. 3. 4. 5. Shared governance (page 4) Collaboration on developing assessment guidelines (page 4) Assessment of the Freshman Academies through the Academy Assessment Protocol (page 6) Implementation and dissemination of college-wide assessment (page 7) Institutional support (page 14) Shared Governance As a result of the leadership of the Academic Senate, the official legislative body of the institution and whose 80 senators include 63 faculty members, a standing committee on assessment and institutional effectiveness has been established and charged. The associate dean for accreditation, assessment, and institutional effectiveness is the president’s designee to this committee. The charge to this committee (see Appendix 1) is to oversee campus-wide assessment efforts and processes and to report annually to the Academic Senate the degree to which the institution is closing the assessment loop—in other words, ensuring that assessment processes include collection and analysis of data, discussion, action plans, and follow-up. The first annual report to the Senate will be presented in spring 2011. That this committee was established by the Academic Senate itself is an institutional reaffirmation of the importance of assessment as an integral component of the college’s work. Collaboration An extensive effort was made to collaborate campus-wide to develop and promote a vision for developing, refining, and implementing assessment processes. On October 20, 2009, the associate dean for accreditation, assessment, and institutional effectiveness presented ―Assessment at Queensborough Community College: Institutional Effectiveness through Collaboration,‖ which kicked off the year’s work on developing, refining, and implementing improved assessment processes. This presentation called for the establishment of two task forces charged with developing an Assessment Handbook for use campus-wide. In addition to this presentation, the associate dean also made periodic presentations and reports on the status of assessment efforts to the Academic Senate, the president’s cabinet, the executive team, the academic chairs, and the administrative divisions. Through the work of two task forces, one composed of faculty and the other of administrative staff, an Assessment Handbook was discussed in concept and developed as a reference tool available to the campus as hard copy and as an electronic version on the college’s Web site (see Appendix 2). The Faculty 3 Assessment Task force consisted of one representative from each departmental personnel and budget committee—that is, senior faculty with formal governance roles in the department’s curriculum development, assessment activities, and personnel actions critical in contributing to assessment processes. The Administrative Assessment Task force consisted of representative administrative directors from all divisions of the institution. First convened in fall 2009 and completing their work in spring 2010, the task forces reviewed the wide variety of assessment resources that already existed and came to a consensus about what needed to be clarified, strengthened, or refined. The Assessment Handbook consists primarily of four sections and 18 appendices. The four sections include material on institutional effectiveness, academic assessment of student learning, administrative assessment of services in support of student learning, and institutional support of assessment. Of particular note is ―Institutional Effectiveness‖ (pp. 11-16), which presents a general description of the overall conceptual framework for assessment processes at the institution. The appendices (beginning on p. 29) are a compendium of assessment resources for the campus, including educational objectives, program review schedules, course assessment and program review templates, and rubrics, among other materials. Taken together, the sections and appendices above make the handbook both a theoretical and practical guide. The overall conceptual framework demonstrates the ways in which assessment processes at the institution are comprehensive, integrated, and sustained. It shows the relationship between the academic assessment of student learning represented by course assessment and academic program review (beginning on p. 17) and administrative assessment of services in support of student learning (beginning on p. 23). Both processes, through the year-end reporting process mentioned below, feed into the annual strategic planning completion report, which is submitted to CUNY each summer and which serves as the basis for application of the budget resource and allocation model. The conceptual framework in itself is not new to the institution, but its articulation in this form does give it a new clarity and transparency. A critical part of Queensborough’s assessment processes is the Year-end Report, both for the teaching and non-teaching departments (see Appendix 14 of the Assessment Handbook, beginning on p. 171). The nonteaching report was significantly modified by the work of the Administrative Assessment Task Force to strengthen and clarify the assessment and planning sections. The assessment section has been reorganized around core activities and key performance indicators established by the administrative directors themselves. Built in to the process is a discussion phase in which administrative directors must present to and receive feedback from the corresponding divisional head to ensure that efforts across the division are consistent and that patterns of concern can be discussed and resolved at the divisional level and, if necessary, brought to the campus-wide strategic planning level. In many ways, the Year-end Report is the linchpin of the campus-wide assessment processes because it is the critical point at which, whether by a teaching or non-teaching department, the year’s assessment work is summarized and an action plan is set out to which the subsequent year’s Year-end Report will have to respond, ensuring strategic and sustained assessment efforts year to year. The appendices of the Assessment Handbook provide evidence of comprehensive, integrated, and sustained assessment processes through a variety of reports and forms, including: Program Review Schedules – Appendix 3 (Assessment Handbook, p. 41) – As part of the work of the Faculty Assessment Task Force as it helped to prepare the Assessment Handbook, a new fiveyear program review schedule was established, one that also features the course assessment required by the academic departments to complete the program review. Though the new program 4 review schedule is consistent with the previous cycle of program review, the course assessment portion specifically in support of program review is new. Six-step Assessment Plan for Program Review – Appendix 9 (Assessment Handbook, p. 103)– and Course Assessment Form – Appendix 11 (Assessment Handbook, p. 123) – These are the standard assessment resources for academic departments, originally developed by a large-scale faculty committee in 2004-2005. The first (Appendix 9) is a thorough and comprehensive approach to course assessment in the service of program review. Both take their practitioner through an assessment process that accounts for course, curricular, and general education objectives. As a part of the Assessment Handbook, these tools should become an even more integral part of the faculty work on assessment in the academic departments. Year-end Report templates – Appendix 14 (Assessment Handbook, p. 171) – As indicated above, the teaching department and non-teaching department Year-end Reports link assessment with planning and provide an organizational structure for using assessment results to inform strategic planning at the department or office level. Academy Assessment Protocol As part of the development of the Freshman Academies, described in the Introduction, an Academy Assessment Protocol was developed. The protocol is intended to assess the effectiveness of the new Freshman Academies according to a variety of student success indicators. A principal investigator reporting to the Assessment Office was assigned in a full-time capacity to implement the protocol, to collaborate and provide guidance to faculty and administrators involved in the academies, and to report to the community. The protocol assesses the effectiveness of the Freshman Academies by evaluating three primary areas: the freshman coordinators, who provide special academic advisement to all first-time, full-time academy students; the high-impact instructional activities, including writing-intensive courses, learning communities, cornerstone experiences, e-portfolio, and service learning; and learning outcomes rubrics based on the general education objectives of the college. The protocol describes procedures including the administration of student and faculty surveys and collection and analysis of data for a variety of success measures like credit completion, retention, and course success rates. In addition to the surveys developed exclusively for the protocol, Queensborough also participated, in spring 2010, in the Community College Survey of Student Engagement (CCSSE), a national survey of students in over 700 community colleges. Using this variety of assessment tools, Queensborough is tracking success measures as it pursues the goal of doubling the graduation rate in three years as compared to the 2006 first-time, full-time cohort. To ensure that a culture of collaboration supports this effort, the principal investigator presented an interim report to the college community at a campus convocation in January 2010. The report cited preliminary results of the student and faculty surveys and of success measures. Students indicated that they were overwhelmingly satisfied with the freshman coordinators; faculty indicated that, for those who were involved in high impact instructional strategies, their engagement with the academies and with students was higher, suggesting that the more faculty are involved, the greater the reward. The results of the CCSSE—in which 73 classes at random, or 1,400 students, responded—indicated that, overall, Queensborough students in spring 2010 report learning activities that are increasingly rigorous and require active learning compared to spring 2007. In addition to survey results, average credit completion for the fall 2009 cohort was 7.2 in the fall 2009 semester, up 10.8 percent from the baseline cohort of fall 2006. A similar report will occur in fall 2010 to bring the campus up to date on the first-year results of the academies. Preliminary results indicate that the percentage of first-time, full-time fall 2009 students who persisted to spring 2010 was 88.6 percent, which is 6.4 percent higher than the baseline cohort of fall 2006. 5 As a further support of this effort, both the principal investigator and two part-time faculty learning outcomes facilitators reporting to the Assessment Office met bi-weekly throughout the 2009-2010 academic year with the faculty cohorts leading the academies to assist with the development and implementation of general education rubrics. Reports from the faculty cohorts were submitted to the principal investigator at the end of the fall and spring semesters, and the principal investigator provided feedback to the cohorts to assist with refining the rubrics and understanding the findings. As a result of these consultations, the quality of the submissions from spring 2010 improved over those from fall 2009: rubrics had more dimensions, fewer numerical transcription errors, and more complete data sets. The goal next year is to have better coverage of general education objectives 4-10 (see Appendix 3, Assessment Web site, under ―Educational Objectives‖). In 2010-2011, a full-time faculty fellow working with the Assessment Office will provide the faculty support previously provided by the two part-time faculty learning outcomes facilitators. Consultations with faculty developing and implementing rubrics by the faculty fellow and the principal investigator will continue in the 2010-2011 academic year. Implementation and Dissemination To ensure that assessment processes are transparent to the campus community, the Assessment Web site has been completely reorganized and enhanced (see Appendix 3). It includes the Assessment Handbook and resources for implementing assessment at the course and program level and in the administrative departments. It is also an archive of course assessment, program review, and administrative assessment in support of student learning. Course assessment is archived in the Assessment Database (on the Assessment Web site), which has been modified and enhanced from its original format. The database has two functions: It allows departmental faculty to post assessment reports; it is an archive of course assessment that faculty members can access to help with their own assessment, to rethink pedagogy, or to help inform program review. The Year-end Reports, submitted by all academic and administrative departments and containing sections on assessment and planning, are posted to the Assessment Web site. In addition, the schedule of program review, which includes course assessment in support of program review, is available on the Web site and is included in the handbook to ensure that departmental planning includes appropriate and timely preparation for course assessment and program review. The most noteworthy components of the Assessment Web site are those on academic program reviews, course assessment, and the year-end reports, which include reports from the teaching and the nonteaching departments. These sections both build on past practices and also make use of modifications from the work of the Assessment Task Forces. Academic Program Reviews The section on academic program review provides evidence of program reviews conducted in the 20092010 academic year. Three programs were reviewed: Day Care Assistant Liberal Arts and Sciences and Childhood Education Massage Therapy Because of continued lack of enrollment in the Day Care Assistant certificate program, the sponsoring department, Social Sciences, with the consent of the Office of Academic Affairs and the approval of the Academic Senate, discontinued this program. As a result, no formal report was prepared. 6 In the other two cases, the documentation includes (on the Assessment Web site, under Program Review, listed under each program): ―Priorities for the Future‖ – the concluding section of the full program review report, which summarizes the findings of the report and makes recommendations for improvement External reviewers’ report – the report from the site visit evaluators, featuring recommendations for the academic program and institution Action plan – the report from the sponsoring academic department(s) that includes recommendations for action presented to the Office of Academic Affairs Administrative Response – the official institutional response to the action plan Conclusions and recommendations from the two program review reports follow. Liberal Arts and Sciences and Childhood Education The Dual/Joint Program in Liberal Arts and Sciences and Childhood Education, which also comprises the Education Academy, is healthy: enrollments are up, recent retention is somewhat up, degrees awarded remain relatively stable, and student satisfaction indices are sound. The high quality of instruction and academic and other support services promotes student success. As the Education Academy matures and evolves, a more thorough and effective integration of academic and nonacademic service delivery will further enhance student success. Yet there remain areas of concern, most especially the declining percentage of three-year graduation rates among recent cohorts. It is believed that in the Education Academy, part of the college-wide Freshman Academy model, the program has the institutional vehicle by which some of the weaknesses cited can be corrected. Some recommendations cited in the program report for childhood education include, in four categories: 1. Student Learning Outcomes Improve methodologies for conducting course assessment as pertains to general education objectives. Increase the number of faculty participating in course assessment activities. Explore implementation of early interventions (e.g., peer mentoring) for high-risk lowerand upper-freshmen to improve retention. Initiate curriculum-specific assessment of effectiveness of academic and other support services for Childhood Education students. 2. Curriculum Development Explore options for capstone experiences (e.g., a Special Topics course). Explore more options for additional learning communities. Explore adding psychology course work to the curriculum. Increase collaboration with education department at Queens College. Explore articulations with other public and private colleges. 3. Faculty Recruitment and Development Recruit male education adjunct faculty. Encourage greater faculty participation in professional development activities and the use of innovative pedagogy. 7 4. Equipment and Facilities Provide more computer laboratories and Smart Rooms with fixed instructional technology. Provide more opportunities for students to work with instructional technology. Expand the Education Academy support staff services to follow the student through graduation. Massage Therapy Responding to the demands of the growing job market for massage therapy, the next five years will see the massage therapy program offering focused electives that will meet those demands more effectively than the extensive Eastern curriculum the electives are replacing. As recommended by the program’s Advisory Council, these electives will also offer more opportunities for graduates of the program and licensed massage therapists in the community to come to Queensborough for continuing education classes. These electives will need to be marketed and promoted. As indicated in the section on facilities, issues have been raised concerning the laboratory and clinic. Most of these have been resolved, and Buildings and Grounds is working to resolve the others. Some recommendations cited in the program report for massage therapy include: Coordinate with the Office of Career Services to develop more extensive career guidance to graduates of the Massage Therapy program. Develop with the Marketing Office a marketing plan to promote the electives that will offer more opportunities for graduates of the program and for licensed massage therapists in the community to come to Queensborough for continuing education classes. Revise the curriculum to offer a more medically-based (i.e., outcomes-based) advanced curriculum to strengthen student preparation for the licensure examination and for career paths. Establish a Massage Therapy program assessment committee to implement recommendations from the program review conducted during spring 2010. Conduct assessments of four Massage Therapy courses each academic year. Evaluate possible effectiveness of standardized examinations for cornerstone, milestone, and capstone courses. Course Assessment and Year-end Reports Two other important vehicles for the implementation and dissemination of college-wide assessment are the course assessment and year-end reporting sections of the Assessment Web site. The section on ―Course Assessment‖ has two components: the ―Assessment Database‖ and ―Reports.‖ (See Appendix 3 for the navigational structure pertinent to this part of the Assessment Web site.) Through the work of a special committee of the academic senate charged with reviewing and making recommendations for the modification and further development of a database in place for years, the ―Assessment Database‖ site now has two subareas, ―Post Reports‖ and ―View Database Resources.‖ Each academic department has designated an assessment coordinator to upload the course assessments that 8 have occurred during the academic year, indicating in each case what general education objectives are covered by the course. The ―Assessment Database‖ site, under ―Post Reports,‖ is the place where the assessment coordinators upload course assessment reports. The system has a vetting process, which requires special logins and passwords, both by the assessment coordinator and the department chair, before the reports are officially posted. When officially posted, the course assessments reports appear on the Web site under ―Course Assessment‖ and ―Reports,‖ accessible to all faculty and administrators through their Outlook sign-on and password and a resource for ongoing course assessment and program review efforts. The other subarea under ―Assessment Database,‖ ―View Database Resources,‖ has elements of the original database (which the special committee was charged to review) in the form of reports that display course and learning objectives organized in multiple ways to show relationships among course learning objectives, course learning activities, and general education objectives. The section of the Web site on the Year-End reports, organized by teaching and non-teaching departments, provides a link to the assessment and planning sections from each Year-end Report submitted for the academic year. This archive is meant to be a resource to faculty and administrators, both within departments and across departments, to make assessment and planning efforts more accessible and to encourage discussion and collaboration across campus. Year-end Reports – Teaching Departments Here are representative examples from the teaching departments’ Year-end Reports to demonstrate the ways in which course assessments in 2009-2010 have led to coordinated departmental action and planning for the 2010-2011 academic year. 1. Art and Design The Art and Design department assessed three art history courses—AR 310, AR 312, and AR 316—and one course in desktop publishing—AR 543. In the art history courses, student performance met or exceeded expectations. The only area of disappointment was in rote memorization of facts. It was agreed that this is a less important area than the ability to utilize nomenclature and critical concepts in the interpretation of artworks and in the recognition of art historical contexts. In the assessment of AR 543, Design for Desktop Publishing, most students successfully revised their designs and demonstrated their technical knowledge of InDesign and Illustrator. Based on observation, the revision process of student designs is essential in helping students to develop and achieve competency in the technical skills requisite for artistic self-expression. The most difficult aspect of the assignment for students is creating a dynamic relationship between text and image. In general, students who bring in samples and spend time outside class looking at and researching design score higher than those who do not. Adding a weekly assignment to bring in an interesting design (which the class can discuss) and creating a design bulletin board will be considered for the 2010-2011 academic year. 2. Basic Educational Skills The Basic Educational Skills department assessed BE 112, its upper-level remedial writing course. All sections used a paragraph-length writing sample that was holistically evaluated by a faculty team using a 6point rubric specifically designed for the assignment. Students were shown how to develop a counterargument body paragraph in a lesson that stressed analysis of existing models and drafting practice 9 paragraphs with the instructor’s input. There was also take-home practice, reviewed in class prior to the actual assessment. Students were expected to learn how to organize and support such a paragraph and to apply the semester’s work in control of grammar, punctuation, and diction in an attempt to combine skills honed all semester with a new and sophisticated rhetorical strategy. Results were mixed: 26 percent of students received clearly passing scores, and another 12 percent received scores on the cusp of passing. It is highly likely that this complex new skill requires more than a meeting-and-a-half of practice, given the low level of preparedness of students in this course. Students also may have not tried their best as they knew that the scores would be evaluated only in the aggregate. Further discussion of the results is scheduled for early fall 2010. In addition, as CUNY is instituting a new writing examination in the fall, one that will involve stronger reading comprehension, the syllabus for BE 112 will no doubt have to be reevaluated and modified, as will the skill-complex to assess. 3. Electrical and Computer Engineering Technology The Electrical and Computer Engineering Technology department has implemented a comprehensive assessment plan, which satisfies TAC of ABET standards, and has been assessing its courses and program outcomes for the past four years. The continuous improvement plan utilizes multiple direct and indirect measures of assessment. The assessment process was developed and implemented after extensive research into the study of assessment. Program outcomes for each of the TAC of ABET a-k objectives were developed by the department’s assessment committee and approved by the department faculty, industrial advisory committee, and Office of Academic Affairs. In its 2009-2010 assessment, which included results from advisory board recommendations, certification examinations, employer and alumni surveys, and student surveys, findings were as follows: a) increase student engagement in the laboratory, b) create a Web site with assessment information, c) increase A+ and Webmaster certifications, and d) improve job placement. The action plan includes: a) implementing more building projects into laboratory courses; b) current construction of Web site with assessment information; c) encouraging more students to take certification examinations and finding ways to reduce the cost for the examination; d) exploring ways to improve the student academic experience through the STEM academy, including more student participation in clubs, more outside speakers, improved and more personalized advisement, and more student/faculty get-togethers; and e) refining selected rubrics that assess program outcomes. 4. Foreign Languages and Literatures The Foreign Languages and Literatures department assessed all first-level language courses. This consisted of assessment of written, auditory, and oral skills using rubrics developed by the department’s assessment committee. Data indicate a uniform acquisition scenario among the foreign languages evaluated. All six language groups display similar percentages of students reaching or exceeding the proficiency levels for a Beginning I language course. Contrary to predictions, however, no language group reported more than 75 percent of their students meeting the proficiency standards of the course. In fact, an average of less than two-thirds of learners reached the desired proficiency levels. A closer look at the data reveals an interesting acquisition phenomenon. Language learners seem to encounter less difficulty in developing the required speaking and auditory skills than their reading or writing abilities. Despite their limited amount of instruction and exposure to the target language, a relatively high number of students (averaging 87 percent), are able to speak and converse at the expected level of a beginning course, regardless of the language. 10 The principal issue is mastery of grammar rules, both for the oral and written use of the target language. The groups that have shown a higher incidence of problems are those learning morphologically rich languages like Italian, French, Spanish, and German. At a beginning level, the acquisition of grammar is mostly morphologically based, and morphology is a linguistic component usually acquired very late. But the similarly lower number of students in Italian and Spanish reaching the desired level of lexical knowledge seems to highlight a general behavior of poor study skills and habits. In fact, language learners appear to fall short whenever the skill requires a more attentive participation and thorough analysis of the language. In response, the department will implement a greater variety of grammar activities and tasks in the beginning language courses and a closer monitoring of students' homework and laboratory assignments. 5. Mathematics and Computer Science Among other courses, the Mathematics and Computer Science department assessed MA 010, Elementary Algebra. Findings indicate that an entirely new pedagogical approach is necessary. The department’s assessment committee met and devised major pedagogical changes to be implemented for the fall 2010 semester. Three major changes include: more problem-solving time built into the curriculum, required student use of the Mathematics Learning Center, and online homework in 12 sections as a pilot. 6. Nursing The Nursing department assessed all courses. Faculty scholarship and evidence-based activities related to the program were conducted in the following areas: current trends in nursing practice, curriculum revision, teaching and learning strategies, testing, evaluation and the NCLEX blueprint. Curriculum revisions to date include reorganization of the program according to NLN educational competencies. Teaching strategies were incorporated including e-portfolio, informatics, simulation, and use of clickers. NLN testing solutions were incorporated throughout the program, and the NCLEX test blueprint was revised. Departmental examinations were re-categorized to align with the NCLEX Test Plan. As a result of advisory board recommendations, certification examinations, employer and alumni surveys, and student surveys, the department will devote more time to instruction in gerontology, cultural competencies, and technology and implementation of simulation. Curriculum revision is ongoing to incorporate cultural competence and gerontology content. Teaching/learning strategies to incorporate eportfolios, informatics, simulation, and clickers in the classroom is ongoing. 7. Student Affairs Student Affairs assessed ST 100, Introduction to College Life. Data was collected using the ST 100 Learning Assessment instrument administered in the final session of the course. Students were asked to respond to 11 questions based on the course learning objectives. Three items were selected for analysis. For objective #1, communicate effectively with peers and faculty, 98 percent of students were aware of faculty expectations. Their responses reflected knowledge of communication appropriate to different audiences. For objective # 2, demonstrate awareness of retention standards, 82 percent of students met the expectation. For objective #8, discuss mental health issues including action to take in emergencies, 91 percent of students met or exceeded the expectations. In response to the assessment of Objective #2, it was decided that more activities like calculation of GPA should be used to reinforce minimum retention standards. In response to the assessment of objective # 8, though the results showed that 91 percent met or exceeded the objective, faculty decided that more time should be spent addressing mental health issues. 11 Year-end Reports – Non-teaching Departments Here are representative examples from the non-teaching departments’ Year-end Reports, organized by division, of assessments conducted during the 2009-2010 academic year and corresponding action plans: 1. Academic Affairs Academic initiatives. Coordinated by an associate dean in academic affairs, these revolve primarily around the operations of the Freshman Academies. Queensborough was part of a national study by MDRC (a national organization that specializes in educational research) on learning communities. Learning communities are one of the ―high impact‖ instructional modalities faculty members have designated for the academies. MDRC’s study found that Queensborough’s learning communities had low instances of integration or integrative practices, which MDRC defines as curricular and pedagogical coordination between the courses in the learning community. The action plan developed in response to this finding includes naming a learning communities project director to work with faculty on integrative practices and encouraging greater demonstration of integration. Student Learning Center. The Student Learning Center has been collecting data in an effort to discover whether there was an optimal number of tutoring hours by discipline related to improved course grades. Data suggest that, in some courses, no significant difference occurs between students tutored and not tutored in terms of grade improvement. The plan is to work with department chairs and faculty in these specific courses to identify particularly problematic course areas or topics and to enhance tutor training related to tutoring in these areas. 2. Student Affairs In a departure from past practice, Student Affairs is requiring that its administrative directors’ year-end reports include a data-driven assessment of student learning. This informs both the larger effort of developing and implementing comprehensive, integrated, and sustained assessment processes and also the collaboration with Academic Affairs on the Freshman Academies, which is being assessed through the Academy Assessment Protocol. Academic Advisement. The Academic Advisement Center has invested time and funding into developing and implementing an individualized advisement model to assist students with academic choices. Over the past four years, the center has coordinated with the Office of Institutional Research to develop survey instruments to assess the quality of the center’s services. Results from fall 2009 indicate, for example, that 76.2 percent of students plan to graduate from Queensborough, that 90 percent find academic advisement to be necessary, and that 91 percent find it useful. The center will continue to work with Institutional Research to conduct surveys of students and assess quality of services in support of the improved retention and graduation rates sought by the institution. Career Services. Career Services will conduct pre- and post-surveys to all students receiving advisement and/or completing FOCUS 2, a career advisement tool, in one-to-one and group laboratory settings. Financial Services. Financial Services (or Financial Aid) has taken concrete steps to improve the quality of services to students, including the following actual outcomes: 1) the Call Center has been upgraded by adding more staff and by providing more training, 2) front-desk customer service has been improved by 12 adding counselors to the staff, and 3) lines and wait-time have been greatly reduced. The plan for the 20102011 academic year is to continue the approach from the previous year, which has proven to be successful, and to purchase software that will assist with managing incoming calls. Judicial Affairs. Powerpoint presentations are offered each year to educate the campus community on the academic integrity policy, the disciplinary process, and the Henderson rules. The plan for the 2010-2011 academic year is to develop a reliable and valid pre- and post-survey instrument to assess the effectiveness of the presentations and to administer the surveys. 3. Finance and Administration Business and Financial Services. As part of a more enhanced resource planning and allocation process, monthly reporting to senior executives and academic department heads has been established and will continue. Timely and useful management reporting has helped to prioritize the resources available in the most efficient manner. The plan for the 2010-2011 academic year is to reorganize staff to ensure more consistent accountability and reporting responsibility among funding sources. Writing training will be offered as required. 4. Institutional Advancement Sponsored Programs. Responsible for the preparation and administration of grants on campus, Sponsored Programs has set a target of developing one or more key performance indicators in the following core postaward tasks: hiring and reappointment of staff; processing of faculty release-time and summer salaries; and review and approval of budget modification requests. The assessment plan for 2010-2011 is to assess and project recovery account cash-flow to determine the potential for hiring a new staff member focused on post-award tasks, to establish appropriate key performance indicators and methods of tracking them, and to investigate automated platforms for integrating pre-award and post-award grants administration. Institutional Support To ensure that assessment processes are ongoing and sustained, the institution provides support in multiple ways: The Assessment Office, led by the associate dean for accreditation, assessment, and institutional effectiveness, maintains and updates the Assessment Web site; offers support and guidance to academic and administrative departments for course assessment, program review, and the Yearend Reports; and oversees the Office of Institutional Research. A faculty fellow, who reports to the vice president for academic affairs, provides full-time faculty support on the development and use of learning outcomes rubrics based on the general education objectives; this position is an expanded version of the two part-time learning outcomes faculty facilitator positions from the 2009-2010 academic year. Each academic department has designated an assessment coordinator whose responsibility is to upload course assessment reports each year to the Assessment Database. Institutional Research has hired a new research analyst to assist with data management and to provide more direct support to faculty and the administration with data requests for pedagogical and other research projects. The Office of Academic Affairs has reorganized and expanded its new faculty orientation program to a year-long series of activities that include information and focus on assessment efforts so that new faculty can understand better the culture of collaboration at Queensborough. 13 With the President’s Office and the Faculty Executive Committee, Academic Affairs has promoted convocations and conferences of the college in which assessment efforts and pedagogical research are valued and recognized; the January 2010 convocation, entitled ―What We Have Learned about Student Learning,‖ featured sessions on ―General Education Rubrics and Student Learning Outcomes,‖ ―Student Experiences,‖ and ―The Academy Assessment Protocol.‖ Academic departments have incorporated questions on assessment into the hiring and interviewing process and, as further evidence of a culture of collaboration in which assessment is considered a key component of college service, recognize faculty assessment efforts in considerations of tenure and promotion. The college has provided funding and support for faculty to attend TAC of ABET (Technology Accreditation Commission of the Accreditation Board of Engineering Technology) and NLNAC (National League for Nursing Accrediting Commission) accreditation preps, conferences, and workshops. External advisory boards include those for the Business Department; the sponsoring departments for the technology programs: Electrical and Computer Engineering Technology, Mechanical Engineering Technology and Design Drafting, and Physics; and the Nursing Department. Institutional Response to the Four Middle States Commission Action Items For ease of reference, the major institutional actions discussed above are organized below according to the four topics in the Commission’s June 2009 action letter (in bold), followed by commentary (in italics): 1) Implementation of comprehensive, integrated, and sustained processes to assess institutional effectiveness and the achievement of institutional mission and goals (Standard 7) Establishment of a Senate committee on assessment and institutional effectiveness Creation of the Faculty Assessment Task Force and the Administrative Assessment Task Force charged with developing an Assessment Handbook Creation of an Assessment Handbook Newly modified program review schedule, including course assessment in support of programs by departments not sponsoring program review (e.g., course assessment of an English composition course in support of an engineering science associate degree program); program reviews conducted in 2009-2010, with recommendations for improvement ( see pages 7-9 above) Modified year-end reporting process for administrative departments, including clearly stated core activities, key performance indicators, assessment and evaluation, and assessment plans (see pages 12-14 for representative examples of assessments and actions plans in response) Development and implementation of an Academy Assessment Protocol to assess the effectiveness of the Freshman Academies launched in fall 2009 Redesign and implementation of an Assessment Web Site Commentary: Though the Senate committee and handbook are entirely new, the handbook makes use of resources and assessment tools that were developed over the past ten years by faculty and administration and is, in many ways, a refinement and refocus of processes and efforts that have been occurring since the earlier self-study in 1999. The new Year-end Report maintains much of the form and substance of its previous version but has clarified the focus on assessment and planning. Administrative departments use assessment results to inform assessment plans, which are documented in the Year-end Report and discussed with the appropriate divisional head. The Academy Assessment Protocol was developed 14 concomitantly with the Freshman Academies and is an indication of the institution’s new focus on integrating planning for assessment with overall planning. Finally, the Assessment Web site is a reorganized and much more enhanced version of a Web site that has been in existence for years; part of the Web site, the Assessment Database, was modified in direct response to recommendations from an ad hoc committee of the academic senate that deliberated on requirements and parameters of the system.. 2) Implementation of comprehensive, integrated, and sustained processes to assess the achievement of student learning outcomes at the course, program, and institutional levels (Standard 14) Creation of Faculty Assessment Task Force to help develop an Assessment Handbook Creation of an Assessment Handbook, one section of which is directly related to general guidelines for the assessment of student learning Development and implementation of an Academy Assessment Protocol to assess the effectiveness of the Freshman Academies, including use of learning outcomes rubrics Launch of an Assessment Web Site to provide an archive for assessment and a campus resource Program reviews conducted in 2009-2010: two programs were reviewed—Liberal Arts and Sciences and Childhood Education and Massage Therapy—with recommendations for improvement concerning implementation of early interventions, curricular development, faculty development, and improving facilities; one other program was discontinued for lack of enrollment Courses assessed in 2009-2010: Art and Design: AR 122, AR 310, AR 312, AR 316; Basic Educational Skills: BE 112; Biological Sciences and Geology: BI 201, BI 202; Business: BU 108, BU 201, BU 203, BU 208, BU 401, BU 402, BU 403, BU 404, BU 405, BU 804, BU 805, BU 806, BU 807, BU 900, BU 901, BU 902, BU 914, BU 916; Chemistry: CH 120, CH 121; Electrical and Computer Engineering Technology: all courses; Foreign Languages and Literatures: LC 111, LF 111, LG 111, LH 111, LI 111, LS 111; Mathematics and Computer Science: MA 010, MA 013, MA 120, MA 440; Mechanical Engineering Technology and Design Drafting: MT 111; Music: MU 120; Nursing: NU 101, NU 102, NU 201, NU 202; Physics: all courses; Student Affairs: ST 100 Course assessment results: findings from course assessments have led to departmental and curricular changes, including pedagogical innovations and improved use of technologies Commentary: Course assessment for the academic departments, an ongoing process from past years but now posted on the Assessment Web site, has led to departmental and curricular changes (see pages 1012). Program review has resumed under the new schedule of program review; two programs were reviewed, and one program has been discontinued (see pages 7-9). Results of the program reviews are posted on the Assessment Web site. As part of the Academy Assessment Protocol, led by the principal investigator, faculty cohorts have designed general education rubrics and reported on student learning outcomes to the Assessment Office; the principal investigator has provided feedback on rubric development and use and some guidance on best practices. Faculty cohorts met regularly during the academic year. Faculty representatives from the cohorts have presented their experiences and findings to the campus community at convocations of the college. 3) Evidence of adequate institutional support for and faculty leadership in the assessment of student learning (Standard 14) Evidence of Institutional Support Establishment of Office of Accreditation, Assessment, and Institutional Effectiveness led by an associate dean 15 Development and implementation of an Academy Assessment Protocol to assess the effectiveness of the Freshman Academies Expansion of the new faculty orientation to a year-long program, with a component devoted to learning objectives and the assessment of student learning and ongoing workshops and meetings devoted to rubric development and implementation Funding to attend conferences and training workshops and sessions on assessment sponsored by the Middle States Commission on Higher Education and by ACBSP (Association of Collegiate Business Schools and Programs), TAC of ABET (Technology Accreditation Commission of the Accreditation Board of Engineering Technology), and NLNAC (National League for Nursing Accrediting Commission). Evidence of Faculty Leadership Designating a full-time faculty fellow in the Office of Academic Affairs (previously two part-time faculty learning outcomes facilitators) to support faculty in the design and implementation of rubrics in the Freshman Academies and to assist with assessment efforts Formation of a Senate Committee on Assessment and Institutional Effectiveness Commentary: To ensure institutional support of the assessment of student learning, the president and his cabinet established the Office of Accreditation, Assessment, and Institutional Effectiveness and hired an outside consultant to help to develop the Academy Assessment Protocol to assess the effectiveness of the Freshman Academies. The associate dean of accreditation, assessment, and institutional effectiveness chaired both task forces—the Faculty Assessment Task Force and the Administrative Assessment Task Force—charged with developing an Assessment Handbook. The president and cabinet also established the position of principal investigator to implement the Academy Assessment Protocol. The Office of Academic Affairs revamped the new faculty orientation to include early acculturation for new faculty to the culture of collaboration that makes assessment possible. To ensure adequate faculty leadership, Academic Affairs established learning outcomes facilitator positions, reporting to the Assessment Office, to assist colleagues on a part-time basis with the development and use of general education rubrics. This support has now become a full-time faculty fellow position. Evidence that the faculty group at large has taken ownership of the assessment of student learning is the establishment, by the Academic Senate itself, of a standing committee on assessment and institutional effectiveness. 4) Steps taken to promote a culture in which assessment is understood and valued and in which efforts to assess student learning are recognized and rewarded (Standard 14) Formation of a Senate Committee on Assessment and Institutional Effectiveness Launch of a modified and highly enhanced Assessment Web Site Establishing an assessment coordinator position in each academic department responsible for posting course assessments to the Assessment Database New full-time faculty fellow in the Office of Academic Affairs to provide support to assessment efforts and to the further development and use of rubrics for the 2010-2011 academic year (in 2009-2010, two part-time learning outcomes facilitators, release time assigned) Considering assessment efforts as college service in decisions about tenure and promotion Including assessment as a question in the interviewing process of faculty searches Including assessment in all New Faculty Orientation programs 16 Encouraging and funding faculty involvement in assessment conferences, workshops, and presentations at MSCHE, ACBSP, TAC of ABET, and NLNAC Incorporating assessment requirements in job postings for faculty recruitment Commentary: Perhaps the most significant indication that the college’s culture has changed is the establishment of a standing committee of the Academic Senate on assessment and institutional effectiveness. This was an initiative carried out by the faculty themselves and in direct response to the Middle States Commission’s June 2009 action. Academic departments have brought the issue of assessment into the recruitment and interviewing process so that prospective faculty members understand clearly the value of assessment. The administration has more fully embraced assessment by articulating that assessment efforts undertaken by faculty are valued and will be considered favorably as college service in deliberations about tenure and promotion. In a collaboration between administration and faculty, academic departments have assigned “assessment coordinators” the task of posting the course assessment reports to the Assessment Database, one resource on the Assessment Web site. Course assessment reports posted then become resources for other departments for their own course assessment or for program review. The array of actions described above may also be viewed arranged according to the recommendations of the evaluation team (see Appendix 4) and the action items in the institutional response to the March 2009 team report (see Appendix 5). 17 Appendices Here is an annotated list of the supporting documents to the monitoring report: Appendix 1: Resolution and charge to the Senate Committee on Assessment and Institutional Effectiveness. Approved as a standing committee in spring 2010, will be responsible for overseeing all assessment processes at the college and reporting annually to the Academic Senate; committee charge has been added to the college’s by-laws. Appendix 2: Assessment Handbook. Queensborough Community College’s official guide to assessment, received and approved by the institution’s Academic Senate in May 2010, available as a separate and self-contained document on CD; also available on the college’s Assessment Web site, under ―Assessment Handbook‖ (www.qcc.cuny.edu/assessment). Appendix 3: Assessment Web site (www.qcc.cuny.edu/assessment). Queensborough Community College’s Assessment Web site, a complete repository of resources, information, templates, and reports, including the college’s site for course assessment (under ―Course Assessment‖ and ―Reports‖) and program reviews (under ―Academic Program Reviews‖ and ―Reports.‖ Appendix 4: Recommendations of the Evaluation Team and Status. The comprehensive list, verbatim, from the evaluation team’s official report to the college and the Middle States Commission and the status of actions taken at the institution. Appendix 5: Institutional Response and Status. Queensborough Community College’s official response to the Commission to address the recommendations of the evaluation team and the status of actions taken at the institution. The full documents, or links to them, appear in the Appendices following the Conclusion of the report. 18 Conclusion Queensborough Community College is committed to comprehensive, integrated, and sustained assessment processes as one integral component of a culture of collaboration at the college. It has committed staff and resources to this effort, reorganized offices, created new positions, and established clear access to assessment information, documents, and guidelines. This commitment is one that is shared by faculty and administration, and the organizational structures that have been institutionalized help to ensure that this commitment is ongoing, systematic, and meaningful. In sum, here are the major actions taken by the institution: Academic Senate Committee on Assessment and Institutional Effectiveness – approved and charged in the spring 2010 semester. Assessment Web site – a completely modified and enhanced version of one previously on the college’s Web site; now includes enhanced assessment information, guidelines, documentation, and the Assessment Database on which course assessments are posted. Assessment Handbook – a collaboration between faculty and administration, the official guidebook to assessment at the college, available as hard copy and on the Assessment Web site. Course assessment – reports posted to the Assessment Web site available for all college employees as resources for program review and for informational purposes. Some examples of actions taken in response to course assessment include: the Elementary Algebra course will build more problem solving into the curriculum, require at-risk students to use the Mathematics Learning Center, and incorporate online homework in 12 sections as a pilot; Nursing faculty have infused gerontology throughout the curriculum, and simulation is incorporated in every course; the Introduction to College Life course will concentrate more on mental health issues. (See pages 1012 for a fuller discussion of course assessment findings and improvement strategies.) Program review – three programs were reviewed in 2009-2010, two to completion with reports, site visits, and administrative responses, and one of a program that has been discontinued; a new fiveyear program review schedule has been established and posted to the Assessment Web site; four degree programs are scheduled for review in 2010-2011. Actions taken in response to the two program reviews include: 1) enhanced articulation and collaboration between Queensborough and Queens College for the Childhood Education degree program, refocused and improved methodology for course assessment, committed effort to explore development of capstone experience, and enhanced use of instructional technology; and 2) modifications and improvements to facilities in the massage therapy clinic and establishment of a massage therapy assessment committee. (See pages 7-9 for a fuller discussion of program review findings and improvement strategies.) Assessment Office – an associate dean’s position established and a clear mandate to lead and provide institutional support to assessment efforts. Academy Assessment Protocol – a research design to assess the effectiveness of the Freshman Academies; a full-time principal investigator’s position was established to implement the protocol. Faculty Fellow – only part-time in 2009-2010 (two Learning Outcomes Faculty Facilitators), full-time for 2010-2011, this position, which supports the work of the Assessment Office, provides faculty support in the development and implementation of learning outcomes rubrics, departmental course assessment, and program review. 19 Appendix 1 Academic Senate: May 2010 Whereas, on March 9, 2010 the Senate adopted, among others, the following resolutions: Whereas on June 25th, 2009, the Middle States Commission on Higher Education acted on the recommendations made by the team that visited our College in March 2009 and requested a MONITORING REPORT by October 1, 2010, and Whereas this report must document: 1. Implementation of comprehensive integrated and sustained processes to assess institutional effectiveness and achievement of institutional mission 2. Implementation of comprehensive integrated and sustained processes to assess the achievement of student learning outcomes at the course, program and institutional levels 3. Evidence of adequate institutional support for and faculty leadership in the assessment of student learning; and 4. Steps taken to promote a culture in which assessment is understood and valued and in which efforts to assess student learning are recognized and rewarded. Whereas the College has been engaged in a College-wide discussion through two task forces, one representing all Academic Departments and the other representing Administrative units of the College, and Whereas for some departments, comprehensive and sustained assessment has been occurring, and it has been documented in the year-end reports and used in program reviews and outside accrediting reviews, in others there is a need to prioritize the documentation of assessment efforts; Whereas a comprehensive, integrated and sustained process for assessing student learning outcomes must systematically plan for and implement assessment of courses and academic programs offered by the college and approved by the Academic Senate, but neither mandates nor dictates how a faculty member engages in such assessment; Be it Therefore Resolved, that the Queensborough Community College Academic Senate adopts as College Policy that the College shall have comprehensive, integrated, and sustained processes to assess institutional effectiveness and achievement of institutional mission, and Be it Therefore Further Resolved, that the Queensborough Community College Academic Senate adopt as College Policy that there shall be mechanisms and processes to promote a culture in which assessment is understood and valued and in which efforts to assess student learning are recognized and rewarded, and Be it Further Resolved that the Committee on By-laws prepare a resolution to amend the Senate bylaws to establish a Standing Committee of the Academic Senate on Assessments whose purview will be the comprehensive overview of all assessment activities of the College, and to whom all efforts to assess student learning and institutional effectiveness and any other units or operations of the College would be reported along with the results of those assessments. Such a committee would make its annual report to the senate inclusive of such summary reports and the committee's own assessments of the assessment processes in place along with appropriate recommendations, and Be it finally Resolved, that the Academic Senate at the May 2010 meeting will consider approval of the Handbook on Assessment of Spring 2010 as providing a working guideline for the implementation of comprehensive, sustained 20 and integrated assessments of student learning outcomes, and of institutional effectiveness, and the achievement of institutional mission. To this resolution the following new resolution is offered: Whereas, there is now an Assessment Handbook of Spring 2010, offered for the consideration of the Academic Senate; Be it Resolved, that the Academic Senate receive the Assessment Handbook of Spring 2010, as providing working guidelines for the implementation of comprehensive, sustained, and integrated assessment of student learning outcomes and institutional effectiveness and the achievement of institutional mission, and Be it Resolved that the Academic Senate tenders its grateful and sincere thanks to the following members of the college faculty and staff who gave generously of their time and energy to serve on the Faculty and Administrative Assessment Task Forces: Wilma Anthony Manette Berlinger Mark Berman Belle Birchfield Bobbi Brauer Chiung Chang Arthur Corradetti, Chair Antonio Contant Michele Cuomo Sarah Danielsson Arthur Flug Benami Freier Paul Jean-Pierre Christina Johnson Dimitrios Kokkinos Maan Lin Anissa Mack Bryn Mader José Luis Madrigal, Senate Representative Macarthur Marshall Georgia McGill Devin McKay Bruce Naples Sherri Newcomb Sangeeta Noel Philip Pecorino, Senate Representative Constance Peluso Arthur Perkins Dion Pincus Faustino Quintanilla Linda Reesman Mary Rosa David Sarno Nicholas Simos Matthew Trachman Edward Volchok Clara Wajngurt Craig Weber Lana Zinger 21 By-laws Committee Resolution WHEREAS it is the recommendation of the Middle States Higher Education Commission that Queensborough Community College establish ―comprehensive, integrated, and sustained processes to assess institutional effectiveness and achievement of institutional mission,‖ and also ―assess the achievement of student learning outcomes at the course, program and institutional levels,‖ and WHEREAS ―institutional support for and faculty leadership in the assessment of student learning,‖ as well as the cultivation of a ―culture of assessment,‖ has become a college-wide priority, and WHEREAS it is critical to keep this process of review within the shared governance system of the college, BE IT RESOLVED that the Bylaws Committee approves the proposal from the Steering Committee for the creation of a Committee on Assessment and Institutional Effectiveness, and that the Bylaws of the Academic Senate of Queensborough Community College shall be amended as follows with an addition to Article VII, to be inserted as Section 11, between Section 10, ―Committee on Admissions,‖ and the current Section 11, ―Committee on Awards and Scholarships,‖ necessitating a renumbering of Sections 11-25 as Sections 12-26. [Addition to the By-laws] Section 11. Committee on Assessment and Institutional Effectiveness The Committee on Assessment and Institutional Effectiveness shall consist of seven (7) members of the faculty and eligible staff, reflecting, where possible, a balanced representation of faculty from varied programs and curricula, with no more than one representative from any given department; two (2) students; a designee of the President, and a designee of the Steering Committee. The Committee on Assessment and Institutional Effectiveness shall: a. Receive and review summary reports describing initiatives to assess student learning from academic departments, academies, and academic programs of the college; b. Receive and review documents relating to assessments of institutional effectiveness from all nonacademic units of the college; c. Make annual reports of progress in assessment of data collection, including: 1. The receipt of assessment reports from each department/unit of the college; 2. Courses/college units assessed from each department; 3. Summary of Assessment data gathered from assessments; 4. Any departmental conclusions drawn and/or actions taken as a result. d. Review assessment procedures the College undertakes and make recommendations concerning these assessment initiatives to the Academic Senate, in support of principles of shared governance, academic freedom and transparency. 22 Appendix 2 Assessment Handbook Table of Contents: Page I. II. III. IV. V. Acknowledgements Foreword Introduction A. Handbook Purposes B. Background and History of Assessment at Queensborough C. Current Status Institutional Effectiveness A. Institutional Context B. Strategic Planning C. Assessment Processes 1. Academic Assessment 2. Administrative Assessment 3. Institutional Change and Renewal Academic Assessment of Student Learning A. Overview B. Processes 1. Purpose 2. Departmental Level 3. Course Level 4. Program Level C. Reporting Documentation 1. Assessment Web Site 2. Assessment Database 3. Closing the Academic Assessment Loop Administrative Assessment of Services in Support of Student Learning A. Overview B. Processes 1. Departmental Level 2. Divisional Level C. Reporting Documentation 1. Year-end Report 2. Closing the Administrative Assessment Loop Institutional Support of Assessment A. Assessment Web Site B. Assessment Database C. Support Offices Appendices: Assessment Resources and Tools Glossary Diagrams A Visual Guide to Institutional Effectiveness A Visual Guide to Academic Assessment A Visual Guide to Administrative Assessment 1 5 7 7 8 9 11 11 11 13 13 14 15 17 17 19 19 19 20 20 21 21 22 22 23 23 25 25 25 26 26 26 27 27 27 28 29 211 12 18 24 23 Appendix 2, continued Assessment Handbook Appendices: Appendix 1 Appendix 2 Appendix 3 Appendix 4 Appendix 5 Appendix 6 Appendix 7 Appendix 8 Appendix 9 Appendix 10 Appendix 11 Appendix 12 Appendix 13 Appendix 14 Appendix 15 Appendix 16 Appendix 17 Appendix 18 Mission Educational Objectives (General Education), including Suggestions for Learning Outcomes, by Educational Objective Program Review Schedules, 2003-2010 and 2011-2016 Program Review Template Program Review Guidelines External Reviewer Report Template Program Review – Action Plan Template Curricular Objectives Business Education Liberal Arts Science, Technology, Engineering, Mathematics Visual and Performing Arts Six-step Assessment Plan for Program Review Course Syllabus and Outline Templates Course Assessment Form New Course Proposal Template Rubrics General Education Objectives Gen. Ed. Obj. 1 Gen. Ed. Obj. 2 Gen. Ed. Obj. 3 Gen. Ed. Obj. 5 Gen. Ed. Obj. 6 Gen. Ed. Obj. 7 Others Speaking Effective Writing Year-end Report –Templates Teaching Departments Non-teaching Departments Organizational Charts President’s Office Academic Affairs Student Affairs Finance and Administration Institutional Advancement List of Administrative Offices Senate Resolution on Institutional Assessment List of Useful Web Sites Cited Page 33 37 41 51 67 73 77 81 83 89 91 93 97 103 115 123 151 153 153 155 157 159 161 163 165 167 167 169 171 173 181 187 189 191 193 195 197 201 205 209 [Assessment Handbook available on CD (enclosed with the monitoring report); also available by going to www.qcc.cuny.edu/assessment/ and clicking on the ―Assessment Handbook‖ link.] 24 Appendix 3 Assessment Web Site Navigation: Assessment Home Academic Program Review Templates and Guidelines Schedule of Reviews Reports Academies Academy Assessment Protocol Academy Assessment Reports Accreditation ACBSP MSCHE NLNAC TAC of ABET Assessment Glossary Assessment Handbook Background Course Assessment Assessment Database Post Reports View Database Resources Reports Educational Objectives Fact Book Mission OIRA Self-study Senate Assessment Committee Visual Guide to Institutional Assessment What is Assessment? Year-end Reports Teaching Departments Non-teaching Departments www.qcc.cuny.edu/assessment/ When the Assessment Web site prompts for a logon and password, you may use the temporary credentials below: Logon: Password: mstates Msche2010 25 Appendix 4 Recommendations of the March 2009 Evaluation Team and Status Recommendation Status Create a conceptual framework for Implemented; through the collaboration of the Faculty and assessment describing the relationship Administrative Assessment Task Forces, an Assessment between assessments at all levels. Handbook was developed for use at all levels of assessment; this is available as hard copy in all academic and administrative departments and on the college’s Assessment Web site. Establish criteria for selection of courses to Completed; course assessment in support of program review is be assessed. These criteria may include now clearly articulated and disseminated through the new department specific criteria. schedule of program review. Establish and implement a schedule of Implemented; new schedule of program review was developed, course assessment. including course assessment in support of program review. Include in annual department reports, Implemented and ongoing; the Year-end Reports, both for convincing evidence of course embedded teaching and non-teaching departments, now have more clearly assessment. defined assessment and planning sections and a clearly articulated process, in the Assessment Handbook, for discussion, setting out action plans, and follow-up. Establish clear links between the goals Implemented; the new Assessment Handbook provides general selected for assessment at the course, guidelines and an overall conception; the college’s Assessment department, and program levels. Web site, under ―Course Assessment,‖ is an archive of course assessment reports and includes reports on course and general education objectives; the Academy Assessment Protocol involves general education assessment through rubrics that academy faculty have adopted and are using in the classroom, and reports have been collected and analyzed and discussed with faculty each semester. Utilize an appropriate departmental annual Implemented and ongoing; the Year-end Reports include a more report to report on follow-up assessment of clearly defined assessment section that includes follow-up to actions appearing in the assessment program review and course assessment. component of Program Review reports. 26 Recommendation Status Establish a clear link between mission and assessment of institutional effectiveness. Reporting on institutional effectiveness needs to done on a regular basis. Implemented and ongoing; the Year-end Reports for nonteaching departments, which have been heavily modified, now have clear articulation of departmental mission, core activities, and key performance indicators aligned with the college’s overall mission and goals; a standing committee of the Academic Senate on assessment and institutional effectiveness was charged in spring 2010 with overseeing all assessment efforts and will work in 2010-2011 to articulate the practical ways in which it will carry out its charge and report on institutional effectiveness. Create a central repository for assessment Implemented; the Assessment Web site, under ―Course data. Assessment,‖ has an Assessment Database site, where course assessment reports are uploaded and vetted, and a ―Reports‖ site, where all course assessment reports are posted; the Assessment Web site is also the repository for the sections on assessment and planning for all Year-end Reports, both teaching and non-teaching departments. Assure that the assessment process is Implemented and ongoing; the Assessment Office is charged monitored and reviewed for effectiveness. with providing institutional support for all assessment efforts; the new Senate committee on assessment and institutional effectiveness will monitor and review all assessment efforts on campus and will report annually to the Academic Senate on institutional effectiveness. Ensure that all departments, teaching and non-teaching, are participating in assessment of student learning, including assessment of distance learning. Implemented and ongoing; both the Assessment Handbook and the newly designed Year-end Reports clearly articulate that all departments are in support of the assessment of student learning; the Senate committee on assessment and institutional effectiveness will closely oversee all assessment efforts; all departments have conducted course assessment in 2009-2010, and the reports are available for viewing on the Assessment Web site, under ―Course Assessments‖ and ―Reports.‖ 27 Appendix 5 Institutional Response (April 2009) and Status Institutional Response Status The College will ask the College Advisory Planning Committee to reconstitute the Mission Task Force with the purpose of: 1) delineating the College goals embedded in the current Mission Statement and 2) including a component in the Mission or Goals relative to the efficient use of College resources Completed; college’s long-term goals form the organizational structure of the three-year strategic plan; all administrative departments that submit Year-end Reports have developed a mission statement and list of core activities and key performance indicators. The College will charge the College Advisory Implemented; each year’s strategic planning process Planning Committee with reporting to the begins with this review Academic Senate each September the results of the annual review of the Mission Statement. The College has already responded to the self- Underway through administrative committee study and evaluation team report in the 20092010 Strategic Plan by planning a thorough review of the website. The College administration will consider Completed; the associate dean for accreditation, reorganizing existing personnel and establishing a assessment, and institutional effectiveness has been centralized planning office to organize and charged with facilitating the strategic planning process. integrate a comprehensive planning mechanism that includes the Strategic Plan, Academic Plan, Enrollment Management Plan, and Facilities Master Plan. The College will reserve fiscal resources to Budgeted, with additional revenue to be raised to engage a consulting firm to conduct a implement comprehensive Facilities Master Plan within the next two years. The College has engaged Derek Price from DVP- Developed and implemented by a principal investigator Praxis to assist in the development of an assigned specifically to this task assessment protocol to measure key outcomes associated with the institutionalization of the Freshman Academies. 28 Institutional Response Status The College will reinstitute the five-year-cycle Implemented; also includes schedule timetable for the Program Review process. The assessment in support of program review process calls for an administrative review of the program review report with the president, the academic vice president, and the corresponding department chair or chairs to address administrative requirements and to determine whether significant curricular changes should be proposed to the Curriculum Committee and Academic Senate. The Program Review process was suspended last year to prepare for the selfstudy and Middle States visit. The College will address the recommendations on assessment by reorganizing existing staff and directing the current associate dean of accreditation, assessment, and institutional effectiveness, reporting directly to the academic vice president, to present an assessment plan that: 1) establishes criteria for the selection of courses to be assessed; 2) implements a schedule of course assessment; 3) redesigns annual departmental reports to contain convincing evidence of course assessment and clear links among goals selected for assessment at course, department, and program levels; 4) implements a continuous improvement plan to guide Program Review reporting and the presentation of findings to the president, the academic vice president, and the department chair or chairs (the results of which will be presented to the Academic Senate for information and/or the Academic Senate Steering Committee for actions resulting from significant changes); and 5) organizes a central repository of assessment data for use by teaching and non-teaching departments to discuss student outcomes and to strategize about improvement and action to be taken 1. 2. 3. 4. 5. of course Criteria for the selection of courses to be assessed – implemented; program review schedule, including course assessment in support of program review by supporting departments Schedule of course assessment – implemented; program review schedule – see above Annual departmental reports – implemented; revised year-end reports with much more focused assessment and planning components that include discussion and feedback Continuous improvement plan to guide Program Review reporting – implemented; program review reports, external reviewers’ reports to campus, departmental action plans, and administrative responses—all documentation published to Web site; ongoing assessment of changes and further reporting on the progress of the action plan in subsequent departmental Year-end Reports Central repository of assessment data – implemented; thoroughly modified and expanded Assessment Web site launched The College administration will coordinate Implemented; both associate dean for assessment and assessment and planning to ensure that the principal investigator of Academy Assessment Protocol assessment of institutional effectiveness informs now sit on CAPC the planning for institutional renewal. 29