Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment OIRA: OFFICE OF INSTITUTIONAL RESEARCH AND ASSESSMENT Mission: College strategic objectives: 1. Maintain rigorous and current curriculum and support new program development and experimentation. 2. Fully implement, sustain & assess the 3 components of the Queensborough Academies: advisement, high impact practices, and the use of technology. Division strategic objectives: 1. Ongoing assessment of academic offerings and support services. 2. Comprehensive reporting on the Queensborough Academies through the Queensborough Academies Assessment Protocol. Note: Relevant college-wide strategic objectives should be selected from the college’s strategic plan. Each division strategic objective should align with a college-wide strategic objective. 2014-15 Completion report (outcomes for this past academic year, based on last year’s year-end report, section F) Key performance indicators Ensure that the effects and outcomes of the various components of the Queensborough Academies are monitored and assessed. Target outcomes Freshmen Experience Survey High Impact Practices (HIP) Assessment General Education Critical Courses Analysis (Roadblock Analysis) Actual outcomes All target outcomes were achieved with the exception of the protocol report. The Academies assessment protocol report is written during the summer months and will be ready in September 2015. This is because data for the entire academic year needs to be included and is not ready in time to have a finished report by July 1st. Recommended action plans* Findings from the Assessment of the Academies will be utilized to plan further action in the coming year. All target outcomes were achieved. Additional Early Alert analyses provided insights into possible use of persistence and academic performance prediction models in the future. Findings were shared also at national conferences and a conference paper Further predication analytics and analysis of the usage and effects of Starfish needs to be done in the coming year to better understand how the tool can be used most effectively. The college submitted an IPAS II grant. If the grant is awarded to QCC, the OIRA will be actively involved in the evaluation process and will need to hire an additional part-time analyst with the grant money. Participate in the Strategic Planning of the Academies and the Academies Assessment Protocol Write a comprehensive report of the protocol findings Monitor and assess the use and effects of Starfish Early Alert and Connect. Analyze the impact on student performance and institutional effectiveness Participate in the planning and execution of High Impact Practices assessment as member of the HIP Assessment Team Provide usage reports and course grades for the Academic Learning Centers. Academic impact analyses of Early Alert on course pass rates, completion Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Key performance indicators Assist in the evaluation and assessment of existing and proposed academic programs and College initiatives. Assist the College in making informed decisions. Target outcomes rates, and withdrawal rates. Attend Starfish Team meetings and assist the team with reports as needed. Program Review data packets and assistance (Programs under review in 2014-15: CJ-AS, SF-AS, ME-AAS, EMAAS, EN-CERT Report on remediation and remedial efforts as needed CUNY Justice Academy data report Assist in the strategic planning goals and completion reports Enable the College to better understand the trends of retention, graduation, and other institutional effectiveness measures. Perform predictive analytics to better understand reasons for attrition Retention and graduation predictive analytics Assist the Retention Management Committee with analytical work as needed (Probation workshop attendance, academic performance of readmits and probation students, reverse transfer project etc.) QCC Drop Out Survey: A deeper analysis of the findings from the Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Actual outcomes and a poster proposal were submitted on predictive analytics and Early Alert for the CSRDE National Symposium on Student Retention. All program review data packets and additional customized data were provided to the review committees. The OIRA assisted in the College strategic planning process and completion reports. Reports on remediation efforts provided by OIRA, Academic Literacy, the Mathematics and Computer Science Department, and PreCollege have been presented at the Faculty Executive Committee’s (FEC) public meeting on March 25, 2015. A remediation dashboard with key remediation indicators and trends is currently developed in OIRA with input from the Academic Affairs office. The Director of OIRA Elisabeth Lackner has been appointed to serve on the Institutional Research Implementation Working group of the CUNY Justice Academy. Predictive analytics is ongoing (see above under Starfish). The OIRA assisted the Retention Management Committee with several analytical studies. A deeper analysis of the QCC drop outs was discussed in the committee but no action was taken. Probation workshop attendance (Academic Success Workshop) analysis was provided by the OIRA in collaboration with Elisabeth Recommended action plans* The programs under review in 2015-2016 are the LE-AA Liberal Arts & Sciences & Education program and the TM-AAS Massage Therapy program. OIRA will provide data packets by September, 2015. The remediation dashboard will be refined over the summer and a first version will be published in fall 2015. Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Key performance indicators Enable the College to understand the value of education to our graduates Target outcomes survey and the college career paths of recent drop outs will be performed Actual outcomes Scheiner, who is the workshop coordinator monitored the workshop participation. Recent Graduate Survey Transfer Out counts and reports CUNY central conducted the Recent Graduate Survey online this year. Thus, QCC did not run an additional Survey Monkey version for nonresponding survey participants. CUNY central is currently following up with a second wave paper survey mailing and will compile the data to be released to the colleges by the end of August 2015. Senior college academic performance analysis of recent transfers Labor data analysis Economic impact of QCC education Senior college academic performance reports were provided for the dual joint transfer programs to the Academic Affairs office (Linda Reesman). Enable the College and all constituencies to understand the scope of student enrollment, performance, and outcomes. Present to the college community and the public at large important findings of institutional effectiveness and achievements Internal reports on recent trends in enrollment, retention, graduation, and other institutional effectiveness measures Fact Book and Fast Facts Program Dashboards Annual reports to MSCHE, Nursing, Engineering Technology, Business, and Art accreditation agencies Presentations at the Assessment Institute Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Labor data analysis and economic impact analysis are ongoing. Data on average and median salary by degree offered has been compiled. All target outcomes were achieved. Recommended action plans* Senior college academic performance, labor data, and economic impact analyses will continue in 2015-2016. Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Key performance indicators Enable all departments and offices to make informed decisions based on information gathered by OIRA. Target outcomes Collaborate with AACU/Roadmap school Support enrollment management and prediction Actual outcomes Recommended action plans* All target outcomes were achieved. Assist the Veterans Services Office with data reports for internal and external reporting Reports to the VP for Student Affairs, Academies Advisement, and the various Student Services Offices. Institutional Review Board service for QCC and CUNY-wide manuscript submissions and IRB assistance on campus Enable the College to apply for grants and report on existing grants with the most accurate data required. Incoming requests and ad hoc requests from departments, faculty, and administrators Sponsored Program Office application and reporting needs All target outcomes were achieved: In the Academic Year 2014-2015 OIRA supported 13 grant applications and reports. Perkins Grant reports and institutional profile Support faculty with individual grant proposal and reports Enable the College to participate in CUNY wide assessment efforts. Gates Foundation IPAS grant reports Noel Levitz and Student Experience Survey implementation and reporting on campus Coordination of the Recent Graduate Survey SAM (Student Achievement Measure) Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). All target outcomes were achieved. The report of the Spring 2015 Noel Levitz survey findings is in progress and will be presented to the administration by the end of summer 2015. QCC applied for an IPAS II grant. If the college is awarded this grant, the OIRA will continue to be involved in the IPAS evaluation in 2015-2016. Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Key performance indicators Enable the Office to stay current with CUNY wide research activities and discussions Enable the Office to stay current with methods and theories in Institutional Research and Assessment. Target outcomes participation VFA (Voluntary Framework of Accountability) Attend all CUNY wide OIRA related meetings and events when appropriate. Participate in IR Council, WIKI, and list serve. Attend CUNY wide events related to IR and education policy research Review literature, news, and attend and participate in workshops and conferences Actual outcomes All target outcomes were achieved. All target outcomes were achieved. This year, the OIRA staff participated in several professional training activities and presented at conferences: NEAIR 2014 in November 2014 attended by Elisabeth Lackner and Wenmey Ting. Poster presentation by Elisabeth Lackner; One day training attended by Victor Fichera on survey design and analysis: METRO & ACRL/NY: Analyzing User Feedback Surveys (Nisa Bakkalbasi, Speaker), Tuesday, February 3, 2015 CUNY IR and Assessment Joint Retreat in April 2015, College of Staten Island, attended by Elisabeth Lackner and Wenmey Ting; AIR Forum 2015 in May 2015 in Denver, Colorado: Concurrent (featured) session presentation by Elisabeth Lackner Additional literature review and research activities relating to Early Alert and predictive analytics performed by Patrick Wynne and Elisabeth Lackner. *Column contents should be used to indicate which strategic objectives below address the follow-up necessary. Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Recommended action plans* Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment 2015-16 Department strategic plan (arranged by each division strategic objective that the department supports) 1. Support the evaluation and assessment of current curricula and new program development. Assist in achieving better alignment with baccalaureate programs and the job market through data analysis. 2. Sustain & assess 3 components of Queensborough Academies: advisement, high impact practices (HIPs) & technology (communication, institutional support & learning environment) Department strategic objectives Assist in the evaluation and assessment of existing and proposed academic programs and College initiatives. Assist the College in making informed decisions. Indicator(s) Provide useful, comprehensible, and timely data, analyses, and assessment of academic programs at QCC. Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Expected outcomes Program Review data packets and assistance (Programs under review in 2015-16: LE-AA, TM-AAS) Action plan/timeline Prepare data packets for the programs over the summer 2015 Who Patrick Wynne & Wenmey Ting Remediation Dashboard First draft of a possible dashboard developed over the summer 2015. Revised in fall 2015. Elisabeth Lackner in collaboration with Margot Edlin/Academic Affairs Assist with data reports for the Art and Design department accreditation report requirements Over the summer 2015 and fall 2015 as needed. Assist in the strategic planning goals and completion reports Year long process. Attend the College Advisory and Planning Committee meetings and assist Dean Corradetti as needed. Assessment of the Academic Success program Collaborate with the Registrar office, Elisabeth Scheiner, to evaluate the student feedback, academic progress, and persistence of probation students and students who participated in the Academic Success program. Participate in the Retention Management Team and assist with data analyses as needed. Year round as needed. Specific focus will OIRA staff be on the academic progress evaluation of readmit and probation students. Elisabeth Lackner Elisabeth Lackner and Wenmey Ting Elisabeth Lackner, Wenmey Ting Actual outcomes Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Department strategic objectives Indicator(s) Expected outcomes Action plan/timeline Who Incoming requests and ad hoc Year round as needed (Constitutes fifty reports to assist decision making for percent of the OIRA work load). proposed initiatives. Provide official and accurate information and analysis to support grant proposals and grant related reporting. Provide information to the College community demonstrating the value of education to our graduates Provide information on educational and career attainments of our QCC graduates to the administration of QCC. Current grants such as IPAS II Grant Bridge Grant and Perkins Grants will have all data and information needed to report on grant activities. Grant proposals and Sponsored Programs related activities can be completed with the help of official data and analysis provided by OIRA Recent Graduate Survey Transfer Out counts and reports Senior college academic performance analyses of recent transfers Year round as needed. OIRA staff The CUNY graduate survey will conclude in August 2015. Data analysis in fall 2015. Transfer out reports in Winter 2015 Senior college academic performance analysis in fall 2015 and spring 2016 in collaboration with SPAIE needs and program review needs Elisabeth Lackner, OIRA staff Finalize the 2014-2015 Protocol Report over the summer 2015. Fall 2015: review of findings and possible changes to the protocol. Victor Fichera Academic Strategic Planning committee Labor data analyses Ensure that the effects and outcomes of the various components of the Queensborough Academies are monitored and assessed. Execute the Queensborough Academies Assessment Protocol Actively participate in the Queensborough Academies Strategic Planning process. Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year). Economic impact of QCC education Review the exiting protocol with the Strategic Planning, Assessment, and Institutional Effectiveness division and the Academies Strategic Planning team. Modify the protocol as needed and execute all elements assigned to the Office of Institutional Research and Assessment. Execution of the various elements of the protocol in the academic year 20152016 over the course of the year as data becomes available. Tentatively: Freshmen experience survey: end of fall 2015 Victor Fichera Actual outcomes Strategic Planning, Assessment, Institutional Effectiveness – Year-end Report – Part B – Planning and Assessment Department strategic objectives Indicator(s) Enable the College and all constituencies to understand the scope of student enrollment, performance, and outcomes. Present to the college community and the public at large important findings of institutional effectiveness and achievements Expected outcomes Report the findings to the administration. Internal reports on recent trends in enrollment, retention, graduation, and other institutional effectiveness measures Action plan/timeline Starfish data analysis: winter 2015 and June 2016. Report on the Academies Assessment Protocol findings for AY 2014-2015 in fall 2015 Fact Book and Fast Facts Fast Facts in January 2016 Elisabeth Lackner Wenmey Ting Program Dashboards Program Dashboards in January 2016 OIRA staff Annual reports to MSCHE, Nursing, Technology & Engineering, Business, and Art accreditation agencies, Perkins Grant reports; Trend report in February 2016 Fact Book in May 2016 OIRA staff Incoming requests and ad-hoc reports Notes Who Patrick Wynne Actual outcomes Victor Fichera, Elisabeth Lackner Accreditation and funding reports as needed throughout the year. Add tables for additional division strategic objectives and corresponding department strategic objectives, as needed. For each division strategic objective, the department should have one or more strategic objectives intended to support that department in the achievement of the division objective. Each department strategic objective should have at least one indicator (key performance indicator) and one expected outcome, with an action plan and timeline to achieve that outcome. At the mid-year point, the department reports partial or preliminary outcomes. At the end of the year, the department reports actual outcomes. An indicator is the measurement by which the objective will be assessed. The outcome is the product of the work in support of achieving the objective. For example, if the strategic objective for Admissions is “to expand feeder high schools,” the indicator might be “applications received from new feeder high schools,” and the expected outcome might be “at least one application from five new feeder high schools.” Note: Mid-year status report due end of January. Completion report due early July (specific deadlines may vary by year).