GENERAL EDUCATION ASSESSMENT COMMITTEE End-of-Year Report to the UW-La Crosse Faculty Senate General Education Committee, 2014-15 Focus, Purpose, and Scope The present report focuses on the activities of the General Education Assessment Committee during the 2014-15 Academic Year. Although the General Education Assessment Committee is a “stand-alone” body of the University of Wisconsin-La Crosse Faculty Senate, the committee substantially derives its operating authority and duties from the General Education Committee. And as the UW-La Crosse General Education Committee originally established – in Spring, 2014 – a 2-year, integrated assessment process, the GEAC’s work over the past year focused on Form As which were submitted over the 2014-15 academic cycle by programs which chose to assess in the first year of the new two-year assessment cycle. Additionally, the GEAC also received and processed Form Bs and Cs, which are reports on the results of general education assessment activities. INFORMATION SOURCES. There are two principal information sources upon which the present report is based: a) The experiences of GEAC committee members, all of whom served uninterrupted terms on the committee from September, 2014 to late April, 2015; b) Data on the cycle of assessment processing, from initial Form A submission and approval, to the processing and validation of Form Bs and Cs. MAJOR CHANGE FOR 2014-15 AND THE LIMITS OF DATA IN THE PRESENT REPORT. In Spring 2014, the General Education Committee changed the cycle of assessment activities from a three-year cycle to a two-year cycle. This means that all courses listed in the UW-La Crosse General Education Program are now subject to formalized, university assessment every two years, and the necessary documentation must be developed, submitted and approved by the General Education Assessment Committee in conjunction with this formal process. Information on the activities of the assessment cycle is found within the GEAC data operating and reporting system, and has been retrieved, subjected to analysis, and is 1|Page represented in the present report. The information contained in the present report will allow the GEC and the University of Wisconsin Faculty Senate to make judgments about the extent to which the General Education Assessment process is working as planned and is meeting its principal objectives. Additionally, the present document is also intended to inform the activities and workings of the GEAC in future semesters. Noteworthy Activities of the General Education Assessment Committee The present report focuses on three items, two of which are explicit, and specific to the 201415 Academic Year. These latter items are, namely, the “charge” to the GEAC from the UW-L Faculty Senate, and the transformation of the technology system which supports GEAC assessment activities. In addition, implicit to the committee’s very purpose – and thus ongoing – is the requirement that GEAC approve and manage General Education Program assessment activities. The latter section of the present report is devoted to specific activities in fulfillment of GEAC’s assessment management function. ACTIVITIES AND ISSUES RELATED THE FACULTY SENATE’S COMMITTEE CHARGE FOR 2014-15. The specific Faculty Senate charge to the GEAC for 2014-15 reads as follows: “Consider how to best coordinate with the General Education Committee. The General Education Committee has been charged with finding a way improve communication with the GEAC. It will be your responsibility to help them in determining the necessary level of interaction and involvement that should occur. For example, does a representative from GEC need to attend each meeting of the GEAC? Ought they have voting privileges for actions taken in GEAC?” The General Education Assessment Committee has, since the 2013-14 Academic Year maintained effective, ongoing interaction with the General Education Committee. During 2013-14, this contact involved frequent exchanges of emails between the respective chairs of the two committees. Additionally, one or two members of the GEAC attended GEC meetings. In 2014-15, one member of the General Education Assessment Committee voluntarily attended meetings of the General Education Committee. In addition to informal interaction initiated by GEAC, the University’s Assessment Coordinator [UAC] serves ex officio on the General Education Assessment Committee and regularly attends meetings of the General Education Committee. During meetings of the GEAC, both the UAC and the GEAC member who 2|Page voluntarily attended GEC meetings have consistently reported on ongoing General Education Committee activities. And finally, during Spring, 2014, an individual serving on GEC began to regularly attend GEAC meetings. Clearly, there has been significant interaction between the GEAC and the GEC. In addition to these very substantial arrangements between GEC and GEAC, the General Education Committee proposed and won the approval of the UW-L Faculty Senate [Spring, 2015] of a requirement that one [1] member serving on the GEC serve as ex officio on the GEAC. This action requires a change in the GEC’s by-laws, and the UW-La Crosse Articles and By-Laws [A and B] Committee is currently in the process of adding this representation provision to GEC’s by-laws. [Approval by the Senate is expected in April/May 2015]. ACTIVITIES RELATED TO TRANSITION IN UW-LA CROSSE’S ASSESSMENT TECHNOLOGY SYSTEM. During Fall, 2014 and J-Term, 2015, several members of the GEAC actively participated in UW-La Crosse’s efforts to identify and contract with a new vendor of assessment technology services. Through a collective process initiated and led by staff from UW-L’s Instructional Technology unit, the participants in the RFP process selected Taskstream as the company which will provide assessment technology services to the university, including technology services for activities related to GEAC, the School of Education, and the College of Business Administration [CBA]. The introduction of Taskstream’s technology began in the UW-La Crosse School of Education in early Spring, 2015. In mid-April, the introduction of the Taskstream technology to the General Education Assessment Committee by the university staff [including the University Assessment Coordinator] was begun. Special issues, concerns, and unknowns in conjunction with the transition from the current GEAC software system to the adoption of the Taskstream technology. Several individuals from the GEAC participated in the RFP process that resulted in the selection of Taskstream. During the selection process, it became apparent that, while Taskstream’s capabilities appeared to significantly dovetail with the needs of the GEAC, there were also some issues that would have to be resolved at the point of implementation. In addition, some questions remained. These issues include the following: The need to introduce committee member [GEC as well as GEAC] to the new Taskstream software; The need for specific training of individuals in departments with respect to the new software; 3|Page Insuring that data critical to GEAC’s operations can be readily and effectively transferred to the new software system, and that it is readily retrievable. The need for rectification of the GEAC’s language such as “Forms A, B, and C” with the new expressions, and terms and procedures that are endemic to Taskstream. While any number of additional issues will no doubt arise as the new system is implemented, some issues drew more attention than others. One concern among GEAC members is the fact that– at least to date – indications are that the Taskstream system is incapable of randomly assigning proposals submitted by the departments to GEAC reviewers. The assignment of proposals must be a deliberate procedure carried out by an individual. This represents a significant workload addition is comparison to the existing UW-La Crosse assessment system. ACTIVITIES IN CONJUNCTION WITH THE 2014 GEC ASSESSMENT PROCEDURE REVISION The movement by the GEC of course assessment from a three-year to a two year cycle required a number of adjustments. Most of the adjustments that resulted need little explanation in the present report, as they are rather obvious. Activities in conjunction with the 2024 GEC revision included [but was not limited to] the following: Updating of the GEC assessment web site, mostly in the summer and fall, 2014; Significantly increasing the sheer numbers of assessment proposals [Form A] to be processed within a one year cycle [with the expectation that additional Forms Bs and Cs will have to be processed in the future]; Revising the committee’s procedures for Form A evaluation, to include a new procedure that the course-specific SLO clearly aligns with one of the six General Education SLOs. KEEPING TRACK OF GENERAL EDUCATION COURSES FOR ASSESSMENT PURPOSES For a number of reasons, it is critically important that the General Education Assessment Committee maintain accurate information on all courses which have been approved for the UW-La Crosse General Education Program, including those which have been recently approved but are not yet entered into the assessment technology system, and courses which are no longer active but for which a record should be maintained for historical purposes. In addition, in recent years, a number of “cross-listed” multi-disciplinary and/or interdisciplinary 4|Page courses have been developed. Maintaining proper records of “cross-listed” courses remains a challenge. Keeping Record on Active Courses. Maintaining a list of active General Education courses and manually adding them to the Assessment Software system continues to be an issue. The 2013- 2014 GEAC committee report identified at least 20 General Education courses which were not present in the Assessment Software system. Some of these (9) were added prior to the Fall 2014 submission of form As and one was determined to be inactive, bringing the number of active General Education courses in the system for the 2014-2015 cycle to 145. With the launch of the new CIM (Curriculum Inventory Management) system, GEAC members can readily access a list of active and inactive general education courses. Table 1 [page 9] includes a list of courses which do not appear in the software but will need to be added and assessed in upcoming cycles. Inactive Courses. Another set of courses are those which are not active general education courses but still remain in the software system, mainly in order to maintain archival assessment data. This include: APH102; BIO107; BIO999; ERS110; ANT999; HIS206; HIS220; PHY156; PSY107; HON100; HON205. Cross-Listed Courses. Finally, keeping track of cross-listed courses remain an issue. This include: THA376 and ECO376; CHM201 and ENV201; AST/PHY155; AST/PHY160; UWL100) 5|Page ACTIVITIES RELATED TO FORM A, B AND C PROCESSING, 2014-15 As has been indicated, the GEAC’s assessment-related activities are centered on the processing of three “forms”: Forms A, B, and C. Form A is an assessment protocol which is submitted to the GEAC for approval. Form B is a report on the results of the assessment. Form C Form A Assessments, 2014-2015. GEAC assessed 80 Form As in 2014-2015, with 9 courses that required revision. All 9 of those resubmitted, and 8 of their resubmissions were approved. One re-evaluation is pending, THA 376. In the 2011-2014 cycle, sixteen courses did not have completed Form As. Five of those sixteen courses had a completed Form A in 2014-2015. They are highlighted in yellow and italicized in the Table 2 below [pages 10-12]. Of the 118 courses that submitted assessment plans that included the year in which they plan on assessing, 81 indicated they would assess in the 2014-2015 year. Of those 81, the following 5 courses did not submit Form As: ECO 336; ENG 220; HED 207; MIC 100; MLG 305. However, eight courses that did not submit plans to assess in 2014-2015, did submit Form As. These are indicated by an X in the right column of Table 1. Distribution of SLOs Selected. GEAC’s data for 2014-15 indicate that of GEC’s 6 major SLOs, some were disproportionately assessed, while others were not substantially chosen. As Table 3 shows, the majority of instructors chose three of GECs six SLOs, namely SLO 2, “Critical and Creative Thinking, SLO 3, Aesthetic Perspectives and Meaning, and SLO 4, Effective Communication. These three SLOs combined accounted for 69 of the 80 courses assessed, or more than 86% of the total. Analysis of Form B Submissions. GEAC analyzed the Form B submissions from assessments conducted in the 2013-2014 academic year. There were 48 courses with approved Form A’s in 2013-2014; 41 of those courses had a validated Form B (85%). Initially, only 36 courses had 6|Page validated forms, but notes from the GEAC chair resulted in 5 additional forms being completed. Of the 7 courses that did not have validated form B’s: Five [5] replied to the chair with legitimate reasons for the incomplete forms Two [2] made promises to complete the forms but did not do so (both PHL). GEAC analyzed the Form B submissions as shown in the following table. Note that GEAC made no attempt to evaluate the effectiveness or significance of the assessments but simply categorized the department or program’s own analysis. In the first part of Table 3, GEAC attempted to determine whether a department indicated that they were satisfied with the results of the assessment. GEAC utilized three categories for this analysis: the department reported satisfaction with the results; the department reported dissatisfaction with the results; or it was unclear how the department viewed the results. Note that percentages may not add up to 100% due to rounding of the individual categories. GEAC then attempted to very broadly determine the depth of analysis reported on Form B. GEAC placed each form into one of the following three categories: analysis of results included detailed or specific observations; analysis of results was broad and lacked specificity; and analysis of results was not included. The classification of Form B submissions with respect to their depth of analysis is given in the second part of Table 4. Finally, GEAC attempted to broadly capture the degree to which courses in the General Education program are making changes as a result of GEC assessment activities. Each Form B was placed into one of the following five categories: the form did not indicate a proposed improvement the form proposed improvement(s) in task and/or rubric the form proposed improvement(s) in pedagogy the form proposed improvement(s) in pedagogy and task/rubric the form proposed repeating the assessment with no changes in task or pedagogy. The classification of Form B submissions with respect to suggested improvements is given in the third part of Table 3. Analysis of Form C. GEAC analyzed the Form C submissions from assessments conducted in the 2012-2013 academic year. The results of this analysis are shown in Table 4. For 2013-14, there were 43 courses with approved Form A’s, 39 courses that followed up with validated form B’s (91%), and finally 26 courses with validated Form C’s 60%. Of the 13 courses that had validated form B’s but did not follow up with Form C’s: 7|Page Twelve were ENG courses, which despite notes from the GEAC chair and promises to complete them, did not One was an SAH course GEAC analyzed the Form C submissions as shown in the following table. Note that GEAC made no attempt to evaluate the effectiveness, quality, or significance of the assessment results themselves. GEAC simply categorized the department or program’s reports according to the criteria shown in the tables. In the first part of Table 3, GEAC attempted to determine whether a department reported an actual improvement or change. GEAC utilized five categories for this analysis. GEAC then attempted to very broadly determine the extent to which the changes were a direct result of the analysis reported on Form B. In other words, GEAC attempted to broadly determine whether changes are being driven by assessment activities. GEAC finally attempted to broadly capture the type of improvement that was actually made (as opposed to the type of improvement that was proposed on Form B). 8|Page Table 1. Approved General Education Courses which were not presently in the General Education Assessment Software System. 1 ARC200: World Archaeology: Origins and Development of Human Culture and 2 BIO100: Biology for the Informed Citizen 3 CHI305: Introduction to Modern Chinese Literature 4 CHI320: Introduction to Chinese Civilization 5 EDS203: School and Society 6 EDS206: Multicultural Education 7 ERS207: Multicultural Literature of the United States Society 8 ERS210: Literature of Black America 9 ERS215: African American Authors 10 ESL250: ESL Speaking 11 ESL251: U.S. Culture and Film 12 ESL252: ESL Writing and Grammar 13 ESL253: ESL Research paper 14 GER398: German Thinkers and Popular Culture 15 HP105: Analysis of Health, Wellness, and Disease for the Health Care Consumer 16 INS250: Orientation to Study Abroad 17 INS251: Study Abroad Practicum: Journaling 18 INS252: Cross-Cultural Reentry from Study Abroad 19 POL244: International Relations 20 SOC261: Technology in Society 9|Page TABLE 2. FORM A STATUS AS OF MAY 1, 2015 Course Form A SLO Assessment Plan Year Status 1 ART 102 Approved 3 X 2 ART 160 Approved 3 X 3 ART 172 Approved 3 X 4 ART 301 Approved 3 X 5 ART 302 Approved 4 X 6 C-S 120 Approved 2 2014-2015 7 C-T 100 Approved 2 2014-2015 8 CHE 200 Approved 1 2014-2015 9 CHI 102 Approved 4 2014-2015 10 CHI 201 Approved 4 2014-2015 11 CHI 202 Approved 4 2014-2015 12 CHI 320 Approved 3 2014-2015 13 CHM 100 Approved 6 2014-2015 14 CHM 103 Approved 2 2014-2015 15 CST 271 Approved 2 2014-2015 16 ECO 110 Approved 2 2014-2015 17 ECO 120 Approved 2 2014-2015 18 ENG 110 Approved 4 2014-2015 20 ENG 200 Approved 3 2014-2015 21 ENG 201 Approved 3 2014-2015 22 ENG 202 Approved 3 2014-2015 23 ENG 203 Approved 3 2014-2015 24 ENG 204 Approved 3 2014-2015 25 ENG 205 Approved 3 2014-2015 26 ENG 206 Approved 3 2014-2015 27 ENG 208 Approved 3 2014-2015 28 ENG 210 Approved 3 2014-2015 29 ENG 215 Approved 3 2014-2015 30 ESS 104 Approved 3 2014-2015 31 FIN 207 Approved 2 X 19 10 | P a g e ENG 112 Approved 4 2014-2015 32 FRE 102 Approved 4 2014-2015 33 FRE 201 Approved 4 2014-2015 34 FRE 202 Approved 4 2014-2015 36 GER 102 Approved 4 2014-2015 37 GER 201 Approved 4 2014-2015 38 GER 202 Approved 4 2014-2015 39 GER 398 Approved 3 2014-2015 40 HIS 101 Approved 2 2014-2015 41 HIS 102 Approved 2 2014-2015 42 HIS 205 Approved 6 2014-2015 43 HPR 105 Approved 6 2014-2015 44 MLG 204 Approved 4 2014-2015 45 MTH 135 Approved 2 2014-2015 46 MTH 136 Approved 2 2014-2015 47 MTH 145 Approved 2 2014-2015 48 MTH 150 Approved 2 2014-2015 49 MTH 151 Approved 2 2014-2015 50 MTH 175 Approved 2 2014-2015 51 MTH 207 Approved 2 2014-2015 52 MTH 208 Approved 2 2014-2015 53 MTH 265 Approved 2 2014-2015 54 MUS 105 Approved 3 2014-2015 55 MUS 201 Approved 3 2014-2015 58 PHL 101 Approved 2 2014-2015 59 PHL 335 Approved 5 2014-2015 60 PHY 103 Approved 2 2014-2015 61 PHY 106 Approved 1 2014-2015 62 PHY 125 Approved 1 2014-2015 63 PHY 142 Approved 1 X 64 PHY 155 Approved 2 2014-2015 65 PHY 160 Approved 2 2014-2015 35 56 57 11 | P a g e FRE 220 MUS 209 MUS 317 Approved Approved Approved 3 3 3 2014-2015 2014-2015 2014-2015 66 PHY 203 Approved 2 2014-2015 67 PSY 100 Approved 2 2014-2015 68 RUS 102 Approved 4 2014-2015 69 RUS 201 Approved 4 2014-2015 70 RUS 202 Approved 4 2014-2015 71 SPA 102 Approved 4 2014-2015 72 SPA 103 Approved 4 X 73 SPA 201 Approved 4 2014-2015 74 SPA 202 Approved 4 2014-2015 75 THA 110 Approved 3 2014-2015 76 THA 120 Approved 3 2014-2015 77 THA 376 Resubmitted 2 2014-2015 79 W-S 100 Approved 5 2014-2015 80 W-S 230 Approved 5 2014-2015 78 12 | P a g e UWL 100 Approved 6 2014-2015 Table 3. SLO by Number Selected, 2014-15 # Student Learning Outcome Number Selected 1 Human cultures and the natural world: 4 2 Critical and creative thinking: 25 3 Aesthetic perspectives and meaning: 24 4 Effective communication 20 5 Interaction in intercultural contexts: 3 6 Individual, social, and environmental responsibility 4 Table 4. Form B Evaluations, 2013-2014 Academic Year Department evaluation of results % Department reported satisfaction with results 71% Department reported dissatisfaction with results 10% Unclear how the department views the results 20% Analysis of results Analysis of results included detailed or specific observations 60% Analysis of results was broad and lacked specificity 32% Analysis of results not included 9% Proposed improvements Did not indicate a proposed improvement 16% Proposed improvement(s) in task and/or rubric 6% Proposed improvement(s) in pedagogy 56% Proposed improvement(s) in pedagogy and task/rubric 18% Proposed repeating assessment with no changes in task or pedagogy 4% 13 | P a g e Table 5. Form C Evaluations, 2012-2013 Academic Year Explanation of assessments use in making improvements % Some or all of proposed improvements were made 46% Improvement other than those proposed were made 4% No improvement because Form B indicated none were necessary 15% No improvement although Form B proposed changes 35% Addresses the Form B analysis Form B analysis is addressed 54% Form B analysis is not addressed - although it did indicate action 25% Form B analysis is not addressed - it did not indicate any action. 8% Unclear 13% Type of improvements made Task and/or rubric 12% Pedagogy 35% Pedagogy and task/rubric 4% None given 50% GGC, 1 May 2015 14 | P a g e