G E A C

advertisement
GENERAL EDUCATION ASSESSMENT COMMITTEE
End-of-Year Report to the UW-La Crosse Faculty Senate General Education Committee, 2014-15
Focus, Purpose, and Scope
The present report focuses on the activities of the General Education Assessment Committee
during the 2014-15 Academic Year. Although the General Education Assessment Committee is
a “stand-alone” body of the University of Wisconsin-La Crosse Faculty Senate, the committee
substantially derives its operating authority and duties from the General Education Committee.
And as the UW-La Crosse General Education Committee originally established – in Spring,
2014 – a 2-year, integrated assessment process, the GEAC’s work over the past year focused on
Form As which were submitted over the 2014-15 academic cycle by programs which chose to
assess in the first year of the new two-year assessment cycle. Additionally, the GEAC also
received and processed Form Bs and Cs, which are reports on the results of general education
assessment activities.
INFORMATION SOURCES. There are two principal information sources upon which the present
report is based:
a) The experiences of GEAC committee members, all of whom served uninterrupted terms on
the committee from September, 2014 to late April, 2015;
b) Data on the cycle of assessment processing, from initial Form A submission and approval, to
the processing and validation of Form Bs and Cs.
MAJOR CHANGE FOR 2014-15 AND THE LIMITS OF DATA IN THE PRESENT REPORT. In Spring 2014, the
General Education Committee changed the cycle of assessment activities from a three-year
cycle to a two-year cycle. This means that all courses listed in the UW-La Crosse General
Education Program are now subject to formalized, university assessment every two years, and
the necessary documentation must be developed, submitted and approved by the General
Education Assessment Committee in conjunction with this formal process.
Information on the activities of the assessment cycle is found within the GEAC data
operating and reporting system, and has been retrieved, subjected to analysis, and is
1|Page
represented in the present report. The information contained in the present report will allow
the GEC and the University of Wisconsin Faculty Senate to make judgments about the extent to
which the General Education Assessment process is working as planned and is meeting its
principal objectives. Additionally, the present document is also intended to inform the activities
and workings of the GEAC in future semesters.
Noteworthy Activities of the General Education Assessment Committee
The present report focuses on three items, two of which are explicit, and specific to the 201415 Academic Year. These latter items are, namely, the “charge” to the GEAC from the UW-L
Faculty Senate, and the transformation of the technology system which supports GEAC
assessment activities. In addition, implicit to the committee’s very purpose – and thus ongoing –
is the requirement that GEAC approve and manage General Education Program assessment
activities. The latter section of the present report is devoted to specific activities in fulfillment of
GEAC’s assessment management function.
ACTIVITIES AND ISSUES RELATED THE FACULTY SENATE’S COMMITTEE CHARGE FOR 2014-15. The specific
Faculty Senate charge to the GEAC for 2014-15 reads as follows:
“Consider how to best coordinate with the General Education Committee. The General
Education Committee has been charged with finding a way improve communication with the
GEAC. It will be your responsibility to help them in determining the necessary level of
interaction and involvement that should occur. For example, does a representative from GEC
need to attend each meeting of the GEAC? Ought they have voting privileges for actions taken
in GEAC?”
The General Education Assessment Committee has, since the 2013-14 Academic Year
maintained effective, ongoing interaction with the General Education Committee. During
2013-14, this contact involved frequent exchanges of emails between the respective chairs of
the two committees. Additionally, one or two members of the GEAC attended GEC meetings. In
2014-15, one member of the General Education Assessment Committee voluntarily attended
meetings of the General Education Committee. In addition to informal interaction initiated by
GEAC, the University’s Assessment Coordinator [UAC] serves ex officio on the General
Education Assessment Committee and regularly attends meetings of the General Education
Committee. During meetings of the GEAC, both the UAC and the GEAC member who
2|Page
voluntarily attended GEC meetings have consistently reported on ongoing General Education
Committee activities. And finally, during Spring, 2014, an individual serving on GEC began to
regularly attend GEAC meetings.
Clearly, there has been significant interaction between the GEAC and the GEC. In
addition to these very substantial arrangements between GEC and GEAC, the General Education
Committee proposed and won the approval of the UW-L Faculty Senate [Spring, 2015] of a
requirement that one [1] member serving on the GEC serve as ex officio on the GEAC. This
action requires a change in the GEC’s by-laws, and the UW-La Crosse Articles and By-Laws [A
and B] Committee is currently in the process of adding this representation provision to GEC’s
by-laws. [Approval by the Senate is expected in April/May 2015].
ACTIVITIES RELATED TO TRANSITION IN UW-LA CROSSE’S ASSESSMENT TECHNOLOGY SYSTEM. During Fall,
2014 and J-Term, 2015, several members of the GEAC actively participated in UW-La Crosse’s
efforts to identify and contract with a new vendor of assessment technology services. Through a
collective process initiated and led by staff from UW-L’s Instructional Technology unit, the
participants in the RFP process selected Taskstream as the company which will provide
assessment technology services to the university, including technology services for activities
related to GEAC, the School of Education, and the College of Business Administration [CBA].
The introduction of Taskstream’s technology began in the UW-La Crosse School of
Education in early Spring, 2015. In mid-April, the introduction of the Taskstream technology to
the General Education Assessment Committee by the university staff [including the University
Assessment Coordinator] was begun.
Special issues, concerns, and unknowns in conjunction with the transition from the current
GEAC software system to the adoption of the Taskstream technology.
Several individuals from the GEAC participated in the RFP process that resulted in the selection
of Taskstream. During the selection process, it became apparent that, while Taskstream’s
capabilities appeared to significantly dovetail with the needs of the GEAC, there were also some
issues that would have to be resolved at the point of implementation. In addition, some
questions remained. These issues include the following:

The need to introduce committee member [GEC as well as GEAC] to the new Taskstream
software;

The need for specific training of individuals in departments with respect to the new
software;
3|Page

Insuring that data critical to GEAC’s operations can be readily and effectively
transferred to the new software system, and that it is readily retrievable.

The need for rectification of the GEAC’s language such as “Forms A, B, and C” with the
new expressions, and terms and procedures that are endemic to Taskstream.
While any number of additional issues will no doubt arise as the new system is implemented,
some issues drew more attention than others. One concern among GEAC members is the fact
that– at least to date – indications are that the Taskstream system is incapable of randomly
assigning proposals submitted by the departments to GEAC reviewers. The assignment of
proposals must be a deliberate procedure carried out by an individual. This represents a
significant workload addition is comparison to the existing UW-La Crosse assessment system.
ACTIVITIES IN CONJUNCTION WITH THE 2014 GEC ASSESSMENT PROCEDURE REVISION
The movement by the GEC of course assessment from a three-year to a two year cycle required
a number of adjustments. Most of the adjustments that resulted need little explanation in the
present report, as they are rather obvious. Activities in conjunction with the 2024 GEC revision
included [but was not limited to] the following:

Updating of the GEC assessment web site, mostly in the summer and fall, 2014;

Significantly increasing the sheer numbers of assessment proposals [Form A] to be
processed within a one year cycle [with the expectation that additional Forms Bs and Cs
will have to be processed in the future];

Revising the committee’s procedures for Form A evaluation, to include a new procedure
that the course-specific SLO clearly aligns with one of the six General Education SLOs.
KEEPING TRACK OF GENERAL EDUCATION COURSES FOR ASSESSMENT PURPOSES
For a number of reasons, it is critically important that the General Education Assessment
Committee maintain accurate information on all courses which have been approved for the
UW-La Crosse General Education Program, including those which have been recently
approved but are not yet entered into the assessment technology system, and courses which are
no longer active but for which a record should be maintained for historical purposes. In
addition, in recent years, a number of “cross-listed” multi-disciplinary and/or interdisciplinary
4|Page
courses have been developed. Maintaining proper records of “cross-listed” courses remains a
challenge.
Keeping Record on Active Courses. Maintaining a list of active General Education courses and
manually adding them to the Assessment Software system continues to be an issue. The 2013-
2014 GEAC committee report identified at least 20 General Education courses which were not
present in the Assessment Software system. Some of these (9) were added prior to the Fall 2014
submission of form As and one was determined to be inactive, bringing the number of active
General Education courses in the system for the 2014-2015 cycle to 145. With the launch of
the new CIM (Curriculum Inventory Management) system, GEAC members can readily access a
list of active and inactive general education courses. Table 1 [page 9] includes a list of courses
which do not appear in the software but will need to be added and assessed in upcoming
cycles.
Inactive Courses. Another set of courses are those which are not active general education
courses but still remain in the software system, mainly in order to maintain archival assessment
data. This include:

APH102;

BIO107;

BIO999;

ERS110;

ANT999;

HIS206;

HIS220;

PHY156;

PSY107;

HON100;

HON205.
Cross-Listed Courses. Finally, keeping track of cross-listed courses remain an issue. This
include:

THA376 and ECO376;

CHM201 and ENV201;

AST/PHY155;

AST/PHY160; UWL100)
5|Page
ACTIVITIES RELATED TO FORM A, B AND C PROCESSING, 2014-15
As has been indicated, the GEAC’s assessment-related activities are centered on the processing
of three “forms”: Forms A, B, and C. Form A is an assessment protocol which is submitted to the
GEAC for approval. Form B is a report on the results of the assessment. Form C
Form A Assessments, 2014-2015. GEAC assessed 80 Form As in 2014-2015, with 9 courses
that required revision. All 9 of those resubmitted, and 8 of their resubmissions were approved.
One re-evaluation is pending, THA 376.
In the 2011-2014 cycle, sixteen courses did not have completed Form As. Five of those
sixteen courses had a completed Form A in 2014-2015. They are highlighted in yellow and
italicized in the Table 2 below [pages 10-12].
Of the 118 courses that submitted assessment plans that included the year in which
they plan on assessing, 81 indicated they would assess in the 2014-2015 year. Of those 81, the
following 5 courses did not submit Form As:

ECO 336;

ENG 220;

HED 207;

MIC 100;

MLG 305.
However, eight courses that did not submit plans to assess in 2014-2015, did submit
Form As. These are indicated by an X in the right column of Table 1.
Distribution of SLOs Selected. GEAC’s data for 2014-15 indicate that of GEC’s 6 major SLOs,
some were disproportionately assessed, while others were not substantially chosen. As Table 3
shows, the majority of instructors chose three of GECs six SLOs, namely SLO 2, “Critical and
Creative Thinking, SLO 3, Aesthetic Perspectives and Meaning, and SLO 4, Effective
Communication. These three SLOs combined accounted for 69 of the 80 courses assessed, or
more than 86% of the total.
Analysis of Form B Submissions. GEAC analyzed the Form B submissions from assessments
conducted in the 2013-2014 academic year. There were 48 courses with approved Form A’s in
2013-2014; 41 of those courses had a validated Form B (85%). Initially, only 36 courses had
6|Page
validated forms, but notes from the GEAC chair resulted in 5 additional forms being completed.
Of the 7 courses that did not have validated form B’s:
 Five [5] replied to the chair with legitimate reasons for the incomplete forms
 Two [2] made promises to complete the forms but did not do so (both PHL).
GEAC analyzed the Form B submissions as shown in the following table. Note that GEAC
made no attempt to evaluate the effectiveness or significance of the assessments but simply
categorized the department or program’s own analysis. In the first part of Table 3, GEAC
attempted to determine whether a department indicated that they were satisfied with the
results of the assessment. GEAC utilized three categories for this analysis: the department
reported satisfaction with the results; the department reported dissatisfaction with the results;
or it was unclear how the department viewed the results. Note that percentages may not add up
to 100% due to rounding of the individual categories.
GEAC then attempted to very broadly determine the depth of analysis reported on Form B.
GEAC placed each form into one of the following three categories: analysis of results included
detailed or specific observations; analysis of results was broad and lacked specificity; and
analysis of results was not included. The classification of Form B submissions with respect to
their depth of analysis is given in the second part of Table 4.
Finally, GEAC attempted to broadly capture the degree to which courses in the General
Education program are making changes as a result of GEC assessment activities. Each Form B
was placed into one of the following five categories:

the form did not indicate a proposed improvement

the form proposed improvement(s) in task and/or rubric

the form proposed improvement(s) in pedagogy

the form proposed improvement(s) in pedagogy and task/rubric

the form proposed repeating the assessment with no changes in task or pedagogy.
The classification of Form B submissions with respect to suggested improvements is given in the
third part of Table 3.
Analysis of Form C. GEAC analyzed the Form C submissions from assessments conducted in the
2012-2013 academic year. The results of this analysis are shown in Table 4. For 2013-14,
there were 43 courses with approved Form A’s, 39 courses that followed up with validated
form B’s (91%), and finally 26 courses with validated Form C’s 60%.
Of the 13 courses that had validated form B’s but did not follow up with Form C’s:
7|Page
 Twelve were ENG courses, which despite notes from the GEAC chair and promises to
complete them, did not
 One was an SAH course
GEAC analyzed the Form C submissions as shown in the following table. Note that GEAC
made no attempt to evaluate the effectiveness, quality, or significance of the assessment results
themselves. GEAC simply categorized the department or program’s reports according to the
criteria shown in the tables. In the first part of Table 3, GEAC attempted to determine whether a
department reported an actual improvement or change. GEAC utilized five categories for this
analysis. GEAC then attempted to very broadly determine the extent to which the changes were
a direct result of the analysis reported on Form B. In other words, GEAC attempted to broadly
determine whether changes are being driven by assessment activities. GEAC finally attempted
to broadly capture the type of improvement that was actually made (as opposed to the type of
improvement that was proposed on Form B).
8|Page
Table 1. Approved General Education Courses which were not presently in the General
Education Assessment Software System.
1
ARC200: World Archaeology: Origins and Development of Human Culture and
2
BIO100: Biology for the Informed Citizen
3
CHI305: Introduction to Modern Chinese Literature
4
CHI320: Introduction to Chinese Civilization
5
EDS203: School and Society
6
EDS206: Multicultural Education
7
ERS207: Multicultural Literature of the United States
Society
8
ERS210: Literature of Black America
9
ERS215: African American Authors
10
ESL250: ESL Speaking
11
ESL251: U.S. Culture and Film
12
ESL252: ESL Writing and Grammar
13
ESL253: ESL Research paper
14
GER398: German Thinkers and Popular Culture
15
HP105: Analysis of Health, Wellness, and Disease for the Health Care
Consumer
16
INS250: Orientation to Study Abroad
17
INS251: Study Abroad Practicum: Journaling
18
INS252: Cross-Cultural Reentry from Study Abroad
19
POL244: International Relations
20
SOC261: Technology in Society
9|Page
TABLE 2. FORM A STATUS AS OF MAY 1, 2015
Course
Form A
SLO
Assessment Plan Year
Status
1
ART 102
Approved
3
X
2
ART 160
Approved
3
X
3
ART 172
Approved
3
X
4
ART 301
Approved
3
X
5
ART 302
Approved
4
X
6
C-S 120
Approved
2
2014-2015
7
C-T 100
Approved
2
2014-2015
8
CHE 200
Approved
1
2014-2015
9
CHI 102
Approved
4
2014-2015
10
CHI 201
Approved
4
2014-2015
11
CHI 202
Approved
4
2014-2015
12
CHI 320
Approved
3
2014-2015
13
CHM 100
Approved
6
2014-2015
14
CHM 103
Approved
2
2014-2015
15
CST 271
Approved
2
2014-2015
16
ECO 110
Approved
2
2014-2015
17
ECO 120
Approved
2
2014-2015
18
ENG 110
Approved
4
2014-2015
20
ENG 200
Approved
3
2014-2015
21
ENG 201
Approved
3
2014-2015
22
ENG 202
Approved
3
2014-2015
23
ENG 203
Approved
3
2014-2015
24
ENG 204
Approved
3
2014-2015
25
ENG 205
Approved
3
2014-2015
26
ENG 206
Approved
3
2014-2015
27
ENG 208
Approved
3
2014-2015
28
ENG 210
Approved
3
2014-2015
29
ENG 215
Approved
3
2014-2015
30
ESS 104
Approved
3
2014-2015
31
FIN 207
Approved
2
X
19
10 | P a g e
ENG 112
Approved
4
2014-2015
32
FRE 102
Approved
4
2014-2015
33
FRE 201
Approved
4
2014-2015
34
FRE 202
Approved
4
2014-2015
36
GER 102
Approved
4
2014-2015
37
GER 201
Approved
4
2014-2015
38
GER 202
Approved
4
2014-2015
39
GER 398
Approved
3
2014-2015
40
HIS 101
Approved
2
2014-2015
41
HIS 102
Approved
2
2014-2015
42
HIS 205
Approved
6
2014-2015
43
HPR 105
Approved
6
2014-2015
44
MLG 204
Approved
4
2014-2015
45
MTH 135
Approved
2
2014-2015
46
MTH 136
Approved
2
2014-2015
47
MTH 145
Approved
2
2014-2015
48
MTH 150
Approved
2
2014-2015
49
MTH 151
Approved
2
2014-2015
50
MTH 175
Approved
2
2014-2015
51
MTH 207
Approved
2
2014-2015
52
MTH 208
Approved
2
2014-2015
53
MTH 265
Approved
2
2014-2015
54
MUS 105
Approved
3
2014-2015
55
MUS 201
Approved
3
2014-2015
58
PHL 101
Approved
2
2014-2015
59
PHL 335
Approved
5
2014-2015
60
PHY 103
Approved
2
2014-2015
61
PHY 106
Approved
1
2014-2015
62
PHY 125
Approved
1
2014-2015
63
PHY 142
Approved
1
X
64
PHY 155
Approved
2
2014-2015
65
PHY 160
Approved
2
2014-2015
35
56
57
11 | P a g e
FRE 220
MUS 209
MUS 317
Approved
Approved
Approved
3
3
3
2014-2015
2014-2015
2014-2015
66
PHY 203
Approved
2
2014-2015
67
PSY 100
Approved
2
2014-2015
68
RUS 102
Approved
4
2014-2015
69
RUS 201
Approved
4
2014-2015
70
RUS 202
Approved
4
2014-2015
71
SPA 102
Approved
4
2014-2015
72
SPA 103
Approved
4
X
73
SPA 201
Approved
4
2014-2015
74
SPA 202
Approved
4
2014-2015
75
THA 110
Approved
3
2014-2015
76
THA 120
Approved
3
2014-2015
77
THA 376
Resubmitted
2
2014-2015
79
W-S 100
Approved
5
2014-2015
80
W-S 230
Approved
5
2014-2015
78
12 | P a g e
UWL 100
Approved
6
2014-2015
Table 3. SLO by Number Selected, 2014-15
# Student Learning Outcome
Number Selected
1
Human cultures and the natural world:
4
2
Critical and creative thinking:
25
3
Aesthetic perspectives and meaning:
24
4
Effective communication
20
5
Interaction in intercultural contexts:
3
6
Individual, social, and environmental responsibility
4
Table 4. Form B Evaluations, 2013-2014 Academic Year
Department evaluation of results
%
Department reported satisfaction with results
71%
Department reported dissatisfaction with results
10%
Unclear how the department views the results
20%
Analysis of results
Analysis of results included detailed or specific observations
60%
Analysis of results was broad and lacked specificity
32%
Analysis of results not included
9%
Proposed improvements
Did not indicate a proposed improvement
16%
Proposed improvement(s) in task and/or rubric
6%
Proposed improvement(s) in pedagogy
56%
Proposed improvement(s) in pedagogy and task/rubric
18%
Proposed repeating assessment with no changes in task or pedagogy
4%
13 | P a g e
Table 5. Form C Evaluations, 2012-2013 Academic Year
Explanation of assessments use in making improvements
%
Some or all of proposed improvements were made
46%
Improvement other than those proposed were made
4%
No improvement because Form B indicated none were necessary
15%
No improvement although Form B proposed changes
35%
Addresses the Form B analysis
Form B analysis is addressed
54%
Form B analysis is not addressed - although it did indicate action
25%
Form B analysis is not addressed - it did not indicate any action.
8%
Unclear
13%
Type of improvements made
Task and/or rubric
12%
Pedagogy
35%
Pedagogy and task/rubric
4%
None given
50%
GGC, 1 May 2015
14 | P a g e
Download