Incorporating IDEA into
Assessment of General
Education Learning
Outcomes
Stephanie Oetting, PhD
Director of Institutional Research
University of Saint Francis
Fort Wayne, IN
General Aims

Incorporating IDEA student ratings into
general education assessment

Using IDEA extra question process to collect
data for general education assessment

USF GE Assessment Model

Mix of direct/indirect and internal/external
measures

Custom reports versus aggregate data files

Using extra questions in many classes
Background





General education curriculum in place 20+ years
Intense work for revision initiated in 2003
New GE Framework approved late Spring 2007
Assessment plan work successfully avoided
Faculty Governance Structure



Committee On Assessment Student Academic
Achievement
General Education Committee
Academic Council
General Education Goals
1.
2.
3.
4.
5.
6.
7.
8.
Develop personal awareness of diverse/global society.
Demonstrate leadership, service, social responsibility.
Communicate effectively in personal/professional
interactions.
Demonstrate competence in applying technologies.
Demonstrate personal and social awareness of
importance of lifelong health/wellness.
Think analytically, synthetically, critically, and creatively
in the pursuit of knowledge.
Develop awareness/understanding of
artistic/expressive aspects of the human experience.
Appreciate spiritual dimension of life, be conscious of
own religious perspective within a community context.
Layers of Complexity
8 General Education Goals
operationalized with
31 Learning Outcomes
grouped into
19 Distribution Requirements
State of Assessment
Challenges:
 Several significant
administrative and
structural changes
 New GE framework
lacking assessment plan
 Exhaustion from GE
framework revision
 No Clear leadership,
accountability, authority
for GE assessment and
 Committees searching for
a responsible party!
Strengths:
 Program assessment welldeveloped in some
programs
 Emerging group of faculty
knowledgeable about,
invested in assessment
 Support, encouragement
from University
 AQIP Systems Portfolio in
progress
 Assessment “Opinion
Leaders” vocalizing
Getting Started…
“You don’t
have to see
the whole
staircase,
just take
the first
step.”
--Martin Luther King, Jr.
Task Force Convened

Composition

Goals





BEGIN draft of assessment plan
Reduce fear and avoidance
Identify critical resources for implementation
Cultivate culture of assessment
Compensation & reward
Requirements of Assessment Plan!









Simple, seamless, sustainable
Produce meaningful data, quickly and efficiently
Use what’s already in place, no extra time or work!
Benefit students whenever possible
Empower instructors/dept. to generate artifacts
Need naturally occurring artifacts, not extra work
Compensate faculty who give their time!
No evaluation of faculty, course or program
No policing or punitive approach
IF ONLY…
Comforting thoughts…
“It doesn’t really matter whether you can quantify your
results. What matters is that you rigorously assemble
evidence – quantitative or qualitative – to track your
progress.”
“What matters is not finding the perfect indicator, but
settling upon a consistent and intelligent method of
assessing your output results, and then tracking your
trajectory with rigor.”
Collins, J. (2005). Good to Great and the Social Sectors: A
Monograph…(p. 7, 8)
GE Assessment Plan

Direct measures



Indirect measures



Student perceptions
Inferred measurements
Internal measures


Student demonstration of learning outcomes
Evaluated by faculty
Locally developed
External measures


Standardized assessments
Results comparisons
Indirect Measures
 External


Student self-reported progress on IDEA
objective that aligns with goal
Noel Levitz SSI
 Internal


Student self-reported progress on learning
outcome using IDEA extra question format
Other surveys and assessments
Align Goals to IDEA Objectives
 Develop
a personal awareness of our
diverse and global society

Developing a clearer understanding of, and
commitment to, personal values (#10)
 Demonstrate
leadership, service, and
social responsibility

Acquiring skills in working with others as a
member of a team (#5)
Direct Measure
 Internal




Artifacts generated at course level
Assessed by faculty committee
Holistic rubric using learning outcome
Allows for comparison of faculty and student
ratings
Goal 1
% students
enrolled in
foreign language
LO3
Develop a personal awareness of
our diverse and global society
LO1
IDEA
Extra Q
IDEA
Extra Q
Student satisfaction
Noel Levitz SSI #62:
Commitment to racial
harmony on campus
MLK Survey
Artifact:
iConnect
Progress on IDEA
objective #10:
Understanding of &
commitment to
personal values
LO2
IDEA
Extra Q
= Direct Measure
University of Saint Francis
GE Assessment Model
= Indirect Measure
4-year Assessment Cycle
 2007-2008:
Goals 1 and 2
 2008-2009: Goals 3 and 4
 2009-2010: Goals 5 and 6
 2010-2011: Goals 7 and 8
 2011-2012:
.
..
Goals 1 and 2
Preliminary Results
 IDEA Objectives


Percent selecting as important/essential
Progress on relevant objectives
 Learning


Outcomes
IDEA Extra Questions – student perception
Artifact assessment – faculty assessment
Results - Learning Outcomes
Goal 1
Indirect Measure: Student Rating
3
2.5
2
1.5
1
0.5
0
Mean
Mean
Goal 1
Direct Measure: Artifact Rating
LO 1
LO 2
LO 3
5
4.5
4
3.5
3
2.5
2
1.5
1
Fall 2007 Mean
Spring 2008 Mean
LO1
Learning Outcomes
Goal 2
Indirect Measure: Student Rating
Mean
Mean
3
2.5
2
1.5
1
0.5
0
LO 5
LO 6
Learning Outcomes
LO3
Learning Outcomes
Goal 2
Direct Measure: Artifact Rating
LO 4
LO2
LO 7
5
4.5
4
3.5
3
2.5
2
1.5
1
Fall 2007 Mean
Spring 2008 Mean
LO4
LO5
LO6
Learning Outcomes
LO7
iConnect Course Summary Ratings

Overall ratings for teacher and for course
5
4.4
4
3.2
iCON Course
Institution
IDEA
3
2
1
Excellence of
Teacher

Excellence of
Course
Student comments examined
 Faculty feedback sought
 Actionable
 Lessons
 New
Results
Learned
Directions
QUESTIONS?
THOUGHTS?
COMMENTS?