Practicing Meaningful
Learning Outcomes Assessment at UGA
Department of Crop and Soil Sciences
August 10, 2015
Dr. Leslie Gordon
Associate Director for Assessment
FAQs about Assessment
Why..
• are programs are required to
assess student learning?
• should all faculty be involved?
What..
• is expected for assessment of
graduate education?
Where..
• do you focus your assessment
efforts?
Two Views of Assessment
Ewell, P. (2009). Assessment, Accountability and Improvement: Revisiting the
Tension. NILOA Occasional Paper #1
http://www.learningoutcomeassessment.org/documents/PeterEwell_005.pd
Why is LOA required of your
program?
Why is LOA required of your
program?
SACS-COC Core Standards
Why is LOA required of your
program?
Why is LOA required of your
program?
Undergraduate, graduate and professional degree and
certificate programs
Why is LOA required of your
program?
UGA Academic Affairs Policy 4.12
http://provost.uga.edu/documents/Assessment_G
uidelines_revised_20140624.pdf
• Faculty-driven
• Comprehensive (grad, undergrad, certificate)
• Faculty-reviewed
 Evaluation as part of 7-yr Program Review
Why is LOA required of your
program?
UGA Academic Affairs Policy 4.12
http://provost.uga.edu/documents/Assessment_G
uidelines_revised_20140624.pdf
• Faculty-driven
• Comprehensive (grad, undergrad, certificate)
• Faculty-reviewed
 Evaluation as part of 7-yr Program Review
Why Is Faculty Involvement
Important?
Faculty are most qualified to
• influence the curriculum and its
enactment
• make meaning of assessment evidence
• use evidence of student learning to
demonstrate effective teaching
• ensure that assessment is integrated
with, not a burden upon, faculty activity
in teaching and scholarship
What is learning outcomes
assessment?
Assessment is the ongoing process of
• establishing clear, measurable expected
outcomes of student learning
• ensuring that students have sufficient
opportunities to achieve the outcomes
• systematically gathering, analyzing, and
interpreting evidence
• using results to improve student
learning
(Suski, 2009, p. 4)
What is learning outcomes
assessment?
Define
student
learning
outcomes
Report and
plan for
future
assessment
Enact
improvements
based on the
evidence
Choose a
measure and
set criteria
Do the
assessment
and analyze
the evidence
The Assessment Cycle
Best Practice Basics*
• Learning outcomes should be clear
(action-focused), measurable, and
student-focused
• Assessment measures should
– clearly address the degree to which students
attain the learning outcomes
– include direct evidence
– be independent from course grades and
evaluations
• Faculty involvement should be clear and
consistent
*See June 11th Presentation to Crop & Soil, which covered this
in more detail
Best Practice for Graduate
Assessment
The same principles apply!
• Graduate learning outcomes focus on
the knowledge, skill/ability, and perhaps
disposition of students.
• The measures should reveal evidence of
learning, not suggest that learning has
taken place.
• Faculty involvement should be broad
and consistent
What Measures to Use?
What is the Focus of Graduate
Education Assessment?
-Comprehensive knowledge
-Research technique
-Communication skill
-Professional behaviors
-Retention & graduation
-Exam pass rates
-Thesis / dissertation completion
-Course / program satisfaction
What Measures to Use?
When deciding what measures to use, first
answer the question
“Where will we see evidence that the
student is demonstrating x,y,z learning
outcomes?”
Where to Find Evidence of
Learning?
Some major approaches to measurement
selection:
• Direct and Indirect
• Formative and Summative
• Embedded and Add-on
Where to Find Evidence of
Learning?
Direct and Indirect Measures
Direct measures yield tangible, visible,
self-explanatory evidence of learning.
Indirect measures yield signs that
students are probably learning.
Where to Find Evidence of
Learning?
Formative and Summative Measures
Formative measures focus on “in
process” learning that can be used to
inform changes (to teaching, to student
behaviors, etc.).
Summative measures focus on the “end
state” of learning, that students attain
after the course or program.
Where to Find Evidence of
Learning?
Embedded and Add-on Measures
Embedded assessments are those that
are embedded into course work but that
also provide information for program or
institutional goals.
Add-on assessments collect information
about student learning independently
from course/program assignments.
Your turn!
Identify the approach(es) for each of the
measures below.
A. Direct B. Summative C. Embedded
1. Exam questions in research methods course
2. Discussion with advisor/major prof. about
literature review
3. Dissertation defense
4. Alumni survey
Where to Find Evidence of
Learning?
Teaching
skills
Job
placement
Engagement,
or Service
Student
Learning
Assessment
Grade
distributions
Scholarship
Retention
Where to Find Evidence of
Learning?
Teaching
skills
Job
placement
Engagement,
or Service
Student
Learning
Assessment
Grade
distributions
Scholarship
Retention
Indicators of success: Related, but not equal to student learning
Recommendations Regarding
Scope and Manageability
• Focus on the essential learning outcomes
of the program
• Assess what is directly under faculty
control (monitor the rest)
• Adjust assessment as dictated by results
or disciplinary demands
What is required for LOA reporting?
Documents
Submitted by
Assessment Plan
All degree and certificate
programs
Brief Report on
Learning Outcomes
Assessment
Undergraduate degree and
certificate programs only
Full report on Learning Undergraduate and graduate
Outcomes Assessment
degree and certificate programs
See also “Program Review Outline for Evaluation of LOA”
What is required for LOA reporting?
The Assessment Plan
Each academic unit should establish and
regularly maintain an assessment plan.
The assessment plan documents a number
of core elements that are defined by the
faculty who teach in the program.
What is required for LOA reporting?
The Assessment Plan
• Defines the learning outcomes for undergraduate
students in your program (usually 4-6).
• Describes the methods the department will use to
collect data that measure the attainment of the
learning outcomes (usually 4-6).
• Describes the methods the department will use to
analyze data that reflect the attainment of the learning
outcomes.
• Describes how faculty will be involved in the various
steps of the assessment process.
• Describes how the department will use information
gathered from the analysis to improve its assessment
practice and, more importantly, student learning.
What is required for LOA reporting?
Examples of assessment plans:
http://oap.uga.edu/assessment/plan/
What is required for LOA reporting?
The Full Report includes
• information on ALL of the program's learning
outcomes
• faculty analysis of the data
• improvements enacted or planned as a result of
that analysis.
• The full report is submitted in the third and
sixth years.
– Graduate programs are asked to submit
only full reports.
– Undergraduate programs will also submit
the full report (years 3 and 6) in addition to
the brief reports (years 1,2,4,5).
What is required for LOA reporting?
Examples of assessment reports:
http://oap.uga.edu/assessment/plan/
What is required for LOA reporting?
Upload plans and reports to the Academic
Planning System (APS) https://aps.uga.edu/APS/
This system houses
• Learning Outcomes Assessment plans and reports
• Program Review self studies
• Strategic plans for your unit/department,
• Annual Reporting documents for schools and colleges.
Access to APS is granted by request. Contact Leslie
Gordon, Associate Director for Assessment, at
gordonls@uga.edu if you think you need access.
When do you report?
October 1st
Annual reporting deadline for all
programs, any type of report.
Spring reminder issued to deans,
academic associate deans, department
heads, assessment coordinators
When do you report?
When do you report?
Assessment
cycle
Items due
Submitted by
Years 1-2
Brief Report on Learning
Outcomes Assessment
Undergraduate degree
programs only
Year 3
Full report on Learning
Outcomes Assessment
Undergraduate and
graduate degree
programs
Years 4-6
Repeat cycle described above
Year 7
Academic Program Review
Assessment reports are
concurrently evaluated by a
Undergraduate and
faculty subcommittee of the
graduate degree
Program Review and Assessment
programs
Committee (PRAC) – evaluation
and recommendations
produced
Who reviews LOA practices?
Subcommittee of Program Review and
Assessment Committee (PRAC)
• 10-12 members
• Work is focused exclusively on LOA
• Follows evaluation criteria drawn from
commonly accepted LOA best practices
• Issues report and recommendations
See also “Sample LOA Evaluation Report”
Where can you find help?
Maki, P. and Borkowski, N. (2006)
Banta, T. Jones, E. and Black, K. (2009)
Where can you find help?
Dr. Leslie Gordon
Associate Director for Assessment
•
•
•
•
•
Assessment (re)design consultation
Informal evaluation of LOA
Department/College presentations
Interactive workshops
Classes and Faculty Learning Communities
And more! Begin your assessment redesign now!
Where can you go for help?
Where can you find help?
OAP website > Student Learning
Outcomes Assessment
http://www.oap.uga.edu/assessment/sloa/
Questions?