Academic Assessment at UTB
Steve Wilson
Director of Academic Assessment
Academic Assessment
Assessment of Student Learning Outcomes,
Non Instructional Outcome Assessment,
Delivery Method Assessment,
5-Year Program Reviews, and
General Education Program Assessment
Now fall under Academic Affairs and is coordinated
by the Director of Academic Assessment (see
handout)
OUTCOME ASSESSMENT
SACS COC Principles of Accreditation
Requirements 3.3.1.1 requires programs to:
1. Identify expected outcomes
2. Assess the extent to which it achieves these
outcomes
3. Provide evidence of improvement based on
analysis of the results
SACS COC also requires assessment to be
continuous and on-going. It should never stop.
UTB, therefore, will need to show that programs
are striving for continuous improvement up to
the moment it no longer exists and is merged
with UTPA into UTRGV
UTB Definition
Program SLO Assessment is the systematic and
ongoing process of gathering and interpreting
information to discover if programs are meeting
intended outcomes and then using the
information to verify good practices and
enhance or improve student learning.
It occurs in Academic
Yearly Cycles
It has three Steps
• Planning
• Analysis
• Closing the Loop
Planning
Analysis
Loop
Closing
Planning
1. Identify an Assessment
Liaison and/or
Assessment Committee
2. Faculty meeting to
discuss program
mission,
knowledge/skills/values
graduating students
should have learned
3. Faculty should then
develop Student
Learning Outcomes or
SLOs
SLOs are expectations that
describe the knowledge,
skills and values that
students should display
when they complete your
program
SLOs should be realistic,
detailed, measureable, and
should use action verbs
developed from Blooms
Taxonomy.
4. Bring faculty together
to develop a curriculum
map, showing how course
level SLOs are aligned to
program level SLOs
Allows faculty to indicate
which courses support the
achievement of specific
outcomes as well as
where embedded
assessment can occur
5. Develop Assessment
Methodology and
Timelines
• Includes how each
outcomes will be
assessed (embedded
assignments/test, exit
exam, portfolio, survey
of students).
• Benchmarks for success
• Not all outcomes have
to be assessed every AY
– cyclical.
• Use multiple
approaches; don’t rely
on just an exit exam.
• Be comprehensive not
superficial.
6. Report: Form A on SharePoint Site
Data Analysis
1. Collect and Analyze Data
Quantitative: by reporting tallies,
percentages, or sophisticated
statistical results
Qualitative: by reporting
summaries of minute papers,
reflective writing, or open-ended
surveys.
Results should be examined in a
manner that can provide feedback
by verifying what worked well and
identifying areas that need
improvement.
2. Report: top section of Form B on SharePoint
Site
3. Develop an
Improvement Plan
• Faculty meeting at the
end of the academic
year to discuss the
results of the analyses
and reflect on ways to
improve student
learning during the next
academic year cycle.
4. Report AY 2013-2014 Continuous Plan on
bottom section of Form B
AY 2014-2015 Exception
Normally improvement plans are completed on second
section of Form B
Because the Merger will take place in 8/2015, AY 2014-2015
Continuous Improvement Plans are not necessary. However,
to wrap UTB assessment up into a nice package, we want
departments to write a “lessons learned” report, based on
reflections of the 2014-2015 assessment findings.
• Form will be available on SharePoint Site
Closing the Loop
1. While the improvement
plan is being
implemented, programs
should collect data to
assess whether the plan
worked, effectively
improving student
learning in those areas
identified during the
analyses.
2. Report Form C on SharePoint site
You WILL need to submit a Form C to close the
AY 2013-2014 loop
You will not close the AY 2014-2015 loop
School/College Assessment Committee
Each department Assessment Liaison will be part of
a college level assessment committee. The
Associate Dean will chair the committee. The
committee will meet once a month to discuss
assessment related activities, including SLO
assessment, to ensure sufficient progress is being
made to meet the appropriate deadlines, and to
report any issues or problems to the relevant
officers, like the Dean of the College, Director of
Academic Assessment or the Executive Director of
the ORIPE.
9/2014
10/2014 to 4/2015
5/2015
Close the AY 2013-14 loop
AY 2014-2015 Lessons Learned
Report
Faculty meeting to discuss AY 20132014 analyses, the AY 2013-2014
Improvement Plan, and the AY 20142015 Assessment Plan
Action Steps
Faculty meeting to discuss AY 20142015 analyses , if the AY 20132014Improvement plan was
successful , and lessons learned from
the 2014-2015 analysis
Collect Data and Analyze for AY 2014-2015 Assessment Plan
AY 2014-2015 Analysis Report
AY 2013 – 2014 Improvement Plan
AY 2014-2015 Assessment Plan
Deliverables
8/2014
Collect Data and Analyze for AY 2013-2014 Improvement Plan
6/2015
NON-INSTRUCTIONAL
GOAL ASSESSMENT
UTB Definition
Non-Instruction Goal Assessment is the systematic
and ongoing process of gathering and interpreting
information to discover if academic departments
are meeting their intended goals and then using the
information to verify good practices and enhance or
improve services.
Note: This is a Departmental Level Assessment, not
a Program Level Assessment
It occurs in Academic
Yearly Cycles
It has three Steps
• Planning
• Analysis
• Lessons Learned
Planning
Analysis
Lessons
Planning
1. Chairs or Departments
should ID
2-3 short term goals that
need improvement or
attention
2. Normally, goals are
aligned to the University’s
strategic plan – we don’t
have one
3. So 2014-2015 goals
should be aligned to:
• Chancellor’s Framework
for Excellence
• Texas Gov’t Code 2114:
Customer Service
• Improvement of Dept.
Effectiveness/Efficiency
Measures
• Your own value (e.g.
items identified in
Program Review)
SMART Goals are:
Specific (Describe exactly what will be
accomplished)
Measureable (Success can be assessed)
Actionable (Action steps can be implemented to
achieve goal)
Realistic (The endpoint is achievable)
Timely (Can be done in one academic year)
2. Once goals have been identified,
Chairs or Departments should:
• Plan achievable action steps
• Determine how each goal will be
measured to determine success
• Identify the criterion (or target)
that will be used to determine
success
• Identify who is responsible for
overseeing the assessment of
the goal
• Identify required resources
3. Report Form A on SharePoint Site
Data Analysis
1. Collect and Analyze Data
Quantitative: by reporting tallies,
percentages, or sophisticated
statistical results
Qualitative: by reporting
summaries of minute papers,
reflective writing, or open-ended
surveys.
Results should be examined in a
manner that can provide feedback
by verifying what worked well and
identifying areas that need
improvement.
2. Report top section of Form B on SharePoint
Site
UTB/UTPA Merger Exception
Normally you would complete the improvments
and closing the loop steps
Because the Merger will take place in 8/2015, AY
2014-2015 Continuous Improvement Plans are not
necessary. However, to wrap UTB assessment up
into a nice package, we want departments to write
a “lessons learned” report, based on reflections of
the 2014-2015 assessment findings.
School/College Assessment Committee
Each department Assessment Liaison will be part of
a college level assessment committee. The
Associate Dean will chair the committee. The
committee will meet once a month to discuss
assessment related activities, non-instructional goal
assessment, to ensure sufficient progress is being
made to meet the appropriate deadlines, and to
report any issues or problems to the relevant
officers, like the Dean of the College, Director of
Academic Assessment or the Executive Director of
the ORIPE.
10/2014 to 4/2015
5/2015
Close the AY 2013-14 loop
AY 2014-2015 Lessons Learned
Report
9/2014
AY 2014-2015 Analysis Report
AY 2013 – 2014 Non-Instructional Goal
Assessment Plan
Deliverables
8/2014
Chair should identify about 3-5 shortterm goals that need
improvement/attention
Action Steps
Chairs should reflect on lessons
learned from the 2014-2015 analysis
Collect Data and Analyze for AY 2013-2014 Non-Instructional Goal
Assessment Plan
6/2015
COMPARING DELIVERY
METHODS
SACS COC requires Universities to compare
distance and other delivery methods to campusbased courses as part of their program
assessment, including assessments of student
learning outcomes
It occurs in Academic
Yearly Cycles
It has three Steps
• Planning
• Analysis
• Lessons Learned
Planning
Analysis
Lessons
Planning
1. Chairs should appoint
an Assessment Liaison.
Given the large amount
of work required by
assessment, it is
recommended that the
Liaison chair a
departmental
assessment committee
so that the work is
manageable.
2. At the beginning of
each academic year, the
assessment liaison and/or
assessment committee,
working with department
chair, will identify courses
that deliver the same
content using different
delivery methods.
3. The assessment liaison
and/or assessment
committee will then
coordinate with the
instructors of those courses
to identify student work that
can be used to assess
student attainment of
PROGRAM outcomes as well
an assessment methodology
and benchmarks.
PROGRAM
LEARNING
OUTCOMES
3. Report
Data Analysis
1. Collect and Analyze Data
Quantitative: by reporting tallies,
percentages, or sophisticated
statistical results
Qualitative: by reporting
summaries of minute papers,
reflective writing, or open-ended
surveys.
Results should be examined in a
manner that can provide feedback
by verifying what worked well and
identifying areas that need
improvement.
2. Report
UTB/UTPA Merger Exception
Normally, Delivery Methods Assessment would
finish with improvement and closing the loop steps.
Because the Merger will take place in 8/2015, AY
2014-2015 Continuous Improvement Plans are not
necessary. However, to wrap UTB assessment up
into a nice package, we want departments to write
a “lessons learned” report, based on reflections of
the 2014-2015 assessment findings.
School/College Assessment Committee
Each department Assessment Liaison will be part of
a college level assessment committee. The
Associate Dean will chair the committee. The
committee will meet once a month to discuss
assessment related activities, including delivery
method assessment, to ensure sufficient progress is
being made to meet the appropriate deadlines, and
to report any issues or problems to the relevant
officers, like the Dean of the College, Director of
Academic Assessment or the Executive Director of
the ORIPE.
10/2014 to 4/2015
5/2015
Close the AY 2013-14 loop
AY 2014-2015 Lessons Learned
Report
9/2014
AY 2014-2015 Analysis Report
AY 2013 – 2014 Non-Delivery Method
Assessment Plan
Deliverables
8/2014
Faculty will identify courses for
assessment, methodology, and
benchmarks
Action Steps
Chairs should reflect on lessons
learned from the 2014-2015 analysis
Collect Data and Analyze for AY 2013-2014 Non-Instructional Goal
Assessment Plan
6/2015