PROVOST OFFICE ACTION STEPS
PROGRAM ASSESSMENT
NOTE: The Provost’s Office is requesting that Faculty
Senators and Deans read the complete document for the
context of the changes. However, we are particularly
requesting feedback on the details associated with Action
Step #1 detailed on pp. 7-9.
March 10, 2016
Action Steps based on Recommendations in
May 2015 Biennial Assessment Working Group Report
Dr. Sandra Grunwald
interim Associate Vice Chancellor for Academic Affairs
Table of Contents
I.
II.
III.
I.
Executive Summary
Background of the Review of Programmatic Assessment Processes
Provost Office 2015-2017 Programmatic Assessment Initiatives
Executive Summary
Based on substantial review of processes and engagement with individuals highly invested in
programmatic assessment over several years, this report delineates the background and
components of the review, the areas identified for improvement, and several initiatives (with
action steps) in progress. Recommendations guiding the assessment initiatives associated with
the Provost’s Office for the 2015-2017 biennium are grouped into three key areas.
1. Recommendations for Increasing Communication and Knowledge
●
●
●
●
Establish an assessment sharing process that allows departments to benefit
from cross-disciplinary assessment efforts.
Articulate a shared conceptual framework and common language for
assessment.
Provide opportunities for academic programs to showcase program-level
assessment.
Make more public program assessment results
Action Item: Academic Affairs sponsored a university-wide assessment event at UWL on
January 20, 2016 -- Assessment Commons: Asking Questions, Sharing Practices, Finding
Answers. Approximately 180 faculty and staff attended the event.
2. Recommendation on Reward and Accountability Structures:
● Incentivize assessment through the Provost Office.
Action Item: In January 2016 the Provost Office put out a call for Program Assessment Initiative
Grant proposals to provide monetary support for gathering, analyzing, discussing and acting on
evidence of student learning in programs. Ten proposals were received from
departments/programs that spanned a range of level of program assessment development (as
self reported). Funding requests spanned proposals associated with department/program
retreats, conference attendance, and support for the development of assessment tasks,
evaluation rubrics or plans.
3. Recommendations on Consistency of Feedback:
● Create an Assessment Associates review group for a university-level, facultybased review process for program assessment reports.
● Develop a university-level program assessment report feedback rubric.
● Establish a common timeline to provide and receive feedback on program
assessment.
Action Items:
1. Consolidate review of program assessment to a university-wide, faculty-based program.
○ includes consultation with Faculty Senate, development of rubrics, timelines and
membership guidelines
2. Move to a 3-year program assessment cycle
3. Move program assessment reporting to Taskstream
1
II. Background of the Review of Programmatic Assessment Processes
In Spring 2015, Provost Macpherson led a comprehensive review of the program assessment
reporting and review process in place at UWL. College biennial assessment reports and
individual program/departmental biennial assessment reports were reviewed in addition to a
meeting dedicated to the topic on February 17, 2015 during which Deans, Associate Deans,
faculty leaders within the colleges, and the UWL Assessment Coordinator shared their views of
the program assessment process. From this comprehensive review, a Working Group on
Biennial Assessment was formulated and charged with providing recommendations in response
to the following guiding questions:
1. Does the timing of review and feedback of assessment within the academic programs
meet the needs of all parties involved?
2. Does the current structure of review and feedback of assessment within the academic
programs meet the needs of all parties involved?
3. If these conditions (both or just one) are not met, what substantive changes could be
taken to improve the process?
The working group surveyed department chairs and reviewed existing information on the
assessment sections of Academic Program Review reports to investigate these questions. In
their summary report submitted to Provost Macpherson in May 2015, the working group stated
Based on a careful review of the feedback from chairs, the working group identified
several barriers to effective biennial program assessment at UW-L:
● Defining assessment and associated expectations
o Understanding what program assessment is
o Unclear communication about program assessment
o Expectations not clearly articulated
● Faculty buy-in/ownership
o Faculty resistance to program assessment
o Limited time to engage in program assessment activities
● Failure to close the loop
o Lack of sufficient communication about the results of program
assessment
o Not using program assessment data to inform decision-making
The working group described the overall state of assessment as follows:
● There is a need to improve the overall culture of program assessment
● Many programs are struggling with program assessment (see Appendix B),
based on an analysis of the past 8 years of APR report program assessment
results
● There is a need for improved communication at all levels, as evidenced by
o Confusion between annual and biennial review (CLS)
o Dissonance between college-level and dean-level review and feedback
o Decentralized processes for program assessment - each college goes
about things a different way that lead to a common biennial report
2
The working group made the following four overarching recommendations to improve the
program assessment process, including specific action steps related to each recommendation.
1. Improve consistency of feedback regarding program assessment
a. Consolidate review process to a university-level faculty peer group.
Identify faculty to serve as Assessment Associates who complete the review of
the assessment reports. See the University of Northern Iowa model.
b. Develop a university-level program assessment report feedback rubric.
c. Establish a common timeline to provide and receive feedback on program
assessment.
2. Encourage increased communication and knowledge about program assessment
at all levels
a. Articulate a shared conceptual framework and common language for
assessment.
b. Communicate feedback about program assessment to entire department.
c. Establish an assessment sharing process that allows departments to benefit from
cross-disciplinary assessment efforts.
d. Clearly articulate the three levels of assessment - course, general education and
program - and the similarities and differences of each.
e. Develop and disseminate clear expectations for program assessment.
f. Clarify the roles of different individuals involved with assessment on campus.
g. Encourage attendance at ongoing professional development opportunities
provided by CATL to enhance the culture of assessment at UW-L.
3. Identify strategies to increase faculty buy-in
a. Incorporate program assessment into faculty promotion files.
b. Incorporate program assessment into digital measures headings.
c. Encourage departments to put assessment issues on agenda at departmental
meetings.
4. Establish a reward and accountability structure
a. Establish an assessment recognition process to identify and celebrate academic
programs for outstanding work in assessing student learning outcomes.
b. Provide opportunities for academic programs to showcase program-level
assessment.
c. Make programs more accountable for program assessment.
d. Incentivize assessment through the Provost Office.
e. Make more public program assessment results
3
III. Provost Office 2015-2017 Programmatic Assessment Initiatives
The Provost Office carefully reviewed the report and has subsequently developed and
supported several initiatives in the 2015-17 biennium that address recommendations within the
communication / knowledge and reward recommendation categories as described below.
Furthermore, thorough investigation into the improving consistency of feedback
recommendation has occurred and a plan has been formulated with action steps as described
below.
1. Recommendations for Increasing Communication and Knowledge
●
Establish an assessment sharing process that allows departments to benefit from crossdisciplinary assessment efforts.
●
Articulate a shared conceptual framework and common language for assessment.
●
Provide opportunities for academic programs to showcase program-level assessment.
●
Make more public program assessment results
Action Item: Academic Affairs sponsored a university-wide assessment event at UWL on
January 20, 2016 -- Assessment Commons: Asking Questions, Sharing Practices, Finding
Answers.
● In Summer and Early Fall 2015, Betsy Knowles (Economics), gathered feedback through
one-on-one discussions and follow-up surveys from a variety of stakeholders at UWL to
inform planning of an assessment day event. A summary of stakeholder input was
submitted to the Provost and informed the faculty-based committee that planned the
event.
● A Planning Committee made up of five faculty members spent several months
developing and carrying out the Assessment Commons event that was made up of a
variety of workshops, panel discussions and presentations that address current
concerns and needs of the university community with respect to assessment practices
(see full event program). The goals of the event were to 1) provide a forum for faculty to
share and collaborate on methods to measure and improve student learning, 2) to
improve understanding of assessment efforts within the context of the university,
including shared language, expectations, and vision, 3) provide an opportunity for
department members to work together on those aspects of assessment that are most
relevant to them.
● As part of the program, attendees were provided a Reference of Assessment Related
Terminology and Resources which included 1) a glossary of common assessment
related terms, 2) a glossary of UWL assessment related acronyms and terminology, and
3) references for two national assessment initiatives and the resources they provide.
These documents will be more broadly distributed.
● Approximately 180 faculty and staff attended the event. Initial feedback and reviews
show that the event was highly successful and very well received.
4
Follow-up: The faculty organizers would recommend offering this type of workshop on a
yearly basis.
2. Recommendation on Reward and Accountability Structures:
● Incentivize assessment through the Provost Office.
Action Item In January 2016 the Provost Office put out a call for Program Assessment
Initiative Grant proposals to provide monetary support for gathering, analyzing, discussing and
acting on evidence of student learning in programs. Ten proposals were received from
departments/programs that spanned a range of level of program assessment development (as
self reported). Funding was requested to support department/program retreats, conference
attendance, and faculty support for development of assessment tasks, evaluation rubrics or
plans.
Follow-up: Pending available funding, the Provost’s Office would like to continue and
refine some form of programmatic assessment grants.
3. Recommendations on Consistency of Feedback:
● Consolidate review process to a university-level faculty peer group.
● Develop a university-level program assessment report feedback rubric.
● Establish a common timeline to provide and receive feedback on program assessment.
Steps taken to involve campus community in the discussion of a consolidation of the
review process:
Step 1: Solicit feedback from Department Chairs
● At the Fall 2015 Chair’s meeting, the Working Group findings regarding program
assessment reporting and review were presented to department chairs, along with all
recommendations.
● Chairs were then asked to discuss and provide their feedback on the value and
challenge of the above three recommendations from the working group report.
● Feedback indicated that chairs found value in having a university-level, faculty peer
group doing program assessment report review especially in regards to 1) consistency
and 2) faculty-based. Several challenges that were identified were 1) finding faculty
with enough knowledge of program assessment to serve as reviewers and 2)
recognizing the programs that are outside-the-norm (i.e. DPI-associated programs,
CBA programs, Library)
Step 2: Video conference with University of Northern Iowa to learn about their universitywide, faculty-based program assessment report review process. Associate Deans,
Working Group faculty members, IRAP members and SEC members were invited to
attend. Also an open invitation was sent to each dean to invite others.
5
Video conference attended by
CLS: Charles Martin-Stanley, Linda Dickmeyer
CBA: Glenn Knowles, Laurie Miller, Betsy Knowles
SAH: Barb Bennie
IRAP: Natalie Solverson, Patrick Barlow
Provost Office: Sandy Grunwald
In discussion with faculty/staff who attended the videoconference, agreement was that such a
centralized process had value and that UWL should move forward with a similar process.
Step 3: Discussion of next steps at the October 2015 Dean’s Council Meeting
6
Action Item #1 - Create an Assessment Associates review group for a university-level,
faculty-based review process for program assessment reports.
Under
development
Detail
Suggested
Deadline
Composition of the
Assessment
Associates review
group
May 15
● Ten faculty members within various ranks,
representing the three academic colleges and SOE,
and also representing both graduate and
undergraduate education.
● Potential Assessment Associates names will be
gathered from the following: 1) an open call from the
Provost to all faculty asking for self-nominations, 2)
soliciting nominations from academic deans, 3)
soliciting nominations from academic chairs. It is
expected that potential Associates will be nominated
based upon their previous involvement in assessment
activities, preferably within the program area in their
department, and their support for assessment of
student learning.
● The Provost Office will review the nominations,
feedback will be solicited from Deans, SEC, etc… and
a final list formulated. The final Assessment Associate
list will be reviewed by the Deans and SEC for
affirmation.
● Committee terms will be three-years, with the first set
of terms being two or three years to provide continuity
in membership at the beginning of the process.
● The initial group of committee members will be named
by the first week of September.
● Training and guided practice in review of reports will
occur by mid-October.
Workload and
recognition of
Assessment
Associates
● It is anticipated that service as an Assessment
Associate will involve approximately 15-20 hours of
time during the fall semester.
● Assessment Associates will be expected to:
■ Be informed and knowledgeable with respect to
the review process;
■ Work to provide supported and supportive reviews
of program assessment reports;
■ Keep ratings confidential and discuss
reports/evaluations only with Assessment
NA
7
Associates or faculty involved with a specific
program area report.
● Assessment Associates should document their work
as a reviewer within Digital Measures to be considered
in both the merit and promotion process.
Work Expectation
● Each Assessment Associate will be assigned about six Oct 20
reports to review.
● There will be two reviewers for each report, one from
the college or the general program discipline.
● Each reviewer will have approximately one month to
review individually the assigned reports. Associates
will utilize a customized template within Taskstream to
conduct the review process. The template will include
ratings and opportunities for comments on each
required section of the report.
● After Associates have completed their assigned
reports, they will confer with the other reviewer for the
same reports, with the goal of creating a single
evaluation of the report for each program area
reviewed.
● Final evaluation reports are to be completed before
December 1st. Evaluation reports will be sent to the
department chair, program director, and dean.
Aggregate reports will be created for college and
university levels. These evaluation reports should also
feed into the centralized program review process.
They could be included as an important appendix
within a program’s 7-year APR self-study.
Development of a
university-level
program
assessment report
feedback rubric.
Need to develop review template within TaskStream
● See possible draft template. (Associates will be
involved in finalizing review template.)
Sept
●Perspectives of the review process will be collected
from departmental programs, Deans, and the
Assessment Associates.
●Assessment Associates will also meet in January to
discuss how the evaluation process worked and make
suggestions for improvement to the process for the
next academic year.
Jan 15
Evaluation of the
new process
8
Explore the need for
an exemption
process for
programs that are
externally
accredited and have
a periodic review
process already in
place.
TBD
May 15
Review of
assessment plans
Involvement of assessment associates in this process?
TBD
Action Item #2 - Move to a 3-year program assessment cycle
● Provides time for gathering at least two years worth of assessment data and then time
for programs to review results and implement action steps as needed.
● Timeline will be created such that certain “tasks” are completed at certain points within
the 3-year cycle to facilitate a full report at end of 3 years.
● At least two complete cycles would be completed within the 7-year APR cycle.
● Submission years of program assessment reports will be staggered so that all reports
are not due in the same year.
● Program Assessment reports will be due October 15th of the submission year (changed
from the current June 30th deadline). Feedback from chairs indicated that the October
submission date would greatly increase communication, feedback, and review of the
report by the whole department before submission (versus just the chair and perhaps
one other faculty member completing, reviewing and submitting the report).
Program Assessment Report review process
● Program assessment reports will be completed in Taskstream by October 15th of
submission year. Submission years will be staggered among programs with
approximately half of program reports due in 2016 and again in 2019, with the other half
due in 2017 and again in 2020. The initial submission year schedule will be based on 1)
the program’s APR schedule and 2) the level of development of the program’s
assessment (based off of the program’s last self study report).
Action Item #3 - Move program assessment reporting to Taskstream
● Taskstream interface will be created to capture information that is currently requested on
biennial assessment form.
● Benefit: Replaces paper/electronic copies of biennial assessment reports that are
currently uploaded into D2L Biennial Assessment site. Taskstream also facilitates the
review of program assessment reports by providing a central repository for reviewers to
access and a central place to provide and access comments.
Next steps to convert program assessment reporting to Taskstream.
9
January/February 2016 - Patrick Barlow completed development of Taskstream program
assessment portal.
February - Sandy Grunwald reviewed portal.
February/March - Group of five faculty representing CBA, CSAH, and CLS (Bonnie
Bratina, Laurie Miller, Glenn Wright, Terry Lilley, Kim Vogt) will be provided training
on the program assessment portal and will review and provide suggested changes
on direction language, etc… Necessary changes will be made per feedback
received.
February - Communicate with SEC regarding changes to process.
March - Deans will be provided update on process.
March 10 - Dept chairs, program directors and assessment coordinators will be advised
to 1) review their program SLOs that have been pre-inputted into the system and
make necessary changes, 2) input current curriculum map into system (if applicable),
and 3) input current program assessment plan into the system by April 1st. During
this communication a draft of the submission year schedule will be circulated for
comment.
March 14-25 - Patrick Barlow will provide four training sessions on the Taskstream
program assessment portal.
10