Rethinking the Assessment Process: A Memo to Departments and Programs
September 2, 2015
To: All academic departments and programs
From: The Colorado College Assessment Committee
We write to explain changes we are making in the biennial cycle of assessment reports we have
been using for the past six years. Currently, the Assessment Committee receives an assessment
report every two years from each department and program with a major. The schedule is based
on the assumption that departments carry out an assessment project every year. However, we
have learned that only some departments actually carry out assessment in their non-report years,
and the current process does not enable them to engage in useful assessment. In order to help you
more effectively scaffold your assessment process the Assessment Committee, with the support
of the Dean of the College/Dean of the Faculty, proposes that a simpler, yet potentially more
effective four-semester assessment process replace the current biennial reporting structure:
Semester
First
Task
Plan an assessment project based on a learning situation of
interest
Short Report Due*
2/1 of semester 2
Second
Gather information about current student learning
7/1 after semester 2
Third
Evaluate and reflect on findings from semester 2
2/1 of semester 4
Fourth
Implement changes based on evaluation, write last report
7/1 after semester 4
First
Plan an assessment project; can repeat prior cycle or not
2/1 of semester 2
* While this process is discussed further below, please note that the short reports in semesters 13 are to be no more than a page and should not take much time to write if the
department/program has carried out its semester task. The report in semester 4 is 1-2 pages with
additional attachments that already exist.
Semester 1: PLAN: Identify a student learning situation in the department/program in which
there is room for improvement, and identify the means by which to investigate it
 This semester is used for planning and discussion. Although information may not be gathered
until late in the second semester, detailed planning occurs during the first semester.
 By February 1 of semester 2 provide the Assessment Committee with a brief update (one
page at most) indicating what situation will be studied as well as the method of information
gathering.
Semester 2: GATHER: Gather relevant information about the nature of the situation
1


This semester is used for design and administration of information-gathering tools, materials,
or instruments. This may include design of specific assignments or the development of
rubrics that can be used to evaluate assignments once in hand.
By July 1 at the end of year 1 provide the Assessment Committee with a brief update (one
page at most) indicating the process that has been used to gather information; departments
and programs are invited to attach specific assignments or rubrics to this update if they wish
but it is not necessary to do so. (Such assignments and rubrics will be included with the
report at the end of the fourth semester regardless of whether they are included at this stage.)
Semester 3: EVALUATE, REFLECT, ADJUST: Review and reflect on the gathered findings
 This aspect of assessment has traditionally been one of the hardest to do well because of time
crunches. By assigning an entire semester to considering the findings from the gathered
information, the Committee hopes that this reflection can become a more thoughtful process,
rather than being rushed.
 During this semester, the department/program also proposes and prepares to implement
changes that should address the situation in need of improvement.
 By February 1 of semester 4 provide the Assessment Committee with a brief update (one
page at most required; departments and programs can elect to write slightly more if that’s
easier) indicating the relevant findings as well as proposed changes for improvement.
Semester 4: IMPLEMENT CHANGES: Implement changes
 Ideas for programmatic changes (curricular, pedagogical, or both) are put into place during
this semester as possible; this can be done if the information was gathered from a spring
semester project or process.
 The assessment report will include a description of the changes being implemented, thus
completing all aspects of the assessment cycle. If possible, include any ideas for the
beginning of the next cycle.
 The assessment report, due by July 1 at the end of year 2, would be restructured to reflect the
new process, including the three updates from the first three semesters and introducing a
fourth brief (1-2 pages) report about how the semester 4 change implementation went.
Materials traditionally requested with assessment reports such as curricular goals, learning
outcomes, and rubrics would be attached as appendices but would not need to be rewritten
from prior reports unless they had changed. Departments and programs would receive
information and training on how to write the modified reports (as discussed below).
This two-year cycle begins again in semester 5.
NOTE: We assume that many departments and programs will be able to carry out this process
over the course of two academic years, starting in the fall of the first year and concluding at the
end of the spring semester of the second year. However, we can imagine situations in which a
department wants to gather its information in a fall semester or over the course of multiple
semesters, and our intent is to develop processes to work individually with such departments to
determine how to modify their two-year cycles to enable them to do the assessment that is most
meaningful to them.
A comparison of the old and new calendar and reporting structure is presented on the next page:
2
Old Calendar
New Calendar
Assessment was to be done consistently but often
is not; two years’ work done in a year or less
Some work done every semester; no “jamup” at the end of the second year
Old Report Structure
New Report Structure
Many required sections; major writing
commitment
Brief reports each semester; 4th semester
report is short; additional materials
appended from earlier documents
This proposed process emerged out of conversations with faculty members who are generally
supportive of assessment but who feel that the most important parts – the reflection and
modification components – do not currently receive enough time to be treated in any but the
most cursory fashion, rendering the entire process far less useful.
The new process would require some work in every semester as well as small additional updates,
but we believe stretching out the process would make it better in a variety of ways:
 It would get us out of the cycle of the “year off that shouldn’t be a year off”/”year on that
ends up requiring two years of work”, which is unsustainable;
 It would make it easier for assessment representatives to bring their departments into
conversation more effectively so that assessment stops being something one person does in
spare time she or he does not actually have;
 It would ultimately make assessment, which we have to do anyway, more useful and
meaningful since a structure would be set up with the express purpose of allowing enough
time to do assessment well;
 While reports would be due more often they would be very brief and would always involve a
lead-up workshop sponsored by the Crown Faculty Center on how to write the report well;
 The new structure would encourage departments to keep their own assessment archives to
maximize ease of the process (though the Assessment Committee would continue to provide
requested earlier documents as possible).
The Assessment Committee would work with departments and programs to support this change,
including developing new materials, offering educational luncheons for all departments and
programs to roll out the approach in fall 2015, and providing workshops once or more during the
semester including a workshop the month before each report is due in order to go over format
and necessary information. An appendix follows this document, demonstrating sample semester
reports for the Colorado College “Department of Criminology” (we felt an imaginary department
might be more useful for this example) so departments and programs can have an immediate
sense of what their reports might look like under the new system.
Departments and programs that turned in assessment reports in summer 2015 would begin the
process in fall 2015, which would serve as semester 1. Departments and programs that have
assessment reports due in summer 2016 would receive individualized attention enabling them to
enter the process in the middle of it and catch up as needed.
3
One other change that would accompany the roll-out of this new schedule and process is that
department/program assessment representatives would be expected to remain in that role for the
entire two years
Please contact Jim Parco, Assessment Committee Chair, or Amanda Udis-Kessler, Director of
Assessment and Program Review, with questions or concerns.
“Assessment is thinking about what you’re doing and doing it better.” – John Riker
Our philosophy:
 While assessment has ties to external entities to which the college is accountable, we carry
out assessment because it helps us become a better educational institution.
 Assessment builds meaningful inquiry and conversation about improving student learning;
our assessment processes prioritize such dialog.
Appendix: Example “Department of Criminology” Reports
Colorado College Department of Criminology Semester 1 Report: Identifying a Student
Learning Situation in Need of Improvement
The Colorado College Department of Criminology has determined that its students, though able
to describe the difference between postmodern, Marxist, feminist, and symbolic interactionist
paradigms of criminology in simple terms, are unable to apply their understanding of these
paradigms to a situation in which they are interpreting new criminological information. That is,
students cannot take a description of a crime situation (qualitative) or a crime data set
(quantitative) and determine how criminologists of different theoretical stripes would make sense
of either the situation or the data set to a sufficiently sophisticated degree.
As this capacity represents several important learning outcomes to be demonstrated at the
capstone level, the Department has determined that it needs to develop an exam that will
demonstrate more clearly how well different aspects of theory application are being learned and
whether there is any clear way to tease apart the theories or theory elements that are easier or
harder for students so that the Department can either modify its pedagogical approach to the
material or change the curricular coverage, or both.
The Department is currently in the middle of developing the exam, which will use both direct
and indirect assessment to gather information about its seniors. The exam will be given during
the block 7 senior seminar, approximately one block before students present their capstone
reports.
The exam will provide several descriptions of crimes and several abbreviated crime data sets and
will ask students to describe in some detail how criminologists of postmodern, Marxist, feminist,
and symbolic interactionist positions would interpret each situation or data set. This part of the
exam is not significantly different from prior assignments, though the questions will be designed
to elicit a new level of detail in the responses. The new aspect of the exam, the “indirect
4
assessment” part, will ask students how difficult it was to answer the questions, which theoretical
perspectives they find most difficult, and other such questions, which may help us understand
their thought processes as well as their knowledge. We will also ask which of their Criminology
courses covered theoretical perspectives, as we are concerned that they may only be covering
this material in the department’s one theory course, a course taught in multiple ways by different
professors.
Colorado College Department of Criminology Semester 2 Report: Gathering Information
As planned last semester, we developed a new exam with the specific intent of learning both how
much our students knew about several theoretical perspectives on criminology and how they
understand their own knowledge of the subject. The exam was given to all 12 senior majors in
the second week of the block 7 senior seminar; we noticed that most students appeared to take
the exam very seriously and took a long time on it. While the exams have been graded and
returned, we asked the students for permission to retain copies for assessment purposes; all 12
students assented. A copy of the exam follows along with the rubric used to determine ability to
apply criminological theories to new situations and a second rubric that addresses student selfunderstanding of their knowledge in these areas. [Since this is an example neither an exam nor a
rubric is actually attached.]
Colorado College Department of Criminology Semester 3 Report: Reflecting on Findings
With rubric in hand, a review of the senior seminar exams enabled us to learn the following:
 Different students have very different levels of understanding of the four criminological
theories of interest and their ability to apply the theories varied not just from theory to theory
but from student to student; only two of the twelve students applied all four theories with
what we would consider sufficient proficiency.
 Students also differed profoundly in their self-understanding regarding this knowledge. Some
who did poorly on the exam expected to have done well; others who did reasonably well
were dissatisfied with their performance.
 Students reported that all four of these theories were only covered in the one theory course,
and covered very differently especially in terms of the amount of coverage.
 Student success with applying the theories generally went as follows:
o Symbolic interactionist: moderately successful with the crime situation, unsuccessful
with the data set
o Postmodern: unsuccessful with both the crime situation and the dataset
o Marxist: moderately successful with both the crime situation and the data set
o Feminist: moderately successful with both the crime situation and the data set
Overall, students were weakest with postmodern criminological theory and with symbolic
interactionism used to make sense of a data set (the latter of which is, admittedly, a harder task).
While we would like our students to be even stronger in the areas where they were mostly
moderately successful, we believe the order of addressing our concerns must be as follows:
 First, strengthen our teaching of postmodern criminology in the theory course and make sure
it is being covered in other courses as well
5



Second, strengthen our teaching of symbolic interactionist theory in regards to data sets in
the theory course and make sure this is also taught in multiple courses
Third, change the way we teach the theory course so that students receive the same coverage
of material regardless of who’s teaching (using the same text book or readings was the most
commonly mentioned suggestion)
Fourth, when the above issues have been addressed, refocus assessment to see how to
improve student learning regarding the application of Marxist and feminist criminology
After some discussion we have decided to implement the following change this coming spring
and assess it as part of next year’s assessment process:
 We have found a textbook that, while not particularly weaker in Marxist or feminist theory,
looks more productive in teaching students how to both understand and apply postmodern
and symbolic interactionist criminology; all of us teaching the theory class will use this book
and in the same way, starting in the spring.
 We have located a book on applying criminological theory with different chapters on
different theories. We will assign several chapters from this book in the theory course.
 We have identified two 300-level courses and one 200-level course where we can introduce
criminological theory starting this spring. Professors teaching those courses have committed
to developing lectures, finding introductory reading materials, and creating at least one
assignment per course.
Colorado College Department of Criminology Semester 4 Report: Implementing Changes
Spring semester was very productive in terms of implementing the changes we reported planning
in our last document to the Committee. Fifteen students took Criminological Theory in block 5;
we used the new textbook and assigned several chapters from the new book on applying
criminological theory. The students were very engaged in the course and did better overall on the
four theoretical areas of interest to us (while we did not do rubric-based assessment for the
course we noted how students did on exams and assignments). Here is our sense of how the new
materials helped with the areas of concern:
 Students were at least as strong this year as in years past on Marxist application and slightly
stronger on feminist application; we think the latter was because of the secondary book,
which included a chapter on using feminist criminology in database research.
 Students were about as strong this year as in years past on symbolic interactionist application
of a crime situation and were notably stronger on using symbolic interactionism to make
sense of a data set. Here, we suspect that a “toolkit” page in the textbook on this topic helped
students improve.
 Our greatest sense of success came with the topic of postmodern criminology. The same
author wrote a subsection on postmodernism in the textbook and wrote the postmodernism
chapter in the applications supplementary book, and the two works were extremely useful in
tandem.
 We did not include any indirect assessment on the assignments in the theory course but noted
in the course evaluations that students almost all found the readings “very helpful” and
agreed or strongly agreed that “the assignments allowed me to demonstrate my understanding
of the readings.”
6
We also introduced both the idea of criminological theory and all four theories discussed in these
assessment reports in three additional courses this spring: 206, Introduction to Criminology
(second of two blocks, which was in block 6), 320, White Collar and Commercial Crime (also
block 6), and 380, Juvenile Delinquency (block 8). We included small assignments to determine
whether students in these three courses could articulate what a criminological theory was as well
as whether they could explain the four theories of interest in simple terms. Students were
overwhelmingly successful in both types of assignments. We also carried out exercises in all
three classes to provide a short introduction to the idea of applying criminological theory to
everyday crime situations; in Juvenile Delinquency we carried out an additional in-class exercise
in which students were given a small data set and asked whether they could read the data set
from the four different theoretical perspectives. As expected, the Marxist and feminist
perspectives were easiest for students, followed by symbolic interactionism, followed by
postmodernism. We believe that even this introduction will be helpful later, when students from
these classes are seeing the materials for the second time.
We are still thinking about whether to continue the assessment process from the past two years as
we start our next assessment cycle in the fall. We will certainly continue the changes we made to
Criminological Theory, which is offered blocks 3 and 6 next year. We will also continue the
changes we made to 206, 320, and 380. At this point we see three possible options moving
forward:
 We could maintain this project and see whether next year’s senior seminar work has
improved compared to the work of this year and years prior. (We did not assess learning in
this past year’s senior seminar because most of the students in it took theory as juniors,
before we changed the course.)
 We could refocus our attention on improving understanding of and capacity to work with
Marxist and feminist criminology – not because the students are doing poorly now but
because they could be doing better.
 We could change our focus and use the fall to consider whether there are any other areas
where student learning is weaker than we would like and develop an entirely new plan for the
next two years.
We are leaning toward the first option, though we may consider both options one and two if we
can pursue them without adding much work.
Attached as appendices please find:
 Our department’s history of assessment
 Our departmental mission statement
 Our curricular goals
 Our learning outcomes
 Copies of any assignments used for assessment
 Copies of any rubrics used in assessment
 A curricular map, showing which courses our learning outcomes are taught in and how
extensively
7