CHPA Departmental Assessment Reports
Criminology Assessment Plan:
In 2012, the Department of Criminology and Criminal Justice established six learning objectives (SLO)
and began to establish the methods by which we would collect data to analyze how well our SLO’s were
being met. In the paragraphs below, the SLO will be listed followed by the data collection method and
designated courses for data collection, data collected and analysis of the data.
SLO 1. Students will demonstrate academic proficiency in the core criminal justice areas (law
enforcement, law and corrections).
Historically, our department used the MFT to measure mastery of content knowledge. As a department,
we determined that the MFT might not be the best measure of content mastery for our purposes. We
determined that we would create our own test and began constructing this in the fall semester of 2012.
Questions were created by instructors responsible for various content areas and sent to the department
head who compiled them into a 100 question exam. We also determined that we would use this as a
pre-test in our Introduction to CJ course. Because we offer our undergraduate degree online, we also
had to create the online version of the test. The exam was given to all sections of CRM 210 and CRM
598 at the end of the Spring 13 semester.
Through the initial trials of the exam we have determined that our measure may not be a valid test of
our student’s learning and that there may be some problems with the exam given to the online
students. A committee was formed in the fall 2013 semester to address these and other issues and to
include identifiers on the test so that we can compare individual tests. We hope to have the revised
exam available for use in the fall 2014 semester.
SLO 2. Students will be able to communicate effectively, orally and in writing, using appropriate
references and technologies.
Our major assessment efforts were directed at gaining consistency in the assignments that are
designated for particular courses. Additionally, we are still working on the creation of standardized
rubrics to be used for targeted assignments.
A course release was given to Mike Ramon for fall 2013 to design an online module that will be used in
all Intro classes (and in the first MSU class for all transfer students). This online module addresses issues
related to criminal justice writing. Students are tested and will have to score 80% or higher in order to
pass. Passage of the module will be linked to passage of the course in which the module is offered. The
pilot run of the module occurred in the spring 2014 semester in one section of CRM 210. Data have
been collected and will be analyzed in summer 2014.
SLO 3. Students will be able to assess the basic quality of research in criminology and criminal justice
publications and other media.
1
College of Humanities and Public Affairs | Missouri State University
Two courses (CRM 340 – Research Methods [a literature review] and CRM 598- Senior Seminar [a policy
analysis paper]) which have substantial writing components have been selected for assessing this SLO.
Currently, scores for the papers are being collected to analyze. Beginning in fall 2014, papers from each
section of the courses will be selected for assessment based on standardized rubrics. Faculty members
teaching these courses have been assigned to create the standardized rubrics to use. Papers will be
assessed during the summer of 2014.
SLO 4. Students will be able to evaluate ethical issues related to the criminal justice system and
criminology.
The primary sources for assessing learning in this area are the exams given in the Senior Seminar (CRM
598). Some instructors used discussion boards to assess student awareness of ethical issues; others
used role playing and essays. We are working toward greater consistency in this area and to identify
the type of data we will collect for analysis
SLO 5. Students will identify issues of diversity and human rights in relation to the workings of the
criminal justice system and criminology.
The primary sources for assessing learning in this area are the exams given in the Senior Seminar. Some
instructors used discussion boards to assess student awareness of cultural differences. Training manuals
created by the ABA were implemented in two sections. The ABA program incorporates a number of role
playing scenarios to heighten awareness of cultural differences and to stimulate discussion of various
ways to increase competency. We are working toward greater consistency in this area.
SLO 6. Students will employ critical thinking skills when evaluating issues in criminology and criminal
justice.
We do not have a specific measure for this. We know that we are encouraging critical thinking across all
of our courses, but our primary focus has been on improving student writing under SLO 2.
Economics Department Assessment Plan:
The ECO Assessment Plan which was developed in 2011 and 2012 was implemented in full for the first
time in CY 2013.
1. Discussion of Assessment Methods
1.1 The Test of Understanding in College Economics (TUCE)—Outcomes 1 & 5
Starting with spring 2012, we are using a new (the newest) version of TUCE produced jointly by the
National Council on Economic Education and the American Economic Association. In contrast to the
previous version, the results are now reported in percentiles for the Microeconomics and
2
College of Humanities and Public Affairs | Missouri State University
Macroeconomics test separately. A combined percentile score is not available. The test is administered
in the spring semester to all students registered in ECO 590 (Senior Research Seminar). This is an
appropriate population since these students are close to completing their undergraduate studies and
are typically expected to graduate within the calendar year (with most graduating in the spring). The
test is offered in two parts—Micro and Macro. The score on the Micro part is used to assess SLO #1,
while the score on the Macro part is used to assess SLO #5. The results for Spring 2013 are summarized
below in Table 1. Nineteen students took the test and the number and percentage of students scoring in
different percentile ranges are indicated in the table.
TABLE 1
SUMMARY OF SPRING 2013 TUCE RESULTS
Percentile Micro
Macro
90-99
13 (68%) 13 (68%)
80-89
4 (21%)
2 (11%)
70-79
1 (5%)
0
60-69
0
3 (16%)
50-59
1 (5%)
1 (5%)
40-49
0
0
The overall performance on both the Micro and Macro sections of the test was excellent. The
percentage of students scoring in the 90-99 percentile range was 68% for both Micro and Macro.
Further, 89% of students scored in the 80 or above percentile range for Micro, while for Macro the
corresponding percentage was 79%. These are very high numbers and indicate that Learning Outcomes
1 and 5 were satisfactorily achieved.
1.2 Rubrics for Assessing Outcomes 2, 3 and 4
Outcomes 2, 3 and 4 are all assessed based on student performance in ECO 590, the Senior Research
Seminar. A rubric has been developed to assess each. A summary of the results is provided below. For
Outcome 2 (Quantitative Analysis), students were assessed on the basis of the econometric analysis
conducted in ECO 590. The components of the assessment (i.e., the categories assessed) are identified
in the first row of Table 2. Each category was assessed on a 4-point scale (Excellent, Good, Fair Poor),
with 4 being high. The averages of the scores achieved in each category are displayed in the second row
of the table, and the overall average for this category was 2.54.
3
College of Humanities and Public Affairs | Missouri State University
TABLE 2
ASSESSMENT OF QUANTITATIVE ANALYSIS (OUTCOME #2), 2013
CATEGORY
Choice of and
Discussion of
Data
Model
Estimation
Analysis and
Organization of Results
Tables and
Charts
AVERAGE
SCORE
2.75
2.39
2.31
2.64
2.64
Overall Average =2.54
Tables 3 and 4 provide the corresponding information for Outcomes 3 and 4. In all cases the overall
average was between Fair and Good, and closer to Good than to Fair. The lowest value was obtained for
Quantitative Analysis and the highest for Analysis of an Article in the Popular Press. The latter result,
together with the TUCE scores, suggests that students are well-grounded in basic theory and have the
ability to apply it to analyze current problems. The comparatively low score in Quantitative Analysis
suggests that students still have some difficulty in conducting econometric analyses that are appropriate
for the problems they are analyzing. (Interestingly, as will be seen later, in their exit interviews many
students identified lack of adequate preparation in Econometrics as one of the weaknesses of the
program).
TABLE 3
ASSESSMENT OF LITERATURE REVIEW (OUTCOME #3), 2013
CATEGORY
Choice of Material
Organization
Economic Analysis
Language
AVERAGE SCORE
2.83
2.64
2.69
3.17
Overall Average =2.83
TABLE 4
ASSESSMENT OF ARTICLE ANALYSIS (OUTCOME #4), 2013
CATEGORY
Choice of Articles
Economic Analysis
Organization and
Language
AVERAGE SCORE
3.03
2.69
3.25
Overall Average =2.99
4
College of Humanities and Public Affairs | Missouri State University
1.3 General Education Assessment
The new General Education program will go into effect in fall 2014. At that time our assessment of the
Gen Ed courses will change significantly both because we will have three Gen Ed courses (ECO 101, 155
and 165) instead of the present one course (ECO 155) and because the SLOs and assessment tools will
be different. However, for now we continue with the Gen Ed assessment that has been in place for
many years.
As before, the existing Gen Ed Goals Assessment Questionnaire was used for evaluation. The
questionnaire contains a set of ten questions designed to assess the degree to which students have
understood the general education goals in the areas of (a) Information Gathering, Reasoning and
Synthesizing Abilities; (b) Reflective, Creative and Critical Dispositions; (c) Communication Skills; (d)
Understanding of Culture and Society. This questionnaire is administered by the faculty during the last
two weeks of the fall and spring semesters in each section of ECO 155. Students are asked to specify
complete agreement (indicated by a value of 1), partial agreement (indicated by a value of 2), or
disagreement (indicated by a value of 3), with a set of statements. Complete agreement would be
indicated by all participants declaring values of 1 for all statements, and average values less than 2
would indicate that the goals are being met to a reasonable degree. The overall average scores in spring
and fall 2013 were 1.39 and 1.46, providing a strong indication that the course does in fact meet the
stated General Education goals.
1.4 Exit Interviews
The last ongoing assessment tool that is implemented every year is the Exit Interview, which is required
of all graduating seniors.
Student responses to the exit interviews have been a key input into some significant program
enhancements. In the last few years many of the exit interviews have indicated that students would like
to be exposed to more math and econometrics. As a direct response to these requests, we have started
teaching (since Spring 2009) two calculus-based theory courses and a second-level econometrics course,
ECO 508 Intermediate Econometrics. This was taught for the first time (in recent memory) in spring
2012, was taught again in spring 2013 and is scheduled to be taught in spring 2014.
There are two parts to the Exit Interview. First, students must complete a brief questionnaire dealing
with specific aspects of the program. Effective in CY 2013, the format of several of the exit interview
questions has been altered so as to make it possible to provide a summary numerical score. However,
the questionnaire still retains several open-ended questions. The new form is reproduced in Appendix 4.
The second part involves an actual interview with the Department Head which includes, but is not
limited to, a discussion of the student’s responses to the questionnaire.
There are two parts to the Exit Interview Form: the first part comprises10 questions which must be
answered either by picking a level of agreement (Strongly Agree, Somewhat Agree, Neutral, Somewhat
Disagree, Strongly Disagree) or a rating (Excellent, Very Good, Good, Fair, Poor). Allotting a score of 5 to
5
College of Humanities and Public Affairs | Missouri State University
the “best” option, 4 to the second best, etc., each response is given a score ranging from 5 to 1 with 5
being high. Based on 18 interviews in CY 2013, we find that the average score in every item but one is
above 4.0. The single exception, in which the average score is 3.83 is the rating for “The variety of
courses taught within the department.” Unfortunately, there is little that we can do to increase the
variety of courses offered with available resources.
The second part of the Exit Interview form consists of 4 open-ended questions. The most striking aspect
of the responses was the frequency (10 out of 18) with which individual identified a weakness of the
ECO as an inadequate focus on statistics/math/econometrics, with several specifically mentioning
calculus-based analysis.
History Department Assessment Report:
As part of our recruitment and retention efforts we measure how well students are mastering the skills
outlined by our department’s learning objectives. Better communicating these objectives to students
and noting their successes in meeting them may keep students invested in their education.
We are revising our assessment in these courses to meet the new general education requirements and
will pilot these changes next year.
We have also begun collecting the assessment data required to measure the success of the changes we
made to our major. The assessment data we collect this year will serve as a baseline as it will measure
the success of the old major. It will be three years before we have sufficient data to measure the new
major because students are just now beginning their sequence of classes. We will also be collecting data
from the praxis exam to compare how well students have mastered content in the new major. We will
use this data to make any necessary adjustments in course requirements (scale from 5 to 1, excellent to
unsatisfactory).
Academic Year 2012-2013
Academic Year, 2013-2014
Topic
5 4
3
2 1
Topic
5
Historical Awareness
8 9
5
1
Historical Awareness
11 0 2 3
Thesis
7 6
9
1
Thesis
9
2 2 1
Argument
8 4
10 1
Argument
9
2 2 3
Primary Source Use
6 12 3
1 2
Primary Source Use
4
3 3 4 1
Secondary Source Use 4 13 5
1
Secondary Source Use 5
3 6 1
Format
0 2
Format
5 2 2
6
7 6
4
College of Humanities and Public Affairs | Missouri State University
7
4 3 2 1
Style and Grammar
8 7
5
3
Style and Grammar
7
5 0 3 1
This data is drawn from four semesters of History 598 papers. It is based on a common rubric that
provides detailed instructions on how to evaluate each category.
Preliminary Data Suggests:




The majority of students obtain excellent or good scores in all categories.
Students are especially strong in historical awareness.
Thesis and argument were the weakest categories in 2012-2013 but evidence seems to be the
weakest in 2013-2014.
The Junior Seminar is designed to help students with using primary source materials and we
should see some improvement as more students take that seminar before history 598.
Praxis Scores:
Average Test Score
2011
2012
2013
MSS
175.50
192.00
BSED
169.17
163.85
166
MSS
2
1
0
BSED
24
33
38
# of Students
Preliminary Assessment Data Suggests:



Our average praxis exam score remains well above the requirements to teach in Missouri. In
order to teach in Missouri, a student must obtain a 152; our undergraduates average over ten
points higher suggesting that they are mastering the content necessary to teach.
Since BSED students take less content than do BA, we can expect that BA students would have
similar scores.
Students are learning the necessary content to teach in high school.
Philosophy Assessment Report:
Our primary instrument is the Exit Interview, in which we sit with graduating students for a good
half-hour to forty-five minutes and have them answer a number of questions about our program.
The questions are probing and provide a great opportunity for criticism. While I think this sort of
assessment is always valuable—who better to give us feedback than students, who have sat through
every one of our classes?—in a small department like ours, where we really know each and every
7
College of Humanities and Public Affairs | Missouri State University
one of our Majors, the process is particularly valuable. The insights that we gather from these exit
interviews provide a terrific basis, for an annual discussion, amongst ourselves, as to the quality of
our offerings, which then informs everything from curricular reform to the development of new
pedagogic strategies.
Political Science Assessment Report:
Total Test
Political
Thought
Assessment
Indicator 3
Methodology
Assessment
Indicator 2
Analytical &
Critical
Thinking
Assessment
Indicator 1
Internat’l
Relations
Subscore 3
Subscore 2
US Govt &
Politics
Subscore 1
Year
Comparative
Govt
For the year 2013, the data from the Major Field Test (MFT) in Political Science is consistent with all
previous data. We are above the national average with respect to the overall mean, the three sub-scores
of U.S. Government and Politics, Comparative Politics, and International Relations, as well as the three
assessment indicators of Analytical and Critical Thinking, Methodology, and Political Thought. That being
said, there was a modest decline in the overall mean, and the three assessment indicators. Although still
above the mean, these lower scores merit further scrutiny.
NATIONAL SCORES FOR MFAT FEBRUARY 2005-JUNE 2011
50.9
51.3
52.1
69.2
61.3
47.3
149.9
Fall07
61
60
56
83
77
51
161
Spr08
59
58
59
80
75
57
161
Fall08
56
56
58
74
75
57
158
Spr09
61
60
57
81
79
56
162
Fall09
66
65
68
87
85
67
169
Spr10
60
57
60
81
80
50
160
Fall10
57
60
59
76
78
52
160
NATIONAL SCORES FOR MFAT 2011-2013
52
52
52
59
45
54
152
Fall 12
64
58
57
69
58
59
161
Spr13
58
59
56
66
49
55
157
Fall13
57
61
59
68
57
48
158
8
College of Humanities and Public Affairs | Missouri State University
Without question, major GPA continues to be highly correlated with success on the MFT. However, as
the data below demonstrate, there has been a decline in the average PLS GPA which could explain the
decline in the overall mean. The most obvious explanation for this decline, related to our overall decline
in majors, is the significant reduction in per-law students. Our drop in majors coincides with the national
average decline in pre-law students; approximately 20%. The loss of these high-achieving students has a
negative impact on our average GPA and, unfortunately, as the national trend continues, we will see a
continued decline in major GPA, already evidenced in spring 2014 GPA and MFT data that will be
accompanied by a decline in overall MFT scores.
GPA Spring 2012
2.98
GPA Spring 2013
2.92
GPA Fall 2012
3.00
GPA Fall 2013
2.47
GPA AVG 2012
2.99
GPA AVG 2013
2.70
A similar explanation could be offered for the decline in the assessment indicators. That being said, a
continued decline in the Methodology scores may warrant a reexamination of curricular changes made
in response to the first program review; eliminating the Empirical field and the subsequent deletion of
PLS 475, as well as abandoning the SOC 302 requirement.
With respect to Political Thought, two hypotheses are worth considering and monitoring in 2014. First,
creating a required intro level theory course, PLS 330, broadened the exposure of all majors to this
subfield. However, fewer students are taking the upper-level theory sequence. It is possible that the
absence of a tenure-track theory professor, at least in fall 2013, contributed to this decline. Having a
new tenure-track professor in 2014 to attract a larger percentage of majors to the upper-level sequence
should result in higher scores on the Political Thought assessment indicator.
With respect to Analytical and Critical Thinking, while likely correlated with the decline in major GPAs,
the decline of this assessment indicator suggest two avenues of analysis. First, data from the Office of
Assessment has demonstrated that, for 2012, PLS majors performed at a much higher level on all
measures of the University Exit Exam; including Critical Thinking. I have requested 2013 data and will
continue to gather this data in 2014. Another measure for future analysis of this particular assessment
indicator would be students’ General Education GPA. In fact, the General Education GPA, may be a
better predictor because of the lack of incentive to perform well on the Exit Exam.
Religious Studies Assessment Report:
The department had been collecting assessment data for several years about its majors, requiring
students to submit short papers at the beginning and end of their careers, requiring faculty to submit
assessment reports on each major in any Religious Studies class, and requiring students to submit
samples of their written work for analysis in an exit interview. This process was revised during the 20112012 academic year and implemented in spring 2012.
9
College of Humanities and Public Affairs | Missouri State University
The department’s full-time faculty held a meeting to examine and discuss the results of the new
assessment plan in fall 2013. The faculty concluded that there were too few signs of students developing
their critical and creative skills in giving oral presentations, and they resolved to work more in this area
in the 2013-2014 academic year. The department also concluded that the assessment plan was
burdensome and not generating sufficient data upon which to act. As such the Department Head
charged the Assessment Committee with examining the department assessment plan and
recommending changes to address these weaknesses. Following input from the Director of Assessment
in the university, a new plan was designed and approved in spring 2014. The results from this plan will
be analyzed in a faculty meeting scheduled for August 2014. The revised Gen Ed. plan led the faculty to
review and revise its assessment procedures for REL 100, REL 101, REL 102, REL 131, and REL 210. Since
the new Gen Ed curriculum is scheduled to go into effect in fall 2014, the faculty was tasked with
preparing their assessment instruments and plan prior to that semester.
Sociology Assessment Report
Graduating seniors in Sociology enroll in a zero credit course SOC 492 Program Assessment and
Career Preparation. They meet with their adviser and complete three instruments: 1) a 13 question
curriculum survey, 2) a 5-factor, 32 item Civic Attitudes and Skills Questionnaire, and 3) a 77
question multiple-choice Assessment Instrument. Data has been collected for 2013-14 and will be
reviewed by the Sociology Committee in Fall 2014.
Anthropology Assessment Report
Anthropology majors give two class presentations, write 12-13 essays, and take a comprehensive
final exam in the capstone course, ANT 595. These tasks provide measures of their public speaking
ability, writing ability, and knowledge of anthropological history and theory. The percentage of
students who were rated as competent in these areas for the past three semesters is as follows:
Spring 2013
Fall 2013
Spring 2014
Oral presentation
94%
84%
79%
Essay writing
86%
86%
87%
Knowledge of theory
71%
20%
90%
Discussion: One trend is a decline in competence in oral presentation. Essay ability seems constant.
There is a marked difference in knowledge scores from semester to semester. Perhaps the main
reason for this is that some semesters students get together and seriously study for the exam and
other semesters they do not. Another factor is there was a different instructor in FA2013. Overall,
however, we can say that students are generally effective writers and speakers.
10
College of Humanities and Public Affairs | Missouri State University
11
College of Humanities and Public Affairs | Missouri State University