ASSOCIATION OF INSTITUTIONAL RESEARCHERS
AND PLANNING OFFICERS (AIRPO)
White Plains, NY
January 7th – 9th, 2009
Brian J. Gorman, M.Sc., J.D.
Assistant Professor, Department of Sociology, Anthropology & Criminal Justice
Towson University, Towson, MD
Catherine J. Wynne, Ph.D.
Office of Institutional Effectiveness,
Suffolk County Community College, Selden, NY
Thought & Action: NEA Higher Education Journal
Volume 24 ~ Fall 2008
“When I’m asked to document assessment, I feel
mildly violated. There is an element of distrust,
of suspicion and doubt, in asking for
documentation….I am resistant to the formal
documentation of assessment. I admit, though,
that my resistance is passive.” (p. 60)
The Assessment Edict and the Love of Teaching
Gregg Primo Ventello
“Usually, good teachers are too busy teaching to
waste time – unless coerced—on what they
consider at best a marginally useful endeavor.”
(p. 66)
Is Outcomes Assessment Hurting Higher Education?
James F. Pontus and Saranna R. Thornton
How can we foster Faculty cooperation with
assessment?
We can assist with answering the following questions:

Does this test assess the assigned course objectives?

If applicable, does this test assess the assigned General Education
objectives?

Does this test assess critical thinking? If so, how?

Do men and women perform differently on this test?

Is this test a measure of one idea (construct) or multiple
constructs?

Does performance on a previous test (or course) predict
performance on this test?

Does this test predict success on a future test?

Are there differences in test performance based on section
assignment? (Time of day? Course venue? Etcetera?)





Identify the sequence of events in the criminal justice process,
criminal law and procedure, including the structure of the system.
Describe the roles of the various actors (offenders and victims, as
well as law enforcement, court system and corrections personnel)
within the American criminal justice system and the impact of
these individuals’ discretion upon the criminal justice process and
its outcomes.
Analyze the operations of criminal justice from a sociological
standpoint by exploring the extralegal factors that influence the
criminal justice process and the disparities that result.
Examine the criminal justice process in the context of the
contradictory concerns for protecting individual rights and
safeguarding community interests.
Interpret existing research on the process of justice.
Criteria 1
 Exceeds…….
Develops a clearly articulated
argument, using evidence and/or systematic logical
reasoning in support of a conclusion or point of view.
 Meets……….
Presents an argument using evidence
and/or logical reasoning in support of a point of view.
 Approaches…
States a conclusion or point of view
but does not organize the evidence or reasons in a
logically adequate way.
 Not Meet……
Does not clearly state a conclusion or
point of view or else little or no supporting reasoning
or evidence I presented.
Criteria 2
 Exceeds…….
Identifies relevant qualifications or
objections or alternative points of view and prioritizes
evidence and/or reasons in support of the conclusion.
 Meets……….
Identifies some qualifications or
objections or alternative points of view.
 Approaches…
Does not clearly identify or respond
to relevant objections or alternative points of view.
 Not Meet……
Makes no attempt to recognize or
respond to objections or alternative points of view.
Objective 2: Students will develop wellreasoned arguments
Sociological/Extralegal Factors
Criteria 1:
Exceeds: Clearly articulated argument,
using evidence.
Meets: Evid-based argument.
Approaches: poorly org evidence.
Not Meet: Unclear cnlsn or pt of view;
poor supp reas or evd.
Criteria 2:
Exceeds: Id’s relevant qualifications or
objections or alt pts of view for cnclsn.
Meets: Ids some quals or objctns or alt
pnts of view.
Approaches: unclear id/resp to rel objs
or alt pts of view.
Not Meet: no id or resp to objs or alt pts
of view.
Rights of the
Individual vs. Safety
of Society

Congratulations! You are approaching the end of the
course and you are now able coalesce aspects of all
that you learned into one paper. You will no doubt be
surprised to see all that you learned this semester once
you complete your paper according to the guidance
below. Your resources for the paper are your
observations of court, class material, and the text. Your
paper gives you an opportunity to draw upon all of the
above. You may go back to Circuit Court for further
observations if you wish. That is to say, you are not
obligated to use your original observation if you go to
court again.
Develop a clearly articulated argument, using
evidence and/or systematic logical reasoning in
support of a conclusion or point of view.
(Application: rights and sociological perspective/extralegal)
Identify relevant qualifications or objections or
alternative points of view and prioritize evidence
and/or reasons in support of the conclusion.
(Application: rights and sociological perspective/extralegal )
 Convergent Validity:
 The
overlap between different tests that
presumably measure the same construct.
 Two
concepts assessed in the focal test and
repeatedly through the semester:
 Rights
&
 Sociological Perspective/ExtraLegal Factors.



Three 50-question objective-style tests were
administered during the course. All test items were
developed by Thomson Wadsworth Hall and supplied
through the CJ textbook test bank.
Faculty member identified relevant test items
operationalizing the “rights” construct and the
“sociological perspective/extralegal” construct.
Assessment associate developed the corresponding
Rights Scale and Sociological Perspective/ExtraLegal
Scale.
Case Processing Summary
N
Cases
Valid
Excluded
Total
55
0
55
%
100.0
.0
100.0
Reliability Statistics
Cronbach's
Alpha
.792
N of Items
5
Rights Scale
Ite m Statistics
T2Q16A
T2Q21A
T2Q22A
T2Q23A
T2Q24A
Mean
.9455
.8182
.9455
.8182
.8000
St d. Deviat ion
.22918
.38925
.22918
.38925
.40369
N
55
55
55
55
55
Case Processing Summary
N
Cases
Valid
Excluded
Total
55
0
55
%
100.0
.0
100.0
Reliability Statistics
Cronbach's
Alpha
.755
N of Items
3
Item Statistics
T2Q9A
T2Q10A
T2Q11A
Mean
Std. Deviation
.82
.389
.93
.262
.80
.404
N
55
55
55
Does Not Meet
Critcal Thinking Obj 2 Criteria 1 Rights
Approaches
Meets
Exceeds
12
10
Count
8
6
4
2
0
0
60
80
100
0
60
80
100
0
60
Rights Scale (%)
80
100
0
60
80
100
Critcal Thinking Obj 2 Criteria 1 Sociological Perspective Extra/Legal
Does Not Meet
Approaches
Meets
Exceeds
12
10
Count
8
6
4
2
0
0
33
67
100
0
33
67
100
0
Soc Scale (%)
33
67
100
0
33
67
100
Does Not Meet
Critcal Thinking Obj 2 Criteria 2 Rights
Approaches
Meets
Exceeds
20
Count
15
10
5
0
0
60
80
100
0
60
80
100
0
60
Rights Scale (%)
80
100
0
60
80
100
Does Not Meet
Critcal Thinking Obj 2 Criteria 2 Rights
Approaches
Meets
Exceeds
20
Count
15
10
5
0
0
60
80
100
0
60
80
100
0
60
Rights Scale (%)
80
100
0
60
80
100



Impression: The experience was worthwhile
and it seems likely that other professors
would benefit from such collaboration.
Impression:-Curiosity over results made
Assessment Activity Appealing. There is
extra work involved, but the curiosity over
the results compensated for the extra work.
Suggestion: Offer the assessment activity on
a voluntary basis before implementing a
mandate.



Impression: -Holding up a mirror to someone’s work
with a genuine question as to the result can either be
anxiety producing or a welcomed event.
Suggestion: Confidentiality. Make results
confidential to encourage engagement. Therefore
the activity will more likely to be welcomed if the
results are confidential.
-Encourage participation. Participation in such a
project would be higher if professors could
engage in the activity without results appearing in
their promotion & tenure file unless desired.


Impression: -Provokes Assignment Improvement.
Working with the rubric is valuable in that it makes one
think more critically about assignment design. More
and better thoughts on improving the assignment came
more readily on the second evaluation of the papers,
i.e. the one done exclusively for this project, not
grading.
Suggestion: It would be best to apply the rubric in
a sitting other than when grading. No doubt
professors would be tempted to read each paper
once then grade and apply the rubric in a split
analysis.


Impression: Complexities. -The SUNY rubric is
comprehensive, but did not fit exactly to the
assignment. The comprehensive nature of the rubric
added to some complexities in applying it.
Suggestion: Streamline.Work with evaluator at the
outset to streamline and adjust the rubric for the
specific assignment.
a-Frame Activity as a Dialogue. Conceptualize the
exercise as a dialogue rather than an evaluation of the
professor’s assessment abilities.
b- Benefit of an Assessment Portfolio. Offer faculty the
opportunity to assemble an assessment portfolio that
allows engagement in the project a number of times,
thus identifying improvements over time.
c-Optional Proof of Effectiveness. Participation would
also improve if professors could use the data from their
assessment portfolio in support of teaching
effectiveness.