To:
David Malik, Interim Executive VCAA
Charles Gallmeier, Chair, Faculty Organization
From: Mary Ann Fischer, Co-chair, General Education / Assessment Committee
Date: August 10, 2009
Re: IU Northwest General Education/Assessment Committee Report to VCAA and the
Faculty Organization
The Faculty Organization adopted the Student Learning Outcomes for Principle 1
at the January 16, 2009 meeting. These outcomes are posted at our web site at
http://www.iun.edu/~genednw/principles/principle_1.shtml
The General Education/Assessment Committee met three times during the spring
2009 semester. Two meetings were devoted to a discussion of the definition and learning
outcomes for Principle 4, Diversity, because the College and Arts and Sciences
Curriculum Committee had had some difficulties interpreting the definition in order to
make course recommendations to the other schools and divisions. At the conclusion of
the second meeting, the General Education / Assessment Committee agreed to let stand
the original definition passed by the Faculty Organization on October 17, 2008. The
definition can be seen at http://www.iun.edu/~genednw/principles/principle_4.shtml.
On May 12, the committee conducted an assessment of writing skills using 46
papers written by students from W131 and advanced classes. A report summarizing the
results of this assessment is attached. The results will serve as a benchmark against
which future writing assessments will be judged and will also form the basis of a general
discussion of writing skills and how to improve them among our students.
The committee chair forwarded to the Interim Executive VCAA a suggested schedule for
the integration of revised general education requirements into school and division
curriculums. The schedule calls for the integration plans to be submitted for approval to
the Survey and Curriculum Committee of the Faculty Organization by November 1,
2009. A form for documenting the planned changes for each degree program was
successfully pilot-tested by the School of Business. A copy of this form, “General
Education Curriculum Integration, 2009” is attached.
An amount of $5000 was designated for stipends and materials needed for a year
of Intensive Writing workshops. The first 3 workshops in the series, sponsored and
advertised through CETL, were presented during the spring 2009 semester. Faculty
writing specialists from the department of English led the workshops, entitled Designing
Writing Assignments (Feb. 24, 25); Responding to Student Writing (Mar. 24, 25); and
Assessing Student Writing (April 14, 15). Additional workshops are planned for fall
2009 and may include an outside speaker on the subject.
Our Committee thanks Interim Executive Vice Chancellor Malik for the help and
support he has provided for the General Education initiative.
Assessment of Writing Skills among Beginning and Advanced Students
IU Northwest General Education/ Assessment Committee
May 12, 2009
Report Date: August 8, 2009
Participating Committee Members: Mary Ann Fischer (co-Chair), Doug Swartz, Barbara
Peat, Latrice Booker, Sheila Marie Trzcinka, Linda Delunas, Karen Evans, Dee Dee Ige,
Jim Thomas,
Eva Mendieta, Cynthia O'Dell, Taylor Lake
Introduction
During May of 2009, the combined General Education/Assessment Committee of
the IU Northwest Faculty Organization completed an assessment of writing skills
revealed in a sample of forty six final papers selected from either W131, Elementary
Composition, or from upper division courses in each of the six schools and colleges.
This report details the procedures used the solicit the papers, to review and evaluate them,
and the results as revealed in quantitative judgments and comments of the faculty
evaluators who served on the committee. We hoped to establish a baseline of evidence
for writing skills and to find evidence of more advanced skills among upper division
papers than among W131 papers.
Selection of Writing Samples
The Associate Vice Chancellor of Academic Affairs requested that the Deans of
the schools and colleges solicit sample papers from their faculty. The request was for at
least two papers along with descriptions of the assignment, from a given instructor, one
judged by the instructor as relatively skilled and the other as relatively weak. All
identifiers except the course number were stripped from the papers before they were
submitted to the AVCAA. The final sample included 23 papers drawn from W131
sections and 23 papers from upper division courses. All schools and colleges submitted
at least 2 papers from upper division courses.
Assessment Procedures
The committee adapted a tool designed by Professor Anne Baylay, Department of
English. It included eight categories, including definitions, of writing skills: research,
persuasion, prose and syntax, counterargument, mechanics, complexity, reader pleasure,
and organization, (See attachment). Each evaluator read three or four papers and judged
them as excellent (3 points), acceptable (2 points), or unacceptable (1 point) in each
category; additional open-ended comments were optional. Each paper was evaluated by
only one rater; thus no inter-rater reliability can be reported.
Results
The mean rating for each skill category among W131 and upper division papers is
shown in the table below. As expected, in every category upper division papers were
given higher ratings than those from W131. Readers knew the course level of the paper,
so expectations for better performance on the upper division papers may have influenced
ratings. Future assessments should remove the course identifiers. Ratings for the W131
papers were low, that is, the means were below 2, defined as “Acceptable.” Mean ratings
for the upper division papers were just above “Acceptable” in all but 2 categories.
Course Level
W131
Upper Division Combined
(N=23)
(N=23)
sample
Mean
Mean
Mean
Research
1.83
1.96
1.89
Persuasion
1.74
2.17
1.96
Prose and syntax
1.65
2.23
1.93
Counterargument
1.61
1.68
1.64
Mechanics
1.61
2.13
1.87
Complexity
1.65
2.13
1.89
Reader pleasure
1.73
2.09
1.91
Organization
1.68
2.02
1.86
Total points
13.70
16.24
14.97
The comments (see attachment 2) about the papers were much more negative than
the mean ratings would have suggested in the case of the upper division papers. It may
be that raters hesitated to classify the upper division papers us “Unacceptable” even
though they were highly critical. If so, future assessment efforts should use a rating scale
with a wider range.
In general, evaluators found the experience of reading student papers valuable and
useful. The committee did not discuss the question of what steps should be taken to
improve the quality of student writing. Comments and suggestions on the process
included the following:
1) Our request to the units for sample papers should be clearer.
2) The rubric should be more universal to any assignment; suggested areas of
universal assessment include: focus, development, mechanics, philosophy
3) Sheila Marie T. brought a rubric that we might consider for next time
rather than the one we used this time.
5) Suggestion that everyone, including faculty, have a Handbook (perhaps purchased by
the
university) that is used in most w130 or w131 classes to encourage students to
keep them and use them in all future classes.
Attachment 1 Writing Assessment Form
General Education/Assessment Committee Assessment Project
May 12, 2009
Reader __________________
Paper #_______________________
This list of desired attributes has been adapted from one designed by Anne G. Balay
(abalay@iun.edu), although she strongly favors holistic assessment rather than evaluating
attributes separately, as we will be doing here. She has argued that after a student has
completed one year of a college-level writing course, a final essay should demonstrate
most of these attributes. After reading each essay please use the definitions which have
been provided to rate each sample paper for each attribute.
Excellent
3 points
Acceptable
2 points
Not
acceptable
1 point
use of research
effectiveness of argument
or persuasion
style reflecting prose and
syntax
use of counterargument
without bias or ethos
attention to mechanics and
punctuation
complexity of ideas
reader pleasure
structure reflecting
organization and focus.
Total points in each
category
Total points: ______________
Additional
comments:_______________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
________________________________________________________________________
Definitions of the categories:
Use of research: Students should execute both library and Internet research in gathering
sources for their projects. Sources appearing in student papers should be drawn from
electronic databases, and contain a combination of academic, public, and primary
sources. Students are expected to demonstrate an ability to summarize, analyze, and
synthesize sources appropriate to academic writing.
Effectiveness of argument or persuasion: Students are expected to establish a research
driven thesis supported by warranted claims that shaped their purpose in writing. Claims
are supported by evidence gleaned from research, primarily, as well as from personal
experience and common knowledge.
Style reflecting prose and syntax: At the surface level, the writing should be clear.
Sentences should reflect a degree of sophistication appropriate to a college writer and
should flow in prose that sustains the argument.
Use of counterargument: The essay should include a counterargument as reflecting a
writer’s sense of ethics and fair-mindedness in working a thesis. Furthermore, a writer
should guide the argument without undue bias, acknowledging opposing opinions and
points of view.
Attention to mechanics and punctuation: Writing should be clear and free of errors. The
writer should demonstrate mechanics that reflect the grammar of Standard American
English, free of slang and jargon, and should attend to the impact of punctuation on the
reader.
Complexity of ideas: Papers should reflect thoughtful interpretations and applications of
research (considered critical thinking). As such, successful papers avoid pedestrian
arguments on mundane subjects.
Reader pleasure: A good essay interests its writer as well as its reader, and is fresh in
ideas and well written.
Structure reflecting organization and focus: Deep issues of organization and
development define this category. A paper’s thesis and claims should guide the reading,
and those claims should be developed through extended discussions and use of research
as support.
Attachment 2 Comments on papers
Total
points
Course level
8
131
8
131
9
131
9
10
10
131
131
131
10
11
131
131
11
12
131
131
13
14
14
15
131
131
131
131
15
131
16
16
16
131
131
131
17
18
19
21
23
131
131
131
131
131
8
upper
division
upper
division
8
upper
division
10
12
upper
division
upper
8
Comments
this essay does not follow assignment, does not provide an adequate or clear
presentation of the sources pursue a thesis, provide fresh ? Or cite sources.
Furthermore, it contains an array of grammatical errors
a lot of serious errors in sentence structure. I get no sense of her sources,
arguments and writer relies too much on personal experience
mostly writer opinion, lack of research support. Fails to cite sources in text
references unacceptable, many writing errors (mechanics)
works cited page has appropriate references but they are not really used in
text as a basis for the "argument" this is simply an "opinion piece" with
attempts to make it appear as if the research supports the writers opinion.
Completely unsophisticated, no fresh ideas, very repetitive
major claim was not clear enough and sub-claims weren't clear and didn't
adequately support and develop major claim. Use of research indicates lack
of critical reading skills.
no central thesis other than title, just a flow of thoughts with personal example
and a few quotes supporting only the authors point of view, problems with
grammar
Very mixed, when formal scores were low, in middle,l when personal, much
better
little depth to analysis
flow more conversational than formal essay - no central thesis and so
supporting TPs don't have connection other than conversational, but sent
structure good and mechanics
one of the sources is not cited. There are many generalizations about
minorities and communities that would seem to require documentation
met the criteria
opening and closing ¶ reflect acceptable writing (whereas middle ¶'s seem
more disorganized and subject to inappropriate sequence of ideas
Basically good, arguments occasionally personal, bit long
this was unacceptable for a 400 level class
Student did not follow instruction in assignment and did not seem to even
understand the point of the assignment
no citations/references, can't tell what kind of assignment this is. Not required
length for a take-home paper. No book identified (except 1984) so not a book
review
If student ? Article o a study to answer the questions in the instructions then,
according o the ? Noted deficiencies in information the ? Should not have ?
This article for review
did not follow instructions of assignment; weak on syntax and grammar which
division
13
upper
division
upper
division
14
upper
division
15
upper
division
12
15
16
18
18
18
18.5
19
20
20
21
22
22
23
23
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
upper
division
hindered reader understanding - was persuasive in argument that good
source for classroom instructor and did connect approp. To the standards
show and tell paper
Frankly, I didn't understand the assignment so evaluated the paper totally on
its own. Was research required?
ineffective use of transitions, spelling errors, and sentence structure errors
detract from this broad look at schizophrenia and cautionary view of certain
medications
the paper is too much driven by summary: it's often not clear why we are
being provided with description of people and events from the book being
reviewed. There needs to be a better sense of purpose in the writing, a
central evolution
incorrect APA format, some misuse of words, assignment calls for figures to
be placed at end of paper but writer includes them with text, conclusion is
vague/cursory
good grammar syntax
Citations in text lack year and ?. Student uses footnotes instead -maybe
okay? Problems with confusing outline structure. Intro poorly developed
well written paper
good writer, little or no evidence of field research visits to the church
Good
there weren't a sufficient number of sources, and I wasn't sure how the source
provided historical background for the memoir. The citation system is also
confusing
well documented, effectively written paper!
General Education Curriculum Integration, 2009
To be completed and submitted to the Survey and Curriculum Committee by
all Schools and Divisions for all degree programs by Nov. 1, 2009
Please use this form to indicate how your programs will meet the revised General
Education Curriculum Requirements
Degree program(s) e.g. Psychology B.S.___________________________
School/Division____________________ Date__________
Principle 1 Foundations and
How will the class requirement and learning
Advanced Courses
goals be met?
W131
Required
S121
Required
One course among the following:
M100, T101, M118, M119, M125,
M215
Natural science course with lab
At least two Intensive Writing courses
Guidelines
At least one Advanced Oral
Communication course Guidelines
At least one Advanced Mathematical
Reasoning course (e.g. K200, E270,
K300, M216 and higher math courses,
etc.) Guidelines
At least one Advanced Scientific
Reasoning course (either natural or
social sciences) Guidelines
At least one Information Literacy
course Guidelines
Principle 2 Breadth of Learning*
Arts and Humanities-6 credits
Cultural and Historical Studies- 6
credits
Social and Behavioral Sciences-6
credits
Mathematical, Physical and Life
sciences-3 credits
How will the class requirement and learning
goals be met?
Principle 3 Critical Thinking **
How will the class requirement and learning
goals be met?
Mandatory capstone course in the
major or equivalent experience as
determined by the unit.
Principle 4 Diversity**
How will the class requirement and learning
goals be met?
Some of the Principle 2 courses and
advanced/writing-intensive courses
from Principle 1 should be designated
as diversity courses.
Principle 5 Ethics and Citizenship**
Some of the Principle 2 courses and
advanced/writing intensive courses
from Principle 1 should be designated
as ethics or citizenship courses.
*Credits used to satisfy Breadth of Learning may also satisfy Advanced Coursework in
the Principle 1. A student may not use the same class to satisfy more than one domain
in Breadth of Learning.
**Units must specify how their students will meet the outcomes for principles 3, 4, and
5. Each unit will provide evidence that students have met the appropriate outcomes.
Units are responsible for creating or designating the necessary courses and/or enriching
experiences that fulfill the requirement and for assessing the outcomes.
**Guidelines: Outcomes for principle 3, 4, and 5 will be met by a class or by an enriching
educational experience such as service learning, an internship, a capstone project,
student teaching, an independent study research/creative activity, study abroad,
creation of an exhibit, or a performance.