2012-2013 Academic Assessment Report
Communication Arts
I. Brief Summary of Assessment Plan
In the Spring of 2013, the Communication Arts faculty collected data in order to assess
Outcome 5B (Students will draw on knowledge of methods to diagnose or analyze real world
communication problems). This Outcome is part of Goal #5 of our curriculum: To employ the
methods of the field.
II. Assessment Methods
In order to assess this outcome, we collected data in the form of papers and final exams from
SPCH-C393 Research Methods and SPCH-S228 Argumentation and Debate, as both of these
courses involve the application of specific methods. The components and performance criteria
for this outcome are listed in Table 1, which also served as our scoring rubric. The assignment
descriptions appear in Appendix A.
Table 1. Assessment Rubric for Outcome 5B
“Students will draw on knowledge of methods to diagnose or analyze real world communication
problems”
Outcome
Components
Performance Criteria
5B. Students will
draw on knowledge
of methods to
diagnose or analyze
real world
communication
problems
Problem identification
a. Incomplete and inappropriate (1 point)
b. Complete and appropriate (2 points)
Presentation of Reasoning
a. Lacks detail and organization (1 point)
b. Some detail and/or organization (2 points)
c. Provides detail and is clearly organized (3 points)
Appropriateness of application
a. Inappropriate (1 point)
b. Appropriate (2 points)
c. Very appropriate (3 points)
For each component, the Communication Arts faculty set a benchmark success rate of 85% of
the students effectively demonstrating the ability to apply the research method in question and
to diagnose real world communication problems. Our benchmark translates to an 85% average
using the rubric in Table 1.
The communication arts faculty collected and evaluated the data in the following manner for
both courses.
1. Collection. Final papers in C393 and final exams in S228 were collected by the
instructor.
2. Evaluation. The papers and exams were assessed by two Communication Arts
resident faculty members according to the performance criteria. The performance
criteria were assessed based on the evaluators’ skills and experiences as instructors
of communication arts courses. The faculty evaluated each component by assigning
a number to each performance criterion as indicated in Table 1.
III. Description of Assessment Results
Students met the 85% benchmark in only the first component (problem identification) with an
average of 96% meeting the criterion of “complete and appropriate.” 80.67% of students met
the criteria of components two and three (presentation of reasoning and appropriateness of
application). These results are shown in Table 2.
Table 2. Results by individual student
Student
Student 1
Student 2
Student 3
Student 4
Student 5
Student 6
Student 7
Student 8
Student 9
Component
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Review
1
2
3
3
2
3
3
2
3
3
2
3
2
2
2
2
2
2
2
2
3
3
1
2
1
2
2
2
Review
2
2
3
3
2
2
3
2
3
3
2
2
3
2
2
2
2
2
2
2
3
3
1
1
1
2
2
2
Average of
Reviewer 1 and 2
2
3
3
2
2.5
3
2
3
3
2
2.5
2.5
2
2
2
2
2
2
2
3
3
1
1.5
1
2
2
2
Student 10
Student 11
Student 12
Student 13
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
Problem identification
Reasoning
Application
2
3
3
2
2
2
2
2
2
2
3
3
2
3
3
2
2
2
2
2
2
2
3
3
2
3
3
2
2
2
2
2
2
2
3
3
IV. Using Assessment for Program Improvement
96% of students demonstrated effectiveness in the component “problem identification” while
80.67% of students demonstrated success in the components “presentation of reasoning” and
“appropriateness of application.”
While students were successful at identifying communication problems, they did not meet the
85% benchmark for explaining their reasoning and applying the chosen method appropriately.
Only three of the items assessed were collected from S228, and given this low number, changes
to the course should be undertaken with caution, and are not planned at this time.
The remainder of these items were collected in C393, so our improvements will be focused on
this course. The faculty will make the following changes:
1) Presentation of reasoning. Higher student performance on this component will be
facilitated through changes in the assignments. Currently, the faculty member teaching the
course divides the class into four major units, focusing on four major research methods.
Each unit includes the reading and discussion of sample journal articles, followed by the
writing of a paper that applies the method. These assignments will be supplemented with
written exercises focusing on the journal readings that specifically analyze the reasoning of
these articles. This will provide students with quality examples and experience discussing
reasoning. For instance, writing prompts could include questions about how the authors
came to the conclusions they came to (the reasoning behind the arguments presented in
the articles).
2) Appropriateness of application. This component focuses on how appropriately and
effectively the chosen method is applied to the communication problem chosen by the
students. These problems include phenomena, interactions and texts. The faculty believe
that students are having trouble applying the correct method to the problem identified—or,
more correctly given the nature of the assignments—choosing the best problem for the
method assigned. This will be addressed by limiting the communication phenomena
available for students to write about. This will provide the instructor the ability to make
sure the application of the method is more appropriate for the phenomenon, interaction or
text.
V. Dissemination of Results
These results will be distributed to the Communication Arts faculty, the Dean of Humanities and
Social Sciences, the Assessment Director, and the Office of Academic Affairs via e-mail, and will
be posted on the Communication Arts webpage. Communication arts faculty will discuss these
results further in an upcoming meeting.
Appendix A. Assignments evaluated
1. S228 Final Exam essay question.
Final Exam
Spring 2013
Directions:
This exam is open book, open note (50 points total). You will answer a total of four (4) questions
from the list below (one question from Part 1, three questions from Part 2).* Length and point value are
described below. Be thorough: make sure you give examples when appropriate and cite specific
passages in the text as appropriate. Please note that several of these questions require you to apply
course concepts to issues outside of the classroom. You will not find all of the answers in the text!! Exams
should be typed and should be no longer than 10 pages total. Exams are due by 10:00 am on Thursday,
May 6. Please email your exam to darrc@iuk.edu . This is an essay exam, not a paper: essays will be
graded on substance, not style. However, do try to use proper spelling, grammar, etc., and do not use
bullet points – write me five essays! (You may wish to underline key terms so as to show me you
understand the course concepts).
You are expected to do your own work. Do not assist other students in any way shape or form.
From this point on I cannot give you any help on this exam, other than to clarify the wording of the
questions. I cannot tell you where to look for an answer, let you know if you are on the right track, etc.
Question: The Toulmin model of argument has been described as an “ideal” form of argument. In your
analysis, why is this model an “ideal”? How well do you think public, interpersonal, business, or other
forms of discourse live up to this ideal?
2. C 393: Communication Research Methods
Ethnography Method Project
Description: The ethnography project is your final method project. Ethnography requires you to spend
time immersed in a culture in order to report what that culture is like to the rest of the world (i.e. the
five grad students who will read your paper). For this assignment, you will find an organization of
interest (a workplace, nonprofit organization, a club of some sort), spend approximately 6 hours
observing the organization, interview a minimum of two members of the organization, and write a
report.
Grading: You will work on your own. The assignment is worth up to 50 points. Papers will be graded
according to how well they achieve the requirements listed below, as well as on stylistic factors and
whether you follow APA style appropriately.
Requirements: You must pick an organization to observe. You are allowed to choose an organization you
belong to (this would make you a participant-observer) with one exception: you cannot pick your
church. This restriction is totally selfish—I don’t want to read 15 papers about people’s churches. Your
organization must be approved by me, and you should get permission from the people you are
observing and interviewing.
As we will discuss in class, two of the main tools of ethnography are observation and interviews.
Ethnographers typically take voluminous field notes while observing a culture or organization, and rely
on interviews with informants for in-depth, insider information. The two should complement each
other: your observations along with the information provided by insiders should help you to paint a
picture of the organization that is interesting and insightful to readers. Your requirements for the
observational stage of the assignment are six hours of observation and two in-depth interviews of at
least 20 minutes each.
Theoretical Perspective: We will be using Schein’s model of organizational culture, which will be
discussed extensively in class. You do not need a literature review per se, but will be expected to explain
Schein’s model in your paper (see below). The important thing for now is to note that you will be
discussing the three “levels of culture” in the organization you observe: artifacts, espoused values, and
basic assumptions. So make sure you take good field notes and ask good interview questions about these
three things.
Parts of the Paper/Requirements: Your paper should be broken down as follows. Notice that your paper
will range from 1,400 to 1,950 words, not including title page and works cited. You will use APA style (as
you did with your Experiment and Rhetorical Criticism Projects). Papers will be turned in via the
Assignments area of the Oncourse site. Papers are due by class time on May 1 (the Final Exam period).
1. Introduction. Introduce the organization and the method. Explain why it is important to analyze
this particular organization. State your thesis and give a preview. (Approximately 200-300
words).
2. Method/Literature review. Explain ethnography and Schein’s model of organizational culture.
(Approximately 300-500 words).
3. Analysis of the organizational culture. Explain what you observed and who you talked to. Discuss
the artifacts, espoused values and basic assumptions that readers need to understand in order
to appreciate the culture of the organization you have observed. Describe what you witnessed
and quote from your interviews. Make an overall argument/observation about the culture of the
organization. (About 800-1,000 words)
4. Conclusion. Give a nice summary. What did we learn from your paper? (100-150 words).