2014-2015 Annual Program Assessment Report
Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to
james.solomon@csun.edu, director of assessment and program review, by September 30, 2015. You may, but are not required to,
submit a separate report for each program, including graduate degree programs, which conducted assessment activities, or you may
combine programs in a single report. Please identify your department/program in the file name for your report.
College: MMAMC
Department: Communication Studies
Program:
Assessment liaison: Aimee Carrillo Rowe
1. Please check off whichever is applicable:
A. ___X___ Measured student work.
B. ___X___ Analyzed results of measurement.
C. _______ Applied results of analysis to program review/curriculum/review/revision.
2. Overview of Annual Assessment Project(s). On a separate sheet, provide a brief overview of this year’s assessment activities,
including:

An explanation for why your department chose the assessment activities (measurement, analysis, and/or application) that it
enacted
This was the fourth year of our five-year assessment plan. As provided by that plan, four faculty members (Assessment Team A)
worked together to conduct the annual program assessment. The target audience for this study was Communication Studies majors
enrolled in undergraduate courses that introduced, emphasized, or developed PSLO 4, as well as graduate students enrolled in the two
courses in which PSLO 4 was introduced and/or emphasized. The Assessment Team developed authentic tasks for assessing both
undergraduate and graduate programs. The team also developed and normed rubrics that we used to evaluate student responses. We
chose authentic tasks appropriate to the graduate and undergraduate PSLOs and to provide an opportunity for faculty to evaluate
student critical thinking.
We also administered a faculty survey to faculty who taught courses that introduced, emphasized, or demonstrated PSLO 4-U to
gather information about actual classroom practices concerning PSLO 4-U. Our goal is to use this data in 2016-17 to update the
department’s assessment plan and to develop methods to share resources on how faculty teach PSLO 4 in their courses.
The department also undertook a smaller, more focused project that targeted Aimee Carrillo Rowe’s COMS 440 (Cultural and
Performance Studies Methods) as a pilot for conducting student-based, self-report, class-specific assessments. We have found that our
assessment over the past several years does not provide us with specific data related to courses and areas of study that might help us
better determine departmental needs, like curriculum development and faculty hires. Our hope was this tool may allow us to make
data-based decisions around curriculum and hiring.

If your department implemented assessment option A, identify which program SLOs were assessed (please identify the SLOs
in full), in which classes and/or contexts, what assessment instruments were used and the methodology employed, the resulting
scores, and the relation between this year’s measure of student work and that of past years: (include as an appendix any and all
relevant materials that you wish to include)
In AY 2014-2015, the Department of Communication Studies (COMS) examined PSLO 4 for both the undergraduate and graduate
programs. Those PSLOs are:
Undergraduate PSLO 4: Identify effective and ethical communication
Graduate PSLO 4: Analyze and critically interpret/evaluate communication practices and research
To distinguish the undergraduate and graduate PSLOs in this report, the undergraduate PSLO will be referred to as PSLO 4-U and the
graduate one will be referred to as PSLO 4-G.
Undergraduate program
Our alignment matrix identifies fifteen undergraduate courses that addressed PSLO 4-U, twelve of which were offered in Spring 2015
semester. All sections of these courses—33 in total—were evaluated. The selected courses addressed PSLO 4-U in varying intensities:
COMS 301 Introduced
COMS 304 Introduced Emphasized
COMS 321 Introduced Emphasized Demonstrated
COMS 323 Introduced
COMS 400C
Emphasized Demonstrated
2
COMS 410
COMS 428
COMS 430
COMS 437
COMS 440
COMS 445
COMS 450
Emphasized Demonstrated
Introduced Emphasized Demonstrated
Introduced Emphasized Demonstrated
Emphasized Demonstrated
Introduced Emphasized Demonstrated
Demonstrated
Introduced Emphasized Demonstrated
The target audience was reached. The assignment prompt was sent to 766 students with 399 responding (a 52% response rate).
Communication Studies majors comprised 76.4% of the total responses. The team limited its evaluation to fifty, randomly-selected
responses by Communication Studies majors.
Given PSLO 4-U concerned students’ ability to identify ethical and effective communication, the Assessment Team designed an
assignment that required students to analyze the ethics and persuasiveness of an anti-domestic violence advertisement/Public Service
Announcement (PSA), which was aired during the 2015 Super Bowl. Students were directed to watch the PSA and to read a Salon
article that explained the National Football League's role in developing the PSA before responding to the following two-part prompt:
1. What visual and verbal content does the PSA use to persuade? What is the PSA's persuasive goal within the context of this
article?
2. Based on what you've learned about ethics in your classes and the perspective the article provides about the creation of this
PSA, assess the ethics of this PSA.
We used assessment software (AMEE) to administer the assignment to students in the identified courses. Students received an
electronically-generated email that included a link to the assignment instructions, directing students to the artifacts and respond to the
prompt either online or by attaching a document. Students had fifteen days to respond. Email reminders were regularly sent to students
with increasing frequency as the deadline approached. After the deadline, the responses were reviewed to ensure the responses
addressed the questions presented. Incomplete responses—those that had no answer—were deleted.
The Team evaluated student responses using an assessment rubric developed in 2011-12 and revised for the current prompt. The rubric
consisted of a three-point scale: (1) Clear expression of ideas; (2) Identifies the persuasive properties of PSA; and (3) Assesses the
ethics of PSA. These three categories were evaluated on the following 4-point scale: Failed to meet PSLO, Approached PSLO, Met
PSLO, and Exceeded PSLO. This three-by-four rubric yielded a total possible score of 12 for each student response. The scoring
criteria of the total points were as follows:
10-12 points—exceeded PSLO
3
7-9 points—met PSLO
4-6 points—approached PSLO
0-3 points—failed to meet PSLO
About one-third (30%) of student responses were rated as meeting or exceeding PSLO 4-U (28% met it and 2% exceeded it). The
benchmark level of 45% was not met. More than half of student responses (52%) were rated as approaching PSLO 4-U, while less
than one-fifth (18%) were rated as failing to meet it.
Our findings revealed students scored much lower this year than in previous assessment years. For instance, last year’s assessment
report (Appendix A) of PSLO 3, “describe and explain the relationship between communication and culture,” in which “three-quarters
(75%) of student responses met or exceeded PSLO 3 and no students failed to meet the PSLO. However, one-quarter of the responses
were rated at only approaching PSLO 3.” Our 2012-13 report (Appendix B) on PSLO 2, “Analyze communication practices,
structures, messages, and outcomes in a variety of contexts,” also exceeded the department benchmark (35%) with almost two-thirds
of responses receiving scores of meeting or exceeding PSLO 2. This year’s report more closely aligns with the study of PSLO 1
(Appendix C), “Identify, describe and explain the role of communication in understanding how practices, concepts and rituals are
socially constructed,” in which “96% of undergraduate students failed to meet the PSLO and that only 1 student actually approached
the PSLO.”
Graduate program
The target audience for this study was Communication Studies graduate students enrolled in graduate courses that introduced,
emphasized, or developed PSLO 4-G. Our alignment matrix identifies five such courses, two of which were offered in Spring 2015.
Those two courses addressed PSLO 4-G with varying intensities:
COMS 601 Introduced
COMS 653 Introduced Emphasized
The target audience was reached. The assignment prompt was sent to all students (20) enrolled in the two courses with fourteen
students completing the assignment, a 70% response rate. The Team evaluated all valid responses (minus one response, which was
used for norming).
Given PSLO 4-G concerned students’ ability to analyze communication practices and research, our assessment assignment required
students to summarize and analyze the strengths and weaknesses of a peer-reviewed academic article from a leading journal in the
discipline, "Soliciting Advice: The Role of Sequential Placement in Mitigating Face Threat" (Daena J. Goldsmith, published in
Communication Monographs). After reading the article, students were to respond to the following prompt:
1. Briefly summarize the article, including its central claim, theory, methods, and results/conclusions. Please limit your
4
response to 250-300 words.
2. Analyze the strengths and weaknesses of the article, the methods employed, the evidence generated, the interpretation of the
evidence, and the theoretical contributions. Please limit your response to 750 words.
We used same assessment software (AMEE) to administer the assignment to graduate students. Students received an electronicallygenerated email that included a link to the assignment instructions, directing students to the article and asking them to respond to the
prompt either online or by attaching a document. Students had fifteen days to respond. Emails were regularly sent to students with
increasing frequency as the deadline approached.
The Team evaluated student responses using an assessment rubric similar to those used in the previous years, revised to evaluate the
prompt. The rubric consisted of three evaluation categories: (1) Clear expression of ideas; (2) Summarizes salient points of the article;
and (3) Analyzes the strengths and weaknesses of the article. As with the undergraduate program, the three categories were evaluated
on a 4-point scale: Failed to meet PSLO, Approached PSLO, Met PSLO, and Exceeded PSLO. This three-by-four rubric yields a total
possible score of 12 for each student response. The scoring criteria of the total points were as follows:
10-12 points—exceeded PSLO
7-9 points—met PSLO
4-6 points—approached PSLO
0-3 points—failed to meet PSLO
A large majority of graduate students (70%) were rated as meeting or exceeding PSLO 4-G, with the balance rated as approaching
PSLO 4-G (30%) and none falling to meet PLSO 4-G. While these results seem positive, the average score (7.82 out of 12 possible
points) is 65%, which is fairly low. Moreover, only one response was rated as exceeding PSLO 4-G.
This year’s findings on graduate students demonstrates a stronger response rate than the last time we assessed graduate student
performance in 2011-2012 on PSLO 1, “Identify, describe and explain the role of communication in understanding how practices,
concepts and rituals are socially constructed,” in which “at least 35% of graduate students enrolled in a research methods course met
the benchmark analysis of 30%. However, 45% failed to meet the PSLO. Of the 35% who met the PSLO, one person exceeded the
PSLO” (see Appendix C).
COMS 440 Data
This is the first year that we undertook a before-after, self-report assessment of a course. Carrillo Rowe offered students extra credit
for filling out a survey indicating their awareness of key concepts and methods in cultural studies (Appendix D). In this targeted study,
students used a 1-3 point scale to rate their awareness of key issues, theories, thinkers, and methods in cultural and performance
5
studies. Students took the survey in the first and the last weeks of class. Findings indicate that students report a stronger sense of the
definition, theoretical legacies, and primary theorists, as well as influences like semiotics, within the fields of cultural studies and
performance studies. The survey also shows their increased mastery over key terms like ideology, hegemony, masculinity, and
representation, as well as more general knowledge like how communication shapes social processes and how we see the world.
Students report an increased awareness of key concepts and theoretical influences within the fields of ethnography and performance
studies as well. Finally, students feel more confident in using a variety of methods, especially textual analysis and ethnography.
Students also report using multiple drafts and peer feedback to improve writing, as well as incorporating course concepts into their
written and embodied/performative work.
Faculty Survey
A survey (see Appendix E) was sent to faculty teaching courses that were being assessed for the undergraduate PSLO 4-U asking
them to report on how well PSLO 4-U (especially ethical communication) aligned with their course content. Faculty indicated that
they taught ethical communication in at least one of the courses they taught. A majority of the faculty indicated they covered ethical
communication at a basic level (66%), while just under one-third do so at an intermediate level (31%). Most faculty expected student
proficiency in ethical communication before taking their course to be at a basic level (79.3%). Only 13.8% expected students entering
their course to be at an intermediate proficiency level, and 6.9% expected students to have extensive proficiency in ethical
communication. Just over two-fifths of faculty rarely or never offer course activities that require students to identify the ethical nature
of communication. However, more than half require students to identify the ethical nature of communication at least sometimes. The
following chart provides more detailed results.
Extent to which course
activities require students to
identify the ethical nature of
communication
% of responses
Never
Rarely
Sometimes
Usually
Always
N/A
31%
10%
24%
17%
14%
3%
Most faculty indicated that the emphasis on ethical and effective communication in PSLO 4 aligned with their goals for their
respective courses. Many explained that their course involved both ethical and effective communication (“There is a strong emphasis
on how performance, language, and culture are used ethically and effectively”; “Yes, because group communication is based upon
‘ethical and effective communication’ in order to accomplish goals”). Others noted that they aren’t “necessarily a priority, but
something that overlaps with other aspects of the course.”
6
While it is not necessary to have course activities that require students to identify the ethical nature of communication, most faculty do
not include such an activity in their courses. All faculty indicated they taught ethics as it pertained to communication in the assessed
courses, but generally at a basic level. Almost 80% of faculty expect students to have proficiency in ethical communication before
taking their courses. This may indicate that more coordination is needed to match faculty expectations with student learning.

If your department implemented assessment option B, identify what conclusions were drawn from the analysis of measured
results, what changes to the program were planned in response, and the relation between this year’s analyses and past and
future assessment activities
Undergraduate: The large majority (70%) of undergraduate students are not achieving competency for PSLO 4-U with just under a
third (30%) meeting or exceeding it. Our expectations of student competency for identifying effective and ethical communication are
not being met.
Student participation rates: This year’s participation rate was comparable to last year (50% to 51%) after three years of steady
increases in participation rates (2013-2014—51%; 2012-2013—30%; and 2011-2012—12%). This leveling off may indicate that we
have reached the peak for participation rates using current methods.
Multiple concepts: This assessment illustrates the difficulty in measuring a PSLO containing two learning concepts. PSLO 4-U
assesses students’ ability to identify ethical communication and effective communication, although the relationship between these
concepts is unclear. For instance, while students responded to two prompts (one on ethics, one on effective communication), the
scores are combined into one score. These results do not reflect students’ ability to demonstrate understanding of either concept (they
generally seemed stronger at identifying ‘effective’ than ‘ethical’ communication). Further, the prompt failed to ask students to
examine the relationship between ethical and effective communication (was it ‘effective’ because it was or was not ethical?). The
faculty should consider revising PSLO 4-U to address just one concept.
The results for the graduate program were stronger than those for the undergraduate program. A large majority of graduate students
(70%) achieved competency for PSLO 4-G, with 30% rated as approaching PSLO 4-G and none failing to meet PLSO 4-G. However,
the average score (7.82) as a percentage of possible points (12) is 65%, is fairly low. Moreover, only one response was rated as
exceeding PSLO 4-G.
Suggestions based on assessment
1. Revise PSLO 4: Revising PSLO 4-U to address only one concept would allow assessment to focus on one concept. When revising
that PSLO, we may examine whether competency in analyzing ethical communication should be included as a PSLO. It may be that
questions of ethics are implicit in other PSLO. For example, the ethical nature of the communication arose when we assessed PSLO 37
U (describe and explain the relationship between communication and culture) in AY 2013-2014.
2. Revise assessment teams: The team structure works well in providing full-time faculty an opportunity to directly engage in
assessment. For instance, the areas in which we rate student answers differently (revealed by the inter-rater reliability score) provide
an opportunity for faculty to discuss what counts as a "good" answer. We should, however, consider reconstituting the faculty teams.
The last time Team A conducted assessment (2011-12) scores were also relatively low. The low scores given by Team A could be a
function of the grouping of the faculty members. Further, assessment teams are currently grouped by concentration/track (rhetorical
theory, performance and cultural studies, and communication theory). We should consider organizing the teams so that a diversity of
research fields is represented on each team to better reflect the disciplinary diversity of our department and bring greater range of
skills and perspectives to the assessment of each PSLO. Moreover, our new faculty should be added to the teams, which would likely
generate an additional team (moving us from three to four assessment teams consisting of three faculty members each).
3. Integrate Lecturers: During this cycle, twenty-four faculty and one teaching associate participated in assessment. Two-thirds of the
full-time faculty (eight of twelve) were involved in some phase of the process. However, the participation of part-time faculty
(Lecturers) is limited to the data collection phrase. Their marginal participation is problematic because most of the courses being
assessed are taught be Lecturers. In addition to making the annual assessment results available to them, we should discuss other
methods to generate more full Lecturer participation. One small step we made this year was offering a faculty survey, which used the
same assessment software we use to assess students. This may increase faculty's understanding of the process and our expectations for
students. Further, we might consider conducting assessment workshops and/or creating a shared website through which Lecturers can
share teaching resources relevant to various PSLOs.
4. Reevaluate Alignment Matrix: The results of the faculty survey indicate that the designation of introducing, emphasizing, or
demonstrating PSLO 4-U may not necessarily be accurate. Only two of the twelve courses assessed (COMS 301 and 323) are
designated as just introducing this PSLO. The remaining ten courses are designated as emphasizing or demonstrating the PSLO as
well. Reviewing the level at which these courses address PSLO 4-U would be important to ensure the classroom activities correspond
to the manner in which they address the PSLO. We might conduct workshops with Lecturers, following up on the faculty survey data,
to gain a better sense of the extent to which course content aligns with PLSOs.

If your department implemented option C, identify the program modifications that were adopted, and the relation between
program modifications and past and future assessment activities
Not applicable
8

In what way(s) do your assessment activities reflect the university’s commitment to diversity in all its dimensions but
especially with respect to underrepresented groups
COMS PSLOs and our application of those PSLOs generally relate to issues of diversity as questions of power and are at the center of
communication-based approaches to social life. For instance, this year’s undergraduate authentic tasks entailed students offering a
critical reading of an NFL-sponsored PSA on the topic of domestic violence. Nuanced student answers recognized the ad as a ploy to
assuage the critiques the NFL has faced in its complicity with domestic violence—an issue shaped by issues of gender, race, ability,
and class. Even less astute readings by students indicated an ability to detect issues of power in interpersonal context as well as in
media representations. Last year’s prompt invited students to evaluate their choice of two Ad Buster culture jam spoofs, Corporate
American Flag or iSuck, and to articulate how these messages comment on U.S. culture and its relationship to communication
practices. As with this year’s prompts, students spoke about how the ads marked the consumerism of U.S. American culture as deviant
from the nation’s ideals of democratic inclusion. In AY 2012-2013, students evaluated a debate between Barack Obama and Mitt
Romney, again commenting on issues of national inclusion, especially in terms of gender, race, and class. These examples suggest that
for COMS, the evaluation of student learning is inseparable from the evaluation of students’ engagement with and competency in
issues of diversity.

Any other assessment-related information you wish to include, including SLO revision (especially to ensure continuing
alignment between program course offerings and both program and university student learning outcomes), and/or the creation
and modification of new assessment instruments
3. Preview of planned assessment activities for next year. Include a brief description and explanation of how next year’s
assessment will contribute to a continuous program of ongoing assessment.
Next year is the final year of our five-year assessment plan during which we will assess PLSO 5 for both the undergraduate and
graduate programs. Undergraduate PLSO 5 asks students to “Identify and apply techniques for effectively communicating in a
multicultural global society.” Graduate PSLO 5 asks students to “Make judgments in writing and orally about the relationship between
communication and culture.” Before beginning this year’s assessment activities, we may need to revamp assessment teams to include
new faculty and create better sub-disciplinary representation across teams.
9