Manship School of Mass Communication
Annual Report on Mass Communication Graduate Student Learning Assessment
May 30, 2011
Amy Reynolds, Associate Dean, Graduate Studies & Research
Background
The Manship School graduate program follows an assessment matrix that covers discipline specific
accreditation (ACEJMC) and SACS accreditation. Student learning is assessed in several ways in the
graduate program, including the following:
(1) review of syllabi by the graduate committee (indirect measure)
(2) external professional review of designated courses (direct measure)
(3) review of all theses, professional projects and dissertations by the graduate committee (direct
measure)
(4) completion of the graduate student assessment instrument by each graduate student’s
advisory committee (direct measure)
(5) exit survey administered to all graduate students who complete their program (indirect
measure)
(6) alumni survey (indirect measure)
These varying measures are used in combination to provide yearly assessments. Not every
measure above is used every year. This year (2010-11 academic year), our assessment did not include the
review of syllabi or the alumni survey.1 Beyond what is noted above, our assessment this academic year
did include responding to the results of our ACEJMC accreditation site team visit. Every six years,
accredited journalism and mass communication programs are evaluated by an ACEJMC site team and
given recommendations for improvement. The specific outcomes from that evaluation are noted in the
next section. During academic year 2010-11, the following assessments occurred:
(1)
(2)
(3)
The graduate committee reviewed all theses, professional projects and dissertations
completed during the 2010-11 academic year.
The school administered the exit survey to all master’s and doctoral students who
attended the graduation ceremony during the spring 2011 semester.
The advisory committee members for master’s and doctoral students who defended
(thesis, professional project, or dissertation) in each semester (2010-11 academic
year) completed the graduate student assessment instrument.
ACEJMC Site Team Evaluation
The Accrediting Council on Education in Journalism and Mass Communications (ACEJMC) is the
agency responsible for the evaluation of professional journalism and mass communication programs in
colleges and universities. Every six years, programs seeking accreditation write an extensive self-study,
and provide program data related to nine standards – mission, governance and administration; curriculum
and instruction; diversity and inclusiveness; full-time and part-time faculty; scholarship: research, creative
1
During the fall semester of 2008, graduate faculty members reviewed all syllabi for graduate
courses and made several recommendations. As a result, the school implemented changes to courses and
program requirements. In fall 2009, four graduate courses were selected for external professional review.
Course work from a sample of students in each class was collected during the semester. A committee of
professional reviewers assessed the course work during the spring semester 2010. The next review of
syllabi is scheduled for fall 2011. The next alumni survey is scheduled for 2012.
and professional activity; student services; resources, facilities and equipment; professional and public
service; and, assessment of learning outcomes. The program submits the self-study to ACEJMC and it is
part of the evaluation process. Also during the accreditation year, a site team of 4-7 trained
administrators, faculty, and professionals from our field visit the program and evaluate it on those same
nine standards. The self-study and the report written by the site team are the primary documents used by
the council to determine re-accreditation. This is done at the undergraduate and master’s degree levels.
In 2009-10, the Manship School received full re-accreditation. On the graduate assessment, the
site team noted that the program had only two weaknesses: “A limited number of graduate courses
outside the core curriculum” and “an overly-broad definition of professional skills courses.”
As a result of the site team’s assessment, the full faculty acted on several recommendations that a
faculty committee put forth during the spring (2010) semester. That committee recommended splitting
the master’s degree into two tracks, one professional and one scholarly. During the summer of 2010,
discussions about reshaping the master’s degree continued, and the two-track idea was refined and
specifically defined. In the fall semester (2010) the full faculty voted to make the following changes to
the master’s degree:
1. Reduce the core by one course (drops from 16 to 13 hours)
2. Create a new professional, skills-based track to better serve the needs of professional
students, and to address the site team’s concern about more clearly defining the professional
skills courses
3. Create five new graduate courses to anchor the professional track. Three of these new
courses – Crisis Communication, Political Communication Writing, and Public Affairs
Reporting – received campus approval during the 2010-11 academic year. Two additional
courses, Strategic Communication I and Strategic Communication II, are in development and
we will seek campus approval and permanent course numbers for them during the 2011-12
academic year. Four of these five courses are scheduled for the 2011-12 school year. We are
offering the two strategic communication courses as special topics courses as we finalize
course proposals to send forward to the campus for approval next year. This addresses the
site team’s concern that we have a limited number of graduate courses outside the core
curriculum.
The School received campus approval of the new degree during the spring 2011 semester, and the
new curriculum is available to students beginning this fall (2011).
Review of Theses, Professional Projects and Dissertations
The review of theses, dissertations and professional projects for the academic year 2010-11
included 14 theses, three professional projects and four dissertations. We also had one master’s student
complete the degree through the comprehensive exam process.
Graduate School and Manship School process require each dissertation committee have five
faculty members (one as the graduate dean’s representative from outside the School, then at least three of
the other four coming from the Manship School) all with standing as graduate faculty members on
campus. At least half the committee must be full members of the graduate faculty. For theses, projects
and exams at the master’s level, committees have three faculty members, all within the Manship School,
and at least one a full/senior member of the graduate faculty. All committee members are graduate
faculty.
The diligence with which we constitute committees is the best safeguard for quality. All of the
dissertations, theses, projects and the one exam from this past academic year highlighted the rigor of the
process. In all cases, students passed their defenses, and spent a minimum of one week making final
adjustments/changes to their work to complete the final project and graduate. On average, students spent
about 3-4 weeks making final adjustments beyond the defense.
Projects
All of the projects evaluated were strong. Each showcased the students’ strength in writing and
communication skills; the ability to incorporate a research element into a professional work; clear and
concrete knowledge of a student’s professional area of training; and, the ability to produce high-quality
professional work that would serve as a component in a student’s portfolio. Based on the high quality of
the projects and the feedback from the student exit survey (see exit survey section later in the report), the
faculty should work to facilitate more professional student projects, particularly if a student chooses the
newly created professional track of the degree.
Theses
Nearly all of the theses were above average, and each was well written and showed the students’
ability to conceptualize a major research project and bring it to fruition. The biggest weakness in the
theses was on the methodological side. About a fifth of the theses were weak in executing quantitative
methodologies, mostly on the design side. Many did not have enough data or significant findings to
support their hypotheses, and in a couple of cases an early re-design of the method might have fixed the
student’s problem. Encouraging committees to ask students to pre-test quantitatively designed projects is
recommended. It’s worth noting that this weakness only showed in a few theses and was not a sufficient
weakness to fail any of the students. It’s possible that some of the students who struggled
methodologically with a thesis would have been better served on a professional track with a professional
project. In most of the cases where students struggled methodologically, these students were professional
students. They didn’t take a heavy load of scholarly courses within the degree program, so the choice of a
thesis at the end of their program may not have been the most appropriate. As scholarly students pursue
the new scholarly track, the preparation for a thesis is much more rigorous and should eliminate the minor
methodological issues observed.
Exam
The comprehensive exam option is new for our program. This year, we had our first student
complete the master’s degree by taking the comprehensive exams. This student struggled with the
process, and barely passed. Her exam committee believes that her struggles were related to a lack of
understanding of what is involved in a comprehensive exam at the master’s level. The recommendation is
to edit the current guide to the comprehensive exams to better explain the process to future students. The
student’s committee believes that had the student been more aware of what the comprehensive exam
process involved she would have fared better. This was particularly true on the written side. The student
effectively “saved” herself in the oral defense, where she showcased the depth and breadth of her
knowledge. The disconnect between her performance in the written and oral components, and subsequent
discussions with the student, leads us to believe that she was not as clear of the written exam expectations
as she should have been. Part of that responsibility rests with the student, but part also rests with the
program.
Dissertation
The dissertations reviewed were disappointing. In no case did a student fail, nor should he/she
have failed. But, the level of overall success was average (see assessment instrument findings in the next
section) for the four students included in the 2010-11 academic year reviews. The biggest weaknesses
came in the area of methodology. Like the few master’s students who struggled methodologically, these
students also had difficulty in methodological design, interpreting data and results and applying statistical
procedures. In some cases – two involving students in which English was a second language – the
writing was weaker than we would normally expect to see.
In all other areas, the dissertations were above average. Students showed the ability to apply
relevant theories; to demonstrate depth of knowledge; to make a contribution to the field; and, to think
critically, creatively and independently.
It’s difficult to make sweeping generalizations based on so few students, but based on the review
of these dissertations, the recommendation is to ask the Graduate Committee to review the doctoral
curriculum and explore ways to enhance the methodological training at the doctoral level.
Graduate Student Assessment Instrument, 2010-11
During the fall semester of 2008, the associate dean for graduate studies and research introduced
a new instrument for the measurement of graduate learning outcomes. The graduate student assessment
instrument consists of ten statements that are rated on a 5-pt scale (1=unsatisfactory, 2=below average,
3=average, 4=above average, 5=superior or N/A if unable to judge). The statements assess the student’s
knowledge of mass communication theories, the understanding and application of research methods and
statistical procedures, core values and competencies2, written and oral communication skills, and mastery
of course content. It also allows for comments by committee members for program improvement. Each
graduate student’s advisory committee completes the instrument after the student’s Final Examination
(defense) every semester.
The results of the student assessment for 2010-11, compared to the previous year’s results, are as
follows:
Average Score for Each Criterion/Item (5-pt. scale)
MMC Students: All
Knowledge and application of mass communication theories
Ability to conceptualize and conduct research
Understanding and application of research methods
Understanding and application of statistical/analytical procedures
Understanding of course content of degree program
Understanding of area of specialization
Understanding of (AEJMC) core values and competencies
Overall quality of thesis or professional project or dissertation
Oral communication skills
Written communication skills
09-10
4.18
4.05
4.00
3.95
4.40
4.43
4.41
4.07
4.24
3.88
10-11
3.88
4.16
3.58
3.64
4.22
4.38
4.26
4.00
4.11
3.83
MMC Students (2010-11): Project, Thesis, Exam
Knowledge and application of mass communication theories
Ability to conceptualize and conduct research
Understanding and application of research methods
Understanding and application of statistical/analytical procedures
Understanding of course content of degree program
Understanding of area of specialization
Understanding of (AEJMC) core values and competencies
Overall quality of thesis or professional project or dissertation
Oral communication skills
Written communication skills
Proj
4.33
5.00
4.00
3.66
4.33
4.66
5.00
4.66
5.00
4.66
Thes
3.85
4.07
3.92
369
4.28
4.42
4.18
4.00
4.00
3.92
Exam
3.00
3.00
2.00
3.00
3.00
3.00
3.00
2.00
3.00
3.00
4.53
4.03
2.80
Overall Average
PhD Students
09-10 10-11
Knowledge and application of mass communication theories
4.53
3.37
2
The core values and competencies are the ACEJMC professional values and competencies, which are
listed at the end of this report. Ability to conceptualize and conduct research
Understanding and application of research methods
Understanding and application of statistical/analytical procedures
Understanding of course content of degree program
Understanding of area of specialization
Understanding of (AEJMC) core values and competencies
Overall quality of thesis or professional project or dissertation
Oral communication skills
Written communication skills
Overall Average (MMC)
Overall Average (PhD)
Overall Average (both programs)
4.63
4.32
4.28
4.53
4.76
4.64
4.61
4.56
4.58
3.25
3.13
3.25
3.37
3.75
4.00
3.37
3.25
3.25
09-10
4.14
4.53
4.32
10-11
4.00
3.39
3.69
In 2009-10, master’s students scored highest in their understanding of their area of specialization,
but lowest in their understanding of statistical and analytical procedures and written communication
skills. In 2009-10, doctoral students ranked highest in their understanding of their area of specialization
and the course content of their degree program, and lowest in their understanding of statistical and
analytical research procedures. This year’s results are similar. Overall, master’s students scored highest
in understanding their area of specialization and lowest in understanding and application of research
methods. Doctoral students scored highest in understanding ACEJMC core values and competencies, and
lowest in understanding and application of research methods.
Previous efforts to address the weaknesses on the methodological side included the addition of
new courses: MC7095 Media History: Research and Writing, MC7014 Qualitative Research Methods,
and MC7202 Experimental Applications in Mass Communication Research. To enhance the theoretical
side, two new theory courses were added as well. But, these courses were only offered in two-year
rotations, so the number of students (particularly master’s students) who could take these courses wasn’t
maximized. These courses need to be offered more often, and the doctoral curriculum needs
reexamination to ensure that doctoral students are receiving the kind of rigorous methodological training
they need.
The observed weakness in methodology in this assessment instrument matches the weakness
observed in the dissertation review. In terms of the master’s degree, the faculty thinks the new tracks will
largely address this issue because a student’s coursework will better prepare and match his/her objectives
and outcomes (professional or scholarly). As noted in the previous section, meaningful differences
existed among projects, theses and the exam outcomes. When the assessment instrument data is
examined in this more precise way, the students who completed the projects demonstrated the strongest
abilities, while the lone student who took the exams fared most poorly. The assessment instrument data
supports the recommendation that the two tracks should independently prepare students for all final
projects, whether professional or thesis. That is, the final project should match the student’s track
(professional = project; scholarly = thesis); and, the coursework on each track should prepare a student for
the appropriate final project. The recommendation to better explain the exam process to students is also
supported.
Exit Survey
The exit survey is administered every May. Seven graduate students received their degrees in the
spring of 2011, although one did not complete the exit survey. No doctoral students graduated in May
2011. The average GPA of these students is 3.66. The average time it took all of the students to graduate
is a little over three years. Four master’s students finished the program in two years, and one of the
master’s students was a part-time student. Of the graduates, one specialized in advertising; four
specialized in public relations, and one in political communication. Of the six graduate students, two are
male and four are female.
Assistance and resources
Only two master’s students completed an internship while in the program, and both had paid
positions. One of the graduate students consulted the internship coordinator. All but one, however, did
consult the counselors, and all five found the counselors to be helpful (the associate dean for graduate
studies is the primary adviser/counselor for graduate students).
Work and funding
All six students said they worked while in school. Four said they worked on campus for 20 hours
per week or less. One said they worked off campus for more than 30 hours per week, and another said
they worked both on and off campus for more than 40 hours per week. Only five students answered the
question about funding sources. Of those five, four listed graduate assistantship or fellowship as their
primary source of funding. One said he or she funded his education through the TOPS program.
Careers
Only four of the six students started their job search prior to graduation. Of these four, half have
found jobs. They will be working as a program coordinator at the Reilly Center at LSU and as a
communications specialist at BASF. Of the three students who state that they will not be entering the
work force full-time, one will be continuing on in a doctoral program, one will continue to law school,
and one will work part-time until full-time work comes along.
Faculty
Overall, the Manship faculty received good reviews from the graduate students. All agreed the
faculty had high expectations for their work and that those expectations were realistic. Five of the six
agreed that the faculty had relevant professional experience, and four of the six agreed that faculty made
them aware of practical applications of the course content, with one student disagreeing. All respondents
said faculty provided adequate feedback. Four students said faculty provided students with opportunities
or involvement in course-related projects that extend beyond the classroom, with one student strongly
disagreeing with that statement. All agreed the faculty treated students with respect.
Career skills
The Manship School stresses teaching skills students can use in their professional careers. The
results of this survey indicate the school does a good job of providing students with these skills. All
agreed faculty expected them to write well, and all said the school enabled them to write well. All six of
the students said the Manship School trained them to speak well. All six agreed that faculty expected
them to think critically, and all said the school enabled them to think critically. All said the school
enabled them to problem solve and to perceive the similarities and differences in ideas. The six students
also said the school enabled them to generate original thought, and all said the school enabled them to
critically evaluate their own work and the work of others. All but one of the students said faculty expected
them to be familiar with media history, and all said faculty expected them to have an understanding of
mass communication theory. All students said the Manship School enabled them to have an
understanding of mass communication theory and an understanding of the freedom of speech and the
press. Five of the six students said that their experiences at the Manship School enabled them to deal with
media ethics. Five of the six also said faculty expected them to understand statistics and to understand
research methods and that the school provided them with an understanding of statistics. All said the
school gave them an understanding of mass communication research methods. The six students said it
was important to have a diverse faculty and it was important to have a diverse student body. All of the
students said they understood issues of diversity and that they work well in groups. Four of the six
students said the school provided them with sufficient job skills to enter their chosen field, with one
student disagreeing.
Technology
The Manship School is dedicated to providing the best technology the University can offer to its
students. Five of six agreed faculty adequately use technology in the classroom. All agreed the school
provides state-of-the-art equipment and facilities. Five of the six said the school enabled them to use
computers effectively.
Conclusions
The graduates gave adequate overall reviews of their Manship education. All of the students said
the quality of instruction was good and their courses were appropriately challenging. All six students
place a high value on their education. Several of the students left comments at the end of the survey.
Many identified the faculty as one of the greatest strengths of the program. However, others also stated
that the faculty should be more open to working with students on professional projects. Other areas
identified for improvement include the core curriculum and promotion of the program on a national level.
Overall, the students voice concerns over the balance of the academic and professional aspects of
the master’s program. They seem to think the program is oriented more toward academic interests than
professional interests. As discussed earlier, they gave relatively low ratings for opportunities to work with
faculty on outside projects, indicating the faculty could do a better job of making students aware of
practical applications for coursework and of professional opportunities outside of the classroom. It is
unknown whether or not these students sought out guidance, and whether or not these students sought out
the professionally oriented courses. To a certain extent, it is possible that these students failed to
understand how to personalize the master’s program to fit their own needs and goals. For example, only
two students had an internship. It seems that students interested in professionally oriented education
would seek out an internship. Only one student used the school’s internship coordinator, and it was to
gather general information. It could be that these students were generally unhappy with the program’s
core curriculum and would have preferred more leeway to take courses in their respective areas of
concentration. These results come from a very small sample size (n = 6). There were not enough
graduating master’s students to get a representative sample on this survey. These results, therefore, cannot
be generalized to the rest of the graduate students.
Use of Results to Improve Program
Several general themes emerge from the data. First, the master’s degree program needs to do a
better job meeting the needs of its professional students in terms of coursework and in terms of
encouraging more professional students to complete a professional project instead of a thesis. The exit
survey suggestions that students felt the program was unbalanced – that the emphasis was too scholarly,
and that they did not have as many professional course options. The two new degree tracks and the
creation of new professional courses outside the core should bring the program back in balance. The
ACEJMC site team, the exit survey and the assessment data all support the idea that the professional side
of the program required attention. The new master’s degree curriculum is a big step in the right direction
to addressing this problem. For next year, the program will ensure that the newly created professional
courses are offered, and that more students will have the option to pursue a professional project as the
final step to degree completion.
Second, the review of projects, theses, dissertations and exams, and the graduate student
assessment instrument both show that our scholarly track for the master’s students and the doctoral
curriculum need to focus on enhancing methodological training. The program needs to make sure it’s
offering methods courses regularly, and it needs to review the doctoral curriculum with an eye toward
improving methodological training within the program.
Finally, the program needs to change the administration of the exit survey. Because graduate
students finish their degrees year-round, we’re missing the feedback from a large number of students by
only administering the exit survey in the spring. We had 6 students out of 21 complete the survey for this
academic year. That number is too small to really be helpful to our assessment. Beginning this summer
(2011), we will administer the exit survey at each graduation – summer, fall and spring.
ACEJMC Professional values and competencies
Individual professions in journalism and mass communication may require certain specialized values and
competencies. Irrespective of their particular specialization, all graduates should be aware of certain core
values and competencies and be able to:
• understand and apply the principles and laws of freedom of speech and press, for the country in which
the institution that invites ACEJMC is located, as well as receive instruction in and understand
the range of systems of freedom of expression around the world, including the right to dissent, to
monitor and criticize power, and to assemble and petition for redress of grievances;
• demonstrate an understanding of the history and role of professionals and institutions in shaping
communications;
• demonstrate an understanding of gender, race, ethnicity, sexual orientation and, as appropriate, other
forms of diversity in domestic society in relation to mass communications.
• demonstrate an understanding of the diversity of peoples and cultures and of the significance and
impact of mass communications in a global society.
• understand concepts and apply theories in the use and presentation of images and information;
• demonstrate an understanding of professional ethical principles and work ethically in pursuit of truth,
accuracy, fairness and diversity;
• think critically, creatively and independently;
• conduct research and evaluate information by methods appropriate to the communications professions
in which they work;
• write correctly and clearly in forms and styles appropriate for the communications professions,
audiences and purposes they serve;
• critically evaluate their own work and that of others for accuracy and fairness, clarity, appropriate style
and grammatical correctness;
• apply basic numerical and statistical concepts;
• apply tools and technologies appropriate for the communications professions in which they work.