Speaking Skills Assessment for SPEECH 211 - SPRING 2014
Speech and Theatre Department – Assessment committee: Dr. Ferrari-Bridgers, Professor Lynch and Professor Vogel
Data analysis and report written by Dr. Franca Ferrari-Bridgers
List your student learning outcomes as described in your syllabus. Please list ALL of the Student Learning Outcomes that are listed in
your syllabus
Gen Ed. Obj.
1. Communicate effectively
through reading, writing,
listening and speaking
Outcome desired
To develop the public
speaking skills
necessary to
effectively present
informative and
persuasive speeches
2. use analytical reasoning to
identify issues or problems and
evaluate evidence in order to
make informed decisions
To develop critical
thinking and problemsolving skills that
enable students to
understand the intricate
link between audience,
speaker, and occasion
Outcome desired
To develop students as
more effective listeners
and evaluators of
communication, in order
to make them, in turn,
more capable learners
and intelligent decisionmakers.
Outcome desired
To learn the major
communication principles in
public speaking, interpersonal
communication, selfcommunication, intercultural
communication, and group
communication. Students
will be able to incorporate
these theories into their own
speaking styles
Outcome desired
To work towards
understanding and
overcoming
communication
apprehension
3. reason quantitatively and
mathematically as required in
their fields of interest and in
everyday life
4. use information management
and technology skills effectively
for academic research and
lifelong learning
To develop skills in
diverse communication
contexts including
small groups,
computer-mediated
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 1
communities and
professional
communities
5. integrate knowledge and skills
in their program of study
To understand the
overwhelming
importance of effective
communication in all
aspects of academic,
professional, and
everyday life
6. differentiate and make
informed decisions about issues
based on multiple value systems
7. work collaboratively in
diverse groups directed at
accomplishing learning
objectives
8. use historical or social
sciences perspectives to examine
formation of ideas, human
behavior, social institutions, or
social processes
9. employ concepts and methods
of the natural and physical
sciences to make informed
judgments
10. apply aesthetic and
intellectual criteria in the
evaluation or creation of works
in the humanities or the arts
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 2
Describe the assessment activity and the (student learning outcome(s) it addresses) that occurred in your course.
Students’ verbal and non-verbal communication skills will be tested twice during the semester in concomitance with students’ informative and
students’ persuasive speeches. The first speech is usually delivered between the 5 th and the 7th week of the semester while the second speech is
delivered towards the end of the semester between the 12 th and the 14th week. The instructor will use the rubric attached below to grade students’
verbal and non-verbal performance during the two speeches. At the end of the speech assessment the instructor will assign a grade. The grade will
determine whether and how the student has met the course and the Gen Ed objectives tested for this assignment. The objectives that need to be met
with this assignment are listed below in bold. Final grade for this assessment needs to be converted by the instructor into 1-5 Assessment
Measuring Scale (AMS) that will be used by the department to track student progress.
Objective of the Course:
1) To give the student, as part of a liberal education, a greater understanding and appreciation of speech communication and its functions in
contemporary society.
2) To develop students as more effective listeners and evaluators of communication, in order to make them, in turn, more capable learners
and intelligent decision-makers.
3) To develop the student as a speaker, in interpersonal communications, problem-solving group discussion, and as a “public” speaker.
General Education Objectives addressed by the course:
1. Communicate effectively through reading, writing, listening and speaking.
4. Use information management and technology skills effectively for academic research and lifelong learning.
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 3
List the data collection instrument (s) used for assessment.
Students’ verbal and non-verbal skills will be assessed twice in the semester using the following rubric.
Speaking Assessment Rubric
Speaker: ___________________________________________
LANGUAGE
Appropriate
language;
Appropriatelanguage
Minors errors
in grammar.
Proper grammar.
Proper
grammar.
Simple
vocabulary or
jargon.
10/40
College level
vocabulary.
Acceptable
vocabulary.
Several
errors in
grammar or
Poor
grammar and
vocabulary.
Vocabulary.
Uses
figurative
language.
4 points
2 points
8 points
Fluency
DELIVERY
Uses excellent
pronunciation and
diction. Free of
vocal fillers
0 points
Adequate
pronunciation
and diction.
Free of vocal
fillers
Few
pronunciation
and diction
errors/vocal
fillers
Several
pronunciation
and diction
errors/vocal
fillers
4 points
3 points
2 points
5 points
15/40
Offensive
Language
6 points
10 points
VOCAL
Poor language/
grammar.
Figurative
language/
Many errors
in
pronunciation
and
diction/vocal
fillers
Not
understandable
0 points
1 points
Volume
Can be heard in all
sections of
audience,
modulates to add
emphasis
Adequately
projects for
audience
understanding
Can be heard
in all sections
of audience
3 points
4 points
With some
audience
effort can be
heard in all
sections of
audience
Inadequate
projection
1 points
Cannot be
heard
anywhere in
the audience
5 points
2 points
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
0 points
Page 4
Rate
Can creatively
modulate rate to
add emphasis
5 points
Adequately
uses rate
for audience
understanding
Uses a rate
acceptable to
the audience
3 points
4 points
PHYSICAL
DELIVERY
Eye
contact
15/40
Speaker uses
excellent eye
contact and scans
entire room
Speaks
slightly too
fast or too
slowly for the
audience
Speaks much
too fast or
too slowlyfor
the audience
Rate
unacceptable
1 points
0 points
Speaker has
intermittent
eye contact
Speaker
exhibits poor
eye contact
Speaker
exhibits no eye
contact
(reading)
2 points
1 points
2 points
Speaker uses
good eye
contact and
scans entire
room
Speaker uses
good eye
contact and
scans most of
room
4 points
3 points
Speaker uses
good facial
and body
gestures to
reinforce
meaning
Speaker uses
adequate
facial and
body gestures
to reinforce
meaning
4 points
3 points
Speaker uses
good body
control
Speaker uses
adequate body
control
4 points
3 points
5 points
Gestures
Speaker uses
excellent facial and
body gestures to
reinforce meaning
5 points
Posture
Speaker uses
excellent body
control: avoids
slouching and
extraneous
movements
0 points
Facial and
body gestures
are stilted
Speaker has
flat affect
uses few
body gestures
Speaker does
not use either
facial or body
gestures
1 points
0 points
Speaker has
little body
control
Speaker has no
body control
2 points
Speaker leans
or shuffles,
dances or has
other
extraneous
body
movements
0 points
1 points
5 points
2 points
LEGEND: “0” = no mastery; “1” = minimal mastery; “2” = developing mastery; “3” = adequate mastery; “4”= good mastery
“5”= exceptional mastery
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 5
Legend to convert letter or percentage point grades into 1-5 Assessment Measuring Scale (AMS)
“1” means that the student did not meet course objective (3) as public speaker and GenEd objective (1) for speaking.
“2” means that the student has a minimal mastery of course objective (3) as public speaker and GenEd objective (1) for speaking.
“3” means that the student is developing mastery course objective (3) as public speaker and GenEd objective (1) for speaking.
“4” means that the student has an adequate mastery of course objective (3) as public speaker and GenEd objective (1) for speaking.
“5” means that the student masters course objective (3) as public speaker and GenEd objective (1) for speaking.
Conversion Scale
AMS
Letter
Percentage
“1”
F
59% below
“2”
D
60-69%
“3”
C
70-79%
“4”
B
80-89%
“5”
A
90% above
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 6
Provide an analysis (and summary) of the assessment results that were obtained.
In this study 157 students1 of Speech 211 were tested on their competence in the following verbal and non-verbal skills: LANGUAGE, VERBAL
SKILLS (FLUENCY, VOLUME, RATE) AND NON VERBAL SKILLS (EYE CONTACT, GESTURE, POSTURE). Students’ raw scores and
means for each of the tested components show that students improved from the first to the second speech in each of seven assessed constructs as
summarized in the following tables.
RAW
Speech1
Speech2
SCORES
LANGUAGE
1112
1258
FLUENCY2
514
635
VOLUME
590
681
RATE
554
669
EYECON
450
605
GESTURE
492
653
POSTURE
512
660
%Change
Speech1
Speech2
MEANS
12.7
LANGUAGE
7.1
8.0
3
24.6
FLUENCY
6.5
8.1
16
VOLUME
7.5
8.7
19.7
RATE
7.1
8.5
35.1
EYECON
5.7
7.7
31.7
GESTURE
6.3
8.3
29.2
POSTURE
6.5
8.4
TOTAL
6.7
8.2
22.4
1
The slightly inferior number of students tested in Spring 2014, 157, versus the 213 students of Fall 2013, is due to the fact that only the data of FerrariBridgers, Lynch, Thompson and Vogel were available at the end of the semester.
2
Fluency, volume, rate, eye contact, gesture and posture are based on a 5 point grading scale, whereas language is on a 10 point grading scale.
3
Means for fluency, volume, rate, eye contact, gesture and posture have been recalculated on a 10 point grading scale from an initial 5 point grading scale to
be comparable with the 10 point grading scale for Language.
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 7
We conducted a repeated measure ANOVA. As indicated in the table, the test is significant, the obtained F values are well above the critical F
values. This suggests that there is an overall significant change in performance from the first to the second speech as a whole as well as there is
significant improvement for each of the seven speaking components between Speech 1 and Speech 2.4
4
Source of
Variation
Sample
Columns
Interaction
Within
SS
339.6251
87.62966
21.28571
2566.013
Df
Total
3014.553
2197
1
6
6
2184
MS
F
P-value
339.6251 289.0637
5.6E-61
14.60494 12.43065 8.17E-14
3.547619 3.019471 0.006087
1.174914
F crit
3.84572
2.10273
2.10273
Values for languages have been converted from a 10 point scale to a 5 point scale to be comparable with the other 5 point values before running the ANOVA.
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 8
Describe how the assessment results that were obtained affected (or did not affect) the student learning outcomes you identified. As part of
your discussion, describe any plans you have to address the areas where students need to improve.
The overall growth rate shows that students improved their ability to speak in front of an audience at an acceptable rate in all the in all seven
assessed speaking skills as depicted in the following chart.
Students overall improved 15 % between Speech 1 and Speech 2, with the largest increase for EYECONTACT (35.1%) and GESTURE (31.7%)
and POSTURE (29.2%). Good improvement was also shown for FLUENCY (24.6%), RATE (16%) and VOLUME (19.7%). The least amount
of improvement was registered for LANGUAGE (12.7%).
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 9
Students’ average grades for each of seven assessed skills in SPEECH 1 and SPEECH 2 are reported in the following table.
SPEECH1
SPEECH2
Average Grade
Average Grade
-C
-B
LANGUAGE
71
80
5
D
-B
FLUENCY
65
81
C
+B
VOLUME
75
87
-C
B
RATE
71
85
F
+C
EYECON
57
77
D
-B
GESTURE
63
83
D
B
POSTURE
63
84
+D
-B
OVERALL
67
82
The grade results show that on an average our students’ scores changed from an initial F, D or C- to a final C+, B and B+ suggesting that students
dramatically improved their speaking skills on an average of two letter grades from the beginning to the end of the semester.
Comparison between Fall 2013 and Spring 2014 results
A comparison between the results of the second speech of students from Fall 13 and Spring 14 was established. Means for each of the seven
speech constructs were compared as indicated in the table below.
MEANS Speech2_Fall13 Speech2_Spring14
8.06
Language
8.01
8.03
Fluency
8.08
8.55
Volume
8.67
8.12
Rate
8.52
7.17
Eye Cont
7.7
7.3
Gesture
8.31
8.27
Posture
8.4
TOTAL
7.9
8.2
5
Means for fluency, volume, rate, eye contact, gesture and posture have been recalculated on a 10 point grading scale from an initial 5 point grading scale to
be comparable with the 10 point grading scale for Language.
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 10
A T-test on the means of the scores of each construct was performed. The t-test was significant (t-stat 2.24, p =.03, t-critical 1.94) indicating that
students from Spring 2014 slightly improved in their overall final speech performance with respect to Fall 2013’s students. We attribute this
improvement to the changes we made to our teaching curriculum by adding more exercises directed to improve students’ verbal and non-verbal
skills, especially eye contact and gestures.
As part of your discussion, describe any plans to improve/modify the speech 211 course to address the speaking skills’ results obtained
through the assessment.
In order to achieve better coordination among the full time speech faculty, Professors Carrey, Ferrari-Bridgers, Lynch, Rada, Thompson and Vogel
met twice during the Spring semester and we also held a two days faculty retreat in May to discuss various matters related to speech 211 and
remedial speech courses. During the retreat Professors Carrey, Ferrari-Bridgers, Lynch, Rada, Thompson and Vogel agreed on the creation of a
speech club to help students to improve their speaking skills outside the classroom. The speech club/lab is designed as a scheduled conversational
time (similar to the ‘conversation teas’ sponsored by the foreign language department) to practice and improve their speaking skills. Professor
Thompson together with a group of interested students will be in charge of the planning of the speech club. We plan to have the speech club ready
to run either in Fall 2014 or Spring 2015. Moreover, we will discuss the results of this report with all the speech faculty during our first Fall 2014
meeting and we will also continue to update our common Blackboard library with more speaking activities to use in class.
Dr. Franca Ferrari-Bridgers (09/04/2014)
Speech 211
Page 11