Speaking Skills Assessment for SPEECH 211 - SPRING 2015
Speech and Theatre Department –
Data collection, Data analysis and report written by Dr. Franca Ferrari-Bridgers
List your student learning outcomes as described in your syllabus. Please list ALL of the Student Learning Outcomes that are listed in
your syllabus
Gen Ed. Obj.
1. Communicate effectively
through reading, writing,
listening and speaking
Outcome desired
To develop the public
speaking skills
necessary to
effectively present
informative and
persuasive speeches
2. use analytical reasoning to
identify issues or problems and
evaluate evidence in order to
make informed decisions
To develop critical
thinking and problemsolving skills that
enable students to
understand the intricate
link between audience,
speaker, and occasion
Outcome desired
To develop students as
more effective listeners
and evaluators of
communication, in order
to make them, in turn,
more capable learners
and intelligent decisionmakers.
Outcome desired
To learn the major
communication principles in
public speaking, interpersonal
communication, selfcommunication, intercultural
communication, and group
communication. Students
will be able to incorporate
these theories into their own
speaking styles
Outcome desired
To work towards
understanding and
overcoming
communication
apprehension
3. reason quantitatively and
mathematically as required in
their fields of interest and in
everyday life
4. use information management
and technology skills effectively
for academic research and
lifelong learning
To develop skills in
diverse communication
contexts including
small groups,
computer-mediated
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 1
communities and
professional
communities
5. integrate knowledge and skills
in their program of study
To understand the
overwhelming
importance of effective
communication in all
aspects of academic,
professional, and
everyday life
6. differentiate and make
informed decisions about issues
based on multiple value systems
7. work collaboratively in
diverse groups directed at
accomplishing learning
objectives
8. use historical or social
sciences perspectives to examine
formation of ideas, human
behavior, social institutions, or
social processes
9. employ concepts and methods
of the natural and physical
sciences to make informed
judgments
10. apply aesthetic and
intellectual criteria in the
evaluation or creation of works
in the humanities or the arts
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 2
Describe the assessment activity and the (student learning outcome(s) it addresses) that occurred in your course.
Students’ verbal and non-verbal communication skills will be tested twice during the semester in concomitance with students’ informative and
students’ persuasive speeches. The first speech is usually delivered between the 5th and the 7th week of the semester while the second speech is
delivered towards the end of the semester between the 12th and the 14th week. The instructor will use the rubric attached below to grade students’
verbal and non-verbal performance during the two speeches. At the end of the assessment, the instructor will assign a grade. The grade will
determine whether and how the student has met the course and the Gen Ed objectives tested for this assignment. The objectives that need to be met
with this assignment are listed below in bold. Final grade for this assessment needs to be converted by the instructor into 1-5 Assessment
Measuring Scale (AMS) that will be used by the department to track student progress.
Objective of the Course:
1) To give the student, as part of a liberal education, a greater understanding and appreciation of speech communication and its functions in
contemporary society.
2) To develop students as more effective listeners and evaluators of communication, in order to make them, in turn, more capable learners
and intelligent decision-makers.
3) To develop the student as a speaker, in interpersonal communications, problem-solving group discussion, and as a “public” speaker.
General Education Objectives addressed by the course:
1. Communicate effectively through reading, writing, listening and speaking.
4. Use information management and technology skills effectively for academic research and lifelong learning.
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 3
List the data collection instrument (s) used for assessment.
Students’ verbal and non-verbal skills will be assessed twice in the semester using the following rubric.
Speaking Assessment Rubric
Speaker: ___________________________________________
LANGUAGE
Appropriate
language;
Appropriatelanguage
Minors errors
in grammar.
Proper grammar.
Proper
grammar.
Simple
vocabulary or
jargon.
10/40
College level
vocabulary.
Acceptable
vocabulary.
Several
errors in
grammar or
Poor
grammar and
vocabulary.
Vocabulary.
Uses
figurative
language.
4 points
2 points
8 points
Fluency
DELIVERY
Uses excellent
pronunciation and
diction. Free of
vocal fillers
0 points
Adequate
pronunciation
and diction.
Free of vocal
fillers
Few
pronunciation
and diction
errors/vocal
fillers
Several
pronunciation
and diction
errors/vocal
fillers
4 points
3 points
2 points
5 points
15/40
Offensive
Language
6 points
10 points
VOCAL
Poor language/
grammar.
Figurative
language/
Many errors
in
pronunciation
and
diction/vocal
fillers
Not
understandable
0 points
1 points
Volume
Can be heard in all
sections of
audience,
modulates to add
emphasis
Adequately
projects for
audience
understanding
Can be heard
in all sections
of audience
3 points
4 points
With some
audience
effort can be
heard in all
sections of
audience
Inadequate
projection
1 points
Cannot be
heard
anywhere in
the audience
5 points
0 points
2 points
Rate
Can creatively
modulate rate to
Dr. Franca Ferrari-Bridgers (06/05/2015)
Adequately
Uses a rate
acceptable to
Speaks
slightly too
Speech 211
Speaks much
too fast or
Rate
Page 4
add emphasis
uses rate
the audience
5 points
for audience
understanding
3 points
fast or too
slowly for the
audience
too slowlyfor
the audience
unacceptable
1 points
0 points
2 points
4 points
PHYSICAL
DELIVERY
Eye
contact
15/40
Speaker uses
excellent eye
contact and scans
entire room
Speaker uses
good eye
contact and
scans entire
room
Speaker uses
good eye
contact and
scans most of
room
4 points
3 points
Speaker uses
good facial
and body
gestures to
reinforce
meaning
Speaker uses
adequate
facial and
body gestures
to reinforce
meaning
4 points
3 points
Speaker uses
good body
control
Speaker uses
adequate body
control
4 points
3 points
Speaker has
intermittent
eye contact
Speaker
exhibits poor
eye contact
2 points
1 points
5 points
Gestures
Speaker uses
excellent facial and
body gestures to
reinforce meaning
5 points
Posture
Speaker uses
excellent body
control: avoids
slouching and
extraneous
movements
Speaker
exhibits no eye
contact
(reading)
0 points
Facial and
body gestures
are stilted
Speaker has
flat affect
uses few
body gestures
Speaker does
not use either
facial or body
gestures
1 points
0 points
Speaker has
little body
control
Speaker has no
body control
2 points
Speaker leans
or shuffles,
dances or has
other
extraneous
body
movements
0 points
1 points
5 points
2 points
LEGEND: “0” = no mastery; “1” = minimal mastery; “2” = developing mastery; “3” = adequate mastery; “4”= good mastery
“5”= exceptional mastery
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 5
Legend to convert letter or percentage point grades into 1-5 Assessment Measuring Scale (AMS)
“1” means that the student did not meet course objective (3) as public speaker and GenEd objective (1) for speaking.
“2” means that the student has a minimal mastery of course objective (3) as public speaker and GenEd objective (1) for speaking.
“3” means that the student is developing mastery course objective (3) as public speaker and GenEd objective (1) for speaking.
“4” means that the student has an adequate mastery of course objective (3) as public speaker and GenEd objective (1) for speaking.
“5” means that the student masters course objective (3) as public speaker and GenEd objective (1) for speaking.
Conversion Scale
AMS
Letter
Percentage
“1”
F
59% below
“2”
D
60-69%
“3”
C
70-79%
“4”
B
80-89%
“5”
A
90% above
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 6
Provide an analysis (and summary) of the assessment results that were obtained.
In this study 341 students of Speech 211 were tested on their competence in the following verbal and non-verbal skills: LANGUAGE, VERBAL
SKILLS (FLUENCY, VOLUME, RATE) AND NON VERBAL SKILLS (EYE CONTACT, GESTURE, POSTURE). Students’ raw scores and
means for each of the tested components showed that students improved from the first to the second speech in each of seven assessed constructs as
summarized in the following tables.
RAW
SCORES
LANGUAGE
FLUENCY 1
VOLUME
RATE
EYECON
GESTURE
POSTURE
Speech1
Speech2
2369
2712
1101
1246
1162
1360
1121
1216
962
1173
1003
1153
1119
1357
Percent
Speech1
Speech2 Change
MEANS
13.5%
LANGUAGE
7.4
8.5
2
13.0%
FLUENCY
6.9
7.8
18.05%
VOLUME
7.2
8.4
17.14%
RATE
7.0
8.1
21.66%
EYECON
6.0
7.3
16.12%
GESTURE
6.2
7.1
21.42%
POSTURE
7.0
8.5
TOTAL
6.8
8.0
17.19%
Fluency, volume, rate, eye contact, gesture and posture are based on a 5 point grading scale, whereas language is on a 10 point grading scale.
Means for fluency, volume, rate, eye contact, gesture and posture have been recalculated on a 10 point grading scale from an initial 5 point grading scale to
be comparable with the 10 point grading scale for Language.
1
2
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 7
We conducted a two tail t-test on the means. The t-test was significant (t=-14, p<0.01) indicating that students improved their overall speaking
skills during the course of the semester.
Describe how the assessment results that were obtained affected (or did not affect) the student learning outcomes you identified. As part of
your discussion, describe any plans you have to address the areas where students need to improve.
The data showed that students improved their ability to speak in front of an audience at an acceptable rate in all seven assessed speaking skills as
indicated in the following chart.
Spring 2015 Speech Assessment
100%
%
80%
60%
40%
20%
0%
PRE
Language
74%
Fluency
69%
Volume
72%
Rate
70%
Eye Cont
60%
Gesture
62%
Posture
70%
POST
84%
78%
85%
82%
73%
72%
85%
Students overall improved 17 % between Speech 1 and Speech 2, with the largest increase for EYECONTACT (21.6%) and POSTURE (21.4%).
Good improvement was registered for GESTURE (16.1.), RATE (17.1%) and VOLUME (18%). LANGUAGE (13.5%). FLUENCY (13%),
instead, improved modestly.
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 8
Students’ average grades for each of seven assessed skills in SPEECH 1 and SPEECH 2 are reported in the following table.
LANGUAGE
FLUENCY 3
VOLUME
RATE
EYECON
GESTURE
POSTURE
OVERALL
SPEECH1
Average
74
69
72
70
60
62
70
68
Grade
C
D+
-C
-C
-D
-D
-C
+D
SPEECH2
Average
85
78
84
81
73
71
85
80
Grade
B
C+
+B
-B
-C
-C
B
-B
The grades showed that, on an average, our students’ scores changed from an initial -D, D or -C- to a final C+, B and B+ . This suggested that
students’ performance improved two letter grades from the beginning to the end of the semester.
Comparison between Fall 2013, Spring 2014 and Spring 2015 results
The table below showed a comparison between the means of the seven speech constructs for the 2nd speech of Fall 13. Spring 14 and Spring 15
students.
MEANS Speech2_Fall13 Speech2_Spring14 Speech 2_ Spr15
8.5
8.06
Language
8.01
7.8
8.03
Fluency
8.08
8.4
8.55
Volume
8.67
8.1
8.12
Rate
8.52
7.3
7.17
Eye Cont
7.7
7.1
7.3
Gesture
8.31
8.5
8.27
Posture
8.4
TOTAL
7.9
8.2
8.0
Means for fluency, volume, rate, eye contact, gesture and posture have been recalculated on a 10 point grading scale from an initial 5 point grading scale to
be comparable with the 10 point grading scale for Language.
3
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 9
The overall and specific means for the last three years suggest that also this semester students overall performance moved from an initial D to a
final B-. Given the consistency of our results across time and the large sample of assessed students, we can deduce that the data reflect our
students’ learning curve: from an initial F or D- level to a final B- level.
As part of your discussion, describe any plans to improve/modify the speech 211 course to address the speaking skills’ results obtained
through the assessment.
Although we were able to create and run a Speech Club, at the moment we are not able to identify what impact the speech club had on our
students’ speaking skills. We envisioned the Speech Club as a scheduled conversational time (similar to the ‘conversation teas’ sponsored by the
foreign language department) for students to practice and improve their speaking skills. However, we are not sure whether the students running the
speech club embraced our vision.
Until we are not able to determine the impact of the speech club on our students’ speaking skills, we recommend that the faculty add more in class
activities to help students to improve the seven speaking skills here assessed.
In addition, in order to effectively improve our students’ performance in each of the seven speaking skills, it is beneficial to design three additional
ad hoc assessments that measure specifically (i) Language Growth; (2) Verbal Skills (volume, fluency and rate) and; (3) Non-Verbal Skills (eyecontact, gesture, posture). These three instruments will allow the faculty to gain a better understanding on how to select students’ targeted
activities that could effectively help students to improve their speaking skills.
Dr. Franca Ferrari-Bridgers (06/05/2015)
Speech 211
Page 10