Dr. Franca Ferrari-Bridgers, Professor Barbara Lynch and Professor Rosanne Vogel _ Speech and Theatre
Data Analysis and Report by Dr. Franca Ferrari-Bridgers
Middle States Listening Skills Report for SP 211-Spring 2014
List your student learning outcomes as described in your syllabus. Please list ALL of the Student Learning Outcomes that are listed in
your syllabus
Gen Ed. Obj.
1. Communicate effectively
through reading, writing,
listening and speaking
Outcome desired
To develop students as
more effective listeners
and evaluators of
communication, in order
to make them, in turn,
more capable learners and
intelligent decisionmakers.
2. use analytical reasoning to
identify issues or problems
and evaluate evidence in
order to make informed
decisions
To develop critical
thinking and problemsolving skills that enable
students to understand the
intricate link between
audience, speaker, and
occasion
Outcome desired
To develop the public
speaking skills necessary
to effectively present
informative and
persuasive speeches
Outcome desired
To learn the major
communication theories in
public speaking, interpersonal
communication, selfcommunication, intercultural
communication, and group
communication. Students will
be able to incorporate these
theories into their own
speaking styles
Outcome desired
To work towards
understanding and
overcoming
communication
apprehension
3. reason quantitatively and
mathematically as required in
their fields of interest and in
everyday life
4. use information
management and technology
skills effectively for academic
research and lifelong learning
To develop skills in
diverse communication
contexts including small
groups, computer-
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 1
mediated communities and
professional communities
5. integrate knowledge and
skills in their program of
study
To understand the
overwhelming importance
of effective
communication in all
aspects of academic,
professional, and everyday
life
6. differentiate and make
informed decisions about
issues based on multiple
value systems
7. work collaboratively in
diverse groups directed at
accomplishing learning
objectives
8. use historical or social
sciences perspectives to
examine formation of ideas,
human behavior, social
institutions, or social
processes
9. employ concepts and
methods of the natural and
physical sciences to make
informed judgments
10. apply aesthetic and
intellectual criteria in the
evaluation or creation of
works in the humanities or
the arts
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 2
Describe the assessment activity and the (student learning outcome(s) it addresses ) that occurred in your course.
Assignment: In order to test the reliability of our assessment tool, this semester we divided students in two groups. Students in each group
will watch and listen twice during the course of the semester one video recorded speeches from Lucas’s DVD included in “The Art of Public
Speaking”. The first group will watch “Medical Robots: From Science Fiction to Science Fantasy” (Needs Improvement) and the second
group “The Hidden World of Chili Peppers” (Needs Improvement). The two speeches are substandard, i.e., they do not have all the required
speech components. Students will use the speech evaluation form included here to evaluate the two speeches indicating which speech
components are present and which are missing, indicating how many topics and transitions the speaker introduce and how many sources are
cited. Moreover, the students will answer three listening and comprehension questions about the content of the each speech. The form used is
intended to facilitate student critical listening by identifying missing parts and by processing and understanding what was heard.
This assignment is designed primarily with the intention to test student listening skills as far as their ability to identify relevant speech outline
components (QUESTIONS 1-12) and show understanding of the content of the speeches (QUESTIONS 13-15). Because the goal is to track
student’s listening, each student should repeat this exercise twice during the semester.
Instructions: Instructors will administer the listening test twice during the semester. The first time the test should be administered after the
instructor has taught the class how to create a speech outline for their first informative (or persuasive) speech approximately between the 5th
and the 6th week of class. The second time the test should be administered after students have been instructed on how to create the speech
outline for their second persuasive (or informative) speech or preferably, after they have delivered their second speech between the 12th and
the 14th week of class.
All instructors need to follow the speech outline format indicated in your speech textbook or conform to it while instructing students on how to
create an informative or persuasive speech outline. Students will first complete the attached scoring SPEECH EVALUATION form and then
they will transfer their results on the scantron sheet.
List the data collection instrument (s) used for assessment.
Speech evaluation forms used by students for the assessment are given below. Each student will complete two forms one in connection with
the first speech and the second one with the second speech. Students will transfer their results to a scantron sheet. All the scantron sheets will
be graded at the end of the semester in two batches. The first batch contains the scantron sheets for the first speech and the second batch
contains the scantron sheets for the second speech.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 3
ASSESSMENT#1
SPEECH EVALUATION FORM
First Name________________________ Surname___________________________________
Speech Title: _____________________________________________________________________________
Listen carefully indicating which speech components are present in this speech and rating the speaker on each point. Remember that your goal
is to identify which speech components are missing from the speaker’s speech.
INTRODUCTION
NO
YES
1. Attention getter


2. Reason to listen


3. Reveal Topic


4. Established credibility


5. Preview body of speech


BODY
0
1
2
>2
6. How many main ideas




7. How many sources are properly cited1







8. How many transitions
1
Please remember that for a source to be cited properly, you need to hear one of the following combinations of information: (1) author and year of publication; (2) author and
title of article/ book; or (3) author, title of article/book and year of publication. Mentioning just the author or a title does not count as a proper citation.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 4
CONCLUSION
NO
YES
9. Cues the audience


10. Review of main points


11. Restate thesis


12. Vivid ending/Closure


QUESTIONS
13. This speech is about
A. Chili peppers
B. The shortage of good peppers
C. A brief history of chili peppers
D. a and c
14. If you eat a pepper that is too hot you should use:
A. Water and salt
B. Milk or ice cream
C. Ice water
D. Milk or yogurt
15. Chili peppers can be used for:
A. Protection from muggers
B. Food storage
C. Fatigue
D. Fertility
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 5
ASSESSMENT # 2
SPEECH EVALUATION FORM
Speech Title: _____________________________________________________________________________
Listen carefully indicating which speech components are present in this speech and rating the speaker on each point. Remember that your goal
is to identify which speech components are missing from the speaker’s speech.
INTRODUCTION
NO
YES
1. Attention getter


2. Reason to listen


3. Reveal Topic


4. Established credibility


5. Preview body of speech


BODY
0
1
2
>2
6. How many main ideas










7. How many sources are properly cited 2
8. How many transitions

CONCLUSION
NO
YES
9. Cues the audience


10. Review of main points


11. Restate thesis


12. Vivid ending/Closure


2
Please remember that for a source to be cited properly, you need to hear one of the following combinations of information: (1) author and year of publication; (2) author and title of article/ book; or
(3) author, title of article/book and year of publication. Mentioning just the author or a title does not count as a proper citation.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 6
COMPREHENSION
13. Orderly3 robots can do the following:
A. let doctors see patients from a remote location
B. communicate with each other
C. allow a doctor to see a patient’s vital signs
D. none of the above
14. Robots called ‘Da Vinci’4 are used for:
A. delivering items in a hospital
B. allowing patients to create art
C. helping hospital staff accomplishing routine tasks
D. surgery
15. The speaker told a story about a woman in New Jersey who:
A. had heart surgery
B. was operated on by a robot
C. had an orderly robot
D. invented a medical robot
3
4
Definition of ‘orderly’: an attendant in a hospital responsible for the nonmedical care of patients and the maintenance of order and cleanliness.
Leonardo Da Vinci was an Italian Renaissance polymath: painter, sculptor, architect, musician, mathematician, engineer, inventor, anatomist, geologist, cartographer, botanist, and writer.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 7
Provide an analysis (and summary) of the assessment results that were obtained.
GROUP_1,
93 students of group 1 watched “Medical Robots: From Science Fiction to Science Fantasy” (Needs Improvement). First, we will report the results
of Questions 1-12 testing ‘critical listening’ and then Question 13-15 testing comprehension.
QUESTIONS 1-12: Critical Listening
The results for number of correct answers for each questions (raw scores) and percentages are summarized in the following table.
LISTENING_1
INTRO 1
INTRO 2
INTRO 3
INTRO 4
INTRO 5
BODY 1
BODY 2
BODY 3
CONCL 1
CONCL 2
CONCL 3
CONCL 4
TOTAL
LISTENING _2
55
54
80
75
76
26
54
31
45
69
71
74
710
% CORRECT_1
64
63
76
79
77
27
54
51
49
82
78
77
777
% CORRECT_2
59%
58%
86%
81%
82%
28%
58%
33%
48%
74%
76%
80%
64%
69%
68%
82%
85%
83%
29%
58%
55%
53%
88%
84%
83%
70%
% CHANGE
16.9
17.2
-4.7
4.9
1.2
3.6
0.0
66.7
10.4
18.9
10.5
3.7
9.4
My only comment relates to your calculation of the percent change. The percentage change tells us how much a variable has increased or
decreased over time. For example, on page 13 of your Middle States report, on the first row of your table you calculated a change of 6% (44%38%). However the percentage change (which I think is a better measure of change because it considers the magnitude of the scores) is
16%. The formula for percentage change is (f2-f1/f1)*100. (44-38/38)*100=16%. Please let me know if my explanation does not make
sense. Thank you…..
The following charts visually show the students’ total number and total percentage of correct answers for both listening assessments.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 8
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 9
LIST_1 vs. LIST_2
% Correct Answers
90%
% Correct Answers
80%
70%
60%
50%
40%
30%
20%
10%
0%
INTR
O1
LIST_1 59%
INTR
O2
58%
INTR
O3
86%
INTR
O4
81%
INTR
O5
82%
BODY BODY BODY CONC CONC CONC CONC TOTA
1
2
3
L1
L2
L3
L4
L
28% 58% 33% 48% 74% 76% 80% 64%
LIST_2 69%
68%
82%
85%
83%
29%
58%
55%
53%
88%
84%
83%
70%
We also performed a repeated measure ANOVA to see whether there was a significant difference between the overall results for the assessments
taken at Time 1 and at Time 2; whether there was a significant difference between the results for each individual question at Time 1 and Time 2
and; finally, if there was significant difference between the interaction of questions and Time. The results are presented in the following table.
ANOVA
Source of Variation
Time
Questions
Interaction
SS
2.011201
69.26299
2.499552
Dr. Franca Ferrari-Bridgers - 09/04/2014
df
1
11
11
MS
2.011201
6.296636
0.227232
Speech 211
F
10.50914
32.90184
1.187356
P-value
0.001206
2.42E-65
0.290138
F crit
3.845674
1.792976
1.792976
Page 10
As the F values in the table indicate there was an overall significant difference for Time and Questions which together with the positive direction
of change indicated by the mean values, suggest that students improved their overall critical listening skills between the first and second
administering of the assessment. However, there was no significant difference for the Interaction between Time and Questions, suggesting that for
certain questions the change overtime was only minimal.
QUESTIONS 13-15: COMPREHENSION
The results for number of correct answers, means and percentages are summarized in the following table and in the chart.
COMPREHENSION 1
COMPREHENSION 2
COMPREHENSION 3
TOTAL
LIST_1
42
70
69
181
LIST_2
41
75
73
189
% Correct Answers
45%
75%
74%
65%
% Correct Answers
44%
81%
78%
68%
The following two charts visually show the students’ number and percentage of correct answers for both listening assessments.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 11
LIST_1 vs LIST_2
% Correct Answers
100%
80%
60%
40%
20%
0%
LIST_2
LIST_1
LIST_1
COMP_ 1
45%
COMP_ 2
75%
COMP_3
74%
TOTAL
65%
LIST_2
44%
81%
78%
68%
As will be discussed in the next section, for the first time since Fall 2012, in Spring 2014 students’ listening comprehension skills improved
notably from one assessment to the next.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 12
GROUP_2
55 students in group one watched “The Hidden World of Chili Peppers” (Needs Improvement). First we will report the results of Questions 1-12
testing ‘critical listening’ and then Question 13-15 testing comprehension.
QUESTIONS 1-12: Critical Listening
The results for number of correct answers for each questions (raw scores) and percentages are summarized in the following table.
INTRO 1
INTRO 2
INTRO 3
INTRO 4
INTRO 5
BODY 1
BODY 2
BODY 3
CONCL 1
CONCL 2
CONCL 3
CONCL 4
TOTAL
LIST_1
20
33
33
36
26
10
26
20
20
30
31
23
308
LIST_2
23
30
42
32
31
10
27
27
24
39
38
16
339
Dr. Franca Ferrari-Bridgers - 09/04/2014
%Correct_1
38%
63%
63%
69%
50%
19%
50%
38%
38%
58%
60%
44%
49%
Speech 211
%Correct_2
44%
58%
81%
62%
60%
19%
52%
52%
46%
75%
73%
31%
54%
Change
15.8
-7.9
28.6
-10.1
20.0
0.0
4.0
36.8
21.1
29.3
21.7
-29.5
10.2
Page 13
The following charts visually show the students’ total number and total percentage of correct answers for both listening assessments.
# Correct Answers
LIST_1 vs LIST_2
45
40
35
30
25
20
15
10
5
0
INTRO
1
LIST_1
20
LIST_2
23
INTRO
2
33
INTRO
3
33
INTRO
4
36
30
42
32
Dr. Franca Ferrari-Bridgers - 09/04/2014
INTRO BODY 1 BODY 2 BODY 3 CONCL
5
1
26
10
26
20
20
31
10
Speech 211
27
27
24
CONCL
2
30
CONCL
3
31
CONCL
4
23
39
38
16
Page 14
% Correct Answers
List_1 vs. List_2
% of correct answers
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
INTRO INTRO INTRO INTRO INTRO BODY BODY BODY CONC CONC CONC CONC TOTAL
1
2
3
4
5
1
2
3
L1
L2
L3
L4
LIST_1 38% 63% 63% 69% 50% 19% 50% 38% 38% 58% 60% 44% 49%
LIST_2
44%
58%
81%
62%
60%
19%
52%
52%
46%
75%
73%
31%
54%
We also performed a repeated measure ANOVA to see whether there was a significant difference between the overall results for the assessments
taken at Time 1 and at Time 2; whether there was a significant difference between the results for each individual question at Time 1 and Time 2
and; finally, if there was significant difference between the interaction of questions and Time. The results are presented in the following table.
ANOVA
Source of Variation
Time
Questions
Interaction
SS
0.770032
27.20112
2.931891
Dr. Franca Ferrari-Bridgers - 09/04/2014
df
1
11
11
MS
0.770032
2.472829
0.266536
Speech 211
F
3.358068
10.78387
1.162347
P-value
0.067119
3.44E-19
0.308882
F crit
3.849067
1.796459
1.796459
Page 15
As the F values in the table indicate there was not an overall significant difference for Time and for the Interaction between Time and Questions,
but there was a significant difference for Questions.
QUESTIONS 13-15: COMPREHENSION
The results for number of correct answers, means and percentages are summarized in the following table and in the chart.
COMPREHENSION_1
COMPREHENSION_2
COMPREHENSION_3
TOTAL
LIST_1
33
45
37
115
LIST_2
31
45
50
126
%Correct_1
63%
87%
71%
74%
%Correct_2
60%
87%
96%
81%
The following charts visually show the students’ number and percentage of correct answers for both listening assessments.
# Correct Answers
LIST_1 vs LIST_2
50
45
40
35
30
25
20
15
10
5
0
LIST_1
COMPRE_1
33
COMPRE_2
45
COMPRE_3
37
LIST_2
31
45
50
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 16
List_1 vs. List_2 % Correct Answers
% Correct Answers
100%
80%
60%
40%
20%
0%
LIST_1
COMPRE_1
63%
COMPRE_2
87%
COMPRE_3
71%
TOTAL
74%
LIST_2
60%
87%
96%
81%
As will be discussed in the next section, for the first time since Fall 2012, in Spring 2014 students listening comprehension skills notably improved
from one assessment to the next.
ASSESSMENT TOOL’S RELIABILITY
In order to test the reliability of our assessment tools we performed a Cronbach’s Alpha Test based on items 1 through 12 obtaining a very high
correlation Cronbach’s Alpha value of .953 for ‘Medical Robots’ and .970 for ‘Chili Peppers’. The high correlation values suggest that our
instrument reliably assesses listening skills and therefore can be used for future assessments. In both speeches, however, we noticed that the
individual mean value and the Item total correlation value of the BODY 1 question were very low in comparison with the other items. We believe
that these values reflect a potential problem in the way we formulated our question “How many main ideas”. Possibly students are not able to
semantically distinguish between the meaning of the words ‘idea’,‘topic’ and ‘point’. Therefore we need to reformulate BODY 1 question for the
next round of assessment and to add more in class activities to make sure students understand the semantic differences between the two terms.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 17
OVERALL ANALYSIS for GROUP 1 and GROUP 2
Data from both groups were compiled together. The total number of the students participating to both listening assessments is 148.
QUESTIONS 1-12: Critical Listening
The results for number of correct answers for each questions (raw scores) and percentages are summarized in the following table.
INTRO 1
INTRO 2
INTRO 3
INTRO 4
INTRO 5
BODY 1
BODY 2
BODY 3
CONC 1
CONC 2
CONC3
CONC 4
TOTAL
LIST_1
LIST_2
% CORRECT
% CORRECT
75
87
51%
60%
86
93
60%
64%
113
118
78%
81%
111
111
77%
77%
102
108
70%
74%
36
37
25%
26%
80
81
55%
59%
51
78
35%
54%
65
73
44%
50%
99
121
68%
83%
102
116
70%
80%
97
93
69%
64%
1018
1116
59%
64%
CHANGE
17.6
6.7
3.8
0.0
5.7
4.0
7.3
54.3
13.6
22.1
14.3
-7.2
8.5
The following charts visually show the students’ total number and total percentage of correct answers for both listening assessments.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 18
List_1 vs. List_2
140
# of correct answers
120
100
80
60
40
20
0
INTRO INTRO INTRO INTRO INTRO BODY BODY BODY CONC CONC CONC CONC
1
2
3
4
5
1
2
3
1
2
3
4
LIST_1 75
86
113
111
102
36
80
51
65
99
102
97
LIST_2
87
93
Dr. Franca Ferrari-Bridgers - 09/04/2014
118
111
108
37
Speech 211
81
78
73
121
116
93
Page 19
List_1 vs. List_2 - % Correct Answers
90%
80%
70%
Axis Title
60%
50%
40%
30%
20%
10%
0%
INTRO INTRO INTRO INTRO INTRO BODY BODY BODY CONC CONC CONC CONC TOTAL
1
2
3
4
5
1
2
3
1
2
2
4
LIST_1 51%
60%
78%
77%
70%
25%
55%
35%
44%
68%
70%
69%
59%
LIST_2
60%
64%
81%
77%
74%
26%
59%
54%
50%
83%
80%
64%
64%
We also performed a repeated measure ANOVA to see whether there was a significant difference between the overall results for the assessments
taken at Time 1 and at Time 2; whether there was a significant difference between the results for each individual question at Time 1 and Time 2
and; finally, if there was significant difference between the interaction of questions and Time. The results are presented in the following table.
ANOVA
Source of Variation
Time
Questions
Interaction
SS
2.75977
85.6954
3.212644
Dr. Franca Ferrari-Bridgers - 09/04/2014
df
1
11
11
MS
2.75977
7.790491
0.292059
Speech 211
F
12.99912
36.6949
1.375659
P-value
0.000316
3.57E-75
0.17712
F crit
3.844151
1.791413
1.791413
Page 20
As the F values in the table indicate there is an overall significant difference for Time and Questions, but there is not a significant difference for
the interaction between Time and Questions.
QUESTIONS 13-15: COMPREHENSION
The following table and charts visually show the students’ number and percentage of correct answers for both listening assessments.
LIST_1
COMPREHENSION1 75
COMPREHENSION 2 115
COMPREHENSION 3 106
TOTAL
296
LIST_2
72
120
123
315
LIST_1
52%
79%
73%
68%
LIST_2
50%
83%
85%
72%
CHANGE
-3.8
5.1
16.4
5.9
LIST_1 vs LIST_2
# Correct Answers
140
120
100
80
60
40
20
0
LIST_1
COMP1
75
COMP2
115
COMP3
106
LIST_2
72
120
123
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 21
% Correct Answers
LIST_1 vs. LIST_2 %
Correct Answers
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
LIST_1
COMP1
52%
COMP2
79%
COMP3
73%
TOTAL
68%
LIST_2
50%
83%
85%
72%
Describe how the assessment results that were obtained affected (or did not affect) the student learning outcomes you identified. As part of
your discussion, describe any plans you have to address the areas where students need to improve.
We are looking at the overall results of Group 1 and Group 2 together. Students were tested on the individuation of main speech components here
listed as ‘intro’, ‘body’, ‘conclusion’ and ‘comprehension’ of the speech twice.
Question 1-12 measured students' ability to recognize parts of speech, a concept reinforced by the syllabus of SP 211. On the ability to recognize
parts of a speech students showed improvement on 10 areas, i.e., (#1) attention getter (+17.6%), (#2) reason to listen (+6.7%),(#3) reveal topic
(+3.8%) , (#5) preview body of speech (+5.7%), (#6) main ideas (+4%), (#7) number of cited sources (7.3%), (#8) number of transitions
(+54.3%), (#9) cues the audience (+13.6%), (#10) review main points (+22.1%) and (#11) restate topic (+14.3%).
There was no change for (#4) credibility statement between Time 1 and Time2. There was instead deterioration in one area only: (#12) vivid
ending (-7.2%).
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 22
Overall Spring 2014 students’ cohort improved their abilities to identify speech components showing a 8.5% gain score from the first to the
second administering of the assessment. Students also deteriorated only in one area. With respect to Spring 2013 and Fall 2013 students’ cohorts,
students of the Spring 2014 cohort improved substantially on (#7) number of cited sources [(+7.3%) vs.( -14%, Spring 13) and (-4% , Fall 13)],
on (#8) transitions [(54.3%) vs. (6% ,Spring 13) and (5% ,Fall 13)], on (#10) review main points [(+22.1%) vs. (2% ,Spring 13) and (4% ,Fall
13)] and (#11) restate thesis [(+14.3%) vs. (2% ,Spring 13) and (3% ,Fall 13)].
We believe students’ improvement in these areas is the result of the recommendations we made to all the faculty in the action plan of our Fall 2013
report. Especially the data showed that students can better discern if a source has been cited properly or not and if the speaker uses transitions in
the speech. Students also substantially improved on most of the content related components such as ‘reveal topic’, ‘restate topic’, ‘preview’ and
‘review’ of main points. There is little improvement on question (#6) ‘How many main ideas”. As stated above we believe that these values
reflect a potential problem with the way we formulated the question, we will address this problem for the next round of assessments.
The deterioration on identify the vivid ending is surprising given that in the past two cohorts students substantially improved in this area (39% in
Spring 13 and 35% in Fall 13) between the two assessments. If we look at the individual Group 1 and Group 2 results, we notice that only Group 2
shows deterioration for this question (-29.5%). Group 1 instead shows a ( 3.7%) improvement in this area. We believe that the deterioration is
related to the way the speaker of the ‘Chili Peppers’ concluded his speech rather than to the students’ inability to recognize and to understand such
a component.
Question 13-15 measured the student's ability to extract content from a spoken message. Students’ comprehension for the first time in three
semester improved overall (5.9%). We registered a minimal deterioration (-3.8%) for question #13. Questions #14 and 15# improved
respectively (5.1%) and (16.4%).
If we compare Spring 14 students’ results with Fall 13 and Spring13 students’ results, we can definitely conclude that Spring 14 students improved
their performance at comprehension level possible due to the increased number of listening activities faculty members performed in their classes
during the semester.
Q13
Q14
Q15
Spring13
-10%
-23%
-17%
Fall13
-11%
-14%
1%
Spring14
-3.8%
5.1%
16.4%
Looking at the comprehension questions for the individual speeches, we notice that in both speeches questions #13 is the most problematic. We
believe that possibly the way the questions are formulated does not help students to answer them correctly. Moreover, as the literature on listening
research points out improvement in students’ listening and comprehension skills take longer than a 6-8 week period and is influenced by many
individual and environmental factors. In our case one the fact that students were assessed by different instructors under different circumstances
might have well played a role in determining the results.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 23
Action plan:
On ability to recognize parts of the speech, we believe faculty should give more emphasis to the concept of topic identification and main ideas
identification, credibility statement and citing sources. With regards to citing sources faculty should organize for their classes more than one trip to
the library and create target exercises to identify citation in speeches and to incorporate citation while delivering speeches following the MLA
citation style for oral citations.
With regards to topic, point and main ideas identification together with the ability to extract content from spoken language (listening and
comprehension) we believe that faculty should assign more in class activities asking students to summarize, analyze and report on content from
spoken language. Moreover as stated above Question # 6 “ how many main ideas” needs to be reformulated possibly in “how many main points”,
though a final decision on this point will be taken after having discussed the issue among the speech faculty members.
Similarly, faculty should make sure that students understand what a credibility statement is and the different ways the speaker has to establish
his/her credibility.
However, as speeches were both substandard, the skill of recognizing parts of the speech at or above 80 % level from the first to the second
assessment might be too advanced for students and it might require longer than a semester to see a substantial improvement, especially with
regards to comprehension. To test if students’ deterioration in comprehension is due to the different content of the two speeches, for Spring 2014
we modified how students were assessed without altering the assessment tool for the moment. Keeping the same testing conditions used in the
Fall, this semester we divided instructors in two groups and each group will test students’ listening skills using one speech only. Group 1 used
“Medical Robots: From Science Fiction to Science Fantasy” (Needs Improvement) while group 1 tested their students twice using the speech “The
Hidden World of Chili Peppers” (Needs Improvement). As the data analysis indicates students’ performance was worst for Group 2 than for Group
1, though the reliability of both assessment is very high as the Cronbach’s Alpha values above suggest. We believe that the difference in the results
is mostly due to the sample size differences between group 1 (93 students) and group 2 (55 students).
Next semester all faculty members participating in the assessment will use one speech only, possibly the Medical Robots’ one due to the good
results obtained in this cohort. To conclude the results of this assessment will be discussed at the September Faculty Meeting and future plan and
possible recommendations will be distributed to all faculty once approved at the meeting.
Dr. Franca Ferrari-Bridgers - 09/04/2014
Speech 211
Page 24