CENTRAL WASHINGTON UNIVERSITY
FALL 2013, WINTER 2014 and SPRING 2014 STUDENT EVALUATIONS OF INSTRUCTION (SEOI)
A COMPARISON OF RESPONSES
FROM FACE-TO-FACE, INTERACTIVE VIDEO, AND ONLINE COURSES
I.
II.
III.
IV.
Abstract ………………………………………………………………………………………………. 2.
Executive Summary …………………………………………………………………………….. 2.
Background ………….……………………………………………………………………………… 2.
Results ……………….……………………………………….…………………….………………… 4.
A. Course Ratings ……………………………………………………………………….. 4.
B. General Information ………………………………………………….…………… 7.
V. Summary …………………………………………………………………………………………….. 7.
Appendix 1. Terms, Acronyms, and Abbreviations …………………………….... 9.
Appendix 2. Fall 2013 SEOI Comparison…………………………………………..….. 10.
Appendix 3. Winter 2014 SEOI Comparison……………………………………..….. 24.
Appendix 4. Spring 2014 SEOI Comparison……………………………….……..….. 38.
Tom Henderson
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 2 of 53
I. ABSTRACT
This report compares the Student Evaluations of Instruction (SEOIs) of students enrolled in
courses taught face-to-face in traditional classrooms (F2F), via synchronous interactive video
(ITV), and “online” or to students enrolled in courses taught 100% online. The main research
questions this report analyzes are:
 How do CWU students rate their instruction?
 Do course evaluations indicate any major differences in learning outcomes between
courses taught 100% online, via interactive video, or F2F lecture courses taught in-class?
II. EXECUTIVE SUMMARY


CWU students’ ratings of instruction during the winter 2014 quarter were, on average,
very positive.
Student evaluations of online courses and interactive video courses are very similar to
the ratings of students in traditional face-to-face courses. All differences were small.
III. BACKGROUND
During the spring 2013/14 year Central Washington University administered all course
evaluations online. The questions on forms designed for online students and face-to-face
lecture students are similar.
Mock ups of the course evaluation forms can be found at:






Form A - Lecture
Form C - Skills Acquisition
Form D - Laboratory
Form E - Visual and Performing Arts
Form F - Field Experience / Student Teaching
Form W - Online
Note 1: this analysis does not compare online or ITV SEOIs to Forms C, D, E, or F because: (a)
the questions on those evaluation forms are not as similar as Form A and Form W, and (b)
because many of the courses assessed with those forms do not use traditional classroom
settings.
Note 2: this analysis does not include summer SEOI results. Not all summer results were
available when this report was written.
During the 2013/14 academic year Form A and Form W accounted for 85% of all course
evaluations.
Table 1 summarizes the number of responses to all SEOI forms during the fall 2013, winter
2014, and spring 2014 terms.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 3 of 53
Table 1 Responses to Course Evaluations by Evaluation Form by Term
Form A - Lecture
Form C - Skills Acquisition
Form D - Lab
Form E - Visual & performing Arts
Form F - Field Experience
Form W - Online
Total
% of Annual
Fall '13 Winter ' 14 Spring '14
13,769
12,031
8,960
919
893
928
993
699
470
772
613
482
101
38
153
2,017
2,348
1,955
18,571
16,622
12,948
39%
35%
27%
2013/14
34,760
2,740
2,162
1,867
292
6,320
48,141
100%
% of All
72%
6%
4%
4%
1%
13%
100%
The SEOI web site has information about the online course evaluation process and links to the
final SEOI Task Force report: http://www.cwu.edu/seoi Many of the questions on CWU’s
course evaluations use a five point Likert scale arranged in a table format (a visual analog scale).
For this analysis a response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1 (see “Likert scale” in appendix 1).
The CWU-wide response rates decreased from the fall 2013 term. The spring 2013 term was
the first quarter that SEOIs were administered for just one week to students (as opposed to two
weeks in prior quarters). Another major difference was that faculty could no longer download a
list of students who did or did not respond to SEOIs. The decrease may have been due to
CWU’s switch from Blackboard to Canvas. Canvas courses did not allow “pop-up” reminders.
Table 2 Response Rates to SEOIs by Term and Form (with length of time of administered)
Form A - Lecture
Form C - Skills Acquisition
Form D - Lab
Form E - Visual & Perform. Arts
Form F - Field Experience
Form W - Online
Overall Average
Spring
Fall
Winter Spring
Fall
Winter
Spring
2012
2012
2013
2013
2013
2014
2014
2 weeks 2 weeks 2 weeks 1 week 1 week 2 weeks 2 weeks
56%
57%
51%
41%
44%
44%
33%
48%
48%
41%
33%
39%
36%
31%
55%
51%
45%
37%
46%
38%
32%
49%
45%
37%
27%
38%
32%
27%
53%
51%
56%
53%
62%
64%
62%
53%
46%
49%
43%
56%
56%
50%
41%
44%
43%
34%
The average responses to SEOIs didn’t seem to be affected by running them one week before
finals or two weeks prior to finals. Table 2 shows weighted averages for the first two question
blocks on the survey: Student Learning Environment and Teaching for Student Learning.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 4 of 53
Table 3 Average Ratings by Course Evaluation Form for Fall 2013, Winter 2014 and Spring
2014
Note: These are the weighted average ratings from two questions banks: Teaching for Student
Learning and Student Learning Environment. All questions had a five point Likert scale assigned
values from 1 (for Strongly Disagree) to 5 (for Strongly Agree).
Form A - Lecture
Student Learning Environment
Teaching for Student Learning
Form C - Skills Acquisition
Student Learning Environment
Teaching for Student Learning
Form D - Laboratory
Student Learning Environment
Teaching for Student Learning
Form E - Visual & Performing Arts
Student Learning Environment
Teaching for Student Learning
Form F - Field Experience
Student Learning Environment
Teaching for Student Learning
Form W - Online
Student Learning Environment
Teaching for Student Learning
F2013
1 week
W2014
2 weeks
S2014
2 weeks
4.5
4.3
4.5
4.3
4.4
4.3
4.6
4.4
4.5
4.3
4,5
4.4
4.5
4.3
4.6
4.4
4.5
4.3
4.5
4.4
4.5
4.4
4.6
4.4
4.5
4.4
4.6
4.6
4.6
4.5
4.3
4.3
4.3
4.3
4.3
4.3
IV. RESULTS
A. COURSE RATINGS
The ratio of course types by college varied quite a bit from one delivery mode to another.
Table 4 shows the percent of Face-to-Face (F2F), Interactive Video (ITV) and Online courses
taught by the four colleges.
In summary:
 The College of Business had 51% of the total ITV course evaluation responses
 The College of Education & Professional Studies along with the College of the Sciences
had 80% of all online course evaluation responses for 2013/14
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 5 of 53
Table 4 Percent of Lecture, ITV and Online SEOI Replies by “Course Types” by College
(for example, 51% of 2013/14 ITV SEOI responses were from the College of Business)
CAH
CB
CEPS
COTS
Other
Total
---------- F2F ---------# SEOIS % of F2F % Bars
7,434
22%
2,783
8%
7,798
23%
13,170
39%
2,528
7%
33,713
100%
---------- ITV ---------# SEOIS % of ITV % Bars
34
3%
536
51%
144
14%
333
32%
0%
1,047
100%
---------- Online ---------# SEOIS % of Online % Bars
555
9%
527
8%
2,799
44%
2,294
36%
145
2%
6,320
100%
CAH
CB
CEPS
COTS
Other
Total
Note: Table 4 only summarizes responses from Form A – Lecture and Form W – Online course
evaluations. It does not include responses from Form C – Skills Acquisition, Form D – Lab, Form
E - Visual & Performing Arts, or Form F – Field Experience.
CWU students’ evaluation of instruction continues to be very positive. The first two question
banks on course evaluation forms relate to “Student Learning Environment” and “Teaching for
Student Learning.” All medians for face-to-face and online courses were 5.0, i.e. more than
50% of the students gave their instructors the highest possible rating.
Table 4 presents a comparison of results from face-to-face, interactive-video, and online
courses. Only the questions that have the same or very similar text on both SEOI forms (Form A
– Lecture and Form W – Online) are listed and compared. All of the differences in averages
were small.



The first two question banks relate to “Student Learning Environment” and “Teaching
for Student Learning.” All of the medians for online sections were 5’s (the best possible
rating). At least 50% of the students gave the best possible rating.
The differences in the average responses between face-to-face and interactive video
courses are all small as measured by Cohen’s d.
The differences in the average responses between face-to-face and online courses are
all small as measured by Cohen’s d.
Note: Cohen’s d effect size is a method of quantifying the difference or “distance” between the
means of two groups. Effect sizes of 0.2 or less are often considered “small,” 0.5 are “medium”
and 0.8 or higher are large. The differences are qualitative. Differences were computed
between face-to-face and interactive video courses as well as between face-to-face and online
courses. Cohen’s d was computed using “pooled standard deviations.” This method gives more
weight to the standard deviation of larger sample sizes. It is used when sample sizes are quite
different between two groups.
A Likert scale was used on all of the above questions:
-5-4-3Strongly agree
Agree
Neutral
8/7/14
-2Disagree
CWU 2013/14 Course Evaluation Summary
-1Strongly disagree
p. 6 of 53
IT
V
F2
F
2013/14 Academic Year SEOI Results
Face-to-face (F2F) vs. Interactive
Video (ITV) vs. 100% Online
Medians less than 5.0 are highlighted
M
ed
ia
ns
M
ed
ia
On
l in ns
e
M
ed
F2
ia
ns
F
Av
er x
ag
IT
es
V
Av
er
ag
es
On
l in
e
Av
er
Co
ag
es
he
n'
sd x
:F
Co
2F
he
vs
n'
.I
sd
TV
:F
2
Di
Fv
ffe
s.
re
On
nc
lin
e
e
in
F2
Di
F
ffe
v
s.
re
IT
nc
V
e
in
F2
Fv
s.
On
lin
e
Table 5 2013/14 Medians, Averages, and Comparison of Averages for Face-to-Face,
Interactive Video, and Online Courses (medians less than 5.0 are highlighted)
1 - STUDENT LEARNING ENVIRONMENT To what extent do you agree or disagree that the…
instructor fostered a fair and respectful
1.a./1.a.
5.0 5.0 5.0
4.5
4.4
4.4
learning environment?
instructor seemed genuinely concerned
1.b./1.b.
5.0 5.0 5.0
4.4
4.3
4.3
with whether students learned?
standards of online behavior were clearly
1.c./1.c.
5.0 5.0 5.0
4.4
4.3
4.4
communicated and enforced?
1.g.
If YES, did the instructor provide help?
5.0 5.0 5.0
4.4
4.3
4.3
2. TEACHING FOR STUDENT LEARNING To what extent do you agree or disagree that the….
course objectives were clearly
2.a./2.a.
5.0 5.0 5.0
4.3
4.3
4.4
communicated?
overall course content was presented in an
2.b./2.b.
5.0 4.0 5.0
4.3
4.2
4.3
understandable sequence?
instructor used a variety of methods, as
2.c./2.c.
5.0 4.0 5.0
4.2
4.1
4.2
needed, to make content clear?
assignments and tests were connected to
2.d./2.e.
5.0 5.0 5.0
4.4
4.4
4.5
course content?
evaluation and grading techniques were
2.e./2.f.
5.0 5.0 5.0
4.3
4.2
4.3
clearly explained?
instructions for class activities were clearly
2.f./2.g.
5.0 5.0 5.0
4.3
4.2
4.3
communicated?
instructor provided useful feedback on
2.g./2.h.
5.0 4.0 5.0
4.2
4.1
4.1
student work?
instructor provided timely feedback on
2.h./2.i.
5.0 4.0 5.0
4.2
4.1
4.1
student progress?
class sessions (online activities) were well
2.i./2.j.
5.0 4.0 5.0
4.3
4.2
4.3
organized?
out-of-class (online) work was useful in
2.j./2.k.
5.0 4.0 5.0
4.3
4.2
4.3
understanding course content?
2.k./2.l.
instructor encouraged students to connect
5.0 5.0 5.0
course content to issues beyond classroom?
4.3
4.3
4.3
x
0.1
0.1
SMALL
SMALL
0.1
0.1
SMALL
SMALL
0.1
x
x 0.0
x
0.0
SMALL
SMALL
0.1
SMALL
SMALL
0.1
-0.1
SMALL
SMALL
0.1
-0.1
SMALL
SMALL
0.1
0.0
SMALL
SMALL
0.0
-0.1
SMALL
SMALL
0.0
-0.1
SMALL
SMALL
0.1
0.0
SMALL
SMALL
0.1
0.0
SMALL
SMALL
0.1
0.1
SMALL
SMALL
0.1
0.0
SMALL
SMALL
0.1
-0.1
SMALL
SMALL
0.0
0.1
SMALL
SMALL
x
x
course activities challenged students to
5.0 5.0 5.0
4.3
4.4
4.4
0.0 -0.1 SMALL
think critically?
GENERAL INFORMATION How would you compare this course with other courses of similar credits
at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the . . .
amount of work OUTSIDE of class / online
3.a.
3.0 3.0 3.0
3.4
3.6
3.5 -0.2 -0.1 SMALL
environment
level of engagement/active learning IN class
3.b.
3.0 3.0 3.0
3.5
3.5
3.4
0.0
0.1 SMALL
/ online environment
3.c.
intellectual challenge presented to you
3.0 4.0 3.0
3.5
3.7
3.5 -0.2 0.0 SMALL
2.l./2.m.
8/7/14
CWU 2013/14 Course Evaluation Summary
SMALL
SMALL
SMALL
SMALL
p. 7 of 53
B. GENERAL INFORMATION
On average, students enrolled in face-to-face courses: study less, have lower class standing, and
take more courses required for General Education credits. Students’ expected grades are
similar whether they take courses face-to-face, via interactive video, or online. See Table 3.
Table 6 General Information Averages and Percentage Responses by Students in Face-toFace, Interactive Video, and Online Courses (Major differences are highlighted in gray)
For this class, about how many hours outside of the class did you spend in a typical 7-day week studying, reading,
conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities
(and participating online)? Note: The average is estimated from category midpoints.
Replied
0
1-3
4-6
7-10
11-15 16-20
21+
Average
Face-to-face
30,996
3%
34%
35%
18%
7%
2%
1%
4.8
approx. hours per week
Interactive Video
1,020
1%
15%
33%
27%
14%
7%
4%
7.5
approx. hours per week
100% online
6,272
0%
13%
34%
29%
15%
6%
3%
7.5
approx. hours per week
Why did you take this course? Please mark all that apply.
Replied
CertiGen
Instruc- Time /
Major
Minor
ficate
Ed
tor
Online Interest Other
to >= 1
Face-to-face
30,957
54%
8%
4%
28%
7%
8%
17%
5%
Interactive Video
1,021
69%
12%
3%
6%
7%
11%
17%
4%
100% online
6,267
60%
11%
3%
12%
5%
25%
20%
5%
What is your class standing?
Replied 1st yr = 1 Soph. = 2 Junior=3 Senior=4 Grad.=5 Other
Avg.
Face-to-face
23%
15%
28%
29%
3%
2%
2.7 Sophomore - Junior
30,772
Interactive Video
1,017
2%
2%
39%
45%
8%
4%
3.4 Junior - Senior
100% online
6,229
2%
6%
38%
47%
6%
2%
3.4 Junior - Senior
What grade do you expect to earn in this course?
Replied
A=4
B=3
C=2
D=1
F = 0 Other
Avg.
Face-to-face
30,894
50%
36%
11%
1%
0%
1%
3.3
expected grade
Interactive Video
1,020
48%
40%
9%
1%
0%
1%
3.3
expected grade
100% online
6,238
expected grade
55%
33%
8%
1%
0%
2%
3.4
It is interesting that the three instructional methods have: demographic differences in students,
differences in the mixture of courses from colleges, and difference in course delivery methods
but still have strong and very similar results in course ratings.
V. SUMMARY
A. How did CWU students evaluate their instruction during the winter 2014 term?
On average, student evaluations at CWU are very positive on all questions relating to
“Student Learning Environment” and “Teaching for Student Learning.” Both Form A face-toface and Form W online students gave median ratings of 5.0 to all questions relating to
Student Learning Environment and Teaching for Student Learning.
B. Do course evaluations indicate any major differences in learning outcomes between courses
taught face-to-face, via interactive video, or online?
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 8 of 53
No, all differences were small when students were asked questions about “Student Learning
Environment” or “Teaching for Student Learning.” Note: there are quite a few covariates:





students in online and ITV courses were almost one year further advanced in class
standing
students in online and ITV courses reported that they studied more hours per week
students in online and ITV courses were less likely to take a course because it was a
General Education requirement
Fifty one percent of responses to ITV courses came from the College of Business
Eighty percent of the online responses came from students enrolled in courses from
the College of Education & Professional Studies (CEPS) and the College of the
Sciences (COTS)
C. CWU must improve response rates to online SEOIs to have reliable results. Response rates
have been trending down since CWU first offered university –wide online course
evaluations during the spring 2012 term.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 9 of 53
APPENDIX 1
Terms, Acronyms, and Abbreviations
CAH
CB
CEPS
COTS
Cohen’s d
F2F
FAQ
Form A
Form W
ITV
Likert scale
Median
Online
SEOI
St Dev
8/7/14
College of Arts and Humanities
College of Business
College of Education and Professional Studies
College of the Sciences
A statistical measure of “effect size.” “Effect size is a method of quantifying
the difference between two groups that has some advantages over the use of
tests of statistical significance alone.” A pooled standard deviation is used as
the denominator in this analysis. A pooled standard deviation is called for
when the sample sizes of the variables studied are significantly different. See
this conference presentation on effect size:
http://www.leeds.ac.uk/educol/documents/00002182.htm
Note: Some studies have indicated that Effect Size calculations with ordinal
data may over-state the difference in means.
“Face-to-Face” refers to courses taught in traditional classrooms with students
face-to-face with an instructor
Frequently asked questions
Form A is the SEOI form developed for students to evaluate lecture courses
that meet face-to-face.
Form W is the SEOI (or course evaluation) form developed for students in
courses taught 100% online using Blackboard.
Courses taught via synchronous, interactive video.
Many of the questions on CWU’s course evaluations use a five point Likert
scale arranged in a table format (a visual analog scale). For this analysis a
response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1. Whether individual Likert items can be
considered as interval-level data, or whether they should be considered
merely ordered-categorical data is the subject of disagreement. This analysis
assumes that the responses can be considered as interval data, especially since
the Likert scale items are arranged in a visual analog format. The average or
mean is a parametric statistic that is robust to ordinal data. Cohen’s d may
tent to overstate distances when ordinal data is used.
Mid-point of ranked data. It is a statistical measure of central tendency. For
example if the median is 5 on a five point scale, at least 50% of students
answered 5.
Courses taught 100% online via Blackboard or Canvas, usually asynchronously.
“Student Evaluation of Instruction” CWU’s term for course evaluations
An abbreviation for “standard deviation” a statistical measure of variability
CWU 2013/14 Course Evaluation Summary
p. 10 of 53
APPENDIX 2
Fall 2013 SEOI COMPARISON OF RESPONSES
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 11 of 53
CENTRAL WASHINGTON UNIVERSITY
FALL 2013 STUDENT EVALUATIONS OF INSTRUCTION (SEOI)
A COMPARISON OF RESPONSES
FROM FACE-TO-FACE, INTERACTIVE VIDEO, AND ONLINE COURSES
I.
II.
III.
IV.
Abstract …………………………………………………………………………………………….. 2.
Executive Summary ……………………………………………………………………………. 2.
Background ……………………………………………………………………………………….. 2.
Overall Results …………………………………………….…………………….……………… 4.
C. Course Ratings ……………………………………………………………………… 4.
D. General Information ………………………………………………….…………. 6.
V. Summary …………………………………………………………………………………………….. 7.
Appendix 1. Terms, Acronyms, and Abbreviations …………………………….. 8.
Appendix 2. SEOI Response Rates by Form by Department ……………….. 9.
Appendix 3. Mockup of Form F – Field Experience…………………………….. 13.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 12 of 53
I. ABSTRACT
All CWU course evaluations were administered online during the fall 2013 quarter. This report
compares the evaluations of students enrolled in courses taught face-to-face in traditional
classrooms (F2F), via synchronous interactive video (ITV), and “online” to students enrolled in
courses taught 100% online. The main research questions this report analyzes are:
 How do CWU students rate their instruction?
 Do course evaluations indicate any major differences in learning outcomes between
courses taught 100% online, via interactive video, or lecture courses taught in-class,
face-to-face?
II. EXECUTIVE SUMMARY
 CWU students’ ratings of instruction during the fall 2013 quarter were, on average, very
positive.
 Student evaluations of online courses and interactive video courses are very similar to
the ratings of students in traditional face-to-face courses.
III. BACKGROUND
During the spring 2012 quarter Central Washington University administered all course
evaluations online for the first time. New forms developed by an SEOI Task Force were also
used for the first time institution-wide. The course evaluation forms for online and face-to-face
courses ask many of the same questions.
Mock ups of the new course evaluation forms can be found at:
 Form A 2010 - Lecture
 Form C 2010 - Skills Acquisition
 Form D 2010 - Laboratory
 Form E 2010 - Visual and Performing Arts
 Form W 2010 - Online
 A mock-up of the new Form F – Field Experience is in Appendix 3.
An Online SEOI FAQ has information about the online course evaluation process and links to the
final SEOI Task Force report: http://www.cwu.edu/associate-provost/Online-SEOI. The FAQ
was updated based on questions that arose during the initial spring 2012 quarter. Many of the
questions on CWU’s course evaluations use a five point Likert scale arranged in a table format
(a visual analog scale). For this analysis a response of “Strongly agree” is assigned a value of 5.
A response of “Strongly disagree” is assigned a value of 1 (see “Likert scale” in appendix 1).
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 13 of 53
The CWU-wide response rates increased slightly from the spring 2013 term. Spring 2013 was
the first quarter that SEOIs were administered for just one week to students (as opposed to two
weeks in prior quarters). Another major difference was that faculty could no longer download a
list of students who did or did not respond to SEOIs. ITV course responses were all pulled from
Form A – Lecture SEOIs.
Table 1. Response Rates to SEOIs by Form
CWU Online SEOI Response Rates
Spring
2012
Form A - Lecture
56%
Form C - Skills Acquisition
48%
Form D - Lab
55%
Form E - Visual & Performing Arts
49%
Form F - Field Experience
Form W - Online
62%
CWU Average - All Forms
56%
Fall
Winter Spring
Fall
2012
2013
2013*
2013*
57%
51%
41%
44%
48%
41%
33%
39%
51%
45%
37%
46%
45%
37%
27%
38%
53%
51%
64%
62%
53%
46%
56%
50%
41%
44%
CWU Online SEOI Response Rates - % Change from Prior Term
Fall
Winter Spring
Fall
2012
2013
2013*
2013*
Form A - Lecture
2%
-12%
-23%
6%
Form C - Skills Acquisition
1%
-19%
-24%
20%
Form D - Lab
-6%
-14%
-22%
23%
Form E - Visual & Performing Arts
-9%
-21%
-37%
40%
Form F - Field Experience
-4%
Form W - Online
3%
-5%
-15%
-14%
% Change from Prior Quarter
1%
-12%
-22%
6%
* SEOIs were only administered for one week during these terms.
The response rates varied quite a bit by faculty and by department. Appendix 2 shows average
response rate by SEOI forms for all CWU departments.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 14 of 53
IV. OVERALL RESULTS
A. COURSE RATINGS
The ratio of courses by college varied quite a bit from one delivery mode to another. For
example, the highest percentage of Interactive Video courses was from the College of Business,
a relatively small college. The College of Education and Professional Studies and the College of
the Sciences accounted for over 80% of online SEOI responses.
Chart 1. Percent of SEOI Responses by Delivery Method by College
% SEOI Responses by College
50%
40%
30%
20%
10%
0%
F2F
ITV
CAH
CB
CEPS
Online
COTS
Other
CWU students’ evaluation of instruction continues to be very positive. The first two question
banks on course evaluation forms relate to “Student Learning Environment” and “Teaching for
Student Learning.”
Table 2 presents a comparison of results from face-to-face, interactive-video, and online
courses. Only the questions that have the same or very similar text on both SEOI forms (Form A
– Lecture and Form W – Online) are listed. All of the differences in averages were small.



The first two question banks relate to “Student Learning Environment” and “Teaching
for Student Learning.” All of the medians for face-to-face and online sections were 5’s
(the best possible rating). At least 50% of the students gave the best possible rating.
The different in the average responses between face-to-face and interactive video
courses are all small as measured by Cohen’s d.
The differences in the average responses between face-to-face and online courses are
all small as measured by Cohen’s d.
Table 2. Medians, Averages, and Comparison of Averages for Face-to-Face, Interactive Video,
and Online Courses
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 15 of 53
e
M
ed
ia
M ns
ed
ia
On
l in ns
e
M
ed
F2
ia
ns
F
Av
er x
ag
IT
es
V
Av
er
ag
es
On
l in
e
Av
er
Co
ag
es
he
n'
sd x
:F
Co
2F
he
vs
n'
.I
sd
TV
:F
2
Di
Fv
ffe
s.
re
On
nc
lin
e
e
in
F2
Di
F
ffe
v
s.
re
IT
nc
V
e
in
F2
Fv
s.
On
lin
IT
V
F2
F
Fall 2013 SEOI Results
Face-to-face (F2F) vs. Interactive
Video (ITV) vs. 100% Online
Medians less than 5.0 are highlighted
1 - STUDENT LEARNING ENVIRONMENT To what extent do you agree or disagree that the…
instructor fostered a fair and respectful
1.a./1.a.
5.0 5.0 5.0 x 4.50 4.43 4.47
learning environment?
instructor seemed genuinely concerned
1.b./1.b.
5.0 5.0 5.0 x 4.38 4.25 4.33
with whether students learned?
standards of online behavior were clearly
1.c./1.c.
5.0 5.0 5.0 x 4.40 4.30 4.43
communicated and enforced?
1.g.
If YES, did the instructor provide help?
5.0 5.0 5.0 x 4.37 4.28 4.31
2. TEACHING FOR STUDENT LEARNING To what extent do you agree or disagree that the….
course objectives were clearly
2.a./2.a.
5.0 4.0 5.0 x 4.34 4.25 4.42
communicated?
overall course content was presented in an
2.b./2.b.
5.0 4.0 5.0 x 4.26 4.18 4.36
understandable sequence?
instructor used a variety of methods, as
2.c./2.c.
5.0 4.0 5.0 x 4.19 3.99 4.19
needed, to make content clear?
assignments and tests were connected to
2.d./2.e.
5.0 5.0 5.0 x 4.43 4.40 4.53
course content?
evaluation and grading techniques were
2.e./2.f.
5.0 4.0 5.0 x 4.26 4.16 4.33
clearly explained?
instructions for class activities were clearly
2.f./2.g.
5.0 4.0 5.0 x 4.28 4.14 4.30
communicated?
instructor provided useful feedback on
2.g./2.h.
5.0 4.0 5.0 x 4.17 3.96 4.16
student work?
instructor provided timely feedback on
2.h./2.i.
5.0 4.0 5.0 x 4.20 3.97 4.14
student progress?
class sessions (online activities) were well
2.i./2.j.
5.0 4.0 5.0 x 4.26 4.13 4.30
organized?
out-of-class (online) work was useful in
2.j./2.k.
5.0 4.0 5.0 x 4.25 4.20 4.32
understanding course content?
2.k./2.l.
x
x 0.08
0.04 SMALL SMALL
x 0.13
0.04 SMALL SMALL
x 0.11 (0.04) SMALL SMALL
x 0.10 0.07 SMALL SMALL
x
0.11 (0.09) SMALL SMALL
0.07 (0.11) SMALL SMALL
0.18 (0.00) SMALL SMALL
0.03 (0.12) SMALL SMALL
0.11 (0.07) SMALL SMALL
0.15 (0.02) SMALL SMALL
0.20
0.01 SMALL SMALL
0.22
0.05 SMALL SMALL
0.13 (0.04) SMALL SMALL
0.06 (0.07) SMALL SMALL
instructor encouraged students to connect
course content to issues beyond classroom?
5.0 5.0 5.0 x 4.33 4.30 4.28 0.03 0.05 SMALL SMALL
course activities challenged students to
2.l./2.m.
5.0 5.0 5.0 x 4.33 4.36 4.42 -0.03 (0.09) SMALL SMALL
think critically?
x
GENERAL INFORMATION
How would you compare this course with other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the...
amount of work OUTSIDE of class / online
3.a.
environment
3.0 3.0 3.0 x 3.31 3.53 3.43 -0.23 (0.14) SMALL SMALL
level of engagement/active learning IN class
3.b.
/ online environment
3.0 3.0 3.0 x 3.43 3.34 3.34 0.09 0.09 SMALL SMALL
3.c.
intellectual challenge presented to you
3.0 4.0 3.0 x 3.46 3.66 3.51 -0.20 (0.05) SMALL SMALL
Note: Cohen’s d effect size is a method of quantifying the difference or “distance” between the
means of two groups. Effect sizes of 0.2 or less are often considered “small,” 0.5 are “medium”
and 0.8 or higher are large. Differences were computed between face-to-face and interactive
video courses as well as between face-to-face and online courses. Cohen’s d was computed
using “pooled standard deviations.” This method gives more weight to the standard deviation
of larger sample sizes. It is used when sample sizes are quite different.
A Likert scale was used on all of the above questions:
-5-4-3-2-1Strongly agree
Agree
Neutral
Disagree
Strongly disagree
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 16 of 53
The scale was also a “visual analog scale” with equal distances between the radio buttons. The
Likert scale was also balanced with the same adjectives for positive and negative responses.
B. GENERAL INFORMATION
On average, students enrolled in face-to-face courses: study less, have lower class standing, and
take more courses required for General Education credits. See Table 3.
Table 3. General Information Averages and Percentage Responses by Students in Face-toFace, Interactive Video, and Online Courses. Major differences are highlighted.
For this class, about how many hours outside of the class did you spend in a typical 7-day week studying, reading,
conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities
(and participating online)? Note: The average is estimated from category midpoints.
Replied
0
1-3
4-6
7-10
11-15 16-20
21+
Average
Face-to-face
12,003
5%
34%
34%
17%
7%
3%
1%
5.48
approx. hours per week
Interactive Video
325
1%
13%
32%
28%
14%
8%
4%
8.33
approx. hours per week
100% online
2,001
0%
12%
34%
29%
15%
7%
3%
8.27
approx. hours per week
Why did you take this course? Please mark all that apply.
Replied
CertiGen
Instruc- Time /
Major
Minor
ficate
Ed
tor
Online Interest Other
to >= 1
Face-to-face
12,003
42%
6%
4%
29%
3%
6%
17%
5%
Interactive Video
325
59%
10%
2%
7%
5%
9%
16%
4%
100% online
1,998
52%
8%
3%
10%
3%
20%
20%
5%
What is your class standing?
Replied 1st yr = 1 Soph. = 2 Junior=3 Senior=4 Grad.=5 Other
Avg.
Face-to-face
2.6 Sophomore - Junior
11,920
28%
14%
26%
28%
3%
2%
Interactive Video
325
6%
5%
37%
40%
10%
3%
3.3 Junior - Senior
100% online
1,987
1%
7%
39%
44%
7%
2%
3.4 Junior - Senior
What grade do you expect to earn in this course?
Replied
A=4
B=3
C=2
D=1
F = 0 Other
Avg.
Face-to-face
11,968
51%
36%
11%
1%
0%
1%
3.3
Interactive Video
325
43%
44%
9%
1%
1%
2%
3.2
100% online
1,988
54%
34%
9%
1%
0%
2%
3.4
It is interesting that the three instructional methods have: demographic differences in students,
differences in the mixture of courses from colleges, and difference in course delivery methods
but still have strong and very similar results in course ratings.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 17 of 53
V. SUMMARY
A. How did CWU students evaluate their instruction during the fall 2013 term?
On average, student evaluations at CWU are very positive on all questions relating to
“Student Learning Environment” and “Teaching for Student Learning.”
D. Do course evaluations indicate any major differences in learning outcomes between courses
taught 100% online and lecture courses taught in-class, face-to-face?
No – not when students were asked questions about “Student Learning Environment” or
“Teaching for Student Learning.”
E. How can CWU improve the response rates to online SEOIs?
CWU should go back to administering SEOIs over a two week period to improve response
rates. CWU should also allow faculty to view the names of students who have or have not
completed SEOIs while they are in progress. Faculty can personalize requests to students
who have not yet completed SEOIs.

8/7/14
CWU 2013/14 Course Evaluation Summary
p. 18 of 53
Fall 2013 APPENDIX 1
Terms, Acronyms, and Abbreviations
Cohen’s d
F2F
FAQ
Form A
Form W
ITV
Likert scale
Median
Online
SEOI
St Dev
8/7/14
A statistical measure of “effect size.” “Effect size is a method of quantifying
the difference between two groups that has some advantages over the use of
tests of statistical significance alone.” A pooled standard deviation is used as
the denominator in this analysis. A pooled standard deviation is called for
when the sample sizes of the variables studied are significantly different. See
this conference presentation on effect size:
http://www.leeds.ac.uk/educol/documents/00002182.htm
Note: Some studies have indicated that Effect Size calculations with ordinal
data may over-state the difference in means.
“Face-to-Face” refers to courses taught in traditional classrooms with students
face-to-face with an instructor
Frequently asked questions
Form A is the SEOI form developed for students to evaluation lecture courses
that meet face-to-face.
Form W is the SEOI (or course evaluation) form developed for students in
courses taught 100% online using Blackboard.
Courses transmitted via synchronous, interactive video.
Many of the questions on CWU’s course evaluations use a five point Likert
scale arranged in a table format (a visual analog scale). For this analysis a
response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1. Whether individual Likert items can be
considered as interval-level data, or whether they should be considered
merely ordered-categorical data is the subject of disagreement. This analysis
assumes that the responses can be considered as interval data, especially since
the Likert scale items are arranged in a visual analog format. The average or
mean is a parametric statistic that is robust to ordinal data. Cohen’s d may
tent to overstate distances when ordinal data is used.
Mid-point of ranked data. It is a statistical measure of central tendency. For
example if the median is 5 on a five point scale, at least 50% of students
answered 5.
Courses taught 100% online via Blackboard or Canvas, usually asynchronously.
“Student Evaluation of Instruction” CWU’s term for course evaluations
An abbreviation for “standard deviation” a statistical measure of variability
CWU 2013/14 Course Evaluation Summary
p. 19 of 53
Fall 2013 APPENDIX 2
SEOI RESPONSE RATES BY FORM BY DEPARTMENT
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 20 of 53
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM A – LECTURE Overall Response Rate = 44%
Note: Continuing Education (CNED) used online SEOIs for high school classes for the first time.
CWU needs to improve contact with high school faculty to emphasize the importance of SEOIs.
Dept.
CNED
ABS
FCS
COM
WS
NEHS
CS
MLS
MUS
GEOL
TEACH
MANA
AFRO
LAJ
AIS
ECON
STEP
ACCT
GEOG
TH
BIOL
MATH
PSY
FIN
LC
LLSE
IET
SOC
SCED
PHYS
ART
PESPH
POSC
ITAM
PHIL
ENG
ANTH
CHEM
FNLA
HIST
CDS
AVIA
EFC
MCNA
REM
ACSK
DHC
PRIM
LLAS
Form A
8/7/14
Enrolled Replies
41
535
11
52
232
708
230
668
11
31
346
973
217
608
39
106
194
493
210
521
172
423
299
733
40
98
381
931
24
58
226
543
18
43
326
777
322
760
401
945
370
867
834
1,920
652
1,475
252
569
190
426
448
1,004
386
863
280
623
57
125
166
363
64
139
360
781
152
325
285
608
383
805
809
1,695
368
766
409
848
282
580
218
438
12
24
131
254
354
684
9
17
18
31
947
1,625
102
160
5
7
27
36
12,310
28,064
Response
Rate
8%
21%
33%
34%
35%
36%
36%
37%
39%
40%
41%
41%
41%
41%
41%
42%
42%
42%
42%
42%
43%
43%
44%
44%
45%
45%
45%
45%
46%
46%
46%
46%
47%
47%
48%
48%
48%
48%
49%
50%
50%
52%
52%
53%
58%
58%
64%
71%
75%
44%
% Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 21 of 53
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM C – SKILLS ACQUISITION Overall Response Rate = 39%
Dept.
COM
TH
PESPH
IET
GEOG
NEHS
GEOL
CWU
ENG
AVIA
LIB
PHYS
CHEM
Form C
Enrolled Replies
248
71
200
64
1,362
522
163
67
17
7
69
29
15
7
13
7
101
56
102
61
20
13
9
6
12
9
2,331
919
Response
Rate
29%
32%
38%
41%
41%
42%
47%
54%
55%
60%
65%
67%
75%
39%
% Data
Bars
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM D – LAB Overall Response Rate = 46%
Dept.
PRIM
COM
TH
AFRO
GEOL
NEHS
GEOG
CHEM
ANTH
EFC
Form D
Enrolled Replies
7
2
123
41
20
8
91
39
247
106
813
358
29
13
634
311
35
19
173
96
2,172
993
Response
Rate
29%
33%
40%
43%
43%
44%
45%
49%
54%
55%
46%
% Data
Bars
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM F – FIELD EXPERIENCE Overall Response Rate = 51%
Dept.
EFC
8/7/14
Enrolled Replies
198
101
Response
Rate
51%
% Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 22 of 53
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM E – VISUAL & PERFORMING ARTS Overall Response Rate = 38%
Dept.
MUS
TH
ART
PESPH
Form E
Enrolled Replies
1,354
474
147
55
401
173
139
70
2,041
772
Response
Rate
35%
37%
43%
50%
38%
% Data
Bars
FALL 2013 SEOI RESPONSE RATES BY DEPARTMENT
FORM W – ONLINE Overall Response Rate = 46%
Dept.
WS
AVIA
CWU
MANA
MATH
ITAM
LLSE
NEHS
ACCT
PESPH
LAJ
TH
COM
PHIL
CNED
GEOL
EFC
LIB
FCS
ENG
SOC
PSY
IDS
ANTH
IET
FIN
AP
HIST
Form W
8/7/14
Enrolled Replies
27
8
44
15
32
12
251
98
54
22
1,455
594
46
19
149
62
38
16
219
94
328
147
24
11
101
47
39
19
88
43
46
23
87
44
37
19
151
78
127
67
351
188
350
189
87
48
109
62
19
12
36
23
47
34
31
25
4,373
2,019
Response
Rate
30%
34%
38%
39%
41%
41%
41%
42%
42%
43%
45%
46%
47%
49%
49%
50%
51%
51%
52%
53%
54%
54%
55%
57%
63%
64%
72%
81%
46%
% Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 23 of 53
Fall 2013 APPENDIX 3
A MOCK UP FOR FORM F – FIELD EXPERIENCE
See: http://www.cwu.edu/seoi/sites/cts.cwu.edu.seoi/files/documents/FORM_F.pdf
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 24 of 53
APPENDIX 3
WINTER 2014 COMPARISON OF SEOI RESPONSES
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 25 of 53
CENTRAL WASHINGTON UNIVERSITY
WINTER 2014 STUDENT EVALUATIONS OF INSTRUCTION (SEOI)
A COMPARISON OF RESPONSES
FROM FACE-TO-FACE, INTERACTIVE VIDEO, AND ONLINE COURSES
I.
II.
III.
IV.
Abstract ……………………………………………………………………………………………..
Executive Summary …………………………………………………………………………….
Background ………………………………………………………………………………………..
Overall Results …………………………………………….…………………….………………
E. Course Ratings ………………………………………………………………………
F. General Information ………………………………………………….………….
V. Summary ……………………………………………………………………………………………..
Appendix 1. Terms, Acronyms, and Abbreviations ……………………………..
Appendix 2. SEOI Response Rates by Form by Department ………………..
8/7/14
CWU 2013/14 Course Evaluation Summary
2.
2.
2.
4.
4.
7.
8.
8.
9.
p. 26 of 53
I. ABSTRACT
All CWU course evaluations were administered online during the winter 2014 quarter. This
report compares the evaluations of students enrolled in courses taught face-to-face in
traditional classrooms (F2F), via synchronous interactive video (ITV), and “online” or to students
enrolled in courses taught 100% online. The main research questions this report analyzes are:
 How do CWU students rate their instruction?
 Do course evaluations indicate any major differences in learning outcomes between
courses taught 100% online, via interactive video, or F2F lecture courses taught in-class?
II. EXECUTIVE SUMMARY
 CWU students’ ratings of instruction during the winter 2014 quarter were, on average,
very positive.
 Student evaluations of online courses and interactive video courses are very similar to
the ratings of students in traditional face-to-face courses. All differences were small.
III. BACKGROUND
During the spring 2012 quarter Central Washington University administered all course
evaluations online for the first time. New forms developed by an SEOI Task Force were also
used for the first time institution-wide. The course evaluation forms for online and face-to-face
courses ask many of the same questions.
Mock ups of the new course evaluation forms can be found at:






Form A - Lecture
Form C - Skills Acquisition
Form D - Laboratory
Form E - Visual and Performing Arts
Form F - Field Experience / Student Teaching
Form W - Online
A SEOI web site has information about the online course evaluation process and links to the
final SEOI Task Force report: http://www.cwu.edu/seoi Many of the questions on CWU’s
course evaluations use a five point Likert scale arranged in a table format (a visual analog scale).
For this analysis a response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1 (see “Likert scale” in appendix 1).
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 27 of 53
The CWU-wide response rates decreased from the fall 2013 term. The spring 2013 term was
the first quarter that SEOIs were administered for just one week to students (as opposed to two
weeks in prior quarters). Another major difference was that faculty could no longer download a
list of students who did or did not respond to SEOIs. The decrease may have been due to
CWU’s switch from Blackboard to Canvas. Canvas courses did not allow “pop-up” reminders.
Although Canvas pop-up reminders were implemented during the summer 2014 terms.
Table 1 Response Rates to SEOIs by Form with length of administration
Form A – Lecture
Form C – Skills Acquisition
Form D – Lab
Form E – Visual & Perform. Arts
Form F – Field Experience
Form W – Online
Overall Average
Fall
2013
1 week
44%
39%
46%
38%
51%
46%
44%
Winter
2014
2 weeks
44%
36%
38%
32%
56%
48%
43%
Spring
2014
2 weeks
33%
31%
32%
27%
53%
43%
34%
The average responses to SEOIs didn’t seem to be affected by running them one week before
finals or two weeks prior to finals. Tables 2 through 7 show weighted averages for the first two
question blocks on the survey: Student Learning Environment and Teaching for Student
Learning. Note, this report compares averages between Form A – Lecture (face-to-face and
interactive video) to Form W – Online.
Table 2 Form A - Lecture CWU Averages
F2013
1 week
Student Learning Environment
4.5
Teaching for Student Learning
4.3
W2014
2 weeks
4.5
4.3
S2014
2 weeks
4.4
4.3
Table 3 Form C - Skills Acquisition CWU Averages
F2013
W2014
1 week 2 weeks
Student Learning Environment
4.6
4.5
Teaching for Student Learning
4.4
4.3
S2014
2 weeks
4,5
4.4
Table 4 Form D – Laboratory CWU Averages
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 28 of 53
W2014
2 weeks
4.6
4.4
S2014
2 weeks
4.5
4.3
Table 5 Form E - Visual & Performing Arts CWU Averages
F2013
W2014
1 week
2 weeks
Student Learning Environment
4.5
4.5
Teaching for Student Learning
4.4
4.4
S2014
2 weeks
4.6
4.4
Table 6 Form F - Field Experience CWU Averages
F2013
W2014
1 week
2 weeks
Student Learning Environment
4.5
4.6
Teaching for Student Learning
4.4
4.6
S2014
2 weeks
4.6
4.5
Table 7 Form W – Online CWU Averages
F2013
1 week
Student Learning Environment
4.3
Teaching for Student Learning
4.3
S2014
2 weeks
4.3
4.3
Student Learning Environment
Teaching for Student Learning
F2013
1 week
4.5
4.3
W2014
2 weeks
4.3
4.3
The response rates varied quite a bit by faculty and by department. Appendix 2 shows average
response rate by SEOI forms for all CWU departments.
IV. OVERALL RESULTS
A. COURSE RATINGS
The ratio of course types by college varied quite a bit from one delivery mode to another.
Chart 1 shows the % of Face-to-Face (F2F), Interactive Video (ITV) and % of Online courses
taught by the four colleges.
Note 1: A small minority of courses each quarter are taught outside of the colleges.
Note 2: The charts below don’t break down percentages of Skills Acquisitions, Lab, Visual &
Performing Arts, or Field Experience courses.
In summary:
 The College of Business administered over 50% of the total ITV courses offered by the
four colleges.
 The College of Education & Professional Studies along with the College of the Sciences
administered over 80% of the online courses taught by the four colleges.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 29 of 53

The College of the Sciences administered 43% of the face-to-face courses.
Chart 1 Percent of lecture and Online SEOIs by one of three “Course Types” by College
for example, the College of Business administer 57% of ITV courses at CWU
60%
50%
40%
30%
20%
10%
0%
CAH
CB
CEPS
COTS
F2F
24%
9%
25%
43%
ITV
5%
56%
12%
27%
Online
11%
8%
44%
38%
TOTAL
21%
10%
27%
42%
F2F
ITV
Online
TOTAL
Note: Chart 1 only summarizes responses from Form A – Lecture and Form W – Online course
evaluations. It does not include responses from Form C – Skills Acquisition, Form D – Lab, Form
E - Visual & Performing Arts, or Form F – Field Experience.
Chart 2 Percent of Lecture & Online SEOIs by Three Course Types by College
for example, 91% of the College of Arts & Humanities Form A and Form W course
evaluations were from face-to-face courses
100%
75%
50%
25%
0%
CAH
CB
CEPS
COTS
F2F
91%
72%
73%
84%
ITV
1%
14%
1%
2%
Online
8%
14%
26%
15%
F2F
8/7/14
ITV
Online
CWU 2013/14 Course Evaluation Summary
p. 30 of 53
CWU students’ evaluation of instruction continues to be very positive. The first two question
banks on course evaluation forms relate to “Student Learning Environment” and “Teaching for
Student Learning.”
Table 8 presents a comparison of results from face-to-face, interactive-video, and online
courses. Only the questions that have the same or very similar text on both SEOI forms (Form A
– Lecture and Form W – Online) are listed and compared. All of the differences in averages
were small.



The first two question banks relate to “Student Learning Environment” and “Teaching
for Student Learning.” All of the medians for online sections were 5’s (the best possible
rating). At least 50% of the students gave the best possible rating.
The differences in the average responses between face-to-face and interactive video
courses are all small as measured by Cohen’s d.
The differences in the average responses between face-to-face and online courses are
all small as measured by Cohen’s d.
Note: Cohen’s d effect size is a method of quantifying the difference or “distance” between the
means of two groups. Effect sizes of 0.2 or less are often considered “small,” 0.5 are “medium”
and 0.8 or higher are large. The differences are qualitative. Differences were computed
between face-to-face and interactive video courses as well as between face-to-face and online
courses. Cohen’s d was computed using “pooled standard deviations.” This method gives more
weight to the standard deviation of larger sample sizes. It is used when sample sizes are quite
different between two groups.
A Likert scale was used on all of the above questions:
-5-4-3Strongly agree
Agree
Neutral
8/7/14
-2Disagree
CWU 2013/14 Course Evaluation Summary
-1Strongly disagree
p. 31 of 53
M
F
IT
V
F2
Winter 2014 SEOI Results
Face-to-face (F2F) vs. Interactive
Video (ITV) vs. 100% Online
Medians less than 5.0 are
highlighted
ed
ia
M ns
ed
ia
O
ns
nl
in
e
M
ed
F2
ia
ns
F
Av
er
a
ge x
IT
V
s
Av
er
ag
es
O
nl
in
e
Av
er
C
ag
oh
es
en
's
x
d:
C
F2
oh
F
en
vs
's
.I
d:
TV
F2
D
iff
F
er
vs
en
.O
ce
nl
in
in
D
e
F2
iff
F
er
v
en
s.
ce
IT
V
in
F2
F
vs
.O
nl
in
e
Table 8. Medians, Averages, and Comparison of Averages for Face-to-Face, Interactive Video,
and Online Courses
1 - STUDENT LEARNING ENVIRONMENT To what extent do you agree or disagree that
x the…
instructor fostered a fair and respectful
1.a./1.a.
5.0 5.0 5.0 x 4.5
4.5
4.5 x 0.03 0.04 SMALL SMALL
learning environment?
instructor seemed genuinely concerned
1.b./1.b.
5.0 5.0 5.0 x 4.4
4.3
4.3 x 0.01 0.07 SMALL SMALL
with whether students learned?
standards of online behavior were clearly
1.c./1.c.
5.0 5.0 5.0 x 4.4
4.4
4.4 x 0.02 (0.02) SMALL SMALL
communicated and enforced?
1.g.
If YES, did the instructor provide help?
5.0 5.0 5.0 x 4.4
4.4
4.3 x -0.03 0.10 SMALL SMALL
2. TEACHING FOR STUDENT LEARNING To what extent do you agree or disagree that
x the….
course objectives were clearly
2.a./2.a.
5.0 5.0 5.0 x 4.3
4.3
4.4 0.04 (0.10) SMALL SMALL
communicated?
overall course content was presented in
2.b./2.b.
5.0 4.0 5.0 x 4.2
4.2
4.4 0.08 (0.12) SMALL SMALL
an understandable sequence?
instructor used a variety of methods, as
2.c./2.c.
4.0 4.0 5.0 x 4.2
4.1
4.2 0.05 (0.05) SMALL SMALL
needed, to make content clear?
assignments and tests were connected to
2.d./2.e.
5.0 5.0 5.0 x 4.4
4.4
4.5 0.00 (0.11) SMALL SMALL
course content?
evaluation and grading techniques were
2.e./2.f.
5.0 5.0 5.0 x 4.2
4.3
4.3 -0.05 (0.06) SMALL SMALL
clearly explained?
instructions for class activities were
2.f./2.g.
5.0 4.0 5.0 x 4.3
4.2
4.3 0.07 (0.04) SMALL SMALL
clearly communicated?
instructor provided useful feedback on
2.g./2.h.
4.0 4.0 5.0 x 4.2
4.0
4.1 0.10 0.03 SMALL SMALL
student work?
instructor provided timely feedback on
2.h./2.i.
5.0 4.0 5.0 x 4.2
4.1
4.1 0.09 0.07 SMALL SMALL
student progress?
class sessions (online activities) were well
2.i./2.j.
5.0 4.0 5.0 x 4.3
4.1
4.3 0.14 (0.05) SMALL SMALL
organized?
out-of-class (online) work was useful in
2.j./2.k.
5.0 4.0 5.0 x 4.2
4.2
4.3 0.02 (0.08) SMALL SMALL
understanding course content?
instructor encouraged students to connect
2.k./2.l. course content to issues beyond
5.0 5.0 5.0 x 4.3
4.3
4.3 0.00 0.05 SMALL SMALL
classroom?
course activities challenged students to
2.l./2.m.
5.0 5.0 5.0 x 4.3
4.4
4.4 -0.05 (0.06) SMALL SMALL
think critically?
GENERAL INFORMATION
x
How would you compare this course with other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the...
amount of work OUTSIDE of class /
3.a.
online environment
3.0 3.0 3.0 x 3.4
3.6
3.5 -0.23 (0.09) SMALL SMALL
level of engagement/active learning IN
3.b.
3.0 3.0 3.0 x 3.5
3.5
3.4 -0.06 0.09 SMALL SMALL
class / online environment
3.0 4.0 3.0 x 3.5
3.8
3.5 -0.33 (0.01) SMALL SMALL
3.c.
intellectual challenge presented to you
B. GENERAL INFORMATION
On average, students enrolled in face-to-face courses: study less, have lower class standing, and
take more courses required for General Education credits. Students’ expected grades are
similar whether they take courses face-to-face, via interactive video, or online. See Table 3.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 32 of 53
Table 9. General Information Averages and Percentage Responses by Students in Face-toFace, Interactive Video, and Online Courses. Major differences are highlighted in gray.
For this class, about how many hours outside of the class did you spend in a typical 7-day week studying, reading,
conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities
(and participating online)? Note: The average is estimated from category midpoints.
Replied
0
1-3
4-6
7-10
11-15
16-20
21+
Average
Face-to-face
2%
33%
36%
18%
7%
2%
1%
5.6
approx. hours / week
10,988
Interactive Video
342
0%
16%
27%
28%
16%
9%
4%
8.6
approx. hours / week
100% online
2,328
0%
13%
35%
29%
15%
6%
3%
8.1
approx. hours / week
Why did you take this course? Please mark all that apply.
CertiGen
Instruc- Time /
Replied
Major
Minor
ficate
Ed
tor
Online Interest
Other
Face-to-face
10,966
59%
9%
4%
28%
9%
9%
17%
5%
Interactive Video
342
73%
12%
3%
6%
11%
13%
18%
5%
100% online
13%
2%
13%
6%
27%
21%
6%
2331 64%
What is your class standing?
Replied 1st yr = 1 Soph. = 2 Junior=3 Senior=4 Grad.=5 Other
Avg.
Face-to-face
16%
28%
30%
3%
2%
2.8
Sophomore - Junior
10,914 20%
Interactive Video
341
1%
1%
36%
49%
9%
4%
3.8
Junior - Senior
100% online
2,314
2%
5%
38%
47%
6%
2%
3.4
Junior - Senior
What grade do you expect to earn in this course?
Replied
A=4
B=3
C=2
D=1
F=0
Other
Avg.
Face-to-face
10,840
50%
38%
11%
1%
0%
1%
3.3
Interactive Video
343
52%
36%
9%
1%
0%
1%
3.3
100% online
2,308
55%
34%
8%
2%
0%
1%
3.4
It is interesting that the three instructional methods have: demographic differences in students,
differences in the mixture of courses from colleges, and difference in course delivery methods
but still have strong and very similar results in course ratings.
V. SUMMARY
A. How did CWU students evaluate their instruction during the winter 2014 term?
On average, student evaluations at CWU are very positive on all questions relating to
“Student Learning Environment” and “Teaching for Student Learning.”
F. Do course evaluations indicate any major differences in learning outcomes between courses
taught face-to-face, via interactive video, or online?
No – not when students were asked questions about “Student Learning Environment” or
“Teaching for Student Learning.”
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 33 of 53
Winter 2014 APPENDIX 1
Terms, Acronyms, and Abbreviations
Cohen’s d
F2F
FAQ
Form A
Form W
ITV
Likert scale
Median
Online
SEOI
St Dev
8/7/14
A statistical measure of “effect size.” “Effect size is a method of quantifying
the difference between two groups that has some advantages over the use of
tests of statistical significance alone.” A pooled standard deviation is used as
the denominator in this analysis. A pooled standard deviation is called for
when the sample sizes of the variables studied are significantly different. See
this conference presentation on effect size:
http://www.leeds.ac.uk/educol/documents/00002182.htm
Note: Some studies have indicated that Effect Size calculations with ordinal
data may over-state the difference in means.
“Face-to-Face” refers to courses taught in traditional classrooms with students
face-to-face with an instructor
Frequently asked questions
Form A is the SEOI form developed for students to evaluate lecture courses
that meet face-to-face.
Form W is the SEOI (or course evaluation) form developed for students in
courses taught 100% online using Blackboard.
Courses taught via synchronous, interactive video.
Many of the questions on CWU’s course evaluations use a five point Likert
scale arranged in a table format (a visual analog scale). For this analysis a
response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1. Whether individual Likert items can be
considered as interval-level data, or whether they should be considered
merely ordered-categorical data is the subject of disagreement. This analysis
assumes that the responses can be considered as interval data, especially since
the Likert scale items are arranged in a visual analog format. The average or
mean is a parametric statistic that is robust to ordinal data. Cohen’s d may
tent to overstate distances when ordinal data is used.
Mid-point of ranked data. It is a statistical measure of central tendency. For
example if the median is 5 on a five point scale, at least 50% of students
answered 5.
Courses taught 100% online via Blackboard or Canvas, usually asynchronously.
“Student Evaluation of Instruction” CWU’s term for course evaluations
An abbreviation for “standard deviation” a statistical measure of variability
CWU 2013/14 Course Evaluation Summary
p. 34 of 53
Winter 2014 APPENDIX 2
SEOI RESPONSE RATES BY FORM AND BY DEPARTMENT
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 35 of 53
WINTER 2014 FORM A – LECTURE
Department
CNED
MLS
ABS
CS
AST
TH
MUS
STEP
PHYS
NEHS
AVIA
FCS
GEOL
LLAS
MATH
ART
ITAM
LC
COM
TEAC
GEOG
ECON
PE
AFRO
MANA
IET
LAJ
LLSE
POSC
ENG
ACCT
FINO
BIOL
SOC
AIS
ACSK
CHEM
ANTH
PSY
FNLA
HIST
CDS
SCED
EDFC
MCNA
SHP
PRIM
PHIL
REM
DHC
8/7/14
Total
184
137
55
614
28
801
532
35
329
1,224
323
733
558
35
1,931
120
338
290
529
702
755
528
723
78
768
1,041
823
680
342
1,759
755
577
1,068
710
61
278
957
695
1,282
479
438
45
174
651
7
7
26
664
30
119
26,018
SEOI RESPONSE RATES BY DEPARTMENT
Replied
41
31
13
204
10
288
193
13
124
467
124
285
220
14
795
50
141
123
226
301
325
230
315
34
336
456
366
304
154
806
349
270
500
336
29
133
462
337
622
233
214
22
90
343
4
4
15
389
18
73
11,432
Rate
22%
23%
24%
33%
36%
36%
36%
37%
38%
38%
38%
39%
39%
40%
41%
42%
42%
42%
43%
43%
43%
44%
44%
44%
44%
44%
44%
45%
45%
46%
46%
47%
47%
47%
48%
48%
48%
48%
49%
49%
49%
49%
52%
53%
57%
57%
58%
59%
60%
61%
44%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 36 of 53
WINTER 2014 FORM C - SKILLS ACQUISITION
Dept.
MLS
COM
NEHS
ETSC
PE
GEOL
ITAM
TH
CHEM
GEOG
ENG
Total
7
449
33
70
1,310
34
269
241
6
21
23
2,463
Winter 2014 FORM D - LAB
DEPT
CNED
MLS
COM
AVIA
PHYS
GEOL
NEHS
AFRO
CHEM
ANTH
EDFC
8/7/14
TOTAL
8
21
95
40
14
206
546
72
704
25
112
1,843
Replied
0
122
9
21
452
13
129
118
3
11
15
893
SEOI RESPONSE RATES BY DEPARTMENT
Rate
0%
27%
27%
30%
35%
38%
48%
49%
50%
52%
65%
36%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
0
5
24
11
4
59
199
29
298
12
58
699
Rate
0%
24%
25%
28%
29%
29%
36%
40%
42%
48%
52%
38%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 37 of 53
WINTER 2014 FORM E – VISUAL & PERFORMING ARTS
SEOI RESPONSE RATES BY DEPARTMENT
Dept.
MUS
COM
TH
PE
ART
Total
1,208
10
175
151
387
1,931
Replied
335
3
59
59
157
613
WINTER 2014 FORM F – FIELD EXPERIENCE
Dept.
CNED
EDFC
PRIM
Total
15
48
5
68
WINTER 2014 FORM W – ONLINE
Dept.
LIB
GEOL
EDFC
LLSE
MATH
COM
ITAM
LAJ
CWU
MANA
FNLA
FCS
ADPR
ACCT
ENG
IDST
IET
NEHS
CNED
FINO
HIST
LIB
ANTH
8/7/14
Total
20
21
100
66
57
181
1,475
426
33
247
10
109
48
33
165
107
49
78
85
82
32
20
18
3,462
Rate
28%
30%
34%
39%
41%
32%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
1
33
4
38
Rate
7%
69%
80%
56%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
7
8
40
27
24
78
660
193
15
115
5
59
26
18
91
60
28
47
54
56
22
15
14
1,662
Rate
35%
38%
40%
41%
42%
43%
45%
45%
45%
47%
50%
54%
54%
55%
55%
56%
57%
60%
64%
68%
69%
75%
78%
48%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 38 of 53
APPENDIX 4
COMPARISON OF SPRING 2014 SEOI RESPONSES
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 39 of 53
CENTRAL WASHINGTON UNIVERSITY
WINTER 2014 STUDENT EVALUATIONS OF INSTRUCTION (SEOI)
A COMPARISON OF RESPONSES
FROM FACE-TO-FACE, INTERACTIVE VIDEO, AND ONLINE COURSES
I.
II.
III.
IV.
Abstract ……………………………………………………………………………………………..
Executive Summary …………………………………………………………………………….
Background ………………………………………………………………………………………..
Overall Results …………………………………………….…………………….………………
G. Course Ratings ………………………………………………………………………
H. General Information ………………………………………………….………….
V. Summary ……………………………………………………………………………………………..
Appendix 1. Terms, Acronyms, and Abbreviations ……………………………..
Appendix 2. SEOI Response Rates by Form by Department ………………..
8/7/14
CWU 2013/14 Course Evaluation Summary
2.
2.
2.
4.
4.
7.
8.
8.
9.
p. 40 of 53
I. ABSTRACT
All CWU course evaluations were administered online during the winter 2014 quarter. This
report compares the evaluations of students enrolled in courses taught face-to-face in
traditional classrooms (F2F), via synchronous interactive video (ITV), and “online” or to students
enrolled in courses taught 100% online. The main research questions this report analyzes are:
 How do CWU students rate their instruction?
 Do course evaluations indicate any major differences in learning outcomes between
courses taught 100% online, via interactive video, or F2F lecture courses taught in-class?
II. EXECUTIVE SUMMARY
 CWU students’ ratings of instruction during the winter 2014 quarter were, on average,
very positive.
 Student evaluations of online courses and interactive video courses are very similar to
the ratings of students in traditional face-to-face courses. All differences were small.
III. BACKGROUND
During the spring 2012 quarter Central Washington University administered all course
evaluations online for the first time. New forms developed by an SEOI Task Force were also
used for the first time institution-wide. The course evaluation forms for online and face-to-face
courses ask many of the same questions.
Mock ups of the new course evaluation forms can be found at:






Form A - Lecture
Form C - Skills Acquisition
Form D - Laboratory
Form E - Visual and Performing Arts
Form F - Field Experience / Student Teaching
Form W - Online
A SEOI web site has information about the online course evaluation process and links to the
final SEOI Task Force report: http://www.cwu.edu/seoi Many of the questions on CWU’s
course evaluations use a five point Likert scale arranged in a table format (a visual analog scale).
For this analysis a response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1 (see “Likert scale” in appendix 1).
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 41 of 53
The CWU-wide response rates decreased from the fall 2013 term. The spring 2013 term was
the first quarter that SEOIs were administered for just one week to students (as opposed to two
weeks in prior quarters). Another major difference was that faculty could no longer download a
list of students who did or did not respond to SEOIs. The decrease may have been due to
CWU’s switch from Blackboard to Canvas. Canvas courses did not allow “pop-up” reminders.
Although Canvas pop-up reminders were implemented during the summer 2014 terms.
Table 1 Response Rates to SEOIs by Form with length of administration
Form A – Lecture
Form C – Skills Acquisition
Form D – Lab
Form E – Visual & Perform. Arts
Form F – Field Experience
Form W – Online
Overall Average
Fall
2013
1 week
44%
39%
46%
38%
51%
46%
44%
Winter
2014
2 weeks
44%
36%
38%
32%
56%
48%
43%
Spring
2014
2 weeks
33%
31%
32%
27%
53%
43%
34%
The average responses to SEOIs didn’t seem to be affected by running them one week before
finals or two weeks prior to finals. Tables 2 through 7 show weighted averages for the first two
question blocks on the survey: Student Learning Environment and Teaching for Student
Learning. Note, this report compares averages between Form A – Lecture (face-to-face and
interactive video) to Form W – Online.
Table 2 Form A - Lecture CWU Averages
F2013
1 week
Student Learning Environment
4.5
Teaching for Student Learning
4.3
W2014
2 weeks
4.5
4.3
S2014
2 weeks
4.4
4.3
Table 3 Form C - Skills Acquisition CWU Averages
F2013
W2014
1 week 2 weeks
Student Learning Environment
4.6
4.5
Teaching for Student Learning
4.4
4.3
S2014
2 weeks
4,5
4.4
Table 4 Form D – Laboratory CWU Averages
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 42 of 53
W2014
2 weeks
4.6
4.4
S2014
2 weeks
4.5
4.3
Table 5 Form E - Visual & Performing Arts CWU Averages
F2013
W2014
1 week
2 weeks
Student Learning Environment
4.5
4.5
Teaching for Student Learning
4.4
4.4
S2014
2 weeks
4.6
4.4
Table 6 Form F - Field Experience CWU Averages
F2013
W2014
1 week
2 weeks
Student Learning Environment
4.5
4.6
Teaching for Student Learning
4.4
4.6
S2014
2 weeks
4.6
4.5
Table 7 Form W – Online CWU Averages
F2013
1 week
Student Learning Environment
4.3
Teaching for Student Learning
4.3
S2014
2 weeks
4.3
4.3
Student Learning Environment
Teaching for Student Learning
F2013
1 week
4.5
4.3
W2014
2 weeks
4.3
4.3
The response rates varied quite a bit by faculty and by department. Appendix 2 shows average
response rate by SEOI forms for all CWU departments.
IV. OVERALL RESULTS
A. COURSE RATINGS
The ratio of course types by college varied quite a bit from one delivery mode to another.
Chart 1 shows the % of Face-to-Face (F2F), Interactive Video (ITV) and % of Online courses
taught by the four colleges.
Note 1: A small minority of courses each quarter are taught outside of the colleges.
Note 2: The charts below don’t break down percentages of Skills Acquisitions, Lab, Visual &
Performing Arts, or Field Experience courses.
In summary:
 The College of Business administered over 50% of the total ITV courses offered by the
four colleges.
 The College of Education & Professional Studies along with the College of the Sciences
administered over 80% of the online courses taught by the four colleges.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 43 of 53

The College of the Sciences administered 43% of the face-to-face courses.
Chart 1 Percent of lecture and Online SEOIs by one of three “Course Types” by College
for example, the College of Business administer 57% of ITV courses at CWU
60%
50%
40%
30%
20%
10%
0%
CAH
CB
CEPS
COTS
F2F
24%
9%
25%
43%
ITV
5%
56%
12%
27%
Online
11%
8%
44%
38%
TOTAL
21%
10%
27%
42%
F2F
ITV
Online
TOTAL
Note: Chart 1 only summarizes responses from Form A – Lecture and Form W – Online course
evaluations. It does not include responses from Form C – Skills Acquisition, Form D – Lab, Form
E - Visual & Performing Arts, or Form F – Field Experience.
Chart 2 Percent of Lecture & Online SEOIs by Three Course Types by College
for example, 91% of the College of Arts & Humanities Form A and Form W course
evaluations were from face-to-face courses
100%
75%
50%
25%
0%
CAH
CB
CEPS
COTS
F2F
91%
72%
73%
84%
ITV
1%
14%
1%
2%
Online
8%
14%
26%
15%
F2F
8/7/14
ITV
Online
CWU 2013/14 Course Evaluation Summary
p. 44 of 53
CWU students’ evaluation of instruction continues to be very positive. The first two question
banks on course evaluation forms relate to “Student Learning Environment” and “Teaching for
Student Learning.”
Table 8 presents a comparison of results from face-to-face, interactive-video, and online
courses. Only the questions that have the same or very similar text on both SEOI forms (Form A
– Lecture and Form W – Online) are listed and compared. All of the differences in averages
were small.



The first two question banks relate to “Student Learning Environment” and “Teaching
for Student Learning.” All of the medians for online sections were 5’s (the best possible
rating). At least 50% of the students gave the best possible rating.
The differences in the average responses between face-to-face and interactive video
courses are all small as measured by Cohen’s d.
The differences in the average responses between face-to-face and online courses are
all small as measured by Cohen’s d.
Note: Cohen’s d effect size is a method of quantifying the difference or “distance” between the
means of two groups. Effect sizes of 0.2 or less are often considered “small,” 0.5 are “medium”
and 0.8 or higher are large. The differences are qualitative. Differences were computed
between face-to-face and interactive video courses as well as between face-to-face and online
courses. Cohen’s d was computed using “pooled standard deviations.” This method gives more
weight to the standard deviation of larger sample sizes. It is used when sample sizes are quite
different between two groups.
A Likert scale was used on all of the above questions:
-5-4-3Strongly agree
Agree
Neutral
8/7/14
-2Disagree
CWU 2013/14 Course Evaluation Summary
-1Strongly disagree
p. 45 of 53
M
F
IT
V
F2
Winter 2014 SEOI Results
Face-to-face (F2F) vs. Interactive
Video (ITV) vs. 100% Online
Medians less than 5.0 are
highlighted
ed
ia
M ns
ed
ia
O
ns
nl
in
e
M
ed
F2
ia
ns
F
Av
er
a
ge x
IT
V
s
Av
er
ag
es
O
nl
in
e
Av
er
C
ag
oh
es
en
's
x
d:
C
F2
oh
F
en
vs
's
.I
d:
TV
F2
D
iff
F
er
vs
en
.O
ce
nl
in
in
D
e
F2
iff
F
er
v
en
s.
ce
IT
V
in
F2
F
vs
.O
nl
in
e
Table 8. Medians, Averages, and Comparison of Averages for Face-to-Face, Interactive Video,
and Online Courses
1 - STUDENT LEARNING ENVIRONMENT To what extent do you agree or disagree that
x the…
instructor fostered a fair and respectful
1.a./1.a.
5.0 5.0 5.0 x 4.5
4.5
4.5 x 0.03 0.04 SMALL SMALL
learning environment?
instructor seemed genuinely concerned
1.b./1.b.
5.0 5.0 5.0 x 4.4
4.3
4.3 x 0.01 0.07 SMALL SMALL
with whether students learned?
standards of online behavior were clearly
1.c./1.c.
5.0 5.0 5.0 x 4.4
4.4
4.4 x 0.02 (0.02) SMALL SMALL
communicated and enforced?
1.g.
If YES, did the instructor provide help?
5.0 5.0 5.0 x 4.4
4.4
4.3 x -0.03 0.10 SMALL SMALL
2. TEACHING FOR STUDENT LEARNING To what extent do you agree or disagree that
x the….
course objectives were clearly
2.a./2.a.
5.0 5.0 5.0 x 4.3
4.3
4.4 0.04 (0.10) SMALL SMALL
communicated?
overall course content was presented in
2.b./2.b.
5.0 4.0 5.0 x 4.2
4.2
4.4 0.08 (0.12) SMALL SMALL
an understandable sequence?
instructor used a variety of methods, as
2.c./2.c.
4.0 4.0 5.0 x 4.2
4.1
4.2 0.05 (0.05) SMALL SMALL
needed, to make content clear?
assignments and tests were connected to
2.d./2.e.
5.0 5.0 5.0 x 4.4
4.4
4.5 0.00 (0.11) SMALL SMALL
course content?
evaluation and grading techniques were
2.e./2.f.
5.0 5.0 5.0 x 4.2
4.3
4.3 -0.05 (0.06) SMALL SMALL
clearly explained?
instructions for class activities were
2.f./2.g.
5.0 4.0 5.0 x 4.3
4.2
4.3 0.07 (0.04) SMALL SMALL
clearly communicated?
instructor provided useful feedback on
2.g./2.h.
4.0 4.0 5.0 x 4.2
4.0
4.1 0.10 0.03 SMALL SMALL
student work?
instructor provided timely feedback on
2.h./2.i.
5.0 4.0 5.0 x 4.2
4.1
4.1 0.09 0.07 SMALL SMALL
student progress?
class sessions (online activities) were well
2.i./2.j.
5.0 4.0 5.0 x 4.3
4.1
4.3 0.14 (0.05) SMALL SMALL
organized?
out-of-class (online) work was useful in
2.j./2.k.
5.0 4.0 5.0 x 4.2
4.2
4.3 0.02 (0.08) SMALL SMALL
understanding course content?
instructor encouraged students to connect
2.k./2.l. course content to issues beyond
5.0 5.0 5.0 x 4.3
4.3
4.3 0.00 0.05 SMALL SMALL
classroom?
course activities challenged students to
2.l./2.m.
5.0 5.0 5.0 x 4.3
4.4
4.4 -0.05 (0.06) SMALL SMALL
think critically?
GENERAL INFORMATION
x
How would you compare this course with other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the...
amount of work OUTSIDE of class /
3.a.
online environment
3.0 3.0 3.0 x 3.4
3.6
3.5 -0.23 (0.09) SMALL SMALL
level of engagement/active learning IN
3.b.
3.0 3.0 3.0 x 3.5
3.5
3.4 -0.06 0.09 SMALL SMALL
class / online environment
3.0 4.0 3.0 x 3.5
3.8
3.5 -0.33 (0.01) SMALL SMALL
3.c.
intellectual challenge presented to you
B. GENERAL INFORMATION
On average, students enrolled in face-to-face courses: study less, have lower class standing, and
take more courses required for General Education credits. Students’ expected grades are
similar whether they take courses face-to-face, via interactive video, or online. See Table 3.
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 46 of 53
Table 9. General Information Averages and Percentage Responses by Students in Face-toFace, Interactive Video, and Online Courses. Major differences are highlighted in gray.
For this class, about how many hours outside of the class did you spend in a typical 7-day week studying, reading,
conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities
(and participating online)? Note: The average is estimated from category midpoints.
Replied
0
1-3
4-6
7-10
11-15
16-20
21+
Average
Face-to-face
2%
33%
36%
18%
7%
2%
1%
5.6
approx. hours / week
10,988
Interactive Video
342
0%
16%
27%
28%
16%
9%
4%
8.6
approx. hours / week
100% online
2,328
0%
13%
35%
29%
15%
6%
3%
8.1
approx. hours / week
Why did you take this course? Please mark all that apply.
CertiGen
Instruc- Time /
Replied
Major
Minor
ficate
Ed
tor
Online Interest
Other
Face-to-face
10,966
59%
9%
4%
28%
9%
9%
17%
5%
Interactive Video
342
73%
12%
3%
6%
11%
13%
18%
5%
100% online
13%
2%
13%
6%
27%
21%
6%
2331 64%
What is your class standing?
Replied 1st yr = 1 Soph. = 2 Junior=3 Senior=4 Grad.=5 Other
Avg.
Face-to-face
16%
28%
30%
3%
2%
2.8
Sophomore - Junior
10,914 20%
Interactive Video
341
1%
1%
36%
49%
9%
4%
3.8
Junior - Senior
100% online
2,314
2%
5%
38%
47%
6%
2%
3.4
Junior - Senior
What grade do you expect to earn in this course?
Replied
A=4
B=3
C=2
D=1
F=0
Other
Avg.
Face-to-face
10,840
50%
38%
11%
1%
0%
1%
3.3
Interactive Video
343
52%
36%
9%
1%
0%
1%
3.3
100% online
2,308
55%
34%
8%
2%
0%
1%
3.4
It is interesting that the three instructional methods have: demographic differences in students,
differences in the mixture of courses from colleges, and difference in course delivery methods
but still have strong and very similar results in course ratings.
V. SUMMARY
A. How did CWU students evaluate their instruction during the winter 2014 term?
On average, student evaluations at CWU are very positive on all questions relating to
“Student Learning Environment” and “Teaching for Student Learning.”
G. Do course evaluations indicate any major differences in learning outcomes between courses
taught face-to-face, via interactive video, or online?
No – not when students were asked questions about “Student Learning Environment” or
“Teaching for Student Learning.”
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 47 of 53
Spring 2014 APPENDIX 1
Terms, Acronyms, and Abbreviations
Cohen’s d
F2F
FAQ
Form A
Form W
ITV
Likert scale
Median
Online
SEOI
St Dev
8/7/14
A statistical measure of “effect size.” “Effect size is a method of quantifying
the difference between two groups that has some advantages over the use of
tests of statistical significance alone.” A pooled standard deviation is used as
the denominator in this analysis. A pooled standard deviation is called for
when the sample sizes of the variables studied are significantly different. See
this conference presentation on effect size:
http://www.leeds.ac.uk/educol/documents/00002182.htm
Note: Some studies have indicated that Effect Size calculations with ordinal
data may over-state the difference in means.
“Face-to-Face” refers to courses taught in traditional classrooms with students
face-to-face with an instructor
Frequently asked questions
Form A is the SEOI form developed for students to evaluate lecture courses
that meet face-to-face.
Form W is the SEOI (or course evaluation) form developed for students in
courses taught 100% online using Blackboard.
Courses taught via synchronous, interactive video.
Many of the questions on CWU’s course evaluations use a five point Likert
scale arranged in a table format (a visual analog scale). For this analysis a
response of “Strongly agree” is assigned a value of 5. A response of “Strongly
disagree” is assigned a value of 1. Whether individual Likert items can be
considered as interval-level data, or whether they should be considered
merely ordered-categorical data is the subject of disagreement. This analysis
assumes that the responses can be considered as interval data, especially since
the Likert scale items are arranged in a visual analog format. The average or
mean is a parametric statistic that is robust to ordinal data. Cohen’s d may
tent to overstate distances when ordinal data is used.
Mid-point of ranked data. It is a statistical measure of central tendency. For
example if the median is 5 on a five point scale, at least 50% of students
answered 5.
Courses taught 100% online via Blackboard or Canvas, usually asynchronously.
“Student Evaluation of Instruction” CWU’s term for course evaluations
An abbreviation for “standard deviation” a statistical measure of variability
CWU 2013/14 Course Evaluation Summary
p. 48 of 53
Spring 2014 APPENDIX 2
SEOI RESPONSE RATES BY FORM AND BY DEPARTMENT
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 49 of 53
WINTER 2014 FORM A – LECTURE
Department
CNED
MLS
ABS
CS
AST
TH
MUS
STEP
PHYS
NEHS
AVIA
FCS
GEOL
LLAS
MATH
ART
ITAM
LC
COM
TEAC
GEOG
ECON
PE
AFRO
MANA
IET
LAJ
LLSE
POSC
ENG
ACCT
FINO
BIOL
SOC
AIS
ACSK
CHEM
ANTH
PSY
FNLA
HIST
CDS
SCED
EDFC
MCNA
SHP
PRIM
PHIL
REM
DHC
8/7/14
Total
184
137
55
614
28
801
532
35
329
1,224
323
733
558
35
1,931
120
338
290
529
702
755
528
723
78
768
1,041
823
680
342
1,759
755
577
1,068
710
61
278
957
695
1,282
479
438
45
174
651
7
7
26
664
30
119
26,018
SEOI RESPONSE RATES BY DEPARTMENT
Replied
41
31
13
204
10
288
193
13
124
467
124
285
220
14
795
50
141
123
226
301
325
230
315
34
336
456
366
304
154
806
349
270
500
336
29
133
462
337
622
233
214
22
90
343
4
4
15
389
18
73
11,432
Rate
22%
23%
24%
33%
36%
36%
36%
37%
38%
38%
38%
39%
39%
40%
41%
42%
42%
42%
43%
43%
43%
44%
44%
44%
44%
44%
44%
45%
45%
46%
46%
47%
47%
47%
48%
48%
48%
48%
49%
49%
49%
49%
52%
53%
57%
57%
58%
59%
60%
61%
44%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 50 of 53
WINTER 2014 FORM C - SKILLS ACQUISITION
Dept.
MLS
COM
NEHS
ETSC
PE
GEOL
ITAM
TH
CHEM
GEOG
ENG
Total
7
449
33
70
1,310
34
269
241
6
21
23
2,463
Winter 2014 FORM D - LAB
DEPT
CNED
MLS
COM
AVIA
PHYS
GEOL
NEHS
AFRO
CHEM
ANTH
EDFC
8/7/14
TOTAL
8
21
95
40
14
206
546
72
704
25
112
1,843
Replied
0
122
9
21
452
13
129
118
3
11
15
893
SEOI RESPONSE RATES BY DEPARTMENT
Rate
0%
27%
27%
30%
35%
38%
48%
49%
50%
52%
65%
36%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
0
5
24
11
4
59
199
29
298
12
58
699
Rate
0%
24%
25%
28%
29%
29%
36%
40%
42%
48%
52%
38%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 51 of 53
WINTER 2014 FORM E – VISUAL & PERFORMING ARTS
SEOI RESPONSE RATES BY DEPARTMENT
Dept.
MUS
COM
TH
PE
ART
Total
1,208
10
175
151
387
1,931
Replied
335
3
59
59
157
613
WINTER 2014 FORM F – FIELD EXPERIENCE
Dept.
CNED
EDFC
PRIM
Total
15
48
5
68
WINTER 2014 FORM W – ONLINE
Dept.
LIB
GEOL
EDFC
LLSE
MATH
COM
ITAM
LAJ
CWU
MANA
FNLA
FCS
ADPR
ACCT
ENG
IDST
IET
NEHS
CNED
FINO
HIST
LIB
ANTH
8/7/14
Total
20
21
100
66
57
181
1,475
426
33
247
10
109
48
33
165
107
49
78
85
82
32
20
18
3,462
Rate
28%
30%
34%
39%
41%
32%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
1
33
4
38
Rate
7%
69%
80%
56%
Data
Bars
SEOI RESPONSE RATES BY DEPARTMENT
Replied
7
8
40
27
24
78
660
193
15
115
5
59
26
18
91
60
28
47
54
56
22
15
14
1,662
Rate
35%
38%
40%
41%
42%
43%
45%
45%
45%
47%
50%
54%
54%
55%
55%
56%
57%
60%
64%
68%
69%
75%
78%
48%
Data
Bars
CWU 2013/14 Course Evaluation Summary
p. 52 of 53
8/7/14
CWU 2013/14 Course Evaluation Summary
p. 53 of 53