CENTRAL WASHINGTON UNVIVERSITY
2013 NOEL-LEVITZ PRIORITIES SURVEY OF ONLINE LEARNERS
A COMPARISON OF RESPONSES FROM STUDENTS ENROLLED PRIMARILY ONLINE
TO STUDENTS ENROLLED PRIMARILY ON CAMPUS
I. Abstract ………………………………………………….
II. Executive summary ………………………………..
III. Analysis ………………………………………………….
A. Importance …………………………………………
B. Satisfaction …………………………………………
C. Enrollment Factors ……………………………..
D. Summary Questions …………………………..
IV. Methodology & Detailed Results ……………
V. Summary ………………………………………………..
Appendix 1 – Terms and Abbreviations ….
Appendix 2 – An Approximate Mock-Up
of the PSOL Survey …………….
2.
2.
2.
3.
3.
5.
5.
5.
14.
15.
16.
I. ABSTRACT
Central Washington University administered the Noel-Levitz (N-L) Priorities Survey of Online Learners (PSOL)
survey from 4/29/13 through 5/15/13. The survey was sent to all CWU students enrolled in a course being
taught 100% online. 253 students responded from a survey population of 2,392 students for a response rate
of 11%.
The 62nd question on the survey (the 4th demographic question) asked students their current enrollment
status. This analysis compares the responses of the 157 students who selected “primarily online” to the 90
students who selected “primarily on-campus.” The six responses by students who did not answer question 62
were not included in this analysis.
II. EXECUTIVE SUMMARY





IMPORTANCE – only a few differences between online and on-campus students
o Primarily online students’ average response was moderately higher to two of the 36 questions as
compared to primarily on-campus students.
o Primarily online students’ perceive the importance of course availability greater than primarily oncampus students(See Table 1 on page 3)
SATISFACTION – major differences between online and on-campus students
o Primarily online students rated their satisfaction moderately higher on nine of 36 questions
o Primarily online students gave moderately lower responses to 4 of the 36 questions.
o Primarily online student averages were much higher on six of the questions
o Primarily online student averages were much lower on one question. See Tables 2 and 3 on page 4.
ENROLLMENT FACTORS – major differences between online and on-campus students
o Primarily online students rated 6 of the 18 questions moderately higher than primarily on-campus
students
o Primarily online students rated one question moderately lower. See Table 4 on page 5.
THREE SUMMARY QUESTIONS – major differences
Primarily online students were much higher on “has the online experience met your expectations”
but were moderately lower on “overall satisfaction.” than primarily on-campus students.
See Table 5 on page 5.
DEMOGRAPHICS
o Primarily online students tended to be older, working, and have family responsibilities. See Table
11 on page 9.
III. ANALYSIS
The PSOL survey consists of 73 questions:
 26 standard items rated for both importance and satisfaction
 10 campus defined items rated for both importance and satisfaction
 18 items assessing pre-enrollment factors – only rated for importance
 3 summary questions
 15 demographic questions
 1 question asking students their general major, or “group code”
July 26, 2013
Online vs. On-campus PSOL Responses
page 2 of 21
The PSOL is partially designed to provide “gap analysis” between importance and satisfaction. This report
compares responses of students enrolled “primarily online” to students enrolled “primarily on-campus.”
Questions 1 through 36 ask: “Each item below describes an expectation about your experiences with this
program. 'On the left tell us how important it is for your institution to meet this expectation. On the right tell
us how satisfied you are that your institution has met this expectation.
The PSOL “importance” and “satisfaction” questions use a seven-point Likert scale.
The scale for importance is:
1 - not important at all
2 - not very important
3 - somewhat unimportant
4 - neutral
5 – somewhat important
6 – important
7 – very important
The scale for satisfaction is:
1 – not satisfied at all
2 – not very satisfied
3 – somewhat satisfied
4 - neutral
5 – somewhat satisfied
6 - satisfied
7 – very satisfied
A. IMPORTANCE – ONLY MODERATE DIFFERENCES
Primarily online students rated two questions moderately more important than primarily on-campus
students and one questions very much more important than primarily on-campus students. Note:
Cohen’s d statistic was used to estimate the difference in means. See appendix 1 for a short
description of Cohen’s d and “Effect Size.”
Table 1. Differences in “Importance” questions
Primarily Primarily
Online
Onaverage campus
average
12. There are sufficient offerings within my
program of study.
14. I receive timely information on the availability
of financial aid.
36. Campus item: Courses necessary to meet my
degree objectives are offered online.
Primarily
Online
higher or
lower?
6.51
6.12
↑
5.74
5.05
↑
6.71
5.66
↑↑
B. SATISFACTION – QUITE DIFFERENT
Online students are moderately or largely different than on-campus students on 20 of the 36 questions in
this question bank.
July 26, 2013
Online vs. On-campus PSOL Responses
page 3 of 21
13 of the 36 questions were moderately different. Online students rated six of the nine questions higher in
satisfaction
Table 2. Moderate differences in “Satisfaction” questions (a Cohen’s d between 0.3 and 0.8)
Primarily
Primarily
OnOnline
campus
average average
2. My program advisor is accessible by telephone
and e-mail.
4. Faculty provide timely feedback about student
progress.
8. Student-to-student collaborations provide
valuable experiences.
10. This institution responds quickly when I request
information.
12. There are sufficient offerings within my
program of study.
18. Registration processes are convenient.
20. The quality of online instruction is excellent.
22. I am aware of whom to contact for questions
about programs and services.
24. Tutoring services are readily available.
26. The bookstore provides timely service to
students.
29. Campus item: Faculty-to-student interaction is
fostered and encouraged in my online course(s).
31. Campus item: General academic advising is
available to online learners.
36. Campus item: Courses necessary to meet my
degree objectives are offered online.
Primarily
Online
higher or
lower?
3.98
5.42

4.33
5.16

3.96
4.74

5.94
4.75
↑
5.48
4.82
↑
6.37
5.52
5.61
4.61
↑
↑
5.50
4.80
↑
4.64
3.41
↑
5.54
4.91
↑
5.43
4.77
↑
4.52
4.83

5.51
4.68
↑
Table 3. Large differences in “Satisfaction” average responses (a Cohen’s d of 0.8 or larger)
1. This institution has a good reputation.
5. My program advisor helps me work toward
career goals.
6. Tuition paid is a worthwhile investment.
9. Adequate financial aid is available.
14. I receive timely information on the availability
of financial aid.
15. Channels are available for providing timely
responses to student complaints.
July 26, 2013
Primarily
Primarily
OnOnline
campus
average average
6.71
5.36
Primarily
Online
higher or
lower?
↑↑
6.17
4.59
↑↑
2.93
6.32
4.71
4.03

↑↑
6.13
3.97
↑↑
6.10
3.72
↑↑
Online vs. On-campus PSOL Responses
page 4 of 21
16. Appropriate technical assistance is available.
6.63
4.52
↑↑
C. IMPORTANCE ON DECISIONS TO ENROLL IN CURRENT PROGRAM – Online students are quite different
than on-campus students
Table 4. Seven of the 18 questions relating to the “importance of factors in your decision to enroll at
CWU” had average responses that were moderately different.
…tell us how important each of the following
factors were in your decision to enroll in this
program.
40. Web site
43.Contact with current students and / or recent
graduates of the program
44. Ability to transfer credits as factor in decision
to enroll.
46. Financial assistance available as factor in
decision to enroll.
49. Work schedule as factor in decision to enroll.
51. Convenience as factor in decision to enroll.
53. Program requirements as factor in decision to
enroll.
Primarily
Primarily
OnOnline
campus
average average
6.17
5.60
Primarily
Online
higher or
lower?
↑
3.96
4.93

6.32
5.43
↑
5.65
4.56
↑
6.13
6.63
5.20
6.28
↑
↑
6.37
5.91
↑
D. THREE “SUMMARY” QUESTIONS
The PSOL includes three summary questions to provide overall ratings.



the average difference between online and on-campus students on question 55 is large
there is a moderate difference on question 56
there is not a significant different to question 57
Table 5. Differences in average responses to “Summary“ questions
SUMMARY QUESTIONS
-------------- Online --------------nAverage
StDev
----------- On Campus ----------- Difference Cohen's d Cohen's d
-nAverage
StDev in Averages Distance data bars
SUM1
Item 55 – So far, how has the online
experience met your expectations?
154
5.51
1.75
90
3.97
1.53
1.55
0.93
SUM2
Item 56 – Rate your overall satisfaction
with your online experience thus far.
155
3.81
2.54
90
4.57
1.94
-0.75
-0.32
SUM3
Item 57 - If you had it to do over again,
would you enroll in this program again?
153
5.27
1.63
90
5.14
1.94
0.12
0.07
July 26, 2013
Online vs. On-campus PSOL Responses
page 5 of 21
IV. METHODOLOGY and DETAILED RESULTS
The Noel-Levitz Priorities Survey of Online Learners is a national survey tested thoroughly to give reliable and
valid questions. CWU administered the survey to all students enrolled in “WW” courses (100% online) during
the spring 2013 quarter. The responses are self-selected and the response rate (11%) is small so unfortunately
CWU cannot infer any population characteristics from the sample results. However, online student responses
are so different than on-campus student responses in satisfaction and factors in deciding on a program that
further research is merited.
Table 6. The follow screen shot from the Noel-Levitz web site shows the response rates over time to CWU’s
administration of the PSOL.
Appendix 2 has an unofficial mock-up of the PSOL survey,
Demographic question #4 (the 61st question overall) asks students their current enrollment status. Of the 253
completed surveys 157 students selected “primarily online”, 90 students selected “primarily on-campus” and
6 students did no answer the question.
This analysis compares the responses on all other survey questions of the 157 students who select “Primarily
online” to the 90 students who selected “Primarily on-campus.”
Effect size was used to categorize the difference in means. Cohen’s d effect size was used. It is the difference
in means divided by the pooled standard deviation and it may provide better estimates of difference when
sample sizes are different.
Cohen gave a “rule-of-thumb” that an effect size of .2 to .3 might be “small,” between .3 and .8 “moderate,”
and larger than .8 “large.” For this analysis an effect size (absolute value) of .3 to .8 is moderate and .8 or
larger is large.
July 26, 2013
Online vs. On-campus PSOL Responses
page 6 of 21
Table 7. Detailed Average Responses to “Importance” Questions
IMPORTANCE
-------------- Online ------------------------ On Campus -----------Difference Cohen's d Cohen's d
- n - Average StDev
- n - Average StDev in Averages Distance data bars
IMP1
1. This institution has a good reputation.
157
6.00
1.16
90
5.82
1.21
0.18
0.15
IMP2
2. My program advisor is accessible by telephone and
e-mail.
157
6.23
1.31
90
6.07
1.42
0.16
0.12
IMP3
3. Instructional materials are appropriate for program
content.
157
6.50
0.72
90
6.39
0.80
0.11
0.15
IMP4
4. Faculty provide timely feedback about student
progress.
156
6.55
0.76
90
6.31
0.99
0.24
0.28
IMP5
5. My program advisor helps me work toward career
goals.
157
5.91
1.63
90
5.82
1.67
0.09
0.05
IMP6
6. Tuition paid is a worthwhile investment.
154
6.56
0.79
90
6.41
1.06
0.15
0.16
IMP7
7. Program requirements are clear and reasonable.
157
6.56
0.77
90
6.44
0.90
0.12
0.14
IMP8
8. Student-to-student collaborations provide valuable
experiences.
156
4.74
1.91
90
5.07
1.88
-0.32
-0.17
IMP9
9. Adequate financial aid is available.
155
5.89
2.08
90
5.32
2.41
0.57
0.26
10. This institution responds quickly when I request
IMP10 information.
156
6.44
1.00
90
6.21
1.03
0.23
0.23
IMP11
11. Student assignments are clearly defined in the
syllabus.
157
6.59
0.77
90
6.53
0.78
0.05
0.07
IMP12
12. There are sufficient offerings within my program
of study.
156
6.51
0.96
90
6.12
1.42
0.38
0.33
IMP13
13. The frequency of student and instructor
interactions is adequate.
155
6.10
1.01
88
6.06
0.99
0.05
0.05
IMP14
14. I receive timely information on the availability of
financial aid.
156
5.74
2.16
88
5.05
2.49
0.70
0.31
IMP15
15. Channels are available for providing timely
responses to student complaints.
156
5.37
2.26
86
5.27
2.08
0.10
0.04
156
5.90
1.85
87
5.92
1.39
-0.02
-0.01
156
6.19
1.31
85
6.39
0.80
-0.20
-0.17
IMP18 18. Registration processes are convenient.
155
6.60
0.83
87
6.48
0.87
0.12
0.14
IMP19 19. Online career services are available.
155
5.14
2.36
86
5.20
2.17
-0.06
-0.03
IMP20 20. The quality of online instruction is excellent.
154
6.62
0.69
87
6.44
0.80
0.19
0.26
IMP21 21. Adequate online library resources are provided.
155
5.59
1.97
87
5.83
1.42
-0.23
-0.13
155
6.34
1.12
87
6.08
1.23
0.26
0.22
155
6.19
1.44
87
5.92
1.64
0.27
0.18
IMP24 24. Tutoring services are readily available.
155
4.63
2.59
87
4.87
2.14
-0.25
-0.10
IMP25 25. Faculty are responsive to student needs.
155
6.54
1.00
87
6.38
1.14
0.16
0.15
IMP26 26. The bookstore provides timely service to students. 155
5.46
2.27
88
5.57
1.99
-0.10
-0.05
IMP27
27. Campus item: Help desk services and technical
support are adequate to meet my needs.
155
5.46
2.20
86
5.66
1.79
-0.20
-0.10
IMP28
28. Campus item: Student-to-student interaction is
fostered and encouraged in my online course(s).
152
5.12
1.84
88
4.89
1.88
0.23
0.13
IMP29
29. Campus item: Faculty-to-student interaction is
fostered and encouraged in my online course(s).
153
6.03
1.21
88
6.10
1.09
-0.08
-0.07
IMP16 16. Appropriate technical assistance is available.
IMP17
17. Assessment and evaluation procedures are clear
and reasonable.
22. I am aware of whom to contact for questions
IMP22 about programs and services.
IMP23
23. Billing and payment procedures are convenient for
me.
30. Campus item: The online course management
IMP30 system is easy to use.
154
6.49
0.94
87
6.28
1.07
0.22
0.22
IMP31
31. Campus item: General academic advising is
available to online learners.
155
5.88
1.78
87
5.30
2.28
0.58
0.29
IMP32
32. Campus item: I feel I am a member of the Central
Washington University community.
154
5.15
1.91
87
5.49
1.76
-0.34
-0.19
IMP33
33. Campus item: The organization and design of my
online course(s) is conducive to learning.
154
6.50
0.73
88
6.35
0.94
0.15
0.18
34. Campus item: Orientation to university systems
IMP34 and support resources is provided to online learners.
154
5.84
1.54
88
5.33
2.13
0.51
0.29
IMP35
35. Campus item: A fair and equitable learning
environment exists in my online course(s).
155
6.43
0.89
88
6.20
0.97
0.22
0.24
IMP36
36. Campus item: Courses necessary to meet my
degree objectives are offered online.
154
6.71
0.64
88
5.66
1.97
1.05
0.81
July 26, 2013
Online vs. On-campus PSOL Responses
*
*
*
page 7 of 21
Table 8. Detailed Average Responses to “Satisfaction” Questions
SATISFACTION
July 26, 2013
-------------- Online ------------------------ On Campus ----------Difference Cohen's d Dohen's d
- n - Average StDev - n - Average StDev in Averages Distance data bars
SAT1
1. This institution has a good reputation.
154
6.71
0.64
90
5.36
1.51
1.35
1.29
*
SAT2
2. My program advisor is accessible by telephone
and e-mail.
157
3.98
2.29
90
5.42
1.84
-1.44
-0.67
*
SAT3
3. Instructional materials are appropriate for
program content.
156
5.74
1.87
90
5.62
1.32
0.12
0.07
SAT4
4. Faculty provide timely feedback about student
progress.
157
4.33
2.37
90
5.16
1.63
-0.82
-0.39
*
SAT5
5. My program advisor helps me work toward
career goals.
157
6.17
1.39
88
4.59
2.21
1.57
0.91
*
SAT6
6. Tuition paid is a worthwhile investment.
157
2.93
2.10
90
4.71
1.85
-1.78
-0.89
*
SAT7
7. Program requirements are clear and reasonable.
156
5.08
2.31
89
5.43
1.39
-0.35
-0.17
SAT8
8. Student-to-student collaborations provide
valuable experiences.
157
3.96
2.44
90
4.74
1.73
-0.79
-0.36
*
SAT9
9. Adequate financial aid is available.
157
6.32
1.57
89
4.03
2.52
2.28
1.16
*
SAT10
10. This institution responds quickly when I
request information.
157
5.94
1.73
89
4.75
1.96
1.19
0.66
*
SAT11
11. Student assignments are clearly defined in the
syllabus.
157
5.65
2.16
90
5.49
1.48
0.16
0.08
SAT12
12. There are sufficient offerings within my
program of study.
156
5.48
2.16
90
4.82
1.76
0.66
0.33
SAT13
13. The frequency of student and instructor
interactions is adequate.
156
5.61
1.73
87
5.17
1.67
0.44
0.26
SAT14
14. I receive timely information on the
availability of financial aid.
156
6.13
1.79
87
3.97
2.59
2.17
1.03
*
SAT15
15. Channels are available for providing timely
responses to student complaints.
156
6.10
1.57
86
3.72
2.44
2.38
1.24
*
SAT16
16. Appropriate technical assistance is available.
156
6.63
0.75
86
4.52
2.36
2.11
1.38
*
SAT17
17. Assessment and evaluation procedures are
clear and reasonable.
156
5.17
2.46
83
5.45
1.32
-0.28
-0.13
SAT18
18. Registration processes are convenient.
156
6.37
0.93
87
5.61
1.62
0.76
0.62
SAT19
19. Online career services are available.
154
3.39
2.67
85
3.87
2.50
-0.48
-0.18
SAT20
20. The quality of online instruction is excellent.
155
5.52
1.45
87
4.61
2.01
0.91
0.54
SAT21
21. Adequate online library resources are
provided.
156
5.57
1.70
87
5.03
2.01
0.54
0.29
SAT22
22. I am aware of whom to contact for questions
about programs and services.
153
5.50
1.50
87
4.80
1.80
0.69
0.43
SAT23
23. Billing and payment procedures are convenient
for me.
154
4.99
1.74
87
5.23
1.88
-0.24
-0.13
SAT24
24. Tutoring services are readily available.
154
4.64
2.22
87
3.41
2.48
1.22
0.53
SAT25
25. Faculty are responsive to student needs.
155
5.08
1.80
88
5.20
1.66
-0.12
-0.07
SAT26
26. The bookstore provides timely service to
students.
154
5.54
1.56
88
4.91
2.14
0.63
0.35
SAT27
27. Campus item: Help desk services and technical
support are adequate to meet my needs.
154
4.68
1.85
87
4.63
2.29
0.05
0.02
SAT28
28. Campus item: Student-to-student interaction is
fostered
anditem:
encouraged
in my online interaction
course(s).
29.
Campus
Faculty-to-student
155
4.72
2.36
88
4.82
1.58
-0.10
-0.05
SAT29
is fostered and encouraged in my online
course(s).
154
5.43
1.72
88
4.77
1.99
0.66
0.36
SAT30
30. Campus item: The online course management
system is easy to use.
156
5.49
1.56
86
5.20
1.65
0.29
0.18
SAT31
31. Campus item: General academic advising is
available to online learners.
155
4.99
1.79
88
3.86
2.59
1.13
0.54
SAT32
32. Campus item: I feel I am a member of the
Central Washington University community.
155
5.38
1.53
87
5.01
1.85
0.37
0.22
SAT33
33. Campus item: The organization and design of
my
course(s)
is conducive
learning.
34. online
Campus
item: Orientation
to to
university
155
4.52
2.30
88
4.83
1.83
-0.31
-0.15
SAT34
systems and support resources is provided to
online learners.
155
3.78
2.60
88
4.19
2.40
-0.41
-0.16
SAT35
35. Campus item: A fair and equitable learning
environment exists in my online course(s).
155
4.71
2.42
88
5.22
1.68
-0.51
-0.23
SAT36
36. Campus item: Courses necessary to meet my
degree objectives are offered online.
154
5.51
1.66
88
4.68
2.01
0.82
0.46
Online vs. On-campus PSOL Responses
*
*
*
*
*
*
*
*
*
page 8 of 21
Table 9. Detailed Average Responses to “Enrollment Factor” Questions
On the left, tell us how important each of
the following factors were in your decision to -------------- Online ------------------- On Campus ------ Difference
Cohen's d Cohen's d
enroll in this program.
- n - Average StDev - n - Average StDev in Averages Distance data bars
IMP37 37. Catalog and brochures (printed)
157
3.98
2.29
89
4.00
2.53
-0.02
-0.01
IMP38 38. Catalog (online)
156
5.74
1.87
89
5.37
2.14
0.37
0.19
IMP39 39. College representatives
157
4.33
2.37
89
4.51
2.20
-0.17
-0.08
IMP40 40. Web site
157
6.17
1.39
89
5.60
1.96
0.57
0.35
IMP41 41. Advertisements
157
2.93
2.10
89
3.12
2.19
-0.19
-0.09
*
IMP42
42. Recommendation from instructor or
program advisor
156
5.08
2.31
88
5.17
2.25
-0.09
-0.04
IMP43
43.Contact with current students and / or
recent graduates of the program
157
3.96
2.44
89
4.93
2.28
-0.98
-0.41
*
IMP44
44. Ability to transfer credits as factor in
decision to enroll.
157
6.32
1.57
89
5.43
2.44
0.89
0.46
*
157
5.94
1.73
89
5.64
2.02
0.30
0.16
IMP45 45. Cost as factor in decision to enroll.
IMP46
46. Financial assistance available as factor in
decision to enroll.
157
5.65
2.16
89
4.56
2.70
1.09
0.46
IMP47
47. Future employment opportunities as factor
in decision to enroll.
156
5.48
2.16
89
5.72
2.01
-0.24
-0.11
IMP48
48. Reputation of institution as factor in
decision to enroll.
156
5.61
1.73
89
5.44
2.13
0.17
0.09
156
6.13
1.79
88
5.20
2.41
0.93
0.46
156
6.10
1.57
87
5.61
2.01
0.49
0.28
156
6.63
0.75
88
6.28
1.36
0.35
0.35
156
5.17
2.46
88
4.91
2.57
0.26
0.10
156
6.37
0.93
88
5.91
1.71
0.46
0.36
154
3.39
2.67
88
3.16
2.65
0.23
0.09
49. Work schedule as factor in decision to
IMP49 enroll.
50. Flexible pacing for completing a program as
IMP50 factor in decision to enroll.
51. Convenience as factor in decision to
IMP51 enroll.
52. Distance from campus as factor in decision
IMP52 to enroll.
53. Program requirements as factor in
IMP53 decision to enroll.
54. Recommendations from employer as
IMP54 factor in decision to enroll.
*
*
*
*
Table 10. Detailed Average Responses to Three “Summary” Questions
SUMMARY QUESTIONS
-------------- Online --------------nAverage
StDev
----------- On Campus ----------- Difference Cohen's d
-nAverage
StDev in Averages Distance
SUM1
Item 55 – So far, how has the online
experience met your expectations?
154
5.51
1.75
90
3.97
1.53
0.93
SUM2
Item 56 – Rate your overall satisfaction
with your online experience thus far.
155
3.81
2.54
90
4.57
1.94
-0.32
SUM3
Item 57 - If you had it to do over again,
would you enroll in this program again?
153
5.27
1.63
90
5.14
1.94
0.07
July 26, 2013
Online vs. On-campus PSOL Responses
*
*
page 9 of 21
Table 11. Demographic Details
DEMOG1 Item 58 – Gender
Online On-Campus Difference Data bars
72%
67%
5%
28%
32%
-4%
0%
1%
-1%
1 – Female
2 – Male
'
Did not answer
DEMOG2 Item 59 – Age
1 – 18 and under
2 – 19 to 24
3 – 25 to 34
4 – 35 to 44
5 – 45 to 54
6 – 55 to 64
7 – 65 and older
Did not answer
Online On-Campus Difference Data bars
0%
4%
-4%
17%
46%
-28%
33%
22%
11%
22%
14%
7%
20%
11%
9%
7%
0%
7%
1%
0%
1%
1%
2%
-2%
DEMOG3 Item 60 – Ethnicity / Race
Online On-Campus Difference Data bars
1 – African-American
6%
4%
1%
2 – American Indian or Alaskan Native
1%
1%
0%
3 – Asian or Pacific Islander
9%
16%
-7%
4 – Caucasian/White
66%
68%
-2%
5 – Hispanic
6%
6%
0%
6 – Other
5%
1%
4%
7 – Prefer not to respond
6%
3%
3%
Did not answer
1%
1%
0%
DEMOG4 Item 61 – Current enrollment status
1 – Primarily online
2 – Primarily on-campus
Online On-Campus Difference Data bars
100%
0%
100%
0
100%
-100%
DEMOG5 Item 62 – Current class load
1 – Full-time
2 – Part-time
Did not answer
July 26, 2013
Online On-Campus Difference Data bars
72%
89%
-17%
27%
11%
16%
1%
0%
1%
Online vs. On-campus PSOL Responses
page 10 of 21
Table 11. Demographic Details - continued
DEMOG6 Item 63 – Class level
1 – First year
2 – Second year
3 – Third year
4 – Fourth year
5 - Special Student
6 - Graduate / Professional
7 - Other
Did not answer
Online On-Campus Difference Data bars
3%
9%
-6%
2%
6%
-4%
42%
28%
14%
36%
42%
-6%
1%
1%
0%
10%
6%
5%
6%
8%
-2%
0%
1%
-1%
DEMOG7 Item 64 – Educational goal
1 - Associate degree
2 –Bachelor's degree
3 - Master's degree
4 - Doctorate or professional degree
5 - Certification (initial or renewal)
6 - Self-improvement / pleasure
7 - Job-related training
8 - Other
Did not answer
Online On-Campus Difference Data bars
1%
2%
-2%
73%
78%
-5%
19%
9%
10%
2%
6%
-4%
3%
3%
0%
0%
0%
0%
0%
0%
0%
2%
0%
2%
0%
2%
-2%
DEMOG8 Item 65 – Employment
Online On-Campus Difference Data bars
58%
22%
36%
23%
41%
-18%
18%
34%
-17%
1%
2%
-1%
1 – Full-time
2 – Part-time
3 – Not employed
Did not answer
DEMOG9 Item 66 – Current residence
1 – Own house
2 – Rent room/apartment/house
3 – Relative’s house
4 – Residence hall
5 – Other residence
Did not answer
July 26, 2013
Online On-Campus Difference Data bars
48%
21%
27%
41%
53%
-12%
8%
9%
-1%
0%
11%
-11%
3%
3%
-1%
0%
2%
-2%
Online vs. On-campus PSOL Responses
page 11 of 21
Table 11. Demographic Details - continued
DEMOG10 Item 67 – Marital status
1 – Single
2 – Single with children
3 – Married
4 – Married with children
5 – Prefer not to respond
Did not answer
Online On-Campus Difference Data bars
36%
60%
-24%
15%
7%
9%
19%
11%
8%
25%
20%
5%
4%
0%
4%
1%
2%
-2%
DEMOG11 Item 68 – Current plans
1 – Complete online degree program
2 – Complete degree on campus
3 – Transfer credits
4 – Complete this course
Did not answer
Online On-Campus Difference Data bars
81%
1%
80%
12%
79%
-67%
2%
1%
1%
5%
16%
-10%
0%
3%
-3%
DEMOG12 Item 69 – Current online enrollment
1 – 1 to 3 credits
2 – 4 to 6 credits
3 – 7 to 9 credits
4 – 10 to 12 credits
5 – 13 to 15 credits
6 – More than 15 credits
Did not answer
Online On-Campus Difference Data bars
4%
19%
-15%
15%
54%
-39%
19%
8%
11%
21%
4%
17%
30%
8%
22%
10%
4%
5%
1%
2%
-1%
DEMOG13 Item 70 – Previous online enrollment
1 – No classes
2 – 1 to 3 classes
3 – 4 to 6 classes
4 – 7 to 9 classes
5 – 10 to 12 classes
6 – 13 to 15 classes
7 – More than 15 classes
Did not answer
July 26, 2013
Online On-Campus Difference Data bars
8%
32%
-25%
23%
26%
-3%
11%
19%
-7%
9%
7%
2%
15%
8%
7%
17%
2%
15%
17%
4%
12%
1%
2%
-2%
Online vs. On-campus PSOL Responses
page 12 of 21
Table 11. Demographic Details - continued
DEMOG14 Item 71 - My intended degree is
1 - Online undergraduate degree
2 - Online graduate degree
3 - Face-to-face undergrad degree
4 - Face-to-face graduate degree
Did not answer
Online On-Campus Difference Data bars
62%
6%
56%
22%
0%
22%
9%
79%
-70%
7%
14%
-7%
1%
1%
0%
DEMOG15 Item 72 - My home CWU campus is:
Online On-Campus Difference Data bars
1 - Ellensburg
17%
56%
-39%
2 - Des Moines, Kent, or Pierce County
20%
18%
3%
3 - Lynnwood or Everett
31%
19%
12%
4 - Moses Lake, Wenatchee, or Yakima
10%
7%
3%
5 - Online: a fully online degree program 23%
0%
23%
Did not answer
0%
1%
-1%
Item 72 - Group code
Online
1063: Health and Physical Education: Athletic Administration (MS)
1087: Information Tech & Admin Mgt (BAS)
29%
1088: Information Tech & Admin Mgt (BS)
8%
1090: Interdisciplinary Studies: Social Sciences (BS)
25%
1094: Law & Justice (BA)
6%
1107: Paramedicine (BS)
2%
1125: Psychology (BA)
8%
1137: School Administration (MED)
1%
1145: Sociology (BA)
4%
1151: Special Education (MED)
3%
1165: Health and Physical Education: Teaching Physical Education and Health (MS) 1%
1166: Higher Education (MED)
3%
1167: Literacy (MED)
1%
1168: Not enrolled in an online degree program
8%
Did not answer
2%
GRAND TOTAL
July 26, 2013
100%
Online vs. On-campus PSOL Responses
On-Campus
Difference Data bar
#
-1%
#
26%
#
-7%
#
16%
#
-6%
2%
#
5%
1%
#
3%
3%
1%
3%
1%
#
-45%
#
-1%
#
page 13 of 21
IV. SUMMARY
Students who are enrolled online answer questions on Satisfaction, Enrollment Factors, Summary
Ratings, and Demographics quite a bit different than students who are enrolled on-campus.
Students enrolled “primarily online” provide more moderately to much higher (better) responses to
Satisfaction questions than students enrolled on-campus. However, online student average responses
were lower on some Satisfaction questions.
It is worrisome that students enrolled online provide, on average, a moderately lower response to
“Rate your overall satisfaction with your online experience so far” than students enrolled on-campus.
The differences in survey responses to Satisfaction, Enrollment, Summary, and Demographic questions
by online students substantiate CWU’s research and highlight the importance of further studies in this
area.
July 26, 2013
Online vs. On-campus PSOL Responses
page 14 of 21
APPENDIX 1
TERMS AND ABBREVATIONS
N-L PSOL
Cohen’s d
Likert scale
n
On-campus
Online
St Dev
July 26, 2013
Noel-Levitz Priorities Survey of Online Learners
This national online survey has been designed to “Assess the satisfaction and
priorities of students in distance learning and online programs.” See
https://www.noellevitz.com/student-retention-solutions/satisfaction-prioritiesassessments/priorities-survey-for-online-learners
A statistical measure of “effect size.” “Effect size is a method of quantifying the
difference between two groups that has some advantages over the use of tests
of statistical significance alone.” A pooled standard deviation is used as the
denominator in this analysis. A pooled standard deviation is called for when the
sample sizes of the variables studied are significantly different. See this
conference presentation on effect size:
http://www.leeds.ac.uk/educol/documents/00002182.htm
Note: Some studies have indicated that Effect Size calculations with ordinal data
may over-state the difference in means. The PSOL survey uses a seven point
Likert scale for many of its questions. A seven point scale tends to provide
statistics closer to real numbers than a four or five point scale.
Most of the questions on CWU’s course evaluations use a five point Likert scale
arranged in a table format (a visual analog scale). For this analysis a response of
“Strongly agree” is assigned a value of 5. A response of “Strongly disagree” is
assigned a value of 1. Whether individual Likert items can be considered as
interval-level data, or whether they should be considered merely orderedcategorical data is the subject of disagreement. This analysis assumes that the
responses can be considered as interval data, especially since the Likert scale
items are arranged in a visual analog format.
N stands for the number of students who answered a particular question. The
sample size is needed to compute Cohen’s d Effect Size.
For this study “on-campus” students are: (a) physically enrolled and located at
the Ellensburg campus or one of CWU’s centers and (b) taking a course that is
taught 100% online.
Courses taught 100% online via Blackboard or Canvas, usually asynchronously.
An abbreviation for “standard deviation” a statistical measure of variability
Online vs. On-campus PSOL Responses
page 15 of 21
APPENDIX 2
AN APPROXIMATE MOCK-UP OF THE NOEL-LEVITZ PRIORITIES SURVEY OF ONLINE LEARNERS
SURVEY INSTRUMENT
July 26, 2013
Online vs. On-campus PSOL Responses
page 16 of 21
CWU 2013 NOEL-LEVITZ PRIORITIES SURVEY OF ONLINE LEARNERS
APPROXIMATE PSOL MOCK-UP
See: http://tinyurl.com/PSOL-MOCKUP
Each item below describes an expectation about your experiences with this program.
On the left tell us how important it is for your institution to meet this expectation.
On the right tell us how satisfied you are that your institution has met this expectation.
important
My level of satisfaction…
1 - not important at all
2 - not very important
3 - somewhat unimportant
4 - neutral
5 - somewhat important
6 - important
7 - very important
0 - does not apply
1
2
3
4
5
6
7
0
IMP1
o
o
o
o
o
o
o
o
IMP2
o
o
o
o
o
o
o
o
IMP3
o
o
o
o
o
o
o
o
IMP4
o
o
o
o
o
o
o
o
IMP5
o
o
o
o
o
o
o
o
IMP6
o
o
o
o
o
o
o
o
IMP7
o
o
o
o
o
o
o
o
IMP8
o
o
o
o
o
o
o
o
IMP9
o
o
o
o
o
o
o
o
IMP10
o
o
o
o
o
o
o
o
IMP11
o
o
o
o
o
o
o
o
IMP12
o
o
o
o
o
o
o
o
IMP13
o
o
o
o
o
o
o
o
IMP14
o
o
o
o
o
o
o
o
IMP15
o
o
o
o
o
o
o
o
IMP16
o
o
o
o
o
o
o
o
IMP17
o
o
o
o
o
o
o
o
IMP18
o
o
o
o
o
o
o
o
IMP19
o
o
o
o
o
o
o
o
IMP20
o
o
o
o
o
o
o
o
July 26, 2013
1. This institution has a good
reputation.
2. My program advisor is
accessible by telephone and email.
3. Instructional materials are
appropriate for program
content.
4. Faculty provide timely
feedback about student
progress.
5. My program advisor helps
me work toward career goals.
6. Tuition paid is a worthwhile
investment.
7. Program requirements are
clear and reasonable.
8. Student-to-student
collaborations provide
valuable experiences.
9. Adequate financial aid is
available.
10. This institution responds
quickly when I request
information.
11. Student assignments are
clearly defined in the syllabus.
12. There are sufficient
offerings within my program
of study.
13. The frequency of student
and instructor interactions is
adequate.
14. I receive timely
information on the availability
of financial aid.
15. Channels are available for
providing timely responses to
student complaints.
16. Appropriate technical
assistance is available.
17. Assessment and evaluation
procedures are clear and
reasonable.
18. Registration processes are
convenient.
19. Online career services are
available.
20. The quality of online
instruction is excellent.
not available/not used - 0
very satisfied - 7
satisfied - 6
somewhat satisfied - 5
neutral - 4
somewhat dissatisfied - 3
not very satisfied - 2
not satisfied at all - 1
1
2
3
4
5
6
7
0
o
o
o
o
o
o
o
o
SAT1
o
o
o
o
o
o
o
o
SAT2
o
o
o
o
o
o
o
o
SAT3
o
o
o
o
o
o
o
o
SAT4
o
o
o
o
o
o
o
o
SAT5
o
o
o
o
o
o
o
o
SAT6
o
o
o
o
o
o
o
o
SAT7
o
o
o
o
o
o
o
o
SAT8
o
o
o
o
o
o
o
o
SAT9
o
o
o
o
o
o
o
o
SAT10
o
o
o
o
o
o
o
o
SAT11
o
o
o
o
o
o
o
o
SAT12
o
o
o
o
o
o
o
o
SAT13
o
o
o
o
o
o
o
o
SAT14
o
o
o
o
o
o
o
o
SAT15
o
o
o
o
o
o
o
o
SAT16
o
o
o
o
o
o
o
o
SAT17
o
o
o
o
o
o
o
o
SAT18
o
o
o
o
o
o
o
o
SAT19
o
o
o
o
o
o
o
o
SAT20
Online vs. On-campus PSOL Responses
page 17 of 21
1 - not at important at all
continued
2 - Not very important
3 - somewhat unimportant
4 - neutral
5 - somewhat important
6 - Important
7 - Very important
0 - does not apply
1
2
3
4
5
6
7
0
21. Adequate online library
resources are provided.
22. I am aware of whom to
contact for questions about
programs and services.
23. Billing and payment
procedures are convenient for
me.
24. Tutoring services are
readily available.
25. Faculty are responsive to
student needs.
26. The bookstore provides
timely service to students.
IMP21
o
o
o
o
o
o
o
o
IMP22
o
o
o
o
o
o
o
o
IMP23
o
o
o
o
o
o
o
o
IMP24
o
o
o
o
o
o
o
o
IMP25
o
o
o
o
o
o
o
o
IMP26
o
o
o
o
o
o
o
o
IMP27
o
o
o
o
o
o
o
o services and technical support
27. Campus item: Help desk
IMP28
o
o
o
o
o
o
o
o
IMP29
o
o
o
o
o
o
o
o
IMP30
o
o
o
o
o
o
o
o
IMP31
o
o
o
o
o
o
o
o
IMP32
o
o
o
o
o
o
o
o
IMP33
o
o
o
o
o
o
o
o
IMP34
o
o
o
o
o
o
o
o
IMP35
o
o
o
o
o
o
o
o
IMP36
o
o
o
o
o
o
o
o
July 26, 2013
are adequate to meet my needs.
28. Campus item: Student-tostudent interaction is fostered
and encouraged in my online
course(s).
29. Campus item: Faculty-tostudent interaction is fostered
and encouraged in my online
course(s).
30. Campus item: The online
course management system is
easy to use.
31. Campus item: General
academic advising is available to
online learners.
32. Campus item: I feel I am a
member of the Central
Washington University
community.
33. Campus item: The
organization and design of my
online course(s) is conducive to
learning.
34. Campus item: Orientation to
university systems and support
resources is provided to online
learners.
35. Campus item: A fair and
equitable learning environment
exists in my online course(s).
36. Campus item: Courses
necessary to meet my degree
objectives are offered online.
not available/not used - 0
very satisfied - 7
satisfied - 6
somewhat satisfied - 5
neutral - 4
somewhat dissatisfied - 3
not very satisfied - 2
not satisfied at all - 1
1
2
3
4
5
6
7
0
o
o
o
o
o
o
o
o
SAT21
o
o
o
o
o
o
o
o
SAT22
o
o
o
o
o
o
o
o
SAT23
o
o
o
o
o
o
o
o
SAT24
o
o
o
o
o
o
o
o
SAT25
o
o
o
o
o
o
o
o
SAT26
o
o
o
o
o
o
o
o
SAT27
o
o
o
o
o
o
o
o
SAT28
o
o
o
o
o
o
o
o
SAT29
o
o
o
o
o
o
o
o
SAT30
o
o
o
o
o
o
o
o
SAT31
o
o
o
o
o
o
o
o
SAT32
o
o
o
o
o
o
o
o
SAT33
o
o
o
o
o
o
o
o
SAT34
o
o
o
o
o
o
o
o
SAT35
o
o
o
o
o
o
o
o
SAT36
Online vs. On-campus PSOL Responses
page 18 of 21
On the left, tell us how important each of the following sources of information
were in your decision to enroll in this program.
1 - not at important at all
2 - Not very important
3 - somewhat unimportant
4 - neutral
5 - somewhat important
6 - Important
7 - Very important
0 - does not apply
37. Catalog and brochures
IMP37
o
o
o
o
o
o
o
IMP38
IMP39
IMP40
IMP41
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o
o (printed)
o 38. Catalog (online)
o 39. College representatives
o 40. Web site
o 41. Advertisements
IMP42
o
o
o
o
o
o
o
o instructor or program advisor
42. Recommendation from
IMP43
o
o
o
o
o
o
o
o
IMP44
o
o
o
o
o
o
o
o
IMP45
o
o
o
o
o
o
o
o
IMP46
o
o
o
o
o
o
o
o
IMP47
o
o
o
o
o
o
o
o
IMP48
o
o
o
o
o
o
o
o
IMP49
o
o
o
o
o
o
o
o
IMP50
o
o
o
o
o
o
o
o
IMP51
o
o
o
o
o
o
o
o
IMP52
o
o
o
o
o
o
o
o
IMP53
o
o
o
o
o
o
o
o
IMP54
o
o
o
o
o
o
o
o
43.Contact with current
students and / or recent
graduates of the program
44. Ability to transfer credits
as factor in decision to enroll.
45. Cost as factor in decision
to enroll.
46. Financial assistance
available as factor in decision
to enroll.
47. Future employment
opportunities as factor in
decision to enroll.
48. Reputation of institution
as factor in decision to enroll.
49. Work schedule as factor in
decision to enroll.
50. Flexible pacing for
completing a program as
factor in decision to enroll.
51. Convenience as factor in
decision to enroll.
52. Distance from campus as
factor in decision to enroll.
53. Program requirements as
factor in decision to enroll.
54. Recommendations from
employer as factor in decision
to enroll.
SUM2
July 26, 2013
Online vs. On-campus PSOL Responses
page 19 of 21
SUM2
SUM1
SUM3
Item 55 – So far, how has the online experience met your expectations?
1 – Much worse than I expected
2 – Quite a bit worse than I expected
3 – Worse than I expected
4 – About what I expected
5 – Better than I expected
6 – Quite a bit better than I expected
7 – Much better than I expected
Item 56 – Rate your overall satisfaction with your online
experience thus far.
1 - Not satisfied at all
2 - Not very satisfied
3 - Somewhat dissatisfied
4 - Neutral
5 - Somewhat satisfied
6 - Satisfied
7 - Very satisfied
Item 57 - If you had it to do over again, would you enroll in this program again?
1 - Definitely not
2 - Probably not
3 - Maybe not
4 - I don't know
5 - Maybe yes
6 - Probably yes
7 - Definitely yes
Section #3 - Demographic Items 58 - 72
DEMOG1 Item 58 – Gender
1 – Female
2 – Male
DEMOG2
DEMOG3
DEMOG4
Item 59 – Age
1 – 18 and under
2 – 19 to 24
3 – 25 to 34
4 – 35 to 44
5 – 45 to 54
6 – 55 to 64
7 – 65 and older
Item 60 – Ethnicity / Race
1 – African-American
2 – American Indian or Alaskan Native
3 – Asian or Pacific Islander
4 – Caucasian/White
5 – Hispanic
6 – Other
7 – Prefer not to respond
Item 61 – Current enrollment status
1 – Primarily online
2 – Primarily on-campus
DEMOG6
DEMOG5 Item 62 – Current class load
1 – Full-time
2 – Part-time
July 26, 2013
Item 63 – Class level
1 – First year
2 – Second year
3 – Third year
4 – Fourth year
5 - Special Student
6 - Graduate / Professional
7 - Other
DEMOG7
Item 64 – Educational goal
1 - Associate degree
2 –Bachelor's degree
3 - Master's degree
4 - Doctorate or professional degree
5 - Certification (initial or renewal)
6 - Self-improvement / pleasure
7 - Job-related training
8 - Other
Online vs. On-campus PSOL Responses
page 20 of 21
DEMOG8 Item 65 – Employment
1 – Full-time
2 – Part-time
3 – Not employed
DEMOG9
DEMOG10
Item 66 – Current residence
1 – Own house
2 – Rent room/apartment/house
3 – Relative’s house
4 – Residence hall
5 – Other residence
Item 67 – Marital status
1 – Single
2 – Single with children
3 – Married
4 – Married with children
5 – Prefer not to respond
DEMOG11
Item 68 – Current plans
1 – Complete online degree program
2 – Complete degree on campus
3 – Transfer credits
4 – Complete this course
DEMOG14
Item 71 - My intended degree is
Online undergraduate degree
Online graduate degree
Face-to-face undergraduate degree
Face-to-face graduate degree
DEMOG12
DEMOG13
Item 69 – Current online enrollment
1 – 1 to 3 credits
2 – 4 to 6 credits
3 – 7 to 9 credits
4 – 10 to 12 credits
5 – 13 to 15 credits
6 – More than 15 credits
70 – Previous online enrollment
1 – No classes
2 – 1 to 3 classes
3 – 4 to 6 classes
4 – 7 to 9 classes
5 – 10 to 12 classes
6 – 13 to 15 classes
7 – More than 15 classes
DEMOG15
Item 72 - My home CWU campus is:
1 - Ellensburg
2 - Des Moines, Kent, or Pierce County
3 - Lynnwood or Everett
4 - Moses Lake, Wenatchee, or Yakima
5 - Online: a fully online degree program
Group Code
Item 72 - Group code
1063: Health and Physical Education: Athletic Administration (MS)
1087: Information Tech & Admin Mgt (BAS)
1088: Information Tech & Admin Mgt (BS)
1090: Interdisciplinary Studies: Social Sciences (BS)
1094: Law & Justice (BA)
1107: Paramedicine (BS)
1125: Psychology (BA)
1137: School Administration (MED)
1145: Sociology (BA)
1151: Special Education (MED)
1165: Health and Physical Education: Teaching Physical Education and Health (MS)
1166: Higher Education (MED)
1167: Literacy (MED)
1168: Not enrolled in an online degree program
July 26, 2013
Online vs. On-campus PSOL Responses
page 21 of 21