Do On-Line Students Analyze,
Synthesize And Evaluate Better
Than Face-To-Face Students?
Preliminary Evidence
Drs. Steven C. Myers & Michael A. Nelson
myers@uakron.edu
Department of Economics - The University of Akron
Online at http://gozips.uakron.edu/~myers/online
Acknowledgments

Grant from
Carnegie Teaching Academy
Scholarship of Teaching, Assessment and Learning Funds
Institute for Teaching and Learning
The University of Akron

Data collected under signed informed consent from our students
subject to the Institutional Research Board for the Protection of
Human Subjects at The University of Akron.


Very few failed to give informed consent
Data collection instruments validated by a number of
independent reviewers and created by MA grad, Michael
Lovette.
Research Questions
(1)Does the mode of delivery (face-to-face or
web-based) have an influence on learning
outcomes?
(2)Are students in an online environment as
likely to do as well as in face-to-face
classes? Will they be able to equal the
complex problem solving of the face-toface students?
Course under study

Introduction to Economic Analysis





One-semester principles of economics
3 cr. General education course
Required for engineering majors
Both face-to-face and on line
The online course is similar to a graduate
course offered since Fall 2001
Comparison

Online






Offered by
Myers (F02) / Nelson (S03)
No face-to-face meetings
Mastery & competency
based learning
High professor-student
interaction
No student to student
interaction
Full use of the WebCT
environment

Face-to-face





Offered by
Michael Nelson
Regular class times
Lecture and active learning
collaborative techniques
Better than the norm of
83% “chalk and talk”
(Becker and Watts)
Some minor webenhancements such as
online gradebook
Student Characteristics

Online Students
More likely to be
 To be older
 To be non-white
 To be female
 To have taken more
credits
 To be decided in their
major
 More favorably disposed
to economics

Face-to-face Students
More likely
 To be male
 To be a freshman
 To be undecided in their
major
 To spend less time
gathering business and
economic news.
The online course design has
influenced grades, but learning?
Online Course
Face-to-face Course
Grades Distribution
A
B
C/D
F
33%
39%
0%
10%
8%
37%
40%
10%
MC Learning Assessment Breakdown / Online
Modules
Total
Recognition Conceptual Analytical %C&A
2-8 Micro
331
29%
45%
26%
71%
9-15 Macro
563
32%
54%
15%
68%
2-15 Total
894
31%
51%
19%
69%
Research design
Student’s success
Pre-test scores
=f( student’s
total
on recall and recognition
initial endowments,
on simple application problems
awareness of the economy,
on complex problems
attitudes about economics,
GPA
course modality,
Previous attempt of course
student characteristics)
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Survey of economic facts
20 questions…on trends and
facts in the economy.
2 questions:
How does the student collect
business and economic news.
How many hours per week
does the student spend
collecting news
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Three Survey Questions:
Level of student report of
being interested in
economics.
Level of student report of
liking this economics course.
Level of student report about
likelihood of taking more
economics courses.
5=strongly agree
4=agree
3=indifferent/no opinion
2=disagree
1=strongly disagree
Research design
Student’s success
=f( student’s
initial endowments,
awareness of the economy,
attitudes about economics,
course modality,
student characteristics)
Online or face-to-face
Major
Level, e.g., freshman
Credits completed
Age
Gender
Ethnicity
Dependent Variable: Student
Success Outcome Measures

examscore on first 6 chapters



Final examination



(FE, FE_recog, FE_apply, FE_complex)
First 3 or 4 levels of Bloom’s Taxonomy
Writing assignment



In class exam for face-to-face class
Average of first three module quizzes for online class
writing2
4th to 6th level of Bloom
Regular Grades in class
Bloom’s Taxonomy
1.
2.
3.
4.
5.
6.
(from Carla Lane)
Knowledge: arrange, define, duplicate, label, list, memorize, name,
order, recognize, relate, recall, repeat, reproduce state.
Comprehension: classify, describe, discuss, explain, express,
identify, indicate, locate, recognize, report, restate, review, select,
translate,
Application: apply, choose, demonstrate, dramatize, employ,
illustrate, interpret, operate, practice, schedule, sketch, solve, use,
write.
Analysis: analyze, appraise, calculate, categorize, compare,
contrast, criticize, differentiate, discriminate, distinguish, examine,
experiment, question, test.
Synthesis: arrange, assemble, collect, compose, construct, create,
design, develop, formulate, manage, organize, plan, prepare,
propose, set up, write.
Evaluation: appraise, argue, assess, attach, choose compare,
defend estimate, judge, predict, rate, core, select, support, value,
evaluate.
Results
Students’ Initial Endowments



All prior expectations are met
Higher pretest and higher GPA are positive
on examscore, writing2 and less so on FE.
Higher complex pretest scores and higher
GPA are positive for writing2 and
FE_Complex
Results
Economic Awareness & Attitudes



“Survey” is weakly positive for examscore
and writing2, but has no effect on FE.
No combination of “News gathering” is
ever significant.
“Attitudes” are strongly significant for
examscore and writing2, but no effect on
FE


When attitudes matter
take_more econ > interest in econ > will like econ
Results
Student characteristics:



Those with an undeclared or undecided major
do much worse than engineering majors for
examscore and FE.
Older students do worse on examscore, no effect
on writing2 and strongly positive on FE
Females do better in writing2 (strongly
significance), but do worse on FE (almost
significant)
Results
Online Students:


Do much better on examscore
Show no significance difference in scoring on analytical and
complex reasoning tasks.


Insignificant coefficients in writing2
Insignificant and small coefficients in





FE
FE_recog
FE_Apply
FE_complex
There is some evidence of a possible interaction with females

Weak suggestion in the data that


Men in online classes do better and
Women in online classes do worse all things equal
Design of the online course




Built in modules
Modules completed in order
Competency based testing
Use student feedback


For student learning enhancement
For modifying & improving the course
Course
Design



DL requires
planning for
contingencies
14 Content
Modules
Active Learning
vs. Passive
Learning
Pre-class and Module 1:
Preparing Students to Learn

Email me!





http://gozips.uakron.edu/~myers/online/
Is Distance Learning for Me?
VARK – testing learning styles
Orientation Module—‘ How to logon to WebCT’
Orientation Module—‘ How to Use WebCT’
Syllabus


Graduate course
Undergraduate course
Pre-class and Module 1:
Additional Orientation


How to Communicate with Dr. Myers
How to access your online text



http://www.economicsplace.com/econ5e/
Rules of the game
Building a relationship –


Breakdown the anonymity
Survey “Tell me about yourself”
Content Modules 2-15






Module Introduction & Objectives
Chapter Introductions
Content
Supportive Materials
Assessment of Learning - Quizzes on
objectives with multiple trials
Evaluation
2 Research & Writing Assignments –
some objectives




Practice and experience in reflecting on
a topic in the current economy.
Practice in analysis of economic trends.
Gaining of confidence about talking
about the economy.
Ability to know and use the resources of
economic commentary, prior analysis
and data.
Introductions
& Content





Mostly Passive
Learner Centered
Students progress
without
intervention
Micro –
Modules 2-8
Macro –
Modules 9-15
Graded
Assessment





Competency based
Everyone strives to
get a perfect 10
(Mastery)
Three attempts, 15
min. time limit
Questions a mix of
(1) Recognition,
(2) Conceptual, &
(3) Analytic
~70% C&A
Random
intervention by
Professor
Role of
Module Evaluation






Planned intervention
Forced contact
Focus on the learning
Professorial
encouragement
Decreases dropout
rates
Process repeats
Module Evaluation
A Classroom Assessment Technique from Angelo and Cross (1993);
tested by Chizmar and Ostrosky (1998)




What comments do you have on this module and your
experience in completing it?
What main point have you learned that you did not
fully understand before?
What questions … Include any points that still remain
muddy or unclear. Do consider posing the muddy
points to your fellow students in the discussions.
What recommendations do you have for us as we
continue to change and enhance the course?
Student Comments / online

“The fact that this course was completely
internet based, had no bearing on the level
of knowledge I gained from the course. I
think I learned as much, or even more, than
I would have in a traditional classroom. I
contribute this most to the way the
instructor organized the course.”
Student Comments / online

“I believe that the benefits of web-based
courses … far outweigh any disadvantages
such as lack of face-to-face interaction--at
least in this particular course. Dr. Myers'
course was, of the four I took this semester
over the web, truthfully the best organized,
most well-adapted to the web (by his
efforts) of them all.”
Do On-Line Students Analyze, Synthesize And
Evaluate Better Than Face-To-Face Students?
The answer is NO
and they don’t do any worse either.
Contact: Steve Myers myers@uakron.edu
The University of Akron
Or get this paper and presentation
Online at http://gozips.uakron.edu/~myers/