NORTHCENTRAL UNIVERSITY ASSIGNMENT COVER SHEET
Learner: Stephen W Watts
THIS FORM MUST BE COMPLETELY FILLED IN
Academic Integrity: All work submitted in each course must be the Learner’s
own. This includes all assignments, exams, term papers, and other projects required by
the faculty mentor. The known submission of another person’s work represented as that
of the Learner’s without properly citing the source of the work will be considered
plagiarism and will result in an unsatisfactory grade for the work submitted or for the
entire course, and may result in academic dismissal.
EDU7707-8
Dr. Amar Almasude
Planning Dissertation Research in Education
8 Write Draft Concept Paper
Assignment: Your task in this assignment is to write a complete draft of your
Concept Paper, making sure it is properly organized, well-written, and perfectly in accord
with the Concept Paper template and the Minimum Standards for a Concept Paper. You
will continue to work on your Concept Paper in your first DIS course, so this is a draft,
but it should be the very best you can make it at this point. To complete this assignment
well, you need to draw on your learning from previous courses, depending on whether
you are proposing a qualitative or quantitative study. Be sure that: (a) your literature
review is sufficiently thorough to support your claim of a gap in the research literature,
(b) the gap you have identified is the basis for a feasible, interesting, worthy, and relevant
study (c) you describe how the study you propose allows you to address the gap in a way
that contributes to theory, (d) tour problem and purpose statements and your research
questions (and hypothesis) are in the form specified in the Concept Paper template and
are aligned, (e) your proposed design will allow you to answer your research questions.
(Draw on the designs used in the studies you reviewed.), (f) the document precisely
follows the format of the Concept Paper template and APA style, and (g) at the bottom of
your paper, paste the Minimum Standards for a Concept Paper and explain how each
section meets the criteria for the section.
Faculty Use Only
Excellent! Unfortunately, your Problem Statement is still lacking support from the
available research. So, I would strongly recommend reviewing some of the studies fin the
area of Multimedia Principle and Instructional Design for Online Learning. Besides that,
I think you did very well in this class. Thank you for your heard work and dedication!
Dr. Amar Almasude
A
95
May 2, 2012
Technological Tools Impact on Learning in Online Professional Development Courses
Concept Paper
Submitted to Northcentral University
Graduate Faculty of the School of Education
in Partial Fulfillment of the
Requirements for the Degree of
DOCTOR OF EDUCATION
by
Stephen W. Watts
Prescott Valley, Arizona
April 2012
i
Table of Contents
Introduction ......................................................................................................................... 1
Statement of the Problem .................................................................................... 2
Purpose of the Study ........................................................................................... 3
Research Questions ............................................................................................. 4
Hypotheses .......................................................................................................... 5
Definition of Key Terms ..................................................................................................... 5
Brief Review of the Literature ............................................................................................ 6
Adult Learning Theories ..................................................................................... 7
Adult E-Learning ................................................................................................ 9
Measuring Adult Learning ................................................................................ 13
E-Learning Factors ........................................................................................... 15
Summary ........................................................................................................... 16
Research Method .............................................................................................................. 17
Operational Definition of Variables ................................................................. 18
Measurement ..................................................................................................... 19
Summary ........................................................................................................... 19
References ......................................................................................................................... 21
Appendix A: Annotated Bibliography .............................................................................. 29
Seminal Article ................................................................................................. 29
Adult Learning Theory ..................................................................................... 30
Teaching Online – Key Findings ...................................................................... 37
Learner Characteristics – Key Findings............................................................ 44
Application and Instructional Design ............................................................... 51
Research Design ............................................................................................... 60
Appendix B: Concept Paper Minimum Standards ............................................................ 67
ii
WattsSEDU7707-8-8
1
Introduction
In the past decade technological advances in information and communication
technology (ICT) have caused drastic changes in the way many people communicate,
socialize, work, and receive training or education. A decade ago institutions of higher
education, and professional development were beginning to explore the new terrain of
what has become e-learning, but few were successful in delivering quality education
using this relatively new media (Broadbent, 2002). Now there are many colleges,
universities, and professional development firms that have embraced e-learning as an
appendage of their instructional portfolio.
The benefits to learners are abundant with e-learning. Specific benefits include;
improving learning efficiency (Cabrera-Lozoya, Cerdan, Cano, Garcia-Sanchez, & Lujan,
2012; Chen & Lien, 2011; Huang, Lin, & Huang, 2012), affecting the way learners
behave (Bhuasiri, Xaymoungkhoun, Zo, Rho, & Ciganek, 2011; Haythornthwaite, Bruce,
Andrews, Kazmer, Montague, & Preston, 2007), enhancing communication (Abrami,
Bernard, Bures, Borokhovski, & Tamim, 2010; Alshare, Freeze, Lane, & Wen, 2011),
convenience (Anderson, 2008; Desai, Hart, & Richards, 2008), saving of time (Lam &
Bordia, 2008; Pastore, 2012), and improved learning ability (Donavant, 2009; Ismail,
Gunasegaran, & Idrus, 2010). More learners are attending online classes due to these
benefits despite factors that lower learner satisfaction with the delivery of such courses
(McGlone, 2011). The dilemma of e-learning is not all students take advantage of these
benefits; the incidence of dropout or failure in online courses is much larger than for
traditional classes (Al-Fahad, 2010; Pigliapoco & Bogliolo, 2008).
The high rate of dissatisfaction with online courses has led to studies focusing on
WattsSEDU7707-8-8
2
the causes of satisfaction and dissatisfaction with online learning (Gunawardena, LinderVanBerschot, LaPointe, & Rao, 2010; Martinez-Caro, 2009). Despite dissatisfaction
with online learning, studies show that students in online courses learn better, as
indicated by grades or acknowledging perceived learning, as they participate more
(Huang et al., 2012; Martinez-Caro, 2009; Watkins, 2005; Zemke & Zemke, 1995), and
as their personal satisfaction with the course increases (Chen & Lien, 2011; Kozub, 2010;
Martinez-Caro, 2009). Student satisfaction with online courses decreases as learner-tolearner interaction, teacher-to-learner interaction (Martinez-Caro, 2009), and the amount
of reflection allowed within the course decreases (McGlone, 2011; Watkins, 2005). In
this context of student satisfaction and dissatisfaction with online learning the succeeding
study is proposed.
Statement of the Problem
The largest factor of dissatisfaction in adult online learning is the lack of face-toface interaction by the learner with the facilitator or other learners (Alshare, Freeze, Lane,
& Wen, 2011; Boling, Hough, Krinsky, Saleem, & Stevens, 2011; Donavant, 2009;
Pigliapoco & Bogliolo, 2008). Dissatisfaction culminates in higher dropout rates (AlFahad. 2010; Pigliapoco & Bogliolo, 2008), decreased motivation to learn (Omar, Kalulu,
& Belmasrour, 2011; Park & Choi, 2009), less participation, and consequently, less
learning (Jackson, Jones, & Rodriguez, 2010; Martinez‐Caro, 2009; Shea, Fredericksen,
& Pickett, 2006; Zemke & Zemke, 1995). A relationship has been demonstrated between
online participation and learning performance (Huang, Lin, & Huang, 2012;
Martinez‐Caro, 2009; Pelz, 2010; Ruey, 2010), as well as between learning performance
and student satisfaction in online courses (Ali & Ahmad, 2011; Chen & Lien, 2011;
WattsSEDU7707-8-8
3
Ferguson & DeFelice, 2010; Kozub, 2010; Martinez‐Caro, 2009). However, there is little
empirical research regarding adult professional development or appropriate techniques
for teaching and engaging non-traditional learners (Donavant, 2009), or on appropriate
modes of interaction in learning management systems (So & Bonk, 2010). The specific
problem is to investigate whether having a visual element (webcam) in an online adult
professional development learning environment can overcome the lack of face-to-face
interaction and foster increased learner participation, satisfaction, and perceived learning.
Knowledge gained will enlarge the currently small knowledge base regarding online
professional development training (Chen & Lien, 2011; Donavant, 2009), will contribute
a better understanding of facilitating engaging online instruction (Bradley, 2009; Huang
et al., 2012; Watkins, 2005), and assist in identifying the proper level and types of media
for use in the online classroom (Fletcher, Tobias, & Wisher, 2007; Martinez‐Caro, 2009).
Purpose of the Study
The purpose of this quasi-experimental nonequivalent groups study is to investigate
whether the addition of a visual element (webcam) can foster increased learner
participation, increased learner satisfaction, and increased perceived learning in an online
adult professional development learning environment. Eight instructors will teach two
separate live virtual classes (LVC) for a US-based Technology Company. One class each
will be a control class and one will utilize the webcam to promote additional interaction
for the student-instructor relationships, and attempt to mitigate the lack of face-to-face
interaction noted as the primary source of dissatisfaction for online students. The
students of these LVC, who can sign in from any location worldwide, will be surveyed
after each class to ascertain their satisfaction, engagement, and perceived learning with
WattsSEDU7707-8-8
4
the class as measured by sections of the Learner Satisfaction and Transfer-of-learning
Questionnaire (LSTQ; Gunawardena et al., 2010). A one-way analysis of variance
(ANOVA) will be conducted to determine whether the use of the visual element
increased learner participation, satisfaction, or perceived learning in the experimental
classes versus the control classes.
Research Questions
The research questions identified for this study are included to evaluate the
relationships between the independent variables - the introduction and use of a visual
input (webcam) in an adult online professional development learning environment, and
the dependent variables of learner satisfaction, learner engagement, and perceived
learning. Associated with the problem and purpose statements the following research
questions will be addressed.
Q1. How does satisfaction of adult learners, as measured by the Learner
satisfaction subsection of the LSTQ (Gunawardena et al., 2010), vary, if at all, in
an online live virtual classroom (LVC) environment when the learners
continuously see the instructor through visual technology (webcam)?
Q2. How does engagement or participation of adult learners, as measured by the
Learner-learner interaction and Learner-instructor interaction subsections of the
LSTQ (Gunawardena et al., 2010), vary, if at all, in an online LVC environment
when the learners continuously see the instructor through visual technology
(webcam)?
Q3. How does perceived learning of adult learners, as measured by the Ability to
transfer subsection of the LSTQ (Gunawardena et al., 2010), vary, if at all, in an
WattsSEDU7707-8-8
5
online LVC environment when the learners continuously see the instructor
through visual technology (webcam)?
Hypotheses
H10.
Learner satisfaction does not change when the visual (webcam) element is
used continuously as opposed to when it is not in online LVC instruction of adult
technical professional development courses.
H1a.
Learner satisfaction changes when the visual (webcam) element is used
continuously as opposed to when it is not in online LVC instruction of adult
technical professional development courses.
H20.
Learner participation does not change when the visual (webcam) element
is used continuously as opposed to when it is not in online LVC instruction of
adult technical professional development courses.
H2a.
Learner participation changes when the visual (webcam) element is used
continuously as opposed to when it is not in online LVC instruction of adult
technical professional development courses.
H30.
Learner perceived learning does not change when the visual (webcam)
element is used continuously as opposed to when it is not in online LVC
instruction of adult technical professional development courses.
H3a.
Learner perceived learning changes when the visual (webcam) element is
used continuously as opposed to when it is not in online LVC instruction of adult
technical professional development courses.
Definition of Key Terms
LVC. Live Virtual Classes. Class participants are able to communicate
WattsSEDU7707-8-8
6
synchronously using a conference call (phone) connection, while class content is
projected in real-time to individual work stations by using the WebEx learning
management system. WebEx also provides synchronous two-way chat, polls, question
and answers, and video interaction of instructor and students through webcam. Each
class participant has access to a lab environment to practice the skills taught in the class
through a remote connection, and the WebEx environment can project participant’s lab
screens for demonstration or troubleshooting purposes.
Brief Review of the Literature
The rapid advances of technology over the past decade have led to a dramatic
shift in the demographics of post-secondary students, as about 40% are over the age of
25, and a majority of these more mature learners are increasingly choosing e-learning to
pursue higher education (Ke & Xie, 2009) and professional development (Gunawardena
et al., 2010). Adults, or nontraditional students, learn differently than do traditional
students, or younger adults entering post-secondary education straight from secondary
education (Bye, Pushkar, & Conway, 2007; Ke & Xie, 2009; Kenner & Weinerman,
2011; Zemke & Zemke, 1995). Historically, these differences have been ignored in
higher education, and in online courses, where the same pedagogies and curriculum face
both the traditional and non-traditional learner (Ke & Xie, 2009). There has also been
little research outside of higher education regarding how mature adults learn best in a
virtual classroom (Chen & Lien, 2011; Donavant, 2009). In this section various adult
learning theories will be expounded to create a foundation from which to address
research findings on the optimal ways that adults learn online, along with characteristics
that detract from online learning. With a grasp on the characteristics that enhance adult
WattsSEDU7707-8-8
7
learning, various means of measuring learning will be identified and expanded upon,
which will help identify factors contributing to learning.
Adult Learning Theories
There are dozens of learning theories that provide a rich foundation for
understanding the complexity of learning and teaching (Minter, 2011). These theories
often have common characteristics, have strengths and shortcomings, and have their
supporters and detractors. Many of these theories do not differentiate between teaching
adults and teaching children, or are not applicable to adult learners (Minter, 2011). When
working with the adult learner the underlying premise of these theories is adults learn in a
different way; therefore teachers of adults need to use different instructional methods
(Minter, 2011; Zemke & Zemke, 1995). For this reason many authors differentiate the
term andragogy to identify the methods of teaching adult learners, and pedagogy to
identify the methods of teaching children (Commonwealth of Learning, 2000; Karge,
Phillips, Dodson, & McCabe, 2011).
Andragogy, “the art and science of helping adults learn” (Blanchard, Hinchey, &
Bennett, 2011, p. 2; Cercone, 2008, p. 137), is a foundational theory that has many
supporters. The term was originally coined by Alexander Kapp in 1833 and
philosophically flows from Plato’s theory regarding education (Abela, 2009). Malcolm
Knowles was the leading proponent of andragogy in the U.S. and developed a number of
tenets describing the adult learner, and these have been expanded by various authors.
Although originally touted as a complete explanation of how adults learn, Knowles later
acknowledged “pedagogy and andragogy probably represent the ends of a spectrum that
ranges from teacher-directed to student-directed learning. Both approaches, he and
WattsSEDU7707-8-8
8
others now suggest, are appropriate with children and adults, depending on the situation”
(Zemke & Zemke, 1995, para. 12). The main principles of andragogy include:

Adult learners are independent and will not necessarily learn what they are
told but need to understand why they need to learn something and the benefits
it will bring (Baskas, 2011a; Fidishun, 2011; Kenner & Weinerman, 2011;
Strang, 2009).

Adult learners become more self-directed and need to have control over their
learning (Blanchard et al., 2011; Guilbaud & Jerome-D’Emilia, 2008;
McGlone, 2011).

Adult learners have a varied and rich experience base, as well as different
learning styles and motivators. Adult learners want to be acknowledged for
and have their experiences used in learning (Abela, 2009; Blanchard et al.,
2011; Fidishun, 2011; Kenner & Weinerman, 2011).

Adult learners are more motivated to learn when a challenge enters their life;
encouraging them to discover how to handle better (Baskas, 2011a; Donavant,
2009; Zemke & Zemke, 1995).

Adult learners are interested in learning how to solve problems, perform tasks,
or improve their life (Cercone, 2008; Chyung & Vachon, 2005; Kenner &
Weinerman, 2011).

Adult learners become more intrinsically motivated, focusing on aspirations
than extrinsically motivated (Abela, 2009; Donavant, 2009; Minter, 2011).

Adult learners expect a student-centered approach to learning in an
environment of mutual respect between teacher and student, and between
WattsSEDU7707-8-8
9
students (Karge et al., 2011; Kenner & Weinerman, 2011; McGlone, 2011;
Minter, 2011).
There are numerous arguments, discussions, principles propounded regarding
adult learning theory and there is still no single unified model, theory, or set of principles
all subscribe to (Baskas, 2011b; Merriam, Caffarella, & Baumgartner, 2007; Zemke &
Zemke, 1995). The principles of andragogy are accepted by most educators of adults as
foundational even though it is acknowledged several factors important to the teaching of
adults are not included or emphasized (Abela, 2009; Blanchard et al., 2011; Donavant,
2009; Strang, 2009).
Another popular theory in the literature professes adults have certain preferred
learning styles, and this predilection dictates certain behaviors; among these behaviors
are an inclination for receiving instruction in certain ways (Buch & Bartley, 2002;
Kozub, 2010), which acts as a predictor of performance (Huang, Lin, & Huang, 2012;
Kozub, 2010). Whereas the research results regarding learning styles is mixed (Cercone,
2008; Kirschner, Sweller, & Clark, 2006), it does underscore adults learn differently.
Adult E-Learning
The literature consistently identifies six characteristics contributing to optimal elearning for adults (Cercone, 2008). These six characteristics include (a) a strong
student-instructor relationship and facilitation by the instructor (Boling et al., 2011;
Chyung & Vachon, 2005; Jackson et al., 2010; Simonson, Schlosser, & Hanson, 1999),
(b) student-student interaction and collaboration (Abrami et al., 2010; McGlone, 2011;
Pelz, 2010; Sinclair, 2009; Yang & Cornelious, 2005), (c) reflection by the learner to tie
new learning to existing experience (Ali & Ahmad, 2011; Cacciamani, Cesareni, Martini,
WattsSEDU7707-8-8
10
Ferrini, & Fujita, 2012; Cercone, 2008; Ruey, 2010), (d) development of a sense of
community among participants (Andrews & Haythornthwaite, 2007; Sharples, Taylor, &
Vavoula, 2007; Tallent-Runnels, Thomas, Lan, Cooper, Ahern, Shaw, & Liu, 2006), (e)
immediate real world application of learning (Baskas, 2011b; Blanchard et al., 2011; Ke
& Xie, 2009; Segrave & Holt, 2003; Zemke & Zemke, 1995), and (f) student motivation
(Abrami et al., 2010; Baskas, 2011b; Kenner & Weinerman, 2011; Omar et al., 2011).
Research demonstrates that as these characteristics are included and emphasized in online
learning the performance of adult learners increases, as does their participation, and
satisfaction.
Success in distance education has many factors, but key to learning for the student
is development of the student-instructor relationship (Simonson et al., 1999) and the
instructor’s level of interaction with the learner (Jackson et al., 2010; Martinez-Caro,
2011). Chyung and Vachon (2005) identified that four of the seven most significant
factors contributing to a learner’s satisfaction were directly related to an instructor’s skills
or their interaction with the student. The supportive and nurturing relationship of learner
and instructor increases learner satisfaction with online courses (Ali & Ahmad, 2011;
Jackson et al., 2010; Shea et al., 2006), improves motivation (Al-Fahad, 2010; Omar et
al., 2011; Park & Choi, 2009; Pigliapoco & Bogliolo, 2008), and optimized learning
outcomes (Abrami et al., 2010; Boling et al., 2011; Jackson et al., 2010; Pelz, 2010).
Regarding the second critical success factor in e-learning, Boling et al. (2011)
argued today’s technology requires a shift from a teacher-centered to a student-centered
paradigm, which relegates the instructor to the role of mentor, guide, coach, or facilitator
(Blanchard et al., 2011; Cabrera-Lozoya et al., 2012; Oncu & Cakir, 2011). One of the
WattsSEDU7707-8-8
11
most important factors in successfully facilitating online is projecting teaching presence
(Archambault, Wetzel, Fouger, & Williams, 2010; Bradley, 2009; Pelz, 2010); the ability
to connect with students (Ke, 2010), and encourage them and provide the necessary
scaffolding to promote learning and self-reliance in the learner (Anderson, 2008;
Cacciamani et al., 2012; Cercone, 2008; Tallent-Runnels et al., 2006) while staying in the
background as much as possible (Hoic-Bozic, Mornar, & Boticki, 2009; Ke, 2010).
When transitioning from the traditional classroom to online, mastering facilitation can be
a great challenge for the instructor (Allen, Crosky, McAlpine, Hoffman, & Munroe,
2009; Jackson et al., 2010) and can be the key to success or failure (Lombardi, 2007). As
teaching presence increases, so does student satisfaction (Donovant, 2009; Ferguson &
DeFelice, 2010; Gunawardena et al., 2010), engagement (Ke & Hoadley, 2009),
motivation (Diaz & Entonado, 2009), and accomplishments (Ally, 2008) as students
actively participate in learning (Yang & Cornelious, 2005).
Another key element to successful learning is self-reflection by the learner, which
engenders deep learning (Cercone, 2008; Ke & Xie, 2009), high-quality learning (Ke,
2010; Ruey, 2010), meta-learning (Baskas, 2011a; Bradley, 2009), and metacognitive
expertise (Cacciamani et al., 2012). Reflection also allows learners to examine their
biases (Baskas, 2011b), other perspectives (Sinclair, 2009) so they can internalize (Ally,
2008; Strang, 2009), contextualize (Bradley, 2009; Fidishun, 2011), and transform
experience and knowledge into learning (Buch & Bartley, 2002; Chan Mow, 2008), while
boosting motivation (Abela, 2009; Baskas, 2011a), and promoting higher order learning
(Taran, 2006). Studies demonstrate reflection is a key online design dimension (Ali &
Ahmad, 2011; Ke, 2010; Yang & Cornelius, 2005) and students seem to prefer e-learning
WattsSEDU7707-8-8
12
because of their ability to reflect before engaging in discussions (Andrews &
Haythornthwaite, 2007; Ke & Hoadley, 2009; Martinez-Caro, 2011; Sinclair, 2009).
A sense of community is vital for successful online learning (Andrews &
Haythornthwaite, 2007; Boling et al., 2011; Tallent-Runnels et al., 2006). It is the role of
the instructor to lead community-building activities (Ally, 2008; Muirhead, 2004) and his
or her example is key to the establishment of a sense of community (Ally, 2008; Ambrose
& Ogilvie, 2010) through accurate and timely feedback (Desai et al., 2008; TallentRunnels et al., 2006), encouragement of participation and interaction (Boling et al., 2011;
Cornelius, Gordon, & Ackland, 2011; Yang & Cornelius, 2005), nurturing caring and
healthy relationships (Abrami et al., 2010; Caine, 2010), and modeling effective and open
communication (Desai et al., 2008). When students feel a sense of belonging to a
community and care for other members of the group significant benefits have been noted.
The benefits to students are they (a) bond earlier and better than in traditional classrooms
(Pelz, 2010), (b) engage in more reflective thinking (Bradley, 2009), (c) better understand
the material (Bradley, 2009), (d) are more motivated (Abrami et al., 2010; Boling et al.,
2011; Karge et al., 2011) and satisfied (Pigliapoco & Bogliolo, 2008), (e) persist with
their studies (Pigliapoco & Bogliolo, 2008), and (f) learn more (Boling et al., 2011; Fahy,
2008; Moisey & Hughes, 2008; Pigliapoco & Bogliolo, 2008).
An additional factor to successful online courses is addressing real-world
applications. According to andragogy, students are more interested in immediate
problem-centered approaches to learning, so learning can improve their work, family, or
personal life (Abela, 2009; Blanchard et al., 2011; Kenner & Weinerman, 2011). By
encouraging students to bring their experience and problems into the classroom learners
WattsSEDU7707-8-8
13
are able to construct deeper and more robust knowledge, while expanding their abilities
to handle actual problems (Allen et al., 2009; Ruey, 2010). This application of real-world
learning is a motivator (Fidishun, 2011) and enriches learning.
The final factor mentioned regularly in the literature is the need for students to be
motivated. Motivation has been demonstrated to significantly increase in students
because of good student-instructor relationships (Al-Fahad, 2010; Chickering & Gamson,
1987; Lam & Bordia, 2008), strong teaching presence (Diaz & Entonado, 2009), having a
sense of community (Abrami et al., 2010; Boling et al., 2011; Karge et al., 2011),
participating or collaborating in learning (Omar et al., 2011; Park & Choi, 2009;
Pigliapoco & Bogliolo, 2008), being encouraged to reflect on new learning (Abela, 2009;
Baskas, 2011b), having material clearly presented (Abrami et al., 2010; Ali & Ahmad,
2011; Alshare et al., 2011), and working through real-world problems (Fidishun, 2011).
Though student motivation is assumed to be a major factor of adult learning, KiliçCakmak (2010) identified that “little or no attention [has been] paid to presentation
methods that influence” (p. 195) motivation.
As each of these factors is present in an online course it is important to be able to
verify their effects on students. Verification comes in the form of measurement.
Measurement of adult learning takes many forms, which will be discussed next.
Measuring Adult Learning
Learning is a subjective and deeply personal experience. Correlations have been
made between measures of perceived learning and other factors to make it possible to
determine how well an instructor has performed even in situations where students do not
necessarily receive grades. The most common methods of determining the amount of
WattsSEDU7707-8-8
14
learning that has taken place in a class are through measuring performance or satisfaction
(Martinez-Caro, 2009). Recently, structural equation modeling (SEM) has been used to
estimate “causal relationships using a combination of statistical data and qualitative
causal assumptions” (p. 576) to identify how various factors affecting learning work
together (Martinez-Caro, 2009; Strang, 2009).
Performance is usually measured in terms of quantitative assessments. Teachers
attempt to determine the level and amount of learning through tests, quizzes, and papers,
and generally apportion grades in accordance with some rubric identifying how well they
believe a student has learned specific material. Performance as measured by grades is
highly dependent on several factors independent of learning, e.g., writing skills, class
participation, prior knowledge, or grading inconsistency (Martinez-Caro, 2009). In
professional development courses grades are not usually given, so learning of the student
generally comes from self-report data of how much knowledge or skill the learner
believes he or she acquired (Donavant, 2009; Gunawardena et al., 2010; Martinez-Caro,
2009).
Another means of determining learning in adults is through their satisfaction with
a course. Martinez-Caro’s (2009) SEM analysis demonstrated there is a strong positive
correlation between a student’s perceived learning and his or her satisfaction, r = .73 (p.
577). Several other authors have used learner satisfaction as the appropriate measure of
effectiveness of learning in online courses (Gunawardena et al., 2010; Kozub, 2010;
McGlone, 2011). This precedent means that under proper circumstances evaluation of a
student’s satisfaction can be an effective means of determining the effectiveness of the
instruction and of the learning.
WattsSEDU7707-8-8
15
E-Learning Factors
Numerous studies have been conducted to determine which factors affect
effective e-learning. As more learners participate in an online class the higher their
satisfaction and the more they learn (Huang et al., 2012; Gunawardena et al., 2010;
Watkins, 2005). Several studies have found one of the most important factors regarding
performance and satisfaction is the amount of interaction between the student and the
instructor (Gunawardena et al, 2010; Zemke & Zemke, 1995). One study found
“interaction is key to effective e-learning, with teacher-student interaction the strongest
predictor of learning in e-learning” (Martinez-Caro, 2009, p. 578). As mentioned
previously, the interactions and collaboration between learners weigh heavily on the
satisfaction of the student (Cercone, 2008; Ke & Xie, 2009; Martinez‐Caro, 2009;
McGlone, 2011; Sinclair, 2009; Zemke & Zemke, 1995), with one study showing it is
“the highest predictor of transfer of learning” (Gunawardena et al, 2010, p. 223). The
convenience and flexibility of the online experience rank highly in factors adding to
learner satisfaction (Donavant, 2009; Ismail et al., 2010).
Chyung and Vachon (2005) identified factors for dissatisfaction with e-learning,
and recognized that satisfaction is not the inverse of dissatisfaction, and vice versa. Their
study showed because students are not satisfied does not automatically mean they are
dissatisfied; conversely just because students are not dissatisfied does not mean they are
satisfied. For this reason they suggested factors contributing to both should be identified,
and those tending toward satisfaction should be maintained or added, and those causing
dissatisfaction should be eliminated or reduced. In this vein, the number one perceived
disadvantage of e-learning by new students is the lack of face-to-face interaction between
WattsSEDU7707-8-8
16
the student and instructor (Donavant, 2009), and the most significant factors contributing
to learner dissatisfaction in e-learning courses is a perceived lack in (a) the instructors
participation level, (b) the instructors feedback and responsiveness, and (c) the
instructor’s giving of clear directions and setting of expectations (Chyung & Vachon,
2005).
Summary
Outside of higher education there is little research in the area of adult education,
and even within higher education very little research has sought to distinguish the
characteristics, traits, and proclivities of the non-traditional student. Andragogy provides
seven assumptions pertaining to the adult learner, but fails to mention or expand on
several factors important to learning. The most common factors mentioned in the
literature for successful adult online learning are (a) the need for a rich and engaging
student-instructor relationship and facilitation by the instructor, (b) collaboration between
students, (c) reflection by the student to meld new knowledge with past knowledge and
experience, (d) building a sense of community between the participants of an online
class, (e) the application of knowledge to immediate, real-world problems, and (f) the
need to enhance student motivation.
To determine if a treatment is successful it is necessary to accurately measure
whether learning has occurred. Because learning is an internal process is not usually
visible to the outward observer several elements have been researched to determine how
well they measure actual learning, and self-reported satisfaction of the learner has been
demonstrated to correlate closely with other factors representative of adult learning.
Many satisfiers have been verified through research such as participation, interaction
WattsSEDU7707-8-8
17
between student and instructor, and between students, and convenience, but the main
dissatisfier in online classes remains the lack of face-to-face interaction between learner
and instructor.
Research Method
A quasi-experimental design method is chosen because it is not possible to
randomly place learners into control and test groups since learners purchase the
appropriate class for their professional development needs and such other motivators
personal to each student in LVC. Similarly, whether a student purchases a technological
course that is conducted in a traditional, face-to-face environment or in a digital LVC
may not be completely at the discretion of the learner, nor determined by the providing
company or presenting instructor. For this reason any consideration of a true
experimental design to determine cause and effect is not possible.
Using the same instructors to deliver similar courses, one as the control and one
as the experiment, provide good internal and reasonable external validity while focusing
on whether the continuous use of a visual element results in changes in satisfaction, class
participation, and perceived learning among students. The control group allows for
statistical manipulation to roughly approximate randomization effects (Edgington, 1966;
Wright, 2006). Data will be collected at the end of each class using an online survey.
Incomplete and surveys that have the same value for all sixteen answers will be
discarded. The data will then be separated into the constructs of satisfaction,
engagement, and perceived learning and an ANOVA conducted to determine if
significant differences exist between the construct means of the control and experimental
groups.
WattsSEDU7707-8-8
18
Operational Definition of Variables
The independent variable for this study is whether the visual element (webcam) of
the instructor is continuously transmitting to adult learners in a LVC as in the
experimental classes, or not as in the control classes. All instructors engaged in the
research are to conduct and facilitate their classes as they normally would with the sole
exception of the independent variable. Measures of three dependent variables will be
collected using the Learner Satisfaction and Transfer-of-learning Questionnaire (LSTQ);
these dependent variables are learner satisfaction, learner participation, and perceived
learning.
Independent variable - visual element. The webcam in this research allows the
transmitting of limited facial expressions and body language of the instructor to the
student. It is a nominal variable as minimal or no visual transmissions will occur in the
control classes, while there will be continuous visual transmissions during lecture, or
participation cycles in the experimental classes between instructor and students.
Dependent variable - learner satisfaction. Learner satisfaction has been chosen
as a dependent variable due to studies that indicate that as student satisfaction increases
so does participation (Gunarwardena et al., 2010) and learning outcomes (Gunawardena
et al., 2010; Kozub, 2010; Martinez-Caro, 2009; McGlone, 2011). Learner satisfaction is
a construct that will be derived from the Learner satisfaction subscale of the LSTQ and
consists of five questions on a 5-point Likert scale, and is thus an interval variable.
Dependent variable - learner engagement. Learner engagement has been
chosen as a dependent variable due to studies that indicate that as students are interactive
with the instructor, other students, and the content they learn more effectively (). Learner
WattsSEDU7707-8-8
19
engagement is a construct that will be derived from both the learner-learner interaction
and learner-instructor interaction subscales of the LSTQ and consists of six questions on
a 5-point Likert scale, and is thus an interval variable.
Dependent variable - learner perceived learning. The objective of adult
professional development is to enhance the knowledge and skills of adult workers so that
they are more productive and effective in their working environment. Generally, students
in adult professional development courses do not participate in evaluated activities or
receive grades. Multiple studies have identified that a student’s self-perception of
learning is as valuable an indicator of learning as any external measure. Perceived
learning is a construct that will be derived from the ability to transfer subscale of the
LSTQ and consists of five questions on a 5-point Likert scale, and is thus an interval
variable.
Measurement
Collection of data for this research will be done at the culmination of each of the
LVC through an online survey. The 16 questions from the LSTQ will be presented in no
particular order with a 5-point Likert scale. The LSTQ has been previously validated
from similar research regarding student satisfaction and transfer of learning. The learner
satisfaction subscale of the LSTQ has a Cronbach alpha of .83 making it extremely
reliable. The reliability of the learner-learner interaction subscale of the LSTQ has a
Cronbach alpha of .69 for good reliability and the learner-instruction interaction subscale
of the LSTQ has fair reliability with a Chronbach alpha of .52. The ability to transfer
subscale of the LSTQ has fair to good reliability with a Chronbach alpha of .62.
Summary
WattsSEDU7707-8-8
20
This research study proposes a quasi-experimental, nonequivalent groups study to
investigate whether the addition of a visual element (webcam) fosters increased learner
participation, satisfaction, and perceived learning in an online adult professional
development learning environment. Each participating instructor will teach two LVC.
One class will be taught in the normal way as a control; one will be taught using a
continuous webcam feed to students as the experiment. An online survey with questions
from four subscales of the Learner Satisfaction and Transfer-of learning Questionnaire
will be administered at the end of each class; the quantitative data garnered and
evaluated, and finally compared using an ANOVA.
WattsSEDU7707-8-8
21
References
Abela, J. (2009). Adult learning theories and medical education: A review. Malta
Medical Journal, 21(1), 11-18. Retrieved from
http://www.um.edu/mt/umms/mmj/PDF/234.pdf
Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. (2010, July).
Interaction in distance education and online learning: Using evidence and theory
to improve practice. The Evolution from Distance Education to Distributed
Learning. Symposium conducted at Memorial Union Biddle Hotel, Bloomington,
IN. Retrieved from
http://www.aect.org/events/symposia/Docs/InteractionDEnext120510.pdf
Al-Fahad, F. N. (2010). The learners’ satisfaction toward online e-learning implemented
in the college of applied studies and community service, King Saud University,
Saudi Arabia: Can e-learning replace the conventional system of education?
Turkish Online Journal of Distance Education (TOJDE), 11(2), 61-72. Retrieved
from https://tojde.anadolu.edu.tr/
Ali, A., & Ahmad, I. (2011). Key factors for determining students’ satisfaction in
distance learning courses: A study of Allama Iqbal Open University.
Contemporary Educational Technology, 2(2), 118-134. Retrieved from
http://cedtech.net/
Allen, B., Crosky, A., McAlpine, I., Hoffman, M., & Munroe, P. (2009). A blended
approach to collaborative learning: Making large group teaching more studentcentred. The International Journal of Engineering Education, 25(3), 569-576.
Retrieved from http://www.ijee.ie/
Ally, M. (2008). Foundations of educational theory for online learning. In T. Anderson
(Ed.), The theory and practice of online learning (pp. 15-44). Edmonton, AB:
Athabasca University.
Alshare, K. A., Freeze, R. D., Lane, P. L., & Wen, H. J. (2011). The impacts of system
and human factors on online learning systems use and learner satisfaction.
Decision Sciences: Journal of Innovative Education, 9(3), 437-461. Retrieved
from http://www.dsjie.org/dnn/default.aspx
Ambrose, J., & Ogilvie, J. (2010). Multiple modes in corporate learning: Propelling
business IQ with formal, informal and social learning. Journal of Asynchronous
Learning Networks, 14(2), 9-18. Retrieved from
http://sloanconsortium.org/sites/default/files/Multiple_Modes_in_Corporate_Lear
ning_Propelling_Business_IQ_with_Formal,_Informal_and_Social_Learning_0_
0.pdf
Anderson, T. (2008). Teaching in an online learning context. In T. Anderson (Ed.), The
theory and practice of online learning (pp. 343-365). Edmonton, AB: Athabasca
University.
WattsSEDU7707-8-8
22
Andrews, R., & Haythornthwaite, C. (2007). Introduction to e-learning research. In R.
Andrews, & C. Haythornthwaite (eds.). The SAGE handbook of e-learning
research (pp. 1-51). Los Angeles, CA: SAGE.
Archambault, L., Wetzel, K., Fouger, T. S., & Williams, M. K. (2010). Professional
development 2.0: Transforming teacher education pedagogy with 21st century
tools. Journal of Digital Learning in Teacher Education, 27(1), 4-11. Retrieved
from http://www.iste.org/learn/ publications/journals/jdlte.aspx
Baskas, R. S. (2011a). Applying adult learning and development theories to educational
practice. Retrieved from ERIC Database. (ED519926)
Baskas, R. S. (2011b, March 27). Adult learning assumptions. Retrieved from ERIC
Database. (ED517971)
Bhuasiri, W., Xaymoungkhoun, O., Zo, H., Rho, J. J., & Ciganek, A. P. (2011). Critical
success factors for e-learning in developing countries: A comparative analysis
between ICT experts and faculty. Computers & Education, 58(2), 843-855.
doi:10.1016/j.compedu.2011.10.010
Blanchard, R. D., Hinchey, K. T., & Bennett, E. E. (2011, April). Literature review of
residents as teachers from an adult learning perspective. Paper presented at the
annual meeting of the American Educational Research Association, New Orleans,
LA.
Boling, E. C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2011). Cutting the
distance in distance education: Perspectives on what promotes positive, online
learning experiences. Internet and Higher Education (Advance online
publication). doi:10.1016/j.iheduc.2011.11.006
Bradley, J. (2009). Promoting and supporting authentic online conversations – which
comes first – the tools of instructional design? International Journal of
Pedagogies and learning, 5(3), 20-31. Retrieved from http://jpl.econtentmanagement.com/
Buch, K., & Bartley, S. (2002). Learning style and training delivery mode preference.
Journal of Workplace Learning, 14(1), 5-10. doi:10.1108/13665620210412795
Bye, D., Pushkar, D., & Conway, M. (2007). Motivation, interest, and positive affect in
traditional and nontraditional undergraduate students. Adult Education Quarterly,
57, 141‐158. doi:10.1177/0741713606294235
Cabrera‐Lozoya, A., Cerdan, F., Cano, M.‐D., Garcia‐Sanchez, D., & Lujan, S. (2012).
Unifying heterogeneous e‐learning modalities in a single platform: CADI, a case
study. Computers & Education, 58(1), 617‐630.
doi:10.1016/j.compedu.2011.09.014
Cacciamani, S., Cesareni, D., Martini, F., Ferrini, T., & Fujita, N. (2012). Influence of
WattsSEDU7707-8-8
23
participation, facilitator styles, and metacognitive reflection on knowledge
building in online university courses. Computers & Education, 58, 874-884.
Retrieved from http://www.journals.elsevier.com/computers-and-education/
Caine, G. (2010). Making connections between e-learning and natural learning. ELearning Handbook, 33, 1-10. Retrieved from
http://www.cainelearning.com/PRODUCTS/From_natural_learning_to_elearning.
pdf
Cercone, K. (2008). Characteristics of adult learners with implications for online learning
design. Association for the Advancement of Computing in Education Journal
(AACE), 16(2), 137-159. Retrieved from http://www.editlib.org/j/AACEJ
Chan Mow, I. T. (2008). Issues and difficulties in teaching novice computer
programming. In M. Iskander (ed.), Innovative techniques in instruction
technolgy, e-learning, e-assessment, and education (pp. 199-204). London,
England: Springer Science+Business Media.
Chen, L.-C., & Lien, Y.-H. (2011). Using author co-citation analysis to examine the
intellectual structure of e-learning: A MIS perspective. Scientometrics, 89, 867886. doi:10.1007/s11192-011-0458-y
Chickering, A. W., & Gamson, Z. F. (1987). Seven principles for good practice in
undergraduate education. American Association for Higher Education Bulletin,
39(7), 3-7. Retrieved from ERIC database. (ED282491)
Chyung, S. Y., & Vachon, M. (2005). An investigation of the profiles of satisfying and
dissatisfying factors in e-learning. Performance Improvement Quarterly, 59(3),
227-245. doi:10.1177/0741713609331546
Commonwealth of Learning. (2000). An introduction to open and distance learning.
Retrieved from http://www.col.org/PublicationDocuments/pub_ODLIntro.pdf
Cornelius, S., Gordon, C., & Ackland, A. (2011). Towards flexible learning for adult
learners in professional contexts: An activity-focused course design. Interactive
Learning Environments, 19(4), 381-393. doi:10.1080/10494820903298258
Desai, M. S., Hart, J., & Richards, T. C. (2008). E-learning: Paradigm shift in education.
Education, 129(2), 327-334. Retrieved from
http://www.projectinnovation.biz/education_2006.html
Diaz, L. A., & Entonado, F. B. (2009). Are the functions of teachers in e-learning and
face-to-face learning environments really different? Educational Technology &
Society, 12(4), 331-343. Retrieved from http://www.ifets.info/
Donavant, B. W. (2009) The new, modern practice of adult education: Online instruction
in a continuing professional education setting. Adult Education Quarterly, 59(3),
227‐245. doi:10.1177/0741713609331546
WattsSEDU7707-8-8
24
Edgington, E. S. (1966). Statistical inference and nonrandom samples. Psychological
Bulletin, 66(6), 485-487. Retrieved from
http://homepage.mac.com/psychreearch/Sites/site2/psy779readings/Edgington196
6.pdf
Fahy, P. J. (2008). Characteristics of interactive online learning media. In T. Anderson
(Ed.), The theory and practice of online learning (pp. 167-199). Edmonton, AB:
Athabasca University.
Ferguson, J. M., & DeFelice, A. E. (2010). Length of online course and student
satisfaction, perceived learning, and academic performance. International Review
of Research in Open and Distance Learning, 11(2), 73-84. Retrieved from
http://www.irrodl.org/ index.php/irrodl
Fidishun, D. (2011, March). Andragogy and technology: Integrating adult learning
theory as we teach with technology. Retrieved from
http://frank.mtsu.edu/~itconf/proceed00/ fidishun.html
Fletcher, J. D., Tobias, S., & Wisher, R. A. (2007). Learning anytime, anywhere:
Advanced distributed learning and the changing face of education. Educational
Research, 36(1), 96‐102. doi:10.3102/0013189X07300034
Guilbaud, P., & Jerome-D’Emilia, B. (2008). Adult instruction & online learning:
Towards a systematic instruction framework. International Journal of Learning,
15(2), 111-121. Retrieved from
http://ijl.cgpublisher.com/product/pub.30/prod.1638
Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010).
Predictors of learner satisfaction and transfer of learning in a corporate online
education program. The American Journal of Distance Education, 24(1), 207-226.
doi:10.1080/08923647.2010.522919
Haythornthwaite, C., Bruce, B. C., Andrews, R., Kazmer, M. M., Montague, R.-A., &
Preston, C. (2007). Theories and models of and for online learning. First Monday,
12(8). Retrieved from
http://www.uic.edu/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1976/1851
Hoic-Bozic, N., Mornar, V., & Boticki, I. (2009). A blended learning approach to course
design and implementation. IEEE Transactions on Education, 52(1), 19-30.
doi:10.1109/GTE.2007.914945
Huang, E. Y., Lin, S. W., & Huang, T. K. (2012). What type of learning style leads to
online participation in the mixed‐mode e‐learning environment? A study of
software usage instruction. Computers & Education, 58(1), 338‐349.
doi:10.1016/j.compedu.2011.08.003
Ismail, I., Gunasegaran, G., & Idrus, R. M. (2010). Does e-learning portal add value to
adult learners? Current Research Journal of Social Sciences, 2(5), 276-281.
WattsSEDU7707-8-8
25
Retrieved from http://maxwellsci.com/print/crjss/v2-276-281.pdf
Jackson, L. C., Jones, S. J., & Rodriguez, R. C. (2010). Faculty actions that result in
student satisfaction in online courses. Journal of Asynchronous Learning
Networks, 14(4), 78-96. Retrieved from
http://jaln.sloanconsortium.org/index.php/jaln
Karge, B. D., Phillips, K. M., Dodson, T. J., & McCabe, M. (2011). Effective strategies
for engaging adult learners. Journal of College Teaching and Learning, 8(12), 5356. Retrieved from
http://journals.cluteonline.com/index.php/TLC/article/view/6621
Ke, F. (2010). Examining online teaching, cognitive, and social presence for adult
students. Computers & Education, 55, 808-820.
doi:10.1016/j.compedu.2010.03.013
Ke, F., & Hoadley, C. (2009). Evaluating online learning communities. Educational
Technology Research & Development, 57(1), 487-510. doi:10.1007/s11423-0099120-2
Ke, F., & Xie, K. (2009). Toward deep learning for adult students in online courses.
Internet and Higher Education, 12, 136-145. doi:10.1016/j.iheduc.2009.08.001
Kenner, C., & Weinerman, J. (2011). Adult learning theory: Applications to
nontraditional college students. Journal of College Reading and Learning, 41(2),
87-96. Retrieved form http://www.crla.net/journal.htm
Kiliç-Cakmak, E. (2010). Learning strategies and motivational factors predicting
information literacy self-efficacy of e-learners. Australasian Journal of
Educational Technology, 26(2), 192-208. Retrieved from
http://www.ascilite.org.au/ajet/ajet.html
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during
instruction does not work: An analysis of the failure of constructivist, discovery,
problem-based, experiential, and inquiry-based teaching. Educational
Psychologist, 41(2), 75-86. Retrieved from
http://www.tandf.co.uk/journals/titles/0046-1520.asp
Knowles, A. (1984). Andragogy in action: Applying modern principles of adult learning.
San Francisco, CA: Jossey-Bass.
Kozub, R. M. (2010). An ANOVA analysis of the relationships between business
students' learning styles and effectiveness of web based instruction. American
Journal of Business Education, 3(3), 89-98. Retrieved from
http://journals.cluteonline.com/index.php/AJBE
Lam, P., & Bordia, S. (2008). Factors affecting student choice of e-learning over
traditional learning: Student and teacher perspectives. The International Journal
WattsSEDU7707-8-8
26
of Learning, 14(12), 131-139. Retrieved from
http://ijl.cgpublisher.com/product/pub.30/prod.1585
Lombardi, M. M. (2007). Authentic learning for the 21st century: An overview (ELI
Paper 1). In D. G. Oblinger (Ed.), EDUCAUSE Learning Initiative: Advancing
Learning through IT Innovation (pp. 1-12). Retrieved from
http://net.educause.edu/ir/library/pdf/ELI3009.pdf
Martinez‐Caro, E. (2011). Factors affecting effectiveness in e‐learning: An analysis in
production management courses. Computer Applications in Engineering
Education, 19(3), 572‐581. Retrieved from
http://onlinelibrary.wiley.com/doi/10.1002/cae.v19.3 /issuetoc
McGlone, J. R. (2011). Adult learning styles and on‐line educational preference.
Research in Higher Education Journal, 12, 1‐9. Retrieved from
http://www.aabri.com/rhej.html
Merriam, S. B., Caffarella, R. S., & Baumgartner, L. (2007). Learning in adulthood: A
comprehensive guide (3rd ed). San Francisco, CA: Jossey-Bass.
Minter, R., L. (2011). The learning theory jungle. Journal of College Teaching and
Learning, 8(6), 7-15. Retrieved from
http://journals.cluteonline.com/index.php/TLC/article/view/4278/4365
Moisey, S. D., & Hughes, J. A. (2008). Supporting the online learner. In T. Anderson
(Ed.), The theory and practice of online learning (pp. 419-439). Edmonton, AB:
Athabasca University.
Muirhead, B. (2004). Contemporary online education challenges. International Journal
of Instructional Technology & Distance Learning (ITDL), 1(10). Retrieved from
http://itdl.org/journal/oct_04/article05.htm
Omar, A., Kalulu, D., & Belmasrour, R. (2011). Enhanced instruction: The future of elearning. International Journal of Education Research, 6(1), 21-37. Retrieved
from http://www. journals.elsevier.com/international-journal-of-educationalresearch/
Oncu, S., & Cakir, H. (2011). Research in online learning environments: Priorities and
methodologies. Computers & Education, 57, 1098-1108.
doi:10.1016/j.compedu.2010.12.009
Park, J.-H., & Choi, H. J. (2009). Factors influencing adult learners’ decision to drop out
or persist in online learning. Journal of Educational Technology & Society, 12(4),
207-217. Retrieved from http://www.ifets.info/journals/12_4/18.pdf
Pastore, R. (2012). The effects of time-compressed instruction and redundancy on
learning and learners’ perceptions of cognitive load. Computers & Education, 58,
641-651. doi:10.1016/j.compedu.2011.09.018
WattsSEDU7707-8-8
27
Pelz, B. (2010). (My) three principles of effective online pedagogy. Journal of
Asynchronous Learning Networks, 14(1), 103-116. Retrieved from
http://sloanconsortium.org/publications/jaln_main
Pigliapoco, E. E., & Bogliolo, A. A. (2008). The effects of psychological sense of
community in online and face-to-face academic courses. International Journal of
Emerging Technologies in Learning, 3(4), 60-69. Retrieved from
http://www.online-journals.org/i-jet
Ruey, S. (2010). A case study of constructivist instructional strategies for adult online
learning. British Journal of Educational Technology, 41(5), 706-720.
doi:10.1111/j.1467-8535.2009.00965.x
Segrave, S., & Holt, D. (2003). Contemporary learning environments: Designing elearning for education in the professions. Distance Education, 24(1), 7‐24.
doi:10.1080/0158791032000066499
Sharples, M., Taylor, J., & Vavoula, G. (2007). A theory of learning for the mobile age.
In R. Andrews, & C. Haythornthwaite (Eds.), The SAGE handbook of e-learning
research (pp. 219247). Los Angeles, CA: SAGE.
Shea, P., Fredericksen, E., & Pickett, A. (2006). Student satisfaction and perceived
learning with on-line courses: Principles and examples from the SUNY learning
network. Journal of Asynchronous Learning Networks, 4(2), 2-31. Retrieved from
http://sloanconsortium.org/publications/jaln_main
Simonson, M., Schlosser, C., & Hanson, D. (1999). Theory and distance education: A
new discussion. American Journal of distance Education, 13(1), 60-75.
doi:10.1080/08923649909527014
Sinclair, A. (2009). Provocative pedagogies in e-learning: Making the invisible visible.
International Journal of Teaching and Learning in Higher Education, 21(2), 197209. Retrieved from http://www.isetl.org/ijtlhe/
So, H.-J., & Bonk, C. J. (2010). Examining the roles of blended learning approaches in
computer-supported collaborative learning (CSCL) environments: A Delphi
study. Educational Technology & Society, 13(3), 189–200. Retrieved from
http://www.ifets. info/
Strang, K. D. (2009). Measuring online learning approach and mentoring preferences of
international doctorate students. International Journal of Educational Research,
48, 245-257. Retrieved from http://www.journals.elsevier.com/internationaljournal-of-educational-research/
Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper, S., Ahern, T. C., Shaw, S.
M., & Liu, X. (2006). Teaching courses online: A review of the research. Review
of Educational Research, 76(1), 93-135. Retrieved from
http://rer.sagepub.com/content/76/1/93.full.pdf
WattsSEDU7707-8-8
28
Taran, C. (2006). Enabling SMEs to deliver synchronous online training – practical
guidelines. Campus-Wide Information Systems, 23(3), 182-195.
doi:10.1108/10650740610674193
Watkins, R. (2005). Developing interactive e-learning activities. Performance
Improvement, 44(5), 5-7. Retrieved from
http://www.ispi.org/content.aspx?id=154&linkidentifier= id&itemid=154
Wright, D. B. (2006). Comparing groups in a before-after design: When t test and
ANCOVA produce different results. British Journal of Educational Psychology,
76, 663-675. doi:10.1348/000709905X52210
Yang, Y., & Cornelious, L. F. (2005). Preparing instructors for quality online instruction.
Online Journal of Distance Learning Administration, 8(1). Retrieved from
http://www.westga.edu/~distance/ojdla/spring81/yang81.htm
Zemke, R., & Zemke, S. (1995). Adult learning: What do we know for sure? Training,
32, 69-82. Retrieved from ERIC Database. (ED504481)
WattsSEDU7707-8-8
29
Appendix A: Annotated Bibliography
Seminal Article
Zemke, R., & Zemke, S. (1995) Adult Learning: What do we know for sure? Training,
32, 69-82. Retrieved from
http://setur.khi.is/fullordinsfraedsla/NoN/Itarefni/Zemke_WhatDoWeKnow.pdf
In this seminal article regarding adult learning, the authors provided a brief
background into the history of adult learning, or andragogy, as a specific field, and then
presented major trends in the field from more than 300 articles. The authors divided this
task into three sections – the things that the field knows about (a) “adult learners and their
motivation” (b) “designing curricula for adults,” and (c) “working with adults in the
classroom” (para. 17). The major themes they identified regarding adult motivation to
learn focuses on three items, (a) “Adult learning is problem-centered” (Motivation to
Learn, para. 6), (b) adult learners seek learning more often than not because of a lifechanging event, but can be motivated “by appealing to personal growth or gain” (para.
10), and (c) motivation “can be increased” (para. 12). The authors noted eight focuses
related to curriculum design, which are (a) the learning experience itself “should be
problem-centered” (Curriculum Design, para. 2), (b) it is important to access “the entrylevel knowledge and understanding of participants” (para. 3), (c) new knowledge should
be related to existing experience, (d) exercises and activities should “contain a reflective
element” (para. 9) in order for learning to take place, (e) “feedback and recognition
should be planned” (para. 10), (f) the design should “account for learning-style
differences” (para. 12), (g) the design should accommodate “adults’ continued growth
and changing values” (para. 15), and (h) design should include means to take the new
knowledge and apply it to the situation for which they sought learning. The authors
discerned three “common threads . . . [that] suggest some useful guidelines” (In the
WattsSEDU7707-8-8
30
Classroom, para. 3) for in class instruction. These threads are (a) the instructor needs to
“create a safe and comfortable environment” (para. 4), (b) instructors should be more
facilitator and less lecturer, and (c) promote the contribution and collaboration of the
students as part of the discussion and learning. The authors provide an excellent
springboard to determine not only what is known, but where gaps in knowledge occur in
the field of andragogy.
Adult Learning Theory
Andrews, R., & Haythornthwaite, C. (2007). Introduction to e-learning research. In R.
Andrews, & C. Haythornthwaite (eds.), The SAGE handbook of e-learning
research (pp. 1-51). Los Angeles, CA: SAGE.
In this introduction, the authors discuss various aspects of e-learning including
what both the ‘e’ and the learning are coming to mean, a discussion of the theoretical
background for e-learning which they claim has its foundations in Rhetorical theory and
Social informatics, and the methodological challenges for research in the field. Their
purpose was to “articulate a model for examining e-learning that incorporates elements of
rhetorical, communication, and social informatics theories” (p. 33). The authors identify
that the default model of educational research in which context is removed and the effect
of one independent variable on one or more dependent variables is considered, is
problematic when dealing with information and communications technology (ICT) and
learning, in that it may be true that the independent variable effects the dependent
variable, but in turn the dependent variable may then effect the independent variable –
marking them as reciprocally co-evolutionary rather than truly independent or dependent.
In the relationship between ICT and learning a temporal dimension is obvious as the
technology of today is vastly different than that of a couple years ago, but it is also
dialectical in that new technologies tend to “backwash on to older technologies and
WattsSEDU7707-8-8
31
practices” (p. 37). The authors suggest that rather than a default research model the use
of “cross-lagged panel designs” (p. 39) could be useful by “paying attention to the
problem of how continuous (and sometimes erratic) development can be adequately
mapped in staged analyses of reciprocity” (p. 39) using both qualitative and quantitative
data.
Baskas, R. S. (2011, March 27). Adult learning assumptions. Retrieved from ERIC
database. (ED517971)
The author, a professor at Walden University, conducted a literature review in this
reviewed article that indicates that Knowles’ six assumptions can be considered a theory
for how adults learn most successfully. The author focused on two of Knowles
assumptions – the need for adults to be motivated in order to learn, and the role of the
students’ life experience that colors their learning, and identified from the literature that
these two assumptions are supported. In regards to adult motivation, the literature
supports that adults need to be motivated, and that adults realize the importance of their
learning because of changes in positive affect. The literature also confirms that as adults
learn, they make mistakes because of their inexperience in a new realm, but that through
interacting with others they can inculcate their learning even better. Regarding the
assumption that students’ previous experience and knowledge will affect their current
learning, the author identified trends in the literature that suggest that instructors need to
take into account the experiences and potential biases of students to engender reflection.
Baskas, R. S. (2011). Applying adult learning and development theories to educational
practice. Retrieved from ERIC database. (ED519926)
In this partial review of the literature the author identifies that three of the
components of online learning; lectures and discussions, scholarly papers, and
reflections, are supported by both Knowles Phase Theory, and Levinson’s Developmental
WattsSEDU7707-8-8
32
Theory. Since online learning can be self-paced learners are able to use that convenience
in working on and turning in assignments. Learners generally also have writing services
to facilitate the development and improvement of writing skills, and online learning can
encourage learner’s to reflect on their past experiences in the context of their current
learning. The author’s discussion of Knowles and Levinson’s theories reinforces other
findings that confirm ways in which instructors can better teach students.
Blanchard, R. D., Hinchey, K. T., & Bennett, E. E. (2011, April). Literature review of
residents as teachers from an adult learning perspective. Paper presented at the
annual meeting of the American Educational Research Association, New Orleans,
LA. Retrieved from
http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?acc
no=ED521385
In this integrated literature review the authors “identified skills and characteristics
of resident teachers” (abstract) and presented them “as mechanisms for achieving the five
tenets of Knowles (1984) model of adult learning” (abstract). After a review of Knowles
five tenets the authors identify that the skills for success in higher education may not
have the “interpersonal and adult relational skills” (p. 4) necessary to succeed in teaching
adult education. The research question developed was “what are the essential skills and
characteristics of resident teachers consistent with andragogy” (p. 4), and seventeen
articles were found that touched on the research question. The mechanisms found
include (a) “feedback which should be ‘both corrective and reinforcing’ (Heflin,
Pinheiro, Kaminetzky, & McNeill, 2009, p. e234)” (p. 5), (b) “promote autonomy in
learners” (p. 6), (c) “incorporate the prior experiences of their learners” (p. 6), (d)
promote “a positive climate for learning” (p. 7), (e) cause learning to be problemcentered, and (f) general skills including “communication skills, enthusiasm, and ethics”
(p. 7) as well as “understanding of leadership” (p. 7). This article reflects a confirming
WattsSEDU7707-8-8
33
and differing view of adult learning theory.
Chen, L.-C., & Lien, Y.-H. (2011). Using author co-citation analysis to examine the
intellectual structure of e-learning: A MIS perspective. Scientometrics, 89(1),
867-886. doi:10.1007/s11192-011-0458-y
The authors argue that “there is little comprehensive knowledge on e-learning,
especially in the non-educational fields” (p. 867), and conducted a study using author cocitation analysis (ACA) in order to identify “the intellectual structure of specific
knowledge domains through the relationship between two similar authors” (p. 867)
regarding e-learning practices from 1996 to 2009.
By treating bibliographic elements as
conceptual units, ACA helps researchers analyze discipline structure and reduce personal
bias within their results. ACA is based on tracking the number of times that two authors
or documents are cited together, assuming that “the more frequently two authors are cited
together, the closer the relationship is between them” (p. 870). The authors identified the
steps of conducting the study as “(1) identify authors highly cited by research articles; (2)
retrieve co-citation counts for each pair of authors; (3) compile a matrix of raw cocitations; (4) perform clustering through various analytical methods . . . ; (5) interpret the
results” (p. 870). By following this methodology the authors determined six similarities
and four differences between MIS focused e-learning and e-learning from an educational
perspective. The authors call for further research in both “theoretical and practical
discussions of e-learning and its structural breakdown” (p. 883). This is an interesting
approach that I would enjoy further investigating and perform in relationship to the
current literature of e-learning and adult education as I do not believe that the authors
went far enough in their considerations.
WattsSEDU7707-8-8
34
Kenner, C., & Weinerman, J. (2011). Adult learning theory: Applications to
non‐traditional college students. Journal of College Reading and Learning, 41(2),
87‐96. Retrieved from
http://20.132.48.254/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno
=EJ926365
The authors focus is “on how best to understand and teach entry-level adult
learners who are between the ages of 25 and 50, have a high school diploma or a GED,
are financially independent, and have one semester or less of college-level coursework”
(p. 88). They suggest that by understanding differences in these learners from traditional
students specific tools might be created to “increase their chances for success” (p. 88).
The authors review Knowles (1974) four principles, and introduce the metacognitive
frameworks of Schraw and Moshman (1995); tacit theory, informal theory, and formal
theory. The authors deemed that formal theory was not applicable to their purpose and
did not discuss it. According to tacit theory “adult learners acquire their metacognitive
skills from peers, teachers, and the local culture” (p. 89) and they are likely difficult to
change, and don’t usually recognize how they arrive at their beliefs. Informal theory says
adult learners “acquire their metacognitive skills over time from their peers and their
environment, but they have at least a rudimentary conscious thought process” (p. 90).
The authors note that learners may have many skills that are useful to them in their
everyday life, but these skills may not be useful in an academic environment, and that
educators should be aware of this possibility. Educators should then create in their
curriculum activities and assignments that will build the metacognitive skills of the adult
learner so that they succeed in converting abstract theory and thought into practical
application over time. To ingrain these new skills into the learner’s toolbox the authors
suggest competition, placing new strategies in direct contrast to old strategies, and
repetition, ensuring that activities build on each other. This article is purely theoretical
WattsSEDU7707-8-8
35
with no associated study, and will not be directly helpful in my educational path.
Lorge, I. (1956). Learning, motivation and education. In J. E. Anderson, J. E. Anderson
(Eds.), Psychological aspects of aging (pp. 207‐210). American Psychological
Association. doi:10.1037/10032‐022
The author’s primary point in this theoretical discourse is that peoples learning
“depends on what rewards, gratifications and values” (p. 208) they have learned to have.
He comments on the limitations of adult learning theory, and wonders if “we give adults
in our laboratories a chance to learn again,” focusing on what has been learned rather
than what is.
Minter, R. L. (2011). The learning theory jungle. Journal of College Teaching and
Learning, 8(6), 7‐15. Retrieved from
http://journals.cluteonline.com/index.php/TLC/article/view/4278
The author notes the irony that university level instructors by and large have less
of a foundation in pedagogical theory, learning models, and instructional application
skills than do K-12 teachers. The author proposes that not all learning theories apply to
the adult learner, but introduce twenty-seven pedagogical theories. The author notes that
this list is not inclusive, but representative and recommended readings are suggested for
those itemized, and research has found that there is no “one set of major theories that are
generalizable across student-learning environments at the college level” (p. 8). The
author compares the major assumptions of the pedagogical and andragogical models of
teaching, and identifies a number of questions to help teachers understand and identify
the model that they follow in their own teaching, but notes that instructors usually follow
their own intuition “without knowing whether learning theory or research support [their]
instructional initiatives” (p. 11). He sardonically evaluates that though the mission of
higher education is to facilitate learning, it is unusual for colleges to encourage teaching
excellence or sharing of successes or failures within their ranks. This article is purely
WattsSEDU7707-8-8
36
theoretical and does not have an associated study, but suggests that there should be
greater communication between higher education educators to facilitate best practices in
teaching adult learners. The author also identifies that there is no central theory that a
majority, or even a minority, subscribe to in regards to andragogy. This article contains
an excellent listing of current learning theories and is a great jumping off point for further
investigation.
Sharples, M, Taylor, J. & Vavoula, G. (2007). A theory of learning for the mobile age. In
R. Andrews, & C. Haythornthwaite (eds.), The SAGE handbook of e-learning
research (pp. 219-247). Los Angeles, CA: SAGE Publications.
The authors in this theoretical discussion define mobile learning as “the processes
of coming to know through conversations across multiple contexts among people and
personal interactive technologies” (p. 225). Their focus is not on the learner or the
technology but instead on the communication that occurs between them that advances
knowledge. There are three main topics in this definition; the conversation, the context,
and the technologies. The authors identify that “the driving process of learning” (p. 225)
is the conversation, and their discussion was primarily derived from the work of Gordon
Pask (1976), who instead of seeing communication as “the exchange of messages through
an inert and transparent medium” (p. 226) saw communication “as the sharing of
understanding within a pervasive computational medium” (p. 226) which serve as the
“active and responsive systems within which mind-endowed individuals converse” (p.
226). According to Ravenscroft (2000) “the most successful learning comes when the
learner is in control of the activity, able to test ideas by performing experiments, to ask
questions, collaborate with other people, seek out new knowledge, and plan new actions”
(p. 227). The authors noted that “all activity is performed in context” (p. 230) but
learning “not only occurs in a context, it also creates context through continual
WattsSEDU7707-8-8
37
interaction” (p. 230). Since my dissertation will emphasize communication this article
reinforces and aids how important communication is perceived and received when it
comes to learning online.
Teaching Online – Key Findings
Boling, E. C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2011). Cutting the
distance in distance education: Perspectives on what promotes positive, online
learning experiences. Internet and Higher Education [Advance online
publication]. doi:10.1016/j.iheduc.2011.11.006
Data collected from this study was used to determine supportive and hindering
characteristics of effective online learning experiences. In this qualitative study, the
authors explored the online learning and teaching experiences of ten adult students and
six online course instructors using a “disciplined configurative cases study approach” (p.
2) using a convenience sample of individuals “who had participated in a number of online
courses” (p. 2) but represented various fields of study. The Cognitive Apprenticeship
Model was used as the theoretical framework for the deductive and inductive analyses for
coding and analyzing the interview data. The findings for this study indicated that online
students found that programs that “were more interactive and incorporated the use of
multimedia” (p. 3) were more helpful in achieving learning objectives, while courses that
were less helpful were characterized by: (a) emphasizing text-based content, (b)
“individualized learning” (p. 3), and (c) limited interactions with other students, the
instructor, or other faculty. Half of this article included the introduction of an online
masters program that provided modeling, coaching, and scaffolding, and encouraged the
interactivity among students, and between students and the instructor. My dissertation
focuses on student satisfaction in an online learning environment, and this article
supports discussion of how this can come about.
WattsSEDU7707-8-8
38
Bradley, J. (2009). Promoting and supporting authentic online conversations – which
comes first – the tools of instructional design? International Journal of
Pedagogies and learning, 5(3), 20-31. Retrieved from http://jpl.econtentmanagement.com/
This case study was designed to “embed pedagogical theory into practice [while]
blend[ing] the correct online communication tool and effective instructional design” (p.
25). Four participants took part in the study and took part of three activities designed
from a constructivist standpoint. The author argued that using the ADDIE model of
instructional design is problematic for a constructivist online learning environment
because the focus of learning should be on the learning process rather than the content
per se. The elements for the design of a constructivist learning environment focuses on
“keeping students active, constructive, collaborative, intentional, complex, contextual,
conversational, and reflective” (p. 22). Technology is the conduit through which learning
can take place, while the learning is enhanced and facilitated by “well-designed learning
goals and objectives” (p. 28) and in the context of constructivist theory “the skill of the
facilitator is integral to achieving successful outcomes” (p. 28). Based on cognitive load
theory I find it difficult to believe that learners will be able to focus on eight different
elements as formulated in this and previous articles. Rather than having the elements
interacting with each other all of the time, it is more likely that the elements during
learning are used in some sequential way, with perhaps a maximum of four elements
applicable at any one time. This article is helpful in identifying and clarifying the
constructivist theory regarding interaction and its uses, as well as the role of the
facilitator in implementing the theory.
WattsSEDU7707-8-8
39
Ferguson, J. M., & DeFelice, A. E. (2010). Length of online course and student
satisfaction, perceived learning, and academic performance. International Review
of Research in Open and Distance Learning, 11(2), 73-84. Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/772/1547
The authors presented an exceptional review of the literature regarding the factors
effecting satisfaction with courses taught online concluding “that connectedness to the
course, either by participating collaboratively with other students or by interacting with
the professor, will likely impact student satisfaction” (p. 75) the most. Equivalency
theory, introduced by Simonson, Scholosser, & Hanson was used as the theoretical
framework of this study to determine if “there were differences in online student
satisfaction, perceived learning, and performance” (p. 76) when the independent variable
was length of the course, in this case five-weeks versus fifteen-weeks, while all other
pedagogical factors were kept the same. Equivalency theory was supported in this study
because students “learned the same content in” (p. 76) both courses. 75 graduate students
took part in part one of the study which consisted of a 15-question Likert scale survey,
while 114 graduate students final grades from the same four courses was analyzed for the
second part of the study. Students were significantly more satisfied with the interaction
with the teacher in the longer course, while students were significantly more satisfied
with the interaction with fellow students in the shorter course. No significant difference
was found for perceived learning or satisfaction regarding taking additional online classes
was found between the two course lengths. Students in the shorter sessions showed
significantly stronger “academic performance than the full-semester students” (p. 81).
Improvements to the pedagogy of the class were identified for both formats. In regard to
the shorter course a shift needs to be made “to emphasize interaction with the professor”
(p. 81) and several possibilities were proposed. Limitation to the study was that the
WattsSEDU7707-8-8
40
students for the shorter course (summer semester) may have been different from those
attending the longer course (regular semesters), and that students were not randomly
selected. This study is very similar to my proposed study in looking at student
satisfaction and perceived learning, and I may utilize the same data instruments.
Jackson, L. C., Jones, S. J., & Rodriguez, R. C. (2010). Faculty actions that result in
student satisfaction in online courses. Journal of Asynchronous Learning
Networks, 14(4), 78-96. Retrieved from
http://jaln.sloanconsortium.org/index.php/jaln
This quantitative research study correlated faculty actions with student
satisfaction in online classes at two community colleges in Texas. Data for the study was
obtained from student responses to each institution’s existing online course evaluation.
All online students were requested to fill out the online evaluation, and 426 students
(30%) from College 1 and 1004 students (69%) from College 2 participated. Descriptive
statistics, bivariate correlations, and multiple regressions were used to identify the faculty
actions that affected student satisfaction in online courses. The authors determined that
“student satisfaction with online courses appears to be impacted by instructor actions
within the course” (p. 91). The highest correlations with student satisfaction were
“timeliness/accessibility of instructor, clearly stated expectations, instructor enthusiasm,
and comfortable climate” (p. 91) and a moderate correlation existed with activities.
Multiple regression analysis indicated that 69% of the variance of student satisfaction
could be explained by those independent variables. This article identifies specific actions
that instructors can do in their online courses to increase student satisfaction. The article
contains good discussion from the literature regarding faculty roles and student
satisfaction.
WattsSEDU7707-8-8
41
Ke, F., & Hoadley, C. (2009). Evaluating online learning communities. Educational
Technology Research & Development, 57(1), 487‐510.
doi:10.1007/s11423‐009‐9120‐2
In this study of 42 studies on online learning communities (OLC) four items were
evaluated, (a) “purpose of the study” (p. 490), (b) “evaluation approaches” (p. 490), (c)
“indicators or measures” (p. 490) used, and (d) how data was collected and analyzed.
OLC evaluations were categorized as proving, improving, or both. In the studies, four
approaches were identified – summative, formative, participatory, and responsive, and
these approaches “usually influenced the methods employed” (p. 494). The authors
found two major constructs in how studies measured – outcome and process. The authors
identify that this study is preliminary aand offer it as the beginning of “identifying
common patterns and the strengths (and weaknesses) of relationships among the various
process and outcome indicators across OLCs” (p. 507, italics in original) in order to
make progress on whether a learning community is beneficial. This article is preliminary
and foundational to my research but may not be used directly.
Sinclair, A. (2009). Provocative pedagogies in e-learning: Making the invisible visible.
International Journal of Teaching and Learning in Higher Education, 21(2), 197209. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE688.pdf
This qualitative descriptive case study describes and explains the process of a
course called ‘Reflective practice for teachers’ (p. 198). The authors identify some key
characteristics of being online including having an emphasis “on distributed learning
whereby control of the learning is distributed among the community” (p. 197) instead of
a single expert, and that for the students the process is less hierarchical with “more
emphasis on self-regulation and participation” (p. 197). This paper studies a course
based on the notion that true learning is brought about through cognitive dissonance that
results in reflection and new shared knowledge. This pedagogy of difficulty says that
WattsSEDU7707-8-8
42
“teaching and learning are complex, tentative, and difficult” sometimes “the need for
right answers” (p. 201) suppresses the benefit of critical analysis “where ideas are
examined from multiple perspectives” (p. 201). Engle and Conant (2002) suggests that
students who are instructed to engage in knowledge building discussions learn to develop
and justify an argument and will find increasingly sophisticated ways of disagreeing with
others. The authors found that role-playing by learners of situations they would not
normally be placed in led to “a deeper understanding and interpretation of human
behavior and meanings” (p. 202), and that the written, asynchronous nature of this class
“enabled [learners] to give deeper consideration and responses” (p. 202). By
emphasizing the learner as a part of a community this allowed the learners a voice and to
feel valued with greater emphasis “on the learning process and learning experiences” (p.
204) rather than the instructor/expert as provider of knowledge. Online teaching has been
found to “promote higher order thinking, reflection, and rigorous intellectual challenges
leading to more equality between learners and teachers” (p. 204).
Strang, K. D. (2009). Measuring online learning approach and mentoring preferences of
international doctoral students. International Journal of Educational Research,
48(4), 245-257. doi:10.1016/j.ijer.2009.11.002
This article reports on an empirical study involving 254 international doctoral
students in twenty-three different cultures to determine if “there is a ‘right’ teaching or
supervising method, based on pedagogy or andragogy theories, for international student
learning modes” (p. 247). “Hypotheses were tested using structural equation modeling . .
. [found that method, supervision, and quality explained 56% of the variance effect on
candidate performance” (p. 245) and were determined to be critical latent variables. The
author identifies that “there is very little research examining cross-cultural teaching
practices or online higher education supervision” (p. 245), or “higher education research
WattsSEDU7707-8-8
43
across cultures” (p. 245). The interdisciplinary empirical literature was reviewed to
identify “critical factors that interact with . . . [and] impact performance” (p. 246). The
author discusses in great detail the testing of the reliability of his instrument. The
conclusion was that “doctorate supervisors could survey their international student with a
learning style instrument to inform (and possibly improve) their supervision approach for
each unique student” (p. 255), which is feasible because this relationship is one-to-one.
This study did not determine an optimal supervisory approach based on specific cultures
or learning styles but instead suggested five “learning preference indicators” (p. 255) and
three latent variables that could be used by supervisors to inform their supervisory style
with individual “international higher education doctorate students” (p. 255). Although my
dissertation will not emphasize cross-cultural issues, the classes that will be evaluated
will have multi-ethnic students and consideration needs to be made in this regard.
Thompson, L., Jeffries, M., & Topping, K. (2010). E-mentoring for e-learning
development. Innovations in Education and Teaching International, 47(3), 305315. doi:10.1080/14703297.2010.498182
In this phenomenological study the authors received survey information on the
perceptions of the benefits, and suggested improvements between mentors, mentees, and
project leaders regarding improvement in the development of online curriculum. The
focus of the study was "e-mentoring between university academic staff focused on the
development of online learning modules" (p. 313). Two findings were advanced, (a) "a
thorough negotiated assessment of mentee needs at an early point is essential, so that the
mentoring process can be differentiated and adaptive" (p. 313), and (b) "early [face-toface] meetings between mentor and mentee are widely seen as essential - purely
electronic contact appears ineffective" (p. 314). Considerations regarding modes of elearning are important to my dissertation topic, and this article broadens my
WattsSEDU7707-8-8
44
understanding of blended interactions and characteristics.
Watkins, R. (2005). Developing interactive E‐learning activities. Performance
Improvement, 44(5), 5‐7. Retrieved from
http://home.gwu.edu/~rwatkins/articles/PI44-5.pdf
In order to have “engaging e-learning activities that can result in both active
learning and the achievement of course objectives” (p. 5) the author in this commentary
identifies a dearth of “creative ideas that are necessary to create such environments” (p.
5). With the inclusion of interactive e-learning experiences in the classroom five specific
benefits accrue. These benefits are (a) improved retention rates, (b) increased learner
participation, (c) learning objectives being achieved, (d) development of effective online
learning communities, and (e) engaged learners. The author notes certain questions that
can be asked before developing an interactive activity for the online classroom, and then
makes some suggestions of his own suggesting that many can be “adaptations of
activities from the traditional classroom along with imaginative ideas that take advantage
of the unique online technologies” (p. 7). My dissertation will focus specifically on
interactivity, and this article supports the requirement for interaction to foster student
satisfaction and learning.
Learner Characteristics – Key Findings
Bye, D., Pushkar, D., & Conway, M. (2007). Motivation, interest, and positive affect in
traditional and nontraditional undergraduate students. Adult Education Quarterly,
57, 141‐158. doi: 10.1177/0741713606294235
In this study 300 undergraduate students were evaluated using three self-report
questionnaires and the results were compared. The students were evaluated for
motivation strategy by using the Motivation subscale of the Motivated Strategies for
Learning Questionnaire (MSLQ) to determine Intrinsic (participation in a task is an end
in itself) or Extrinsic (tasks are a means to an end) Goal Orientation. The students were
WattsSEDU7707-8-8
45
also evaluated using the Interest subscale of the Differential Emotions Scale IV-A (DWS)
to determine level of interest which has shown to “correlate significantly with
achievement, affiliation, endurance, and understanding” (p. 148). The students were also
evaluated for positive or negative affect by using the Positive and Negative Affect
Schedule (PANAS). The study showed a 2-to-1 ratio between traditional and
nontraditional age students, and had “no significant differences between the two groups”
in terms of other demographic factors, and neither skewness nor outliers were found. The
study focused on “differences between the traditional and the nontraditional student
subgroups” and used “the entire student sample to test hypothesized predictors of
intrinsic motivation and positive affect” (p. 149). A mixed factorial analysis of variance
(ANOVA) was employed to test different intrinsic and extrinsic motivation scores, which
found there is “a greater need among nontraditional students to simply enjoy the process
of mastering new skills in the classroom” (p. 155). The findings related to interest
directly correlate to the topic of my dissertation, and the data instruments could be
potentially used.
Chyung, S. Y., & Vachon, M. (2005). An investigation of the profiles of satisfying and
dissatisfying factors in e‐learning. Performance Improvement Quarterly, 18(2),
97‐113. Retrieved from http://www.blackwellsynergy.com/doi/abs/10.1111/j.1937-8327.2005.tb00335.x
The purpose of this study was to “conduct a theory-based investigation to reveal
comprehensive profiles of satisfying and dissatisfying factors in e-learning” (p. 99). The
theoretical framework chosen is based on Edward Lee Thorndike’s law of effect that said
that animals will form a connection with a stimulus based on consequences – so, if a
stimulus satisfies the connection is strengthened, while if a stimulus annoys the
connection is weakened, and these satisfiers and annoyers are “individual- and context-
WattsSEDU7707-8-8
46
dependent” (p. 99). Another portion of the author’s theoretical framework was Frederick
Herzberg’s motivation-hygiene theory which states that certain factors contribute to
satisfaction (motivation factors), and other factors may contribute to dissatisfaction
(hygiene factors), but these factors are not the opposite of each other. The author’s
performed a content analysis study aimed at building theory and guidelines from course
evaluation data of 17 e-learning courses. After data analysis of qualitative input, 19
categories were derived. These categories were then prioritized based on student
emphasis on whether the category was a satisfying factor, or a dissatisfying factor. The
most frequent satisfying factors were learning oriented (i.e., “interesting and relevant
learning content, effective teaching methods, instructor’s expertise, and effective learning
activities” (p. 107)). The most frequent dissatisfying factors were impediments to goal
achievement (i.e., “lack of their instructor’s participation during class discussions and
lack of clarity in instructional directions or expectations that caused confusion or
frustration” (p. 108)). Although this article is dated it is relevant and key to my
dissertation.
Donavant, B. W. (2009) The new, modern practice of adult education: Online instruction
in a continuing professional education setting. Adult Education Quarterly, 59(3),
227‐245. doi: 10.1177/0741713609331546
In a three phase quazi-experimental quantitative study of American police officers
the author determined the “efficacy of online education for professional development”
and found that learning took place with respect to both online education and traditional
instruction and that there was no “statistically significant difference in the effectiveness
of the two delivery methods” (p. 239). This study also showed no significant difference
between these modes with respect to gender, race, age, number of years on the force, or
previous exposure to online education, but did show a significant association between
WattsSEDU7707-8-8
47
level of formal education and potential success with online learning. Phase one consisted
of historical data of performance from various courses without descriptive information.
In phase three open-ended questions were asked of 150 participants, and they indicated
four attractive features of online education which were; (a) general convenience, (b)
flexibility in scheduling, (c) remote access, and (d) self pacing of learning. The least
attractive feature was “the lack of personal interaction or face-to-face contact with the
facilitator or other learners” (p. 239). The author noted one problem in the field is that
“little research has been conducted within the professional development environment,
that arena involving training relative to the current occupation of the adult learner” (p.
227). My dissertation focuses on adult professional development, and this article is
supportive of the need for such research, but also identifies key characteristics of that
scenario.
Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010).
Predictors of learner satisfaction and transfer of learning in a corporate online
education program. The American Journal of Distance Education, 24(1), 207-226.
doi:10.1080/08923647.2010.522919
In a mixed-methods design, the authors used a survey, as well as open-ended
questionnaire, face-to-face, and phone interviews to gather data on the perceptions of
students, instructors, and instructional designers to determine that “online self-efficacy
[was the] strongest predictor of learner satisfaction; collegial support was the strongest
predictor of transfer of learning” (p. 207). The authors identify that the major problem of
the current literature in terms of distance education in the corporate sector is that they
describe only “specific contexts and programs” (p. 208) and there is a need to move
beyond case studies to determine the characteristics of training that “lead to learning
gains, transfer of learning, and satisfaction” (p. 208). Four independent variables were
WattsSEDU7707-8-8
48
measured in part one of the study, “online self-efficacy, course design, learner-instructor
interaction, and learner-learner interaction” (p. 211), with the dependent variable being
learner satisfaction. The authors noted that the small sample size of the quantitative
portion was the major limitation, and that the qualitative data “yielded information of
great value” (p. 223). This article contains concepts and background of importance to my
dissertation topic.
Huang, E. Y., Lin, S. W., & Huang, T. K. (2012). What type of learning style leads to
online participation in the mixed‐mode e‐learning environment? A study of
software usage instruction. Computers & Education, 58(1), 338‐349.
doi:10.1016/j.compedu.2011.08.003
The authors extended previous research by testing a model that examined the
mediating process of prior knowledge in the relationship between learning style and elearning performance. They posited that (a) learning style is positively related to online
participation, (b) that online participation is positively related to e-learning performance,
and (c) the greater the prior knowledge, the stronger the relationship between online
participation and learning performance. This study measured the learning style of 219
college students in a single course by measuring (a) student learning style using the ILS,
(b) student online participation, (c) student performance, (d) prior knowledge of the tool
used in the course, and (e) the control variables of gender, computer experience, and
Internet experience. Support was found that online participation is a mediating construct
between learning style and performance; further the study found that sensory learning
style individuals tend to participate more frequently and for a longer duration; while prior
knowledge was shown to moderate the relationship between participation and learning
performance only in terms of passive participation. Several recommendations were made
by the authors. First, “although it is difficult to determine the degree of influence of the
WattsSEDU7707-8-8
49
mediating construct, educational institutions should take action to boost ‘students’ online
participation in e-learning courses” (p. 347). Second, “most learners appear to be able to
benefit [from e-learning] immediately” (p. 347). The authors also commented on several
suggestions for further research, namely (a) the model needs to be tested in different
subject contexts, (b) additional mediating processes that link learning styles and learning
performance should be explored, and (c) a more mature, professional, and autonomous
set of online learners should be enlisted. As my dissertation focuses on student
satisfaction this article contributes to a basic understanding of its components.
McGlone, J. R. (2011). Adult learning styles and on‐line educational preference.
Research in Higher Education Journal, 12, 1‐9. Retrieved from
http://www.aabri.com/manuscripts/11859.pdf
In this literature review the author identified that the foremost andragogical theory
to the teaching of adults “requires a process-focused approach” (p. 4) and accepted the
eight-element process model of Knowles, Holton & Swanson (2005) as “one of the most
comprehensive discussions on the theoretical and historical background of” (p. 4)
andragogy, and discusses in some detail each of the elements. The author then identified
that the Kolb Learning Styles Inventory (LSI) is an “instrument that is used widely in
studies related to” (p. 5) evaluating differences between adult students, and presents three
studies that use this instrument. The author reports findings of three studies regarding
adults and on-line education, noting that these studies indicated two things; “more and
more . . . adults want to enroll in these courses, in an effort to improve their employment
or qualification standing” (p. 6), but adult learners were “all relatively dissatisfied with
the on-line learning experience” (p. 6). Another study showed that most online courses
placed less of a focus on the “development of critical-thinking skills or other forms of
critical reflection on the material . . . [making the experience] less impactful” (p. 6). Two
WattsSEDU7707-8-8
50
recommendations were expressed by the author as suggested by the literature; make sure
that the “latest hardware and software improvements” (p. 8) are used in the on-line
training, and either train students on how to utilize the computer, or “adapt the material to
allow for better Internet learning skills” (p. 8) for the students. This article presents adult
student characteristics that will be foundational for my dissertation.
Pigliapoco, E. E., & Bogliolo, A. A. (2008). The effects of psychological sense of
community in online and face-to-face academic courses. International Journal of
Emerging Technologies in Learning, 3(4), 60-69. Retrieved from
http://halshs.archives-ouvertes.fr/docs/00/19/72/37/PDF/71_Final_Paper.pdf
In “a case study of a BS degree program in Applied computer science delivered
both online and face-to-face” (p. 68) regarding the influence of teaching environment on
psychological sense of community (PSoC) the authors determined that “virtuality does
not necessarily impair PSoC, and that the differences in student performance and dropout
rate between online and face-to-face degree programs are mainly explained by the
composition of the corresponding student populations” (pp. 60-61). PSoC has four
dimensions, identified by Rovai (2002) which are spirit, trust, interaction, and common
expectations and is “affected by transactional and geographical distance between students
and instructors” (p. 61). The authors note that other studies student-teacher and studentstudent interactions “increase the effectiveness of learning” (p. 61) and other benefits like
student satisfaction, performance, and reduced dropout rates. The authors also note that
in addition to PSoC factors like slow instructor feedback, confusion on how to use the
technologies, and confusion regarding a different study model could also contribute to
dropout rate. The discussion of the control and experimental group on page 62 will be of
benefit in my dissertation. The authors suggest a refinement of their regressions models
since the models failed to “completely model the behavior of the dependent variables
WattsSEDU7707-8-8
51
under study” (p. 68), and conduct further research to generalize their findings since this
case study only involved 107 students.
Application and Instructional Design
Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. (2010, July).
Interaction in distance education and online learning: Using evidence and theory
to improve practice. The Evolution from Distance Education to Distributed
Learning. Symposium conducted at Memorial Union Biddle Hotel, Bloomington,
IN. Retrieved from
http://www.aect.org/events/symposia/Docs/InteractionDEnext120510.pdf
In a continuation of a meta-analysis of distance and online education, the authors
explored “interactions: among students, between the instructor and students, and between
students and course content” (p. 2), discussed methodological issues with existing
learning research, and suggest methods for improving online instruction. Interaction is
widely accepted as important in online learning and in the meta-analysis the authors
“found the overall positive weighted average effect size of 0.38 for achievement
outcomes favoring more interactive treatments over less interactive ones” (p. 7)
supporting the three types of interaction. Student to student average effect size was
positive and significant at 0.49. Student to content average effect size was positive and
significant at 0.46. Student to instructor average effect size was positive and significant
at 0.32. Instead of comparing online learning with traditional instruction the authors
indicate that new research needs to focus on pitting “different but compatible types of
distance education technologies” (p. 5) against each other to determine if “larger and
more consistently positive effects” (p. 9) are possible. Four suggestions for future
research and development of knowledge tools were included to identify and enhance the
ability for course designers to “facilitate more purposeful interaction” (p. 29). This
article has a wealth of knowledge in regard to my dissertation regarding history, theory,
WattsSEDU7707-8-8
52
past journal articles on interactivity, principles of interactivity and collaborative group
work, as well as suggestions for research and development.
Buch, K., & Bartley, S. (2002). Learning style and training delivery mode preference.
Journal of Workplace Learning, 14(1), 5‐10. doi:10.1108/13665620210412795
In a study of 165 employees from a large financial institution the Kolb Learning
Style Instrument (LSI) and a survey was administered to determine training delivery
mode preference. Correlations were run to determine if there is a relationship between
learning style and delivery mode preference. “Educational research and practice have
demonstrated that learning can be enhanced when the instructional process
accommodates the various learning styles of students” (p. 5). The author’s hypothesis is
that “individuals with specific learning styles would have preferences for specific training
delivery mode formats” (p. 5) namely, divergers would prefer “traditional, classroombased delivery” (p. 6), accommodators and convergers would prefer computer-based
training, and assimilators would prefer print-based delivery. The authors developed an
instrument to determine learner’s preferred training delivery mode which offered, (a)
computer-based, (b) TV-based, (c) print-based, (d) audio-based, and (e) classroom-based
options. Friedman’s chi-square statistic was used by the authors to test for any
differences in learning mode preference across individuals with different learning styles.
The authors also ran t-tests “to compare learning mode preferences within each learning
style” (p. 8). All learning styles preferred classroom learning to other types, but partial
support was found that diverger’s preference for classroom learning was stronger than
other groups, accommodators preferred computer-based learning to other forms except
classroom learning, convergers also preferred computer-based learning second, but this
finding was not significant, and assimilators’ chose print-based learning second though it
WattsSEDU7707-8-8
53
was not significant. The authors suggest that design and delivery of training should take
into consideration the learning style of the learner. They also found that learners with
more experience in computer-based instruction “reported greater satisfaction than those
with less experience” (pp. 9-10). The authors suggest that there is a need to answer the
question, “What can training professionals do to facilitate learning beyond the classroom?
. . . Future research is needed to identify other potential tools and strategies” (p. 10).
Cabrera‐Lozoya, A., Cerdan, F., Cano, M.‐D., Garcia‐Sanchez, D., & Lujan, S. (2012).
Unifying heterogeneous e‐learning modalities in a single platform: CADI, a case
study. Computers & Education, 58(1), 617‐630. doi:
10.1016/j.compedu.2011.09.014
In this quantitative study the authors present a “web-based framework for the
creation, development, and implementation of heterogeneous learning environments” (p.
617) that was tested on a group of senior college students, and used to promote active
learning on any WiFi compliant device. In a thorough literature review of the different
forms of e-learning (i.e., collaborative learning, problem-based learning, blended
learning, and mobile learning), which is beneficial to the theoretical framework of my
dissertation, the authors determined that “there is still no e-learning system serving to
abstract some generic educational principles and put them all into practice in a specific elearning platform” (p. 619). The author’s main suggestion identifies “the establishment
of a generic real-time communications channel between the teacher and the students” (p.
619) that abstracts “the common aspects of CSCL, PBL or BL systems so that it can be
configured to support all kind of activities” (p. 620). In the study, the goal “was to
increase students’ participation . . . and to evaluate the impact of a higher interactive
environment in the students’ academic performance” (p. 624). A one-factor ANOVA
was performed and three elements were determined to significantly improve students
WattsSEDU7707-8-8
54
grades; scores regarding short questions, scores for problem solutions, and the final score
such that the authors determined the use of the framework to provide “an additional
communication channel in the learning methodology improved students’ academic
performance” (p. 625). 81 students were in the control groups with 27 students in the
experimental group.
Fletcher, J. D., Tobias, S., & Wisher, R. A. (2007). Learning anytime, anywhere:
Advanced distributed learning and the changing face of education. Educational
Research, 36(1), 96‐102. doi:10.3102/0013189X07300034
In this article the authors introduced the Advanced Distributed Learning (ADL)
initiative designed “to make learning accessible at anytime, anywhere in the world” (p.
96) and consists of objects that must meet specifications to be accessible, interoperable,
durable, and reusable and are instructional materials developed by industry and
government for reuse globally. Currently the number of these objects associated with
ADL is in excess of 10 million and range in size from entire courses to single graphics or
animations. The authors also identified a number of research projects that need to be
conducted regarding ADL, including; (a)
Determine how the Semantic Web can best be used to identify and expose
semantic linkages between bodies of knowledge that may appear dissimilar; to
create more precise, comprehensive, and substantive models of subject matter
domains and learner’s levels of mastery; and to link learner models to appropriate
instructional objects (p. 97).
(b) “Determine whether the provision of readily accessible instructional materials
by ADL improves students’ learning or enhances adaptations to their learning needs” (p.
100), (c) Cost-benefit studies, and (d) “Whether and to what extent ADL improves
students’ learning when it is used to augment the usual resources available in classrooms”
WattsSEDU7707-8-8
55
(p. 100). This article provided an excellent section on the need for technology in
instruction, showing that based on affordability and cost-effectiveness technology
outstrips other forms of instruction. The authors also presented several studies that
showed that there is no difference between distance instruction and classroom learning,
even though students prefer the latter.
Ismail, I., Gunasegaran, G., & Idrus, R. M. (2010). Does e‐learning portal add value to
adult learners?. Current Research Journal of Social Sciences, 2(5): 276‐281.
Retrieved from http://maxwellsci.com/print/crjss/v2‐276‐281.pdf
This study using 1084 undergraduate adult learners in the School of Distance
Education at Universiti Sains Malaysia gauged the benefits of an E-learning Portal. The
respondents were almost equally distributed between Science, Management, and Social
Science, with fewer respondents from the Arts program. A survey was conducted of the
respondents who were randomly selected from learners who had experience with the
Portal. The authors state that for institutions to truly engage in “educational transaction[s
using an] . . . electronic medium and cyberspace . . . requires the development of a virtual
learning environment” (p. 276) that they labeled as an E-learning Portal. The advantages
of such a portal are that “learners have access to education information and materials at
any time, from any location” (p. 276) and “allows learners and lecturers to communicate
at any time” (p. 277). The authors also reviewed the advantages of e-learning which
includes flexibility in schedule and learning process, convenience of time and place, can
encourage deeper thought on topics, and diminished costs. The results of the study
confirmed these advantages, but have a weakness in that it only shows comparative
approval of the e-learning Portal, but does not attempt to determine relationships between
the proposed advantages or between those advantages and the demographic data
collected, nor does it attempt to determine if the advantages, while positive, are
WattsSEDU7707-8-8
56
significant for specific correlations. The authors do identify how their findings are
supported by other studies, but only propose new studies to determine how the
advantages that had lowest approval could be improved.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during
instruction does not work: An analysis of the failure of constructivist, discovery,
problem-based, experiential, and inquiry-based teaching. Educational
Psychologist, 41(2), 75-86. Retrieved from
http://www.tandf.co.uk/journals/titles/0046-1520.asp
The author’s purpose is to demonstrate that several theories of adult learning
theory are untenable from the perspective of cognitive load. The article contains a fair
representation of the history and concepts of constructivist theory. Experience shows that
teachers rarely use a strictly constructivist approach, so this article is building a straw
man regarding pure minimally guided instruction. The article demonstrates effectively
the rationale of why direct guidance, which minimizes cognitive load, is preferred to
minimally guided instruction, and concludes that “minimally guided instruction is less
effective and less efficient than instructional approaches that place a strong emphasis on
guidance of the student learning process” (p. 75). The authors also stated that the
“epistemology of a discipline should not be confused with a pedagogy for teaching or
learning it” (p. 83). Based on understanding of cognitive architecture, the author’s
demonstrated multiple studies supporting directed, or guided instruction as opposed to
discovery, or interactive methods, concluding “guided instruction not only produced
more immediate recall of facts than unguided approaches, but also longer term transfer
and problem-solving skills” (p. 80). This article will be useful in instructional design,
and in techniques that can minimize cognitive load, and emphasize learning in my
dissertation.
WattsSEDU7707-8-8
57
Kozub, R. M. (2010). An ANOVA analysis of the relationships between business
students’ learning styles and effectiveness of web based instruction. American
Journal of Business Education, 3(3), 89-98. Retrieved from
http://journals.cluteonline.com/index.php/AJBE/index
In a class of upper-class college students, Kozub failed to find a significant
correlation between Kolb’s experiential learning style theory and the efficacy of Web
Based Instruction (WBI). Kozub used an 2x4 Analysis of Variance (ANOVA) analysis
experiment to determine whether students preferred learning styles, as measured by
Kolb’s Learning Style Inventory (LSI-IIa), dictated how well they did in terms of
performance, or likeability, on a classroom module, that was presented in three different
manners – text-only, using WBI, and using enhanced features of WBI. A comparison was
made of the 159 participants and their preferred learning style, their ACT composite
scores, three other in-class examinations, and the score on the in-class test for this
module, and a survey taken after the module to determine their reaction to it. The
findings of this experiment were that while there is a positive correlation between how
well students liked the module and the scores they earned, and between their scores and
their cumulative ACT scores, there is no correlation between learning style, and (a) how
the material is presented, (b) test score, and (c) reaction to how the material is presented.
This study supports the findings of other researchers, notably Hodges and Evans (1983)
and Trout and Crawley (1985) that there is no “predictive utility of the learning styles
construct” (p. 96). The author did state that more research needs to be done to determine
if, how, and when the use of enhancements in WBI would be beneficial to the student.
My dissertation will be testing enhancements in WBI so this article is relevant and
supportive.
WattsSEDU7707-8-8
58
Martinez‐Caro, E. (2011). Factors affecting effectiveness in e‐learning: An analysis in
production management courses. Computer Applications in Engineering
Education, 19(3), 572‐581. Retrieved from
http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291099-0542
425 students in 15 class sections that were delivered using e-learning were
surveyed on their perceived learning and general satisfaction, along with several
independent variables. Structural equation modeling (SEM) was used to estimate causal
relationships between the dependent and independent variables, and “used the LISREL
8.50 maximum likelihood method to measure the fit of the model” (p. 576). This study
showed no significant influence on perceived learning due to either age or gender. The
supported hypotheses in this study were (a) “perceived learning is positively related to
satisfaction in e-learning courses” (p. 573) and a t test was performed to ensure these did
not measure the same factor, (b) “e-learning flexibility is positively related to perceived
learning” (p. 574), (c) “teacher-student interaction is positively related to perceived
learning in e-learning courses” (p. 574), and (d) “student-student interaction is positively
related to perceived learning in e-learning courses” (p. 574). Three hypotheses were
significant in a negative direction (a) “prior experience is [negatively] related to
perceived learning,” (p. 574) (b) “working student status is [negatively] related to
perceived learning in e-learning courses” (p. 574), and (c) “blended e-learning is
[negatively] related to perceived learning” (p. 575). The key indicator for learning is the
relationship between teacher and student. Blended e-learning was found to be beneficial,
but the authors suggest that “the right level of media involvement must be analyzed” (p.
578) in further research. This article has many factors in common with the direction of
my dissertation topic including student satisfaction and perceived learning in a
professional development context.
WattsSEDU7707-8-8
59
Simmons, L. L., Conlon, S., Mukhopadhyay, S., & Yang, J. (2011). A computer aided
content analysis of online reviews. The Journal of Computer Information Systems,
52(1), 43-55. Retrieved from
http://xt6nc6eu9q.search.serialssolutions.com.proxy1.ncu.edu/?sid=
CentralSearch:null&genre=article&atitle=A+COMPUTER+AIDED+CONTENT
+ANALYSIS+OF+ONLINE+REVIEWS&volume=52&issue=1&title=The+Jour
nal+of+Computer+Information+Systems&issn=0887-4417&date=2011-1001&spage=43&aulast= Simmons&aufirst=Lakisha
With the large number of electronic Word of Mouth (eWOM) tools available to
consumers it is important to companies to be able to gather and interpret the flavor of
these messages to assist in better decision making. This article describes a system
whereby such a system is created to perform a content analysis of movie reviews. I am
not interested in the practical implementation, but am interested in the underlying
research method of the content analysis and how that was done. The authors argue that
performing the content analysis by computer “lessens the human resource and increases
the reliability and speed of sentiment analysis” (p. 43).
Content analysis is used to study the content of communication. It involves
examining theoretical definition and empirical measurement. The goal of content
analysis is to create systematic and objective criteria for transforming written text
in highly reliable data that can be analyzed for the symbolic content of
communication. (p. 44)
The authors used an “inductive content analysis approach” which is “often
conducted by first examining patterns in data and then seeking to make sense of those
patterns” (p. 48). Independent raters were trained to create a lexicon of words
representing positive and negative reviews with a proportion of agreement between them
as .94 (.75 is generally accepted as high reliability).
WattsSEDU7707-8-8
60
So, H.-J., & Bonk, C. J. (2010). Examining the roles of blended learning approaches in
computer-supported collaborative learning (CSCL) environments: A Delphi
study. Educational Technology & Society, 13 (3), 189–200. Retrieved from
http://www.ifets.info/journals/13_3/17.pdf
In this study the authors used a Delphi approach which is “a time- and costefficient method to obtain opinions from experts without physically bringing them
together for a face-to-face meeting” (p. 191) and is seen as beneficial to (a) “identify
current practice and perceived obstacles,” (b) “draw consensus on ill-defined problems or
constructs,” and (c) “forecast future events and trends” (p. 191). In this study in round
one eight open-ended questions were presented to 13 experts “to generate a list of
statements for subsequent rounds” (p. 192). In round two “areas of agreement and
disagreement” (p. 192) were identified among the panel members, in addition to selfratings on levels of expertise, and ratings of which statements were the most important
and why. The final round served to focus high consensus which was attained on most of
the 38 items identified as important regarding blended learning. Blended learning
combines “face-to-face instruction with technology-mediated instruction” (p. 189), and is
encouraged because “simply turning classroom lectures into online learning formats does
not necessarily provide students with the opportunities for rich interactions arising from
engagement in activities that make learning experiences meaningful” (p. 189).
Research Design
Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for
comparing the effects of two treatments in a single subject. Journal of Applied
Behavior Analysis, 12(2), 199-210. doi:10.1901/jaba.1979.12-199
The authors proposed a new research design they called alternating treatments
design which allows for “the fast alternation of two or more treatments in a single
subject” (p. 202). This design allows for good internal validity but critics may question
WattsSEDU7707-8-8
61
its external validity. It is through the repetition of internally valid research on additional
groups that external validity is established “rather than statistical inference from groups
to populations” (p. 204). Alternate treatments design counterbalance treatments in
random ways and can improve internal validity by minimizing sequential confounding,
which also allows for statistical analysis. Carry over effects come in two varieties,
contrast (the change in “behavior in a direction opposite to that expected due to a contrast
with another treatment” (p. 205)) and induction (a “positive transfer between treatments
with the behavior during one treatment more closely approximating the behavior during a
second treatment than would occur if the treatments were applied individually” (p. 205)).
Alternate treatment designs are used when comparing treatment to no treatment, and to
compare two distinct treatments. Alternate treatment designs have the advantages of (a)
not requiring a withdrawal of treatment, (b) “comparison can be made more quickly than
in a withdrawal design” (p. 207), and (c) the possibility that a formal baseline does not
have to be conducted. Disadvantages to the alternate treatment design are multiple
treatment interference, and it is “more cumbersome to arrange than the withdrawal
design” (p. 208), meaning that potential confounding variables need to be
counterbalanced.
de Anda, D. (2007). Intervention research and program evaluation in the school setting:
Issues and alternative research designs. Children & Schools, 29(2), 87-94.
Retrieved from
http://puck.naswpressonline.org/vl=730123/cl=18/nw=1/rpsv/cw/nasw/01627961/
v29n2/s4/p87
To overcome the problems of using intact groups the authors propose:
the use of two quasi-experimental designs, a design that allows the individual to
be used as his or her own control and employs a paired analysis and the SeparateSample Pretest-Posttest Control Group Design, which compares outcomes across
WattsSEDU7707-8-8
62
sequential experimental and control groups. (p. 88)
In cases where the participant acts as his or her own control, internal validity is
improved and can be compared by calculating individual differences between pretestposttest and then using a paired t-test analysis to identify significance. The only threat to
internal validity in this case is history in that another “simultaneous event could not be
ruled out as affecting pretest to posttest changes” (p. 89), but by “applying the
intervention to different groups of students at differing times” (p. 89) this threat can be
mitigated, while confounding variables can be identified by an analysis of covariance
(ANCOVA). The authors suggest that combining the previous with a separate-sample
pretest-posttest allows for “intervention in sequential fashion without multiple testing; . . .
[and] control group comparisons using intact classrooms, . . . [offering] cross-validation
of the results of the paired comparison” (p. 90). This combination design “has strong
internal validity, controlling for history, maturation, selection bias, and testing” (p. 90).
Dimitrov, D. M., & Rumrill, P. D. Jr. (2003). Pretest-posttest designs and measurement
of change. Work, 20(2), 159-165. Retrieved from
http://www.phys.lsu.edu/faculty/browne/MNS_Seminar/JournalArticles/Pretestposttest_design.pdf
The underlying context of this article is rehabilitation services and the use of
pretest-posttest designs in that field. I am more interested in the discussion of the
research design and its related reliability issues.
Internal validity is the degree to which the experimental treatment makes a
difference in (or causes change in) the specific experimental settings. External
validity is the degree to which the treatment effect can be generalized across
populations, settings, treatment variables, and measurement instruments. . . .
Factors that threaten internal validity are: history, maturation, pretest effects,
WattsSEDU7707-8-8
63
instruments, statistical regression toward the mean, differential selection of
participants, mortality, and interactions of factors. . . . Threats to external validity
include: interaction effects of selection biases and treatment, reactive interaction
effect of pretesting, reactive effect of experimental procedures, and multipletreatment interference. (p. 159)
The use of a nonrandomized control group pretest-posttest design has the practical
advantage that “it deals with intact groups and thus does not disrupt the existing research
setting” (p. 160) which improves the external validity by reducing “the reactive effects of
the experimental procedure” (p. 160), but potentially dilutes the internal validity because
of “selection and maturation, selection and history, and selection and pretesting” (p. 160).
The authors suggest that for a nonrandomized control group design the comparison
groups are not assumed to be equal on the pretest, and “the data analysis with this design
should use ANCOVA or other appropriate statistical procedure” (p. 163) but if the pretest
scores are not reliable “the treatment effects can be seriously biased” (p. 164) in a
nonrandomized design.
Edgington, E. S. (1966). Statistical inference and nonrandom samples. Psychological
Bulletin, 66(6), 485-487. Retrieved from
http://homepage.mac.com/psychresearch/Sites/site2/psy779readings/Edgington19
66.pdf
“Statistical inferences cannot be made concerning populations that have not been
randomly sampled” (p. 485). The author suggests that while his procedure will not allow
for the drawing of statistical inferences, there’s nothing preventing nonstatistical
inferences to be made based on “logical considerations” (p. 485). The author
demonstrates that with appropriate sample sizes it does not matter whether groups are
randomly selected or not because the probability (based on the Mann-Whitney U
WattsSEDU7707-8-8
64
probability table) of selecting all of the high-level or low-level individuals into a single
group is remotely small. By using a t-test and determining every possible combination of
randomizing two groups – if the t test values of the nonrandomized sample is “a close
approximation to the randomization test” (p. 487) a parametric test can be considered an
approximation of a randomized test. This constitutes a great argument for a
nonrandomized sampling under certain specific conditions.
Ellis, T. J., & Levy, Y. (2011). Framework of problem‐based research: A guide for
novice researchers on the development of a research‐worthy problem. Informing
Science: The International Journal of an Emerging Transdiscipline, 11(1), 17‐33.
Retrieved from http://inform.nu/Articles/Vol11/ ISJv11p017‐033Ellis486.pdf
The authors address the “issue of identifying and establishing the researchworthiness of a [research] problem” (p. 17), by addressing five aspects, (a) the need “for
basing research on a well-defined problem;” (p. 17) (b) what constitutes a research
worthy problem; (c) demonstrating effective problem statements; (d) how to locate
problems that are research worthy; and (e) a “summary and recommendations” (p. 17).
The authors demonstrate through the literature that the problem statement is “a
cornerstone for any quality research” (p. 19) and “serves as the starting point for the
research and is a unifying thread that runs throughout all the elements of the research
endeavor” (p. 19). The authors point out that while the research problem is the starting
point, it is the literature review that serves as the foundation for the research. Regarding
research problems the authors suggested three qualifications which are (a) the problem is
active, “the current state differs from the ideal state” (p. 22); (b) the problem is impactful,
it must be identifiable; and (c) the current solutions are inadequate or nonexistent. To be
considered research the authors identify six means of “creating identifiable new
knowledge” (p. 23). Worthiness is built on “an exhaustive understanding of the body of
WattsSEDU7707-8-8
65
knowledge related to the field or topic of study” (p. 24) and encompasses an expansion
on or filling of current knowledge in the field. The authors define the problem statement
as “the statement of the problem and the argumentation for its viability” (p. 27) and offer
a template that while simple “is far from effortless” (p. 27) to write. The template
“provides an overview of how the six questions of what, how, where, when, why, and
who, should be addressed” (p. 27). The authors propose that novices can locate research
worthy problems by using four steps; look, read, synthesize, and consult. The problem
statement template is not prescriptive and does not allow a paint-by-number context, but
it does propose a means of succinctly identifying the questions that need to be answered
and the support that is needed for the various parts of a problem statement.
Wright, D. B. (2006). Comparing groups in a before-after design: When t test and
ANCOVA produce different results. British Journal of Educational Psychology,
76, 663-675. doi:10.1348/000709905X52210
This article focused on nonrandomly select groups assumed to be non-equivalent.
Identified in this article are the “two most common statistical approaches” (p. 663) used
in this situation; t test on the gain scores and an analysis of covariance (ANCOVA). The
author states that
The only procedure that is always correct in this situation is a scatterplot
comparing the scores at time 2 with those at time 1 for the different groups. In
most cases you should analyse (sic) the data in several ways. If the approaches
give different results . . . think more carefully about the model implied by each.
(Wright, 2003, p. 130)
The paradox occurs because different assumptions are made and different
questions are asked corresponding to each test.
t test:
posti = prei + β1groupi + β0 + ei
WattsSEDU7707-8-8
ANCOVA:
66
posti = β2prei + β1groupi + β0 + ei
The t test “asks whether the average gain in score is different for the two groups”
(p. 666), while ANCOVA “asks whether the average gain, partialling out pre-scores, is
different between the two groups” (p. 666). If the assumption is that without treatment
measures will remain constant or if the assignment tests and is based on ability then the t
test is the best tool, but if the assumption is that even without treatment measures are
changing, or if assignment is from the pretest then the ANCOVA is the best tool.
WattsSEDU7707-8-8
67
Appendix B: Concept Paper Minimum Standards
Introduction
[Text… Dissertation topic is introduced in one or more paragraphs (2 pages
maximum). The study topic is briefly described to establish the main ideas and context.
Note: Topic must reflect doctoral level study and the specific program.]
Statement of the Problem
[Text… Present general issue/observation that in theory or practice leads to the
need for the study (in most cases scholarly citations within the last 5 years are included).
Present focused problem that leads to the need for a research response. Clearly describe
and document the problem that directly leads to the study purpose. For some degree
programs (DBA, EdD) the problem identified might be a practical problem or issue in an
organization or school.]
Purpose of the Study
[Text… Research method is identified as qualitative, quantitative, or mixed
method. Research design is clearly stated and is aligned with the problem statement.
Identification of variables/constructs and/or phenomenon/concept/idea:
Quantitative research variables/constructs are briefly identified (including
potential confounding variables, covariates, mediating variables, etc.). Research
variables/constructs are identified and cited, if appropriate.
Specific population of proposed study is identified. The number of participants
that will serve as the sample should be estimated based on a power analysis
(quantitative/mixed method) or conventions (qualitative) as detailed in chapter 3.
Geographic location of study is identified.]
WattsSEDU7707-8-8
68
Research Questions
[Text…]
Q1.
Q2.
Hypotheses (Quantitative/Mixed Studies Only)
H10. [Null Hypothesis Text…]
H1a. [Alternative Hypothesis Text…]
Definition of Key Terms
[Text (optional)… ]
Term 1. Definition (APA citation).
Brief Review of the Literature
[Text… Discussion has depth and presents a critical analysis and synthesis of the
literature that provides a context for the dissertation study. Discussion is comprehensive,
organized, and flows logically. Use themes and/or subtopics as headings. Identify the
themes or sub-topics around which the literature review has been organized into a
coherent narrative discussion. In the review, at least 7 to 10 of the most important works
or studies that touch upon the dissertation topic or problem are discussed. Be sure to
include works that provide alternate or opposing perspectives on the proposed topic area
to demonstrate unbiased research. Learners focus particularly on those works that
address main ideas in the field, describe areas of controversy, and indicate areas of
incomplete knowledge and relate them to the envisioned study’s problem, purpose, and
research questions. Include historical and germinal works as well as current works
(within the last 5 years).
WattsSEDU7707-8-8
69
Theme/Sub-Topic 1 [Repeat, as needed…]
[Text…]
Summary
[Text…]
Research Method
[Text… Here discuss the proposed research method (quantitative, qualitative, or
mixed). In this section the appropriateness of the method and design are substantiated
and includes a brief discussion of why the method/design(s) was/were chosen over
others. Discussion is not simply a listing and description of research designs; rather,
elaboration demonstrates how the proposed method and design accomplish the study
goals, why the design is the optimum choice for the proposed research, and how the
method aligns with the purpose and research questions. Be sure to provide a brief
discussion of the proposed data collection and analysis procedures. Provide appropriate
foundational support for the proposed study design; for example, refer to Moustakas and
other appropriate authors to describe a phenomenological design.
Note: Avoid introductory research design and analyses descriptions as well as
excessive reference to textbook authors such as Creswell and Trochim.
Operational Definition of Variables (Quantitative/Mixed Studies Only)
[Text (optional)… Identify each of the primary constructs associated with the
proposed topic, problem, research question(s), and hypotheses. Include a brief overview
of how each will be operationally defined for the proposed study]
Construct/Variable 1. Description/Operational Definition.
Describe each variable, the nature of the variable (e.g., nominal, ordinal, interval),
WattsSEDU7707-8-8
70
how each variable will vary (e.g., the range (1 – 5, 0 – 100) or levels (low, medium, high;
male, female) and the data sources (e.g., archival data, survey items, and if appropriate,
how the items will be combined to form the variable construct). Consult research design
sources and ensure that the nature of each variable is appropriate to the proposed
statistical analyses.
Measurement
[Text…Provide a brief description of how study data will be collected, measured
and analyzed. Describe the proposed instrument. Please note that survey selfdevelopment should be considered only after an exhaustive search for an existing
validated instrument and will require a multi-step pilot and validation process. Although
a detailed description is not required at the CP stage, the variables must demonstrate
appropriateness to the study purpose and meet the assumptions of the proposed statistical
tests. For qualitative studies, describe the proposed instrument or collection (e.g.,
interviews, observations), and how concepts will be coded and analyzed as appropriate to
the proposed design. Include appropriate support for the application of the proposed
design. Consult research design and analysis sources including those available in the
Dissertation Center for guidance.]
Summary
[Text…]
WattsSEDU7707-8-8
71
References
Reference 1
Reference 2
Reference n…
Instructions: This section of the Concept Paper is a list of references cited in text,
including the literature review. All resources cited in the concept paper must be included in the
list of references.
List all references in APA format with the exception noted below. For each reference
listed, there should be at least one corresponding citation within the body of the text, and viceversa.
Formatting: Single space each reference citation, along with a .5 inch hanging indent;
double space between consecutive references in the reference list (See the Dissertation
Handbook located in the Dissertation Center for NCU exceptions to APA format).
Tips: Sort in alpha surname/title order. Only capitalize the first word of the title and of
the subtitle, if any. Do not bold the title. Know when to italicize and when not to (i.e.,
periodical/non-periodical/publication versus book/report/paper). Italicize volume (i.e., Journal
Name 4, pp. 12-22.)
Note: APA6 Requires Digital Object Identifier (DOI), if one has been assigned (see page
191).
WattsSEDU7707-8-8
72
Example (note single-space references, with double-spacing in-between):
Winslade, J., & Monk, G. (2001). Narrative mediation: A new approach to conflict
resolution. San Francisco: Jossey-Bass Publishers.