WattsSEDU7707-8-3
0
Thematic Annotated Bibliography
Stephen W. Watts
Northcentral University
WattsSEDU7707-8-3
1
Thematic Annotated Bibliography
Seminal Article
Zemke, R., & Zemke, S. (1995) Adult Learning: What do we know for sure? Training, 32, 6982. Retrieved from
http://setur.khi.is/fullordinsfraedsla/NoN/Itarefni/Zemke_WhatDoWeKnow.pdf
In this seminal article regarding adult learning, the authors provided a brief background
into the history of adult learning, or andragogy, as a specific field, and then presented major
trends in the field from more than 300 articles. The authors divided this task into three sections –
the things that the field knows about (a) “adult learners and their motivation” (b) “designing
curricula for adults,” and (c) “working with adults in the classroom” (para. 17). The major
themes they identified regarding adult motivation to learn focuses on three items, (a) “Adult
learning is problem-centered” (Motivation to Learn, para. 6), (b) adult learners seek learning
more often than not because of a life-changing event, but can be motivated “by appealing to
personal growth or gain” (para. 10), and (c) motivation “can be increased” (para. 12). The
authors noted eight focuses related to curriculum design, which are (a) the learning experience
itself “should be problem-centered” (Curriculum Design, para. 2), (b) it is important to access
“the entry-level knowledge and understanding of participants” (para. 3), (c) new knowledge
should be related to existing experience, (d) exercises and activities should “contain a reflective
element” (para. 9) in order for learning to take place, (e) “feedback and recognition should be
planned” (para. 10), (f) the design should “account for learning-style differences” (para. 12), (g)
the design should accommodate “adults’ continued growth and changing values” (para. 15), and
(h) design should include means to take the new knowledge and apply it to the situation for
which they sought learning. The authors discerned three “common threads . . . [that] suggest
some useful guidelines” (In the Classroom, para. 3) for in class instruction. These threads are (a)
the instructor needs to “create a safe and comfortable environment” (para. 4), (b) instructors
WattsSEDU7707-8-3
2
should be more facilitator and less lecturer, and (c) promote the contribution and collaboration of
the students as part of the discussion and learning. The authors provide an excellent springboard
to determine not only what is known, but where gaps in knowledge occur in the field of
andragogy.
Adult Learning Theory
Andrews, R., & Haythornthwaite, C. (2007). Introduction to e-learning research. In R. Andrews,
& C. Haythornthwaite (eds.), The SAGE handbook of e-learning research (pp. 1-51). Los
Angeles, CA: SAGE.
In this introduction, the authors discuss various aspects of e-learning including what both
the ‘e’ and the learning are coming to mean, a discussion of the theoretical background for elearning which they claim has its foundations in Rhetorical theory and Social informatics, and
the methodological challenges for research in the field. Their purpose was to “articulate a model
for examining e-learning that incorporates elements of rhetorical, communication, and social
informatics theories” (p. 33). The authors identify that the default model of educational research
in which context is removed and the effect of one independent variable on one or more
dependent variables is considered, is problematic when dealing with information and
communications technology (ICT) and learning, in that it may be true that the independent
variable effects the dependent variable, but in turn the dependent variable may then effect the
independent variable – marking them as reciprocally co-evolutionary rather than truly
independent or dependent. In the relationship between ICT and learning a temporal dimension is
obvious as the technology of today is vastly different than that of a couple years ago, but it is
also dialectical in that new technologies tend to “backwash on to older technologies and
practices” (p. 37). The authors suggest that rather than a default research model the use of
“cross-lagged panel designs” (p. 39) could be useful by “paying attention to the problem of how
WattsSEDU7707-8-3
3
continuous (and sometimes erratic) development can be adequately mapped in staged analyses of
reciprocity” (p. 39) using both qualitative and quantitative data.
Baskas, R. S. (2011, March 27). Adult learning assumptions. Retrieved from ERIC database.
(ED517971)
The author, a professor at Walden University, conducted a literature review in this
reviewed article that indicates that Knowles’ six assumptions can be considered a theory for how
adults learn most successfully. The author focused on two of Knowles assumptions – the need
for adults to be motivated in order to learn, and the role of the students’ life experience that
colors their learning, and identified from the literature that these two assumptions are supported.
In regards to adult motivation, the literature supports that adults need to be motivated, and that
adults realize the importance of their learning because of changes in positive affect. The
literature also confirms that as adults learn, they make mistakes because of their inexperience in
a new realm, but that through interacting with others they can inculcate their learning even
better. Regarding the assumption that students’ previous experience and knowledge will affect
their current learning, the author identified trends in the literature that suggest that instructors
need to take into account the experiences and potential biases of students to engender reflection.
Baskas, R. S. (2011). Applying adult learning and development theories to educational practice.
Retrieved from ERIC database. (ED519926)
In this partial review of the literature the author identifies that three of the components of
online learning; lectures and discussions, scholarly papers, and reflections, are supported by both
Knowles Phase Theory, and Levinson’s Developmental Theory. Since online learning can be
self-paced learners are able to use that convenience in working on and turning in assignments.
Learners generally also have writing services to facilitate the development and improvement of
writing skills, and online learning can encourage learner’s to reflect on their past experiences in
the context of their current learning. The author’s discussion of Knowles and Levinson’s
WattsSEDU7707-8-3
4
theories reinforces other findings that confirm ways in which instructors can better teach
students.
Blanchard, R. D., Hinchey, K. T., & Bennett, E. E. (2011, April). Literature review of residents
as teachers from an adult learning perspective. Paper presented at the annual meeting of
the American Educational Research Association, New Orleans, LA. Retrieved from
http://www.eric.ed.gov/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=ED
521385
In this integrated literature review the authors “identified skills and characteristics of
resident teachers” (abstract) and presented them “as mechanisms for achieving the five tenets of
Knowles (1984) model of adult learning” (abstract). After a review of Knowles five tenets the
authors identify that the skills for success in higher education may not have the “interpersonal
and adult relational skills” (p. 4) necessary to succeed in teaching adult education. The research
question developed was “what are the essential skills and characteristics of resident teachers
consistent with andragogy” (p. 4), and seventeen articles were found that touched on the research
question. The mechanisms found include (a) “feedback which should be ‘both corrective and
reinforcing’ (Heflin, Pinheiro, Kaminetzky, & McNeill, 2009, p. e234)” (p. 5), (b) “promote
autonomy in learners” (p. 6), (c) “incorporate the prior experiences of their learners” (p. 6), (d)
promote “a positive climate for learning” (p. 7), (e) cause learning to be problem-centered, and
(f) general skills including “communication skills, enthusiasm, and ethics” (p. 7) as well as
“understanding of leadership” (p. 7). This article reflects a confirming and differing view of
adult learning theory.
Chen, L.-C., & Lien, Y.-H. (2011). Using author co-citation analysis to examine the intellectual
structure of e-learning: A MIS perspective. Scientometrics, 89(1), 867-886.
doi:10.1007/s11192-011-0458-y
The authors argue that “there is little comprehensive knowledge on e-learning, especially
in the non-educational fields” (p. 867), and conducted a study using author co-citation analysis
(ACA) in order to identify “the intellectual structure of specific knowledge domains through the
WattsSEDU7707-8-3
5
relationship between two similar authors” (p. 867) regarding e-learning practices from 1996 to
2009.
By treating bibliographic elements as conceptual units, ACA helps researchers analyze
discipline structure and reduce personal bias within their results. ACA is based on tracking the
number of times that two authors or documents are cited together, assuming that “the more
frequently two authors are cited together, the closer the relationship is between them” (p. 870).
The authors identified the steps of conducting the study as “(1) identify authors highly cited by
research articles; (2) retrieve co-citation counts for each pair of authors; (3) compile a matrix of
raw co-citations; (4) perform clustering through various analytical methods . . . ; (5) interpret the
results” (p. 870). By following this methodology the authors determined six similarities and four
differences between MIS focused e-learning and e-learning from an educational perspective.
The authors call for further research in both “theoretical and practical discussions of e-learning
and its structural breakdown” (p. 883). This is an interesting approach that I would enjoy further
investigating and perform in relationship to the current literature of e-learning and adult
education as I do not believe that the authors went far enough in their considerations.
Kenner, C., & Weinerman, J. (2011). Adult learning theory: Applications to non‐traditional
college students. Journal of College Reading and Learning, 41(2), 87‐96. Retrieved from
http://20.132.48.254/ERICWebPortal/contentdelivery/servlet/ERICServlet?accno=EJ92636
5
The authors focus is “on how best to understand and teach entry-level adult learners who
are between the ages of 25 and 50, have a high school diploma or a GED, are financially
independent, and have one semester or less of college-level coursework” (p. 88). They suggest
that by understanding differences in these learners from traditional students specific tools might
be created to “increase their chances for success” (p. 88). The authors review Knowles (1974)
four principles, and introduce the metacognitive frameworks of Schraw and Moshman (1995);
tacit theory, informal theory, and formal theory. The authors deemed that formal theory was not
WattsSEDU7707-8-3
6
applicable to their purpose and did not discuss it. According to tacit theory “adult learners
acquire their metacognitive skills from peers, teachers, and the local culture” (p. 89) and they are
likely difficult to change, and don’t usually recognize how they arrive at their beliefs. Informal
theory says adult learners “acquire their metacognitive skills over time from their peers and their
environment, but they have at least a rudimentary conscious thought process” (p. 90). The
authors note that learners may have many skills that are useful to them in their everyday life, but
these skills may not be useful in an academic environment, and that educators should be aware of
this possibility. Educators should then create in their curriculum activities and assignments that
will build the metacognitive skills of the adult learner so that they succeed in converting abstract
theory and thought into practical application over time. To ingrain these new skills into the
learner’s toolbox the authors suggest competition, placing new strategies in direct contrast to old
strategies, and repetition, ensuring that activities build on each other. This article is purely
theoretical with no associated study, and will not be directly helpful in my educational path.
Lorge, I. (1956). Learning, motivation and education. In J. E. Anderson, J. E. Anderson (Eds.),
Psychological aspects of aging (pp. 207‐210). American Psychological Association.
doi:10.1037/10032‐022
The author’s primary point in this theoretical discourse is that peoples learning “depends
on what rewards, gratifications and values” (p. 208) they have learned to have. He comments on
the limitations of adult learning theory, and wonders if “we give adults in our laboratories a
chance to learn again,” focusing on what has been learned rather than what is.
Minter, R. L. (2011). The learning theory jungle. Journal of College Teaching and Learning,
8(6), 7‐15. Retrieved from
http://journals.cluteonline.com/index.php/TLC/article/view/4278
The author notes the irony that university level instructors by and large have less of a
foundation in pedagogical theory, learning models, and instructional application skills than do K12 teachers. The author proposes that not all learning theories apply to the adult learner, but
WattsSEDU7707-8-3
7
introduce twenty-seven pedagogical theories. The author notes that this list is not inclusive, but
representative and recommended readings are suggested for those itemized, and research has
found that there is no “one set of major theories that are generalizable across student-learning
environments at the college level” (p. 8). The author compares the major assumptions of the
pedagogical and andragogical models of teaching, and identifies a number of questions to help
teachers understand and identify the model that they follow in their own teaching, but notes that
instructors usually follow their own intuition “without knowing whether learning theory or
research support [their] instructional initiatives” (p. 11). He sardonically evaluates that though
the mission of higher education is to facilitate learning, it is unusual for colleges to encourage
teaching excellence or sharing of successes or failures within their ranks. This article is purely
theoretical and does not have an associated study, but suggests that there should be greater
communication between higher education educators to facilitate best practices in teaching adult
learners. The author also identifies that there is no central theory that a majority, or even a
minority, subscribe to in regards to andragogy. This article contains an excellent listing of
current learning theories and is a great jumping off point for further investigation.
Sharples, M, Taylor, J. & Vavoula, G. (2007). A theory of learning for the mobile age. In R.
Andrews, & C. Haythornthwaite (eds.), The SAGE handbook of e-learning research (pp.
219-247). Los Angeles, CA: SAGE Publications.
The authors in this theoretical discussion define mobile learning as “the processes of
coming to know through conversations across multiple contexts among people and personal
interactive technologies” (p. 225). Their focus is not on the learner or the technology but instead
on the communication that occurs between them that advances knowledge. There are three main
topics in this definition; the conversation, the context, and the technologies. The authors identify
that “the driving process of learning” (p. 225) is the conversation, and their discussion was
primarily derived from the work of Gordon Pask (1976), who instead of seeing communication
WattsSEDU7707-8-3
8
as “the exchange of messages through an inert and transparent medium” (p. 226) saw
communication “as the sharing of understanding within a pervasive computational medium” (p.
226) which serve as the “active and responsive systems within which mind-endowed individuals
converse” (p. 226). According to Ravenscroft (2000) “the most successful learning comes when
the learner is in control of the activity, able to test ideas by performing experiments, to ask
questions, collaborate with other people, seek out new knowledge, and plan new actions” (p.
227). The authors noted that “all activity is performed in context” (p. 230) but learning “not only
occurs in a context, it also creates context through continual interaction” (p. 230). Since my
dissertation will emphasize communication this article reinforces and aids how important
communication is perceived and received when it comes to learning online.
Teaching Online – Key Findings
Boling, E. C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2011). Cutting the distance in
distance education: Perspectives on what promotes positive, online learning experiences.
Internet and Higher Education [Advance online publication].
doi:10.1016/j.iheduc.2011.11.006
Data collected from this study was used to determine supportive and hindering
characteristics of effective online learning experiences. In this qualitative study, the authors
explored the online learning and teaching experiences of ten adult students and six online course
instructors using a “disciplined configurative cases study approach” (p. 2) using a convenience
sample of individuals “who had participated in a number of online courses” (p. 2) but
represented various fields of study. The Cognitive Apprenticeship Model was used as the
theoretical framework for the deductive and inductive analyses for coding and analyzing the
interview data. The findings for this study indicated that online students found that programs
that “were more interactive and incorporated the use of multimedia” (p. 3) were more helpful in
achieving learning objectives, while courses that were less helpful were characterized by: (a)
WattsSEDU7707-8-3
9
emphasizing text-based content, (b) “individualized learning” (p. 3), and (c) limited interactions
with other students, the instructor, or other faculty. Half of this article included the introduction
of an online masters program that provided modeling, coaching, and scaffolding, and encouraged
the interactivity among students, and between students and the instructor. My dissertation
focuses on student satisfaction in an online learning environment, and this article supports
discussion of how this can come about.
Bradley, J. (2009). Promoting and supporting authentic online conversations – which comes first
– the tools of instructional design? International Journal of Pedagogies and learning,
5(3), 20-31. Retrieved from http://jpl.e-contentmanagement.com/
This case study was designed to “embed pedagogical theory into practice [while]
blend[ing] the correct online communication tool and effective instructional design” (p. 25).
Four participants took part in the study and took part of three activities designed from a
constructivist standpoint. The author argued that using the ADDIE model of instructional design
is problematic for a constructivist online learning environment because the focus of learning
should be on the learning process rather than the content per se. The elements for the design of a
constructivist learning environment focuses on “keeping students active, constructive,
collaborative, intentional, complex, contextual, conversational, and reflective” (p. 22).
Technology is the conduit through which learning can take place, while the learning is enhanced
and facilitated by “well-designed learning goals and objectives” (p. 28) and in the context of
constructivist theory “the skill of the facilitator is integral to achieving successful outcomes” (p.
28). Based on cognitive load theory I find it difficult to believe that learners will be able to focus
on eight different elements as formulated in this and previous articles. Rather than having the
elements interacting with each other all of the time, it is more likely that the elements during
learning are used in some sequential way, with perhaps a maximum of four elements applicable
WattsSEDU7707-8-3
10
at any one time. This article is helpful in identifying and clarifying the constructivist theory
regarding interaction and its uses, as well as the role of the facilitator in implementing the theory.
Ferguson, J. M., & DeFelice, A. E. (2010). Length of online course and student satisfaction,
perceived learning, and academic performance. International Review of Research in
Open and Distance Learning, 11(2), 73-84. Retrieved from
http://www.irrodl.org/index.php/irrodl/article/view/772/1547
The authors presented an exceptional review of the literature regarding the factors
effecting satisfaction with courses taught online concluding “that connectedness to the course,
either by participating collaboratively with other students or by interacting with the professor,
will likely impact student satisfaction” (p. 75) the most. Equivalency theory, introduced by
Simonson, Scholosser, & Hanson was used as the theoretical framework of this study to
determine if “there were differences in online student satisfaction, perceived learning, and
performance” (p. 76) when the independent variable was length of the course, in this case fiveweeks versus fifteen-weeks, while all other pedagogical factors were kept the same.
Equivalency theory was supported in this study because students “learned the same content in”
(p. 76) both courses. 75 graduate students took part in part one of the study which consisted of a
15-question Likert scale survey, while 114 graduate students final grades from the same four
courses was analyzed for the second part of the study. Students were significantly more satisfied
with the interaction with the teacher in the longer course, while students were significantly more
satisfied with the interaction with fellow students in the shorter course. No significant difference
was found for perceived learning or satisfaction regarding taking additional online classes was
found between the two course lengths. Students in the shorter sessions showed significantly
stronger “academic performance than the full-semester students” (p. 81). Improvements to the
pedagogy of the class were identified for both formats. In regard to the shorter course a shift
needs to be made “to emphasize interaction with the professor” (p. 81) and several possibilities
WattsSEDU7707-8-3
11
were proposed. Limitation to the study was that the students for the shorter course (summer
semester) may have been different from those attending the longer course (regular semesters),
and that students were not randomly selected. This study is very similar to my proposed study in
looking at student satisfaction and perceived learning, and I may utilize the same data
instruments.
Jackson, L. C., Jones, S. J., & Rodriguez, R. C. (2010). Faculty actions that result in student
satisfaction in online courses. Journal of Asynchronous Learning Networks, 14(4), 78-96.
Retrieved from http://jaln.sloanconsortium.org/index.php/jaln
This quantitative research study correlated faculty actions with student satisfaction in
online classes at two community colleges in Texas. Data for the study was obtained from
student responses to each institution’s existing online course evaluation. All online students
were requested to fill out the online evaluation, and 426 students (30%) from College 1 and 1004
students (69%) from College 2 participated. Descriptive statistics, bivariate correlations, and
multiple regressions were used to identify the faculty actions that affected student satisfaction in
online courses. The authors determined that “student satisfaction with online courses appears to
be impacted by instructor actions within the course” (p. 91). The highest correlations with
student satisfaction were “timeliness/accessibility of instructor, clearly stated expectations,
instructor enthusiasm, and comfortable climate” (p. 91) and a moderate correlation existed with
activities. Multiple regression analysis indicated that 69% of the variance of student satisfaction
could be explained by those independent variables. This article identifies specific actions that
instructors can do in their online courses to increase student satisfaction. The article contains
good discussion from the literature regarding faculty roles and student satisfaction.
WattsSEDU7707-8-3
12
Ke, F., & Hoadley, C. (2009). Evaluating online learning communities. Educational Technology
Research & Development, 57(1), 487‐510. doi:10.1007/s11423‐009‐9120‐2
In this study of 42 studies on online learning communities (OLC) four items were
evaluated, (a) “purpose of the study” (p. 490), (b) “evaluation approaches” (p. 490), (c)
“indicators or measures” (p. 490) used, and (d) how data was collected and analyzed. OLC
evaluations were categorized as proving, improving, or both. In the studies, four approaches
were identified – summative, formative, participatory, and responsive, and these approaches
“usually influenced the methods employed” (p. 494). The authors found two major constructs in
how studies measured – outcome and process. The authors identify that this study is preliminary
aand offer it as the beginning of “identifying common patterns and the strengths (and
weaknesses) of relationships among the various process and outcome indicators across OLCs”
(p. 507, italics in original) in order to make progress on whether a learning community is
beneficial. This article is preliminary and foundational to my research but may not be used
directly.
Sinclair, A. (2009). Provocative pedagogies in e-learning: Making the invisible visible.
International Journal of Teaching and Learning in Higher Education, 21(2), 197-209.
Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE688.pdf
This qualitative descriptive case study describes and explains the process of a course
called ‘Reflective practice for teachers’ (p. 198). The authors identify some key characteristics
of being online including having an emphasis “on distributed learning whereby control of the
learning is distributed among the community” (p. 197) instead of a single expert, and that for the
students the process is less hierarchical with “more emphasis on self-regulation and
participation” (p. 197). This paper studies a course based on the notion that true learning is
brought about through cognitive dissonance that results in reflection and new shared knowledge.
This pedagogy of difficulty says that “teaching and learning are complex, tentative, and difficult”
WattsSEDU7707-8-3
13
sometimes “the need for right answers” (p. 201) suppresses the benefit of critical analysis “where
ideas are examined from multiple perspectives” (p. 201). Engle and Conant (2002) suggests that
students who are instructed to engage in knowledge building discussions learn to develop and
justify an argument and will find increasingly sophisticated ways of disagreeing with others.
The authors found that role-playing by learners of situations they would not normally be placed
in led to “a deeper understanding and interpretation of human behavior and meanings” (p. 202),
and that the written, asynchronous nature of this class “enabled [learners] to give deeper
consideration and responses” (p. 202). By emphasizing the learner as a part of a community this
allowed the learners a voice and to feel valued with greater emphasis “on the learning process
and learning experiences” (p. 204) rather than the instructor/expert as provider of knowledge.
Online teaching has been found to “promote higher order thinking, reflection, and rigorous
intellectual challenges leading to more equality between learners and teachers” (p. 204).
Strang, K. D. (2009). Measuring online learning approach and mentoring preferences of
international doctoral students. International Journal of Educational Research, 48(4), 245257. doi:10.1016/j.ijer.2009.11.002
This article reports on an empirical study involving 254 international doctoral students in
twenty-three different cultures to determine if “there is a ‘right’ teaching or supervising method,
based on pedagogy or andragogy theories, for international student learning modes” (p. 247).
“Hypotheses were tested using structural equation modeling . . . [found that method, supervision,
and quality explained 56% of the variance effect on candidate performance” (p. 245) and were
determined to be critical latent variables. The author identifies that “there is very little research
examining cross-cultural teaching practices or online higher education supervision” (p. 245), or
“higher education research across cultures” (p. 245). The interdisciplinary empirical literature
was reviewed to identify “critical factors that interact with . . . [and] impact performance” (p.
246). The author discusses in great detail the testing of the reliability of his instrument. The
WattsSEDU7707-8-3
14
conclusion was that “doctorate supervisors could survey their international student with a
learning style instrument to inform (and possibly improve) their supervision approach for each
unique student” (p. 255), which is feasible because this relationship is one-to-one. This study did
not determine an optimal supervisory approach based on specific cultures or learning styles but
instead suggested five “learning preference indicators” (p. 255) and three latent variables that
could be used by supervisors to inform their supervisory style with individual “international
higher education doctorate students” (p. 255). Although my dissertation will not emphasize
cross-cultural issues, the classes that will be evaluated will have multi-ethnic students and
consideration needs to be made in this regard.
Thompson, L., Jeffries, M., & Topping, K. (2010). E-mentoring for e-learning development.
Innovations in Education and Teaching International, 47(3), 305-315.
doi:10.1080/14703297.2010.498182
In this phenomenological study the authors received survey information on the
perceptions of the benefits, and suggested improvements between mentors, mentees, and project
leaders regarding improvement in the development of online curriculum. The focus of the study
was "e-mentoring between university academic staff focused on the development of online
learning modules" (p. 313). Two findings were advanced, (a) "a thorough negotiated assessment
of mentee needs at an early point is essential, so that the mentoring process can be differentiated
and adaptive" (p. 313), and (b) "early [face-to-face] meetings between mentor and mentee are
widely seen as essential - purely electronic contact appears ineffective" (p. 314). Considerations
regarding modes of e-learning are important to my dissertation topic, and this article broadens
my understanding of blended interactions and characteristics.
Watkins, R. (2005). Developing interactive E‐learning activities. Performance Improvement,
44(5), 5‐7. Retrieved from http://home.gwu.edu/~rwatkins/articles/PI44-5.pdf
WattsSEDU7707-8-3
15
In order to have “engaging e-learning activities that can result in both active learning and
the achievement of course objectives” (p. 5) the author in this commentary identifies a dearth of
“creative ideas that are necessary to create such environments” (p. 5). With the inclusion of
interactive e-learning experiences in the classroom five specific benefits accrue. These benefits
are (a) improved retention rates, (b) increased learner participation, (c) learning objectives being
achieved, (d) development of effective online learning communities, and (e) engaged learners.
The author notes certain questions that can be asked before developing an interactive activity for
the online classroom, and then makes some suggestions of his own suggesting that many can be
“adaptations of activities from the traditional classroom along with imaginative ideas that take
advantage of the unique online technologies” (p. 7). My dissertation will focus specifically on
interactivity, and this article supports the requirement for interaction to foster student satisfaction
and learning.
Learner Characteristics – Key Findings
Bye, D., Pushkar, D., & Conway, M. (2007). Motivation, interest, and positive affect in
traditional and nontraditional undergraduate students. Adult Education Quarterly, 57,
141‐158. doi: 10.1177/0741713606294235
In this study 300 undergraduate students were evaluated using three self-report
questionnaires and the results were compared. The students were evaluated for motivation
strategy by using the Motivation subscale of the Motivated Strategies for Learning Questionnaire
(MSLQ) to determine Intrinsic (participation in a task is an end in itself) or Extrinsic (tasks are a
means to an end) Goal Orientation. The students were also evaluated using the Interest subscale
of the Differential Emotions Scale IV-A (DWS) to determine level of interest which has shown
to “correlate significantly with achievement, affiliation, endurance, and understanding” (p. 148).
The students were also evaluated for positive or negative affect by using the Positive and
Negative Affect Schedule (PANAS). The study showed a 2-to-1 ratio between traditional and
WattsSEDU7707-8-3
16
nontraditional age students, and had “no significant differences between the two groups” in terms
of other demographic factors, and neither skewness nor outliers were found. The study focused
on “differences between the traditional and the nontraditional student subgroups” and used “the
entire student sample to test hypothesized predictors of intrinsic motivation and positive affect”
(p. 149). A mixed factorial analysis of variance (ANOVA) was employed to test different
intrinsic and extrinsic motivation scores, which found there is “a greater need among
nontraditional students to simply enjoy the process of mastering new skills in the classroom” (p.
155). The findings related to interest directly correlate to the topic of my dissertation, and the
data instruments could be potentially used.
Chyung, S. Y., & Vachon, M. (2005). An investigation of the profiles of satisfying and
dissatisfying factors in e‐learning. Performance Improvement Quarterly, 18(2), 97‐113.
Retrieved from http://www.blackwell-synergy.com/doi/abs/10.1111/j.19378327.2005.tb00335.x
The purpose of this study was to “conduct a theory-based investigation to reveal
comprehensive profiles of satisfying and dissatisfying factors in e-learning” (p. 99). The
theoretical framework chosen is based on Edward Lee Thorndike’s law of effect that said that
animals will form a connection with a stimulus based on consequences – so, if a stimulus
satisfies the connection is strengthened, while if a stimulus annoys the connection is weakened,
and these satisfiers and annoyers are “individual- and context-dependent” (p. 99). Another
portion of the author’s theoretical framework was Frederick Herzberg’s motivation-hygiene
theory which states that certain factors contribute to satisfaction (motivation factors), and other
factors may contribute to dissatisfaction (hygiene factors), but these factors are not the opposite
of each other. The author’s performed a content analysis study aimed at building theory and
guidelines from course evaluation data of 17 e-learning courses. After data analysis of
qualitative input, 19 categories were derived. These categories were then prioritized based on
WattsSEDU7707-8-3
17
student emphasis on whether the category was a satisfying factor, or a dissatisfying factor. The
most frequent satisfying factors were learning oriented (i.e., “interesting and relevant learning
content, effective teaching methods, instructor’s expertise, and effective learning activities” (p.
107)). The most frequent dissatisfying factors were impediments to goal achievement (i.e., “lack
of their instructor’s participation during class discussions and lack of clarity in instructional
directions or expectations that caused confusion or frustration” (p. 108)). Although this article is
dated it is relevant and key to my dissertation.
Donavant, B. W. (2009) The new, modern practice of adult education: Online instruction in a
continuing professional education setting. Adult Education Quarterly, 59(3), 227‐245.
doi: 10.1177/0741713609331546
In a three phase quazi-experimental quantitative study of American police officers the
author determined the “efficacy of online education for professional development” and found
that learning took place with respect to both online education and traditional instruction and that
there was no “statistically significant difference in the effectiveness of the two delivery methods”
(p. 239). This study also showed no significant difference between these modes with respect to
gender, race, age, number of years on the force, or previous exposure to online education, but did
show a significant association between level of formal education and potential success with
online learning. Phase one consisted of historical data of performance from various courses
without descriptive information. In phase three open-ended questions were asked of 150
participants, and they indicated four attractive features of online education which were; (a)
general convenience, (b) flexibility in scheduling, (c) remote access, and (d) self pacing of
learning. The least attractive feature was “the lack of personal interaction or face-to-face contact
with the facilitator or other learners” (p. 239). The author noted one problem in the field is that
“little research has been conducted within the professional development environment, that arena
involving training relative to the current occupation of the adult learner” (p. 227). My
WattsSEDU7707-8-3
18
dissertation focuses on adult professional development, and this article is supportive of the need
for such research, but also identifies key characteristics of that scenario.
Gunawardena, C. N., Linder-VanBerschot, J. A., LaPointe, D. K., & Rao, L. (2010). Predictors
of learner satisfaction and transfer of learning in a corporate online education program.
The American Journal of Distance Education, 24(1), 207-226.
doi:10.1080/08923647.2010.522919
In a mixed-methods design, the authors used a survey, as well as open-ended
questionnaire, face-to-face, and phone interviews to gather data on the perceptions of students,
instructors, and instructional designers to determine that “online self-efficacy [was the] strongest
predictor of learner satisfaction; collegial support was the strongest predictor of transfer of
learning” (p. 207). The authors identify that the major problem of the current literature in terms
of distance education in the corporate sector is that they describe only “specific contexts and
programs” (p. 208) and there is a need to move beyond case studies to determine the
characteristics of training that “lead to learning gains, transfer of learning, and satisfaction” (p.
208). Four independent variables were measured in part one of the study, “online self-efficacy,
course design, learner-instructor interaction, and learner-learner interaction” (p. 211), with the
dependent variable being learner satisfaction. The authors noted that the small sample size of the
quantitative portion was the major limitation, and that the qualitative data “yielded information
of great value” (p. 223). This article contains concepts and background of importance to my
dissertation topic.
Huang, E. Y., Lin, S. W., & Huang, T. K. (2012). What type of learning style leads to online
participation in the mixed‐mode e‐learning environment? A study of software usage
instruction. Computers & Education, 58(1), 338‐349.
doi:10.1016/j.compedu.2011.08.003
The authors extended previous research by testing a model that examined the mediating
process of prior knowledge in the relationship between learning style and e-learning
performance. They posited that (a) learning style is positively related to online participation, (b)
WattsSEDU7707-8-3
19
that online participation is positively related to e-learning performance, and (c) the greater the
prior knowledge, the stronger the relationship between online participation and learning
performance. This study measured the learning style of 219 college students in a single course by
measuring (a) student learning style using the ILS, (b) student online participation, (c) student
performance, (d) prior knowledge of the tool used in the course, and (e) the control variables of
gender, computer experience, and Internet experience. Support was found that online
participation is a mediating construct between learning style and performance; further the study
found that sensory learning style individuals tend to participate more frequently and for a longer
duration; while prior knowledge was shown to moderate the relationship between participation
and learning performance only in terms of passive participation. Several recommendations were
made by the authors. First, “although it is difficult to determine the degree of influence of the
mediating construct, educational institutions should take action to boost ‘students’ online
participation in e-learning courses” (p. 347). Second, “most learners appear to be able to benefit
[from e-learning] immediately” (p. 347). The authors also commented on several suggestions for
further research, namely (a) the model needs to be tested in different subject contexts, (b)
additional mediating processes that link learning styles and learning performance should be
explored, and (c) a more mature, professional, and autonomous set of online learners should be
enlisted. As my dissertation focuses on student satisfaction this article contributes to a basic
understanding of its components.
McGlone, J. R. (2011). Adult learning styles and on‐line educational preference. Research in
Higher Education Journal, 12, 1‐9. Retrieved from
http://www.aabri.com/manuscripts/11859.pdf
In this literature review the author identified that the foremost andragogical theory to the
teaching of adults “requires a process-focused approach” (p. 4) and accepted the eight-element
process model of Knowles, Holton & Swanson (2005) as “one of the most comprehensive
WattsSEDU7707-8-3
20
discussions on the theoretical and historical background of” (p. 4) andragogy, and discusses in
some detail each of the elements. The author then identified that the Kolb Learning Styles
Inventory (LSI) is an “instrument that is used widely in studies related to” (p. 5) evaluating
differences between adult students, and presents three studies that use this instrument. The
author reports findings of three studies regarding adults and on-line education, noting that these
studies indicated two things; “more and more . . . adults want to enroll in these courses, in an
effort to improve their employment or qualification standing” (p. 6), but adult learners were “all
relatively dissatisfied with the on-line learning experience” (p. 6). Another study showed that
most online courses placed less of a focus on the “development of critical-thinking skills or other
forms of critical reflection on the material . . . [making the experience] less impactful” (p. 6).
Two recommendations were expressed by the author as suggested by the literature; make sure
that the “latest hardware and software improvements” (p. 8) are used in the on-line training, and
either train students on how to utilize the computer, or “adapt the material to allow for better
Internet learning skills” (p. 8) for the students. This article presents adult student characteristics
that will be foundational for my dissertation.
Pigliapoco, E. E., & Bogliolo, A. A. (2008). The effects of psychological sense of community in
online and face-to-face academic courses. International Journal of Emerging
Technologies in Learning, 3(4), 60-69. Retrieved from http://halshs.archivesouvertes.fr/docs/00/19/72/37/PDF/71_Final_Paper.pdf
In “a case study of a BS degree program in Applied computer science delivered both
online and face-to-face” (p. 68) regarding the influence of teaching environment on
psychological sense of community (PSoC) the authors determined that “virtuality does not
necessarily impair PSoC, and that the differences in student performance and dropout rate
between online and face-to-face degree programs are mainly explained by the composition of the
corresponding student populations” (pp. 60-61). PSoC has four dimensions, identified by Rovai
WattsSEDU7707-8-3
21
(2002) which are spirit, trust, interaction, and common expectations and is “affected by
transactional and geographical distance between students and instructors” (p. 61). The authors
note that other studies student-teacher and student-student interactions “increase the
effectiveness of learning” (p. 61) and other benefits like student satisfaction, performance, and
reduced dropout rates. The authors also note that in addition to PSoC factors like slow instructor
feedback, confusion on how to use the technologies, and confusion regarding a different study
model could also contribute to dropout rate. The discussion of the control and experimental
group on page 62 will be of benefit in my dissertation. The authors suggest a refinement of their
regressions models since the models failed to “completely model the behavior of the dependent
variables under study” (p. 68), and conduct further research to generalize their findings since this
case study only involved 107 students.
Application and Instructional Design
Abrami, P. C., Bernard, R. M., Bures, E. M., Borokhovski, E., & Tamim, R. (2010, July).
Interaction in distance education and online learning: Using evidence and theory to
improve practice. The Evolution from Distance Education to Distributed Learning.
Symposium conducted at Memorial Union Biddle Hotel, Bloomington, IN. Retrieved
from http://www.aect.org/events/symposia/Docs/InteractionDEnext120510.pdf
In a continuation of a meta-analysis of distance and online education, the authors
explored “interactions: among students, between the instructor and students, and between
students and course content” (p. 2), discussed methodological issues with existing learning
research, and suggest methods for improving online instruction. Interaction is widely accepted
as important in online learning and in the meta-analysis the authors “found the overall positive
weighted average effect size of 0.38 for achievement outcomes favoring more interactive
treatments over less interactive ones” (p. 7) supporting the three types of interaction. Student to
student average effect size was positive and significant at 0.49. Student to content average effect
size was positive and significant at 0.46. Student to instructor average effect size was positive
WattsSEDU7707-8-3
22
and significant at 0.32. Instead of comparing online learning with traditional instruction the
authors indicate that new research needs to focus on pitting “different but compatible types of
distance education technologies” (p. 5) against each other to determine if “larger and more
consistently positive effects” (p. 9) are possible. Four suggestions for future research and
development of knowledge tools were included to identify and enhance the ability for course
designers to “facilitate more purposeful interaction” (p. 29). This article has a wealth of
knowledge in regard to my dissertation regarding history, theory, past journal articles on
interactivity, principles of interactivity and collaborative group work, as well as suggestions for
research and development.
Buch, K., & Bartley, S. (2002). Learning style and training delivery mode preference. Journal of
Workplace Learning, 14(1), 5‐10. doi:10.1108/13665620210412795
In a study of 165 employees from a large financial institution the Kolb Learning Style
Instrument (LSI) and a survey was administered to determine training delivery mode preference.
Correlations were run to determine if there is a relationship between learning style and delivery
mode preference. “Educational research and practice have demonstrated that learning can be
enhanced when the instructional process accommodates the various learning styles of students”
(p. 5). The author’s hypothesis is that “individuals with specific learning styles would have
preferences for specific training delivery mode formats” (p. 5) namely, divergers would prefer
“traditional, classroom-based delivery” (p. 6), accommodators and convergers would prefer
computer-based training, and assimilators would prefer print-based delivery. The authors
developed an instrument to determine learner’s preferred training delivery mode which offered,
(a) computer-based, (b) TV-based, (c) print-based, (d) audio-based, and (e) classroom-based
options. Friedman’s chi-square statistic was used by the authors to test for any differences in
learning mode preference across individuals with different learning styles. The authors also ran
WattsSEDU7707-8-3
23
t-tests “to compare learning mode preferences within each learning style” (p. 8). All learning
styles preferred classroom learning to other types, but partial support was found that diverger’s
preference for classroom learning was stronger than other groups, accommodators preferred
computer-based learning to other forms except classroom learning, convergers also preferred
computer-based learning second, but this finding was not significant, and assimilators’ chose
print-based learning second though it was not significant. The authors suggest that design and
delivery of training should take into consideration the learning style of the learner. They also
found that learners with more experience in computer-based instruction “reported greater
satisfaction than those with less experience” (pp. 9-10). The authors suggest that there is a need
to answer the question, “What can training professionals do to facilitate learning beyond the
classroom? . . . Future research is needed to identify other potential tools and strategies” (p. 10).
Cabrera‐Lozoya, A., Cerdan, F., Cano, M.‐D., Garcia‐Sanchez, D., & Lujan, S. (2012). Unifying
heterogeneous e‐learning modalities in a single platform: CADI, a case study. Computers
& Education, 58(1), 617‐630. doi: 10.1016/j.compedu.2011.09.014
In this quantitative study the authors present a “web-based framework for the creation,
development, and implementation of heterogeneous learning environments” (p. 617) that was
tested on a group of senior college students, and used to promote active learning on any WiFi
compliant device. In a thorough literature review of the different forms of e-learning (i.e.,
collaborative learning, problem-based learning, blended learning, and mobile learning), which is
beneficial to the theoretical framework of my dissertation, the authors determined that “there is
still no e-learning system serving to abstract some generic educational principles and put them all
into practice in a specific e-learning platform” (p. 619). The author’s main suggestion identifies
“the establishment of a generic real-time communications channel between the teacher and the
students” (p. 619) that abstracts “the common aspects of CSCL, PBL or BL systems so that it can
be configured to support all kind of activities” (p. 620). In the study, the goal “was to increase
WattsSEDU7707-8-3
24
students’ participation . . . and to evaluate the impact of a higher interactive environment in the
students’ academic performance” (p. 624). A one-factor ANOVA was performed and three
elements were determined to significantly improve students grades; scores regarding short
questions, scores for problem solutions, and the final score such that the authors determined the
use of the framework to provide “an additional communication channel in the learning
methodology improved students’ academic performance” (p. 625). 81 students were in the
control groups with 27 students in the experimental group.
Fletcher, J. D., Tobias, S., & Wisher, R. A. (2007). Learning anytime, anywhere: Advanced
distributed learning and the changing face of education. Educational Research, 36(1),
96‐102. doi:10.3102/0013189X07300034
In this article the authors introduced the Advanced Distributed Learning (ADL) initiative
designed “to make learning accessible at anytime, anywhere in the world” (p. 96) and consists of
objects that must meet specifications to be accessible, interoperable, durable, and reusable and
are instructional materials developed by industry and government for reuse globally. Currently
the number of these objects associated with ADL is in excess of 10 million and range in size
from entire courses to single graphics or animations. The authors also identified a number of
research projects that need to be conducted regarding ADL, including; (a)
Determine how the Semantic Web can best be used to identify and expose semantic
linkages between bodies of knowledge that may appear dissimilar; to create more precise,
comprehensive, and substantive models of subject matter domains and learner’s levels of
mastery; and to link learner models to appropriate instructional objects (p. 97).
(b) “Determine whether the provision of readily accessible instructional materials by ADL
improves students’ learning or enhances adaptations to their learning needs” (p. 100), (c) Costbenefit studies, and (d) “Whether and to what extent ADL improves students’ learning when it is
used to augment the usual resources available in classrooms” (p. 100). This article provided an
WattsSEDU7707-8-3
25
excellent section on the need for technology in instruction, showing that based on affordability
and cost-effectiveness technology outstrips other forms of instruction. The authors also
presented several studies that showed that there is no difference between distance instruction and
classroom learning, even though students prefer the latter.
Ismail, I., Gunasegaran, G., & Idrus, R. M. (2010). Does e‐learning portal add value to adult
learners?. Current Research Journal of Social Sciences, 2(5): 276‐281. Retrieved from
http://maxwellsci.com/print/crjss/v2‐276‐281.pdf
This study using 1084 undergraduate adult learners in the School of Distance Education
at Universiti Sains Malaysia gauged the benefits of an E-learning Portal. The respondents were
almost equally distributed between Science, Management, and Social Science, with fewer
respondents from the Arts program. A survey was conducted of the respondents who were
randomly selected from learners who had experience with the Portal. The authors state that for
institutions to truly engage in “educational transaction[s using an] . . . electronic medium and
cyberspace . . . requires the development of a virtual learning environment” (p. 276) that they
labeled as an E-learning Portal. The advantages of such a portal are that “learners have access to
education information and materials at any time, from any location” (p. 276) and “allows
learners and lecturers to communicate at any time” (p. 277). The authors also reviewed the
advantages of e-learning which includes flexibility in schedule and learning process,
convenience of time and place, can encourage deeper thought on topics, and diminished costs.
The results of the study confirmed these advantages, but have a weakness in that it only shows
comparative approval of the e-learning Portal, but does not attempt to determine relationships
between the proposed advantages or between those advantages and the demographic data
collected, nor does it attempt to determine if the advantages, while positive, are significant for
specific correlations. The authors do identify how their findings are supported by other studies,
WattsSEDU7707-8-3
26
but only propose new studies to determine how the advantages that had lowest approval could be
improved.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction
does not work: An analysis of the failure of constructivist, discovery, problem-based,
experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.
Retrieved from http://www.tandf.co.uk/journals/titles/0046-1520.asp
The author’s purpose is to demonstrate that several theories of adult learning theory are
untenable from the perspective of cognitive load. The article contains a fair representation of the
history and concepts of constructivist theory. Experience shows that teachers rarely use a strictly
constructivist approach, so this article is building a straw man regarding pure minimally guided
instruction. The article demonstrates effectively the rationale of why direct guidance, which
minimizes cognitive load, is preferred to minimally guided instruction, and concludes that
“minimally guided instruction is less effective and less efficient than instructional approaches
that place a strong emphasis on guidance of the student learning process” (p. 75). The authors
also stated that the “epistemology of a discipline should not be confused with a pedagogy for
teaching or learning it” (p. 83). Based on understanding of cognitive architecture, the author’s
demonstrated multiple studies supporting directed, or guided instruction as opposed to discovery,
or interactive methods, concluding “guided instruction not only produced more immediate recall
of facts than unguided approaches, but also longer term transfer and problem-solving skills” (p.
80). This article will be useful in instructional design, and in techniques that can minimize
cognitive load, and emphasize learning in my dissertation.
Kozub, R. M. (2010). An ANOVA analysis of the relationships between business students’
learning styles and effectiveness of web based instruction. American Journal of Business
Education, 3(3), 89-98. Retrieved from
http://journals.cluteonline.com/index.php/AJBE/index
In a class of upper-class college students, Kozub failed to find a significant correlation
between Kolb’s experiential learning style theory and the efficacy of Web Based Instruction
WattsSEDU7707-8-3
27
(WBI). Kozub used an 2x4 Analysis of Variance (ANOVA) analysis experiment to determine
whether students preferred learning styles, as measured by Kolb’s Learning Style Inventory
(LSI-IIa), dictated how well they did in terms of performance, or likeability, on a classroom
module, that was presented in three different manners – text-only, using WBI, and using
enhanced features of WBI. A comparison was made of the 159 participants and their preferred
learning style, their ACT composite scores, three other in-class examinations, and the score on
the in-class test for this module, and a survey taken after the module to determine their reaction
to it. The findings of this experiment were that while there is a positive correlation between how
well students liked the module and the scores they earned, and between their scores and their
cumulative ACT scores, there is no correlation between learning style, and (a) how the material
is presented, (b) test score, and (c) reaction to how the material is presented. This study supports
the findings of other researchers, notably Hodges and Evans (1983) and Trout and Crawley
(1985) that there is no “predictive utility of the learning styles construct” (p. 96). The author did
state that more research needs to be done to determine if, how, and when the use of
enhancements in WBI would be beneficial to the student. My dissertation will be testing
enhancements in WBI so this article is relevant and supportive.
Martinez‐Caro, E. (2011). Factors affecting effectiveness in e‐learning: An analysis in
production management courses. Computer Applications in Engineering Education,
19(3), 572‐581. Retrieved from
http://onlinelibrary.wiley.com/journal/10.1002/%28ISSN%291099-0542
425 students in 15 class sections that were delivered using e-learning were surveyed on
their perceived learning and general satisfaction, along with several independent variables.
Structural equation modeling (SEM) was used to estimate causal relationships between the
dependent and independent variables, and “used the LISREL 8.50 maximum likelihood method
to measure the fit of the model” (p. 576). This study showed no significant influence on
WattsSEDU7707-8-3
28
perceived learning due to either age or gender. The supported hypotheses in this study were (a)
“perceived learning is positively related to satisfaction in e-learning courses” (p. 573) and a t test
was performed to ensure these did not measure the same factor, (b) “e-learning flexibility is
positively related to perceived learning” (p. 574), (c) “teacher-student interaction is positively
related to perceived learning in e-learning courses” (p. 574), and (d) “student-student interaction
is positively related to perceived learning in e-learning courses” (p. 574). Three hypotheses were
significant in a negative direction (a) “prior experience is [negatively] related to perceived
learning,” (p. 574) (b) “working student status is [negatively] related to perceived learning in elearning courses” (p. 574), and (c) “blended e-learning is [negatively] related to perceived
learning” (p. 575). The key indicator for learning is the relationship between teacher and
student. Blended e-learning was found to be beneficial, but the authors suggest that “the right
level of media involvement must be analyzed” (p. 578) in further research. This article has many
factors in common with the direction of my dissertation topic including student satisfaction and
perceived learning in a professional development context.
Simmons, L. L., Conlon, S., Mukhopadhyay, S., & Yang, J. (2011). A computer aided content
analysis of online reviews. The Journal of Computer Information Systems, 52(1), 43-55.
Retrieved from http://xt6nc6eu9q.search.serialssolutions.com.proxy1.ncu.edu/?sid=
CentralSearch:null&genre=article&atitle=A+COMPUTER+AIDED+CONTENT+ANAL
YSIS+OF+ONLINE+REVIEWS&volume=52&issue=1&title=The+Journal+of+Comput
er+Information+Systems&issn=0887-4417&date=2011-10-01&spage=43&aulast=
Simmons&aufirst=Lakisha
With the large number of electronic Word of Mouth (eWOM) tools available to
consumers it is important to companies to be able to gather and interpret the flavor of these
messages to assist in better decision making. This article describes a system whereby such a
system is created to perform a content analysis of movie reviews. I am not interested in the
practical implementation, but am interested in the underlying research method of the content
analysis and how that was done. The authors argue that performing the content analysis by
WattsSEDU7707-8-3
29
computer “lessens the human resource and increases the reliability and speed of sentiment
analysis” (p. 43).
Content analysis is used to study the content of communication. It involves examining
theoretical definition and empirical measurement. The goal of content analysis is to
create systematic and objective criteria for transforming written text in highly reliable
data that can be analyzed for the symbolic content of communication. (p. 44)
The authors used an “inductive content analysis approach” which is “often conducted by
first examining patterns in data and then seeking to make sense of those patterns” (p. 48).
Independent raters were trained to create a lexicon of words representing positive and negative
reviews with a proportion of agreement between them as .94 (.75 is generally accepted as high
reliability).
So, H.-J., & Bonk, C. J. (2010). Examining the roles of blended learning approaches in
computer-supported collaborative learning (CSCL) environments: A Delphi study.
Educational Technology & Society, 13 (3), 189–200. Retrieved from
http://www.ifets.info/journals/13_3/17.pdf
In this study the authors used a Delphi approach which is “a time- and cost-efficient
method to obtain opinions from experts without physically bringing them together for a face-toface meeting” (p. 191) and is seen as beneficial to (a) “identify current practice and perceived
obstacles,” (b) “draw consensus on ill-defined problems or constructs,” and (c) “forecast future
events and trends” (p. 191). In this study in round one eight open-ended questions were
presented to 13 experts “to generate a list of statements for subsequent rounds” (p. 192). In
round two “areas of agreement and disagreement” (p. 192) were identified among the panel
members, in addition to self-ratings on levels of expertise, and ratings of which statements were
the most important and why. The final round served to focus high consensus which was attained
on most of the 38 items identified as important regarding blended learning. Blended learning
WattsSEDU7707-8-3
30
combines “face-to-face instruction with technology-mediated instruction” (p. 189), and is
encouraged because “simply turning classroom lectures into online learning formats does not
necessarily provide students with the opportunities for rich interactions arising from engagement
in activities that make learning experiences meaningful” (p. 189).
Research Design
Barlow, D. H., & Hayes, S. C. (1979). Alternating treatments design: One strategy for comparing
the effects of two treatments in a single subject. Journal of Applied Behavior Analysis,
12(2), 199-210. doi:10.1901/jaba.1979.12-199
The authors proposed a new research design they called alternating treatments design
which allows for “the fast alternation of two or more treatments in a single subject” (p. 202).
This design allows for good internal validity but critics may question its external validity. It is
through the repetition of internally valid research on additional groups that external validity is
established “rather than statistical inference from groups to populations” (p. 204). Alternate
treatments design counterbalance treatments in random ways and can improve internal validity
by minimizing sequential confounding, which also allows for statistical analysis. Carry over
effects come in two varieties, contrast (the change in “behavior in a direction opposite to that
expected due to a contrast with another treatment” (p. 205)) and induction (a “positive transfer
between treatments with the behavior during one treatment more closely approximating the
behavior during a second treatment than would occur if the treatments were applied individually”
(p. 205)). Alternate treatment designs are used when comparing treatment to no treatment, and
to compare two distinct treatments. Alternate treatment designs have the advantages of (a) not
requiring a withdrawal of treatment, (b) “comparison can be made more quickly than in a
withdrawal design” (p. 207), and (c) the possibility that a formal baseline does not have to be
conducted. Disadvantages to the alternate treatment design are multiple treatment interference,
WattsSEDU7707-8-3
31
and it is “more cumbersome to arrange than the withdrawal design” (p. 208), meaning that
potential confounding variables need to be counterbalanced.
de Anda, D. (2007). Intervention research and program evaluation in the school setting: Issues
and alternative research designs. Children & Schools, 29(2), 87-94. Retrieved from
http://puck.naswpressonline.org/vl=730123/cl=18/nw=1/rpsv/cw/nasw/01627961/v29n2/
s4/p87
To overcome the problems of using intact groups the authors propose:
the use of two quasi-experimental designs, a design that allows the individual to be used
as his or her own control and employs a paired analysis and the Separate-Sample PretestPosttest Control Group Design, which compares outcomes across sequential experimental
and control groups. (p. 88)
In cases where the participant acts as his or her own control, internal validity is improved and
can be compared by calculating individual differences between pretest-posttest and then using a
paired t-test analysis to identify significance. The only threat to internal validity in this case is
history in that another “simultaneous event could not be ruled out as affecting pretest to posttest
changes” (p. 89), but by “applying the intervention to different groups of students at differing
times” (p. 89) this threat can be mitigated, while confounding variables can be identified by an
analysis of covariance (ANCOVA). The authors suggest that combining the previous with a
separate-sample pretest-posttest allows for “intervention in sequential fashion without multiple
testing; . . . [and] control group comparisons using intact classrooms, . . . [offering] crossvalidation of the results of the paired comparison” (p. 90). This combination design “has strong
internal validity, controlling for history, maturation, selection bias, and testing” (p. 90).
Dimitrov, D. M., & Rumrill, P. D. Jr. (2003). Pretest-posttest designs and measurement of
change. Work, 20(2), 159-165. Retrieved from
http://www.phys.lsu.edu/faculty/browne/MNS_Seminar/JournalArticles/Pretestposttest_design.pdf
WattsSEDU7707-8-3
32
The underlying context of this article is rehabilitation services and the use of pretestposttest designs in that field. I am more interested in the discussion of the research design and its
related reliability issues.
Internal validity is the degree to which the experimental treatment makes a difference in
(or causes change in) the specific experimental settings. External validity is the degree to
which the treatment effect can be generalized across populations, settings, treatment
variables, and measurement instruments. . . . Factors that threaten internal validity are:
history, maturation, pretest effects, instruments, statistical regression toward the mean,
differential selection of participants, mortality, and interactions of factors. . . . Threats to
external validity include: interaction effects of selection biases and treatment, reactive
interaction effect of pretesting, reactive effect of experimental procedures, and multipletreatment interference. (p. 159)
The use of a nonrandomized control group pretest-posttest design has the practical
advantage that “it deals with intact groups and thus does not disrupt the existing research setting”
(p. 160) which improves the external validity by reducing “the reactive effects of the
experimental procedure” (p. 160), but potentially dilutes the internal validity because of
“selection and maturation, selection and history, and selection and pretesting” (p. 160). The
authors suggest that for a nonrandomized control group design the comparison groups are not
assumed to be equal on the pretest, and “the data analysis with this design should use ANCOVA
or other appropriate statistical procedure” (p. 163) but if the pretest scores are not reliable “the
treatment effects can be seriously biased” (p. 164) in a nonrandomized design.
WattsSEDU7707-8-3
33
Edgington, E. S. (1966). Statistical inference and nonrandom samples. Psychological Bulletin,
66(6), 485-487. Retrieved from
http://homepage.mac.com/psychresearch/Sites/site2/psy779readings/Edgington1966.pdf
“Statistical inferences cannot be made concerning populations that have not been
randomly sampled” (p. 485). The author suggests that while his procedure will not allow for the
drawing of statistical inferences, there’s nothing preventing nonstatistical inferences to be made
based on “logical considerations” (p. 485). The author demonstrates that with appropriate
sample sizes it does not matter whether groups are randomly selected or not because the
probability (based on the Mann-Whitney U probability table) of selecting all of the high-level or
low-level individuals into a single group is remotely small. By using a t-test and determining
every possible combination of randomizing two groups – if the t test values of the
nonrandomized sample is “a close approximation to the randomization test” (p. 487) a parametric
test can be considered an approximation of a randomized test. This constitutes a great argument
for a nonrandomized sampling under certain specific conditions.
Ellis, T. J., & Levy, Y. (2011). Framework of problem‐based research: A guide for novice
researchers on the development of a research‐worthy problem. Informing Science: The
International Journal of an Emerging Transdiscipline, 11(1), 17‐33. Retrieved from
http://inform.nu/Articles/Vol11/ ISJv11p017‐033Ellis486.pdf
The authors address the “issue of identifying and establishing the research-worthiness of
a [research] problem” (p. 17), by addressing five aspects, (a) the need “for basing research on a
well-defined problem;” (p. 17) (b) what constitutes a research worthy problem; (c) demonstrating
effective problem statements; (d) how to locate problems that are research worthy; and (e) a
“summary and recommendations” (p. 17). The authors demonstrate through the literature that
the problem statement is “a cornerstone for any quality research” (p. 19) and “serves as the
starting point for the research and is a unifying thread that runs throughout all the elements of the
research endeavor” (p. 19). The authors point out that while the research problem is the starting
WattsSEDU7707-8-3
34
point, it is the literature review that serves as the foundation for the research. Regarding research
problems the authors suggested three qualifications which are (a) the problem is active, “the
current state differs from the ideal state” (p. 22); (b) the problem is impactful, it must be
identifiable; and (c) the current solutions are inadequate or nonexistent. To be considered
research the authors identify six means of “creating identifiable new knowledge” (p. 23).
Worthiness is built on “an exhaustive understanding of the body of knowledge related to the field
or topic of study” (p. 24) and encompasses an expansion on or filling of current knowledge in the
field. The authors define the problem statement as “the statement of the problem and the
argumentation for its viability” (p. 27) and offer a template that while simple “is far from
effortless” (p. 27) to write. The template “provides an overview of how the six questions of
what, how, where, when, why, and who, should be addressed” (p. 27). The authors propose that
novices can locate research worthy problems by using four steps; look, read, synthesize, and
consult. The problem statement template is not prescriptive and does not allow a paint-bynumber context, but it does propose a means of succinctly identifying the questions that need to
be answered and the support that is needed for the various parts of a problem statement.
Wright, D. B. (2006). Comparing groups in a before-after design: When t test and ANCOVA
produce different results. British Journal of Educational Psychology, 76, 663-675.
doi:10.1348/000709905X52210
This article focused on nonrandomly select groups assumed to be non-equivalent.
Identified in this article are the “two most common statistical approaches” (p. 663) used in this
situation; t test on the gain scores and an analysis of covariance (ANCOVA). The author states
that
The only procedure that is always correct in this situation is a scatterplot comparing the
scores at time 2 with those at time 1 for the different groups. In most cases you should
WattsSEDU7707-8-3
35
analyse (sic) the data in several ways. If the approaches give different results . . . think
more carefully about the model implied by each. (Wright, 2003, p. 130)
The paradox occurs because different assumptions are made and different questions are asked
corresponding to each test.
t test:
posti = prei + β1groupi + β0 + ei
ANCOVA:
posti = β2prei + β1groupi + β0 + ei
The t test “asks whether the average gain in score is different for the two groups” (p. 666), while
ANCOVA “asks whether the average gain, partialling out pre-scores, is different between the
two groups” (p. 666). If the assumption is that without treatment measures will remain constant
or if the assignment tests and is based on ability then the t test is the best tool, but if the
assumption is that even without treatment measures are changing, or if assignment is from the
pretest then the ANCOVA is the best tool.