1
Candidate’s Evidence of Research & Creative Activities
Allan C. Jeong
In the field of instructional design and distance learning, interest in the pedagogical
applications of computer-mediated communication (CMC) to support collaborative learning
(CSCL) continues to grow with the ever-increasing array of social and communication
technologies now available on the Internet. The significance of CMC and CSCL research has
grown due to the increased use of networked computers and social software within education
and the workplace, the rapid pace of change in today’s world, and the increasing reliance on
collaborative work and problem-solving. Even though the advantages of CMC are commonly
recognized by both researchers and practitioners (e.g., more time to read and compose
substantive responses, reduces social inequalities by diminishing the salience of physical and
social cues), we still have not achieved a full understanding of CMC and its potential because
we have yet to examine how characteristics of the message, messenger, response, responder,
collaborative tasks, and discussion environment affect the processes of critical discourse (e.g.,
argument-challenge-explain) and how changes in discourse processes ultimately affect
learning. We also lack adequate tools and methods to sequentially analyze, computationally
model, and visualize the complex discourse processes that support group learning, decisionmaking, and problem solving.
To meet these challenges, I have
established a programmatic and integrated
line of research centered on developing the
next generation of software tools and
methods to enable researchers to take a
micro-genetic approach to analyzing socialcognitive processes in group discourse,
measured in terms of message-response
sequences that promote/inhibit critical
thinking (e.g., claimchallengeno response
vs. simple agreement vs. counterchallenge
vs. explanation). To this end, all of the
papers I have published in peer-reviewed
journals stem directly from the software
tools (see “ForumManager” and
“Discussion Analysis Tool” in section “Other Scholarly Activity) and computational
algorithms I developed to address the questions examined within each paper.
In this statement, I describe how I have used ForumManager and Discussion Analysis
Tool (DAT) to conduct programmatic and integrated research that seeks to: a) determine
which and to what extent certain characteristics of the messenger (gender, intellectual
openness, learning style), the message (message function, conversational vs. expository style,
intensifiers vs. qualifiers, response time, day of posting), and instructional environment
(prescribed conversational scripts and message tags, pre-structured/unstructured discussion
threads, imposing constraints on message-reply sequences) help to elicit the types of
responses or speech act sequences that produce and sustain critical inquiry; and b) produce
2
visual diagrams of stochastic models to concisely convey how specific factors/characteristics
affect the processes of critical discourse.
Overview of Scholarly Accomplishments
Before I describe my research in detail, here is a brief summary of my accomplishments.
I have a total of 14 published papers or in press (8 single authored, 4 first author), two
submitted papers (both single authored), and four manuscripts in progress (see section
“Other Scholarly Activity”). Six of the papers are published in ISI listed journals – two of
which are published in the number 1 and number 4 ranked educational journals (ranked on
impact factor) listed in the ISI 2006 Social Science index. Five of my papers are published in
journals of international stature. All of the papers I have completed here at FSU are the
products a programmatic research agenda intended specifically to determine ways to measure
and predict how students respond (or do not respond) to messages based on who, when, why,
and how messages and responses are posted in online discussions. In addition, I have 22
refereed conference papers (plus four accepted papers), and have received three invitations to
write book chapters (see “Chapter invitations” in section “Selected Unsolicited Letters”).
These are all indications of how my work has achieved international recognition, and how
researchers find my methods to be unique and an insightful approach to micro-analyzing
where the action is and understanding how discourse supports learning.
Foundational Papers & Creative Works
My first publication, “Sequential analysis of group interaction and critical thinking in
online threaded discussions” (Jeong, 2003), illustrates how DAT was used to sequentially
analyze message-response sequences produced in online debates with business students in a
business ethics course. This paper was published in the premier, highly esteemed, and
internationally recognized journal in the field of distance education, the American Journal of
Distance Education. It was ranked first among all distance learning journals in a 2004 survey
conducted with faculty in Instructional Systems and sister programs. The articles in this
particular journal (from 2001 to 2003) were cited most frequently (n =130) by nine other
distance learning journals among all articles cited in the ten distance learning journals (based
on a 2003 survey conducted by students in my online course, Introduction to Distance
Learning). According to unsolicited emails from professionals, three invitations to write
book chapters (see Selected Unsolicited Letters), and the papers of other researchers in the
field of CMC and distance learning (see “Citation Counts” in Other Scholarly Activity), this
and subsequent studies has been recognized as seminal work that presents a unique approach
to gaining new and important insights on the role of social interactions in online discourse.
Another foundational paper, “A guide to analyzing message-response sequences and
group interaction in computer-mediated communication” (Jeong, 2005), was published in
Distance Education, a highly esteemed journal ranked third among the distance learning
journals (2004 faculty survey) and ranked number 2 in number of times cited in nine other
distance learning journals (n = 55) based on the 2003 survey. This paper describes how to use
DAT to sequentially analyze and determine how characteristics of the message, messenger,
responder, and instructional context help to trigger or elicit responses that either
support/inhibit the critical discourse. The guidelines presented in this paper were formulated,
refined, and based on the methods developed during the time I conducted studies on the
effects of gender, conversational style, response time, and dialog scripts and message tagging
- where students are restricted to posting specific types of messages and required to tag their
3
postings to identify the primary function of each message and responses to messages (e.g.,
argument, challenge, explanation, evidence).
The software program I developed, the Discussion Analysis Tool, was enhanced and
released to the general public in the summer of 2002 (see “DAT” in section “Other Scholarly
Activity”). One of the main enhancements enables users to convert response probabilities
(e.g., the percent of responses to arguments that are challenges vs. explanations vs.
supporting evidence) into state diagrams to visually convey and provide a Gestalt view of
response patterns produced in group discussions (see illustration on page 1). I was awarded
the First-Year Assistant Professor research grant (sponsored by Florida State University) in
2002, which provided full summer salary to further develop the software. I was also awarded
a COFRS summer grant (sponsored by FSU) in 2003, and the CERPS summer grant in 2006
(sponsored by the College of Education at FSU) to write NSF grant proposals to fund further
software development. Additional enhancements were recently made to help users compute
“mean response scores” in addition to response probabilities (e.g., mean number of
challenges, explanations, and evidence elicited per argument).
I also developed ForumManager (see section “Other Scholarly Activity”) to download
threaded discussions from the Blackboard course management system. Once discussions are
downloaded, ForumManager can be used to assess group performance on criteria such as the
percent of messages that elicited responses, number of threads, and the average length of
discussion threads. It can also be used to assess individual performance on criteria such as the
number of postings, replies, number of replies elicited by each students’ posting, number of
times each student reciprocates a reply, number of posting days, and number of postings
containing user-specified keywords like “but” and “however” (keywords that might serve as
indicators of critical thinking). The program also pulls out the message tags (e.g., ARG, BUT,
EXPL, EVID) inserted by students in the subject lines and records the position of each
posting within a thread (thread level). This data can then be loaded into the DAT software to
sequentially analyze and identify patterns in message-response exchanges.
Selected Papers
The three selected papers I describe below (see section “Selected Evidence”) are
studies I have published in highly esteemed peer-reviewed journals. These papers best
illustrate the potential value and insights gained from using sequential analysis (and DAT)
to examine computer-mediated discourse. The first paper, “The effects of conversational
styles of communication on group interaction patterns and argumentation in online
discussions” (Jeong, 2006), is published in Instructional Science (ISI journal, ranked #4
on impact factor among the 100 educational journals listed in the ISI database, 30%
acceptance rate, impact factor = 1.81). This paper demonstrates how DAT can be used to
create computational models to help us understand how characteristics of the message text
(the presence of conversational language like greetings, addressing participants by name,
emoticons, and signatures when used in the absence of non-verbal cues) affect how likely
or how often students respond to messages (with challenges or counterchallenges in
particular) in ways that promote and sustain critical inquiry, and identifies the specific
behaviors that instructors should encourage and discourage to raise the level of inquiry in
group discussions (effect sizes ranging from .00 to +0.74). In a similar study (Jeong, 2005)
where I examine the characteristics of the message, I compared the number of responses
4
elicited by arguments when arguments were presented with qualifiers (e.g., I think, maybe)
vs. intensifiers (e.g., always, never) vs. without qualifiers nor intensifiers (stated factually).
The second paper, Scaffolding Collaborative Argumentation in Asynchronous
Discussions with Message Constraints and Message Labels (Jeong & Juong, 2007), is
published in Computers & Education (ISI journal, impact factor = 1.085, 10% acceptance
rate). This paper provides a thorough introduction to the use of dialogue scripts and message
tags to facilitate online discourse. The study’s findings suggest that message tags alone do
not improve and at times can hinder performance (effect sizes ranging from -.98 to -1.88).
However, the real advantages of using message tags are achieved when the tags and software
programs like DAT are used to assess and diagnose the quality of exchanges produced in
online debates and threaded discussion boards. This paper also serves to illustrate some
potential methods (message tagging) that can be used in online discussion to generate
sufficiently large data sets needed to conduct sequential analysis, and how DAT can be used
to investigate and gain deeper understanding of how dialogue scripts affect the dialogic
processes that produce and sustain critical inquiry. Using the same approach, I have
published three additional papers that examine the effects of the inherent characteristics of
asynchronous threaded discussions - the effects of pre-structuring discussion threads (Brooks
& Jeong, 2006), short vs. long response times (Jeong, 2004), and early-in-the-week vs.
weekend postings (Jeong & Frazier, in press).
The third paper, “The effects of gender interaction patterns on student participation in
computer-supported collaborative argumentation” (Jeong & Davidson-Shivers, 2006), is
published in the journal Educational Technology, Research, and Development (ISI journal,
8% acceptance rate, impact factor = .364 in the ISI 2005 Social Science Edition). This
journal was ranked the highest by 105 faculty within our field in 2005. This paper illustrates
how my tools can be used to determine how characteristics of the messenger and responder
(male vs. female) affect the types of message-response exchanges that are produced by
participants in mostly male discussion groups. This study reveals important insights into the
complex social-psychological nuances that shape social interactions, and insights into the
types and frequencies of exchanges produced between participants of the same vs. opposing
gender (reported effect sizes ranging from +0.12 to +0.32). I also completed three other
papers that examine how characteristics of the messenger and responder affect social
interactions. These studies examined the effects of gender in mostly female discussion
groups (Jeong, 2006), intellectual openness (Jeong, in press), and active vs. reflective
learning style (Jeong & Lee, 2007).
Synergy between Research and Teaching
I am fortunate that my area of research is also the area in which I teach. Large portions of
the data examined in my investigations are generated from the courses I teach online. As a
result, I have been able to apply what I learn from my investigations to make incremental
improvements in the design and implementation of my online class discussions and activities
in my online courses. In addition, I am using DAT and ForumManager in my courses to
assess, diagnose, and improve my students’ ability to engage in more critical discourse in the
weekly discussions and debates. Students in my online course, EME6635 Designing for
Online Collaborative Learning, study and apply my research findings to design and develop
their own computer-supported collaborative learning activities. My students have expressed
to me that they find my class discussions to be better organized and more engaging than the
5
discussions they have had in other online courses, and that the debates have been valuable in
teaching them how to think more systematically and more critically when solving problems
outside of class.
My research has also directly benefited nine doctoral and one master’s student who
worked with me in research apprenticeships and directed independent studies. The
collaborative work has resulted in five papers (three published, one in press, one in review),
five peer-reviewed conference papers, and three accepted conference papers (see “Refereed
article published” and “Refereed conference papers” in the Vitae). For example, one student
in sports psychology worked with me to sequentially analyze dialog between doubles tennis
partners to identify communication patterns that improve team performance and point/match
outcomes. A student in mental health counseling used my tools to identify patterns in
decision-making processes that determine how obese vs. non-obese subjects choose items
from a McDonald menu. Another student in information studies will use my tools to identify
and compare patterns in eye/gaze movements on web pages between Americans vs. Koreans.
These projects demonstrate how my methods and tools can be used across disciplines.
Synergy between Research and Service
My unique approach to studying social interaction, and my recognition as a scholar in
online learning and online discourse, has earned me a position in two editorial boards for
Educational Technology, Research & Development (the most prominent journal among
faculty and practitioners in the area of instructional design and instructional technology) and
Quarterly Journal of Distance Learning. I also review manuscripts for seven other journals:
Journal of Computer-Assisted Learning, Computers & Education, The American Journal of
Distance Education, Tech Trends, Journal of Distance Learning, and IEEE Transactions on
Professional Communication. In addition, I have been invited on several occasions to
demonstrate my research and tools to distance technology support staff at Academic and
Professional Services at FSU. At this time we are continuing to discuss possible ways to
integrate my tools with the Blackboard course management system and disseminate the tools
to teaching faculty here at FSU (see “APS letter” in “Other Scholarly Activity”).
Looking Ahead into the Future
Because sequential analysis has been claimed by some to be the missing factor in the
quest to understand “interactivity” and the role of social interaction in human learning and
behavior, I believe that my research will have broad applications across multiple disciplines
– particularly in the areas of educational research, cognitive psychology, instructional
technology, human-computer interaction, human communications, social psychology, and
sociology. In addition, my research addresses some of the goals of NSF and its cyberinfrastructure initiative to develop data analysis and visualization tools to support
educational research (see “Grant Proposal Pending” in section “Other Scholarly
Activity”). My research also shares some of the goals of the UK e-Social Science initiative
and the European interests in methods for analyzing CSCL, particularly the Kaleidoscope
Network of Excellence, which involves all the major educational research labs in Europe.
With respect to these goals, I look forward to applying my research to the large-scale
development of cyber tools to support educational research and assessment.
___________________________________
Allan C. Jeong