Report on the 2003 Presidential Grant Research Project:
Discussion Board versus Traditional Feedback-Seeking Channels -Learning Performance and Underlying Driving Values
Alvin Hwang, PhD.
Department of Management and Management Science
Lubin Business School
Pace University
One Pace Plaza, NY NY 10038
E-mail: ahwang@pace.edu
Tel: (212) 346-1342
Fax (212) 346-1573
April 15, 2004
Report on the 2003 Presidential Grant Research Project: Discussion Board versus
Traditional Feedback-Seeking Behaviors on Learning Performance -- Media
Effectiveness and Underlying Driving Values
Overview
This 2003 President Research Grant Project was initiated to compare the effectiveness of
different forms of feedback-seeking activities in the learning environment. Specifically,
face-to-face feedback-seeking activities among students, from students to professors in
class and outside of class, and virtually through the electronic discussion board were
tested for their impact on learning performance that was measured by multiple-choice test
questions. The different forms of feedback-seeking activities were also tested for
possible relationships to underlying competitiveness and collaborative attitudes.
Results of this study showed that one cooperative and three competitive attitudes
(Hwang, Ang, & Francesco, 2002; King, Simmons, Welch, & Shimezu, 1995) were
related to face-to-face feedback-seeking behaviors. Also, two competitive attitudes were
marginally related to electronic discussion board participation. Participation on the
electronic discussion board was significantly related to performance on multiple-choice
tests. In light of these findings, educators should consider the central role of the
electronic discussion board in a hybrid learning environment where both face-to-face and
electronic feedback-seeking channels are available to students. Also, underlying
influences of both cooperative and competitive attitudes on feedback-seeking behaviors
should be considered in designing learning systems. Other implications of these findings
are also examined in this study.
The study findings have been written in a paper that was submitted to the 2004 Academy
of Management Conference in New Orleans scheduled from August 6-11, 2004. This
paper has been accepted for presentation at the conference.
Theoretical Basis of Research
There has been increasing interest in the use of various electronic medium for classroom
interactions (Alavi, Yoo, & Vogel, 1997) despite very few empirical studies on their
efficacy in enhancing learning performance (Arbaugh, 2000). This study was initiated to
examine the impact of three traditional feedback-seeking channels (Hwang, Ang &
Francesco, 2002) and participation on the electronic discussion board for their relative
influence on learning performance. The influence of competitive and collaborative
attitudes on both traditional and electronic feedback-seeking channels was also tested in
this study.
While traditional feedback-seeking channels such as asking questions of the
professor in class (InAsk), asking questions of the professor outside class (OutAsk) and
asking questions of fellow-students outside class (OutCheck) are known to have
important influences on learning performance (Fassinger, 1995; Hwang, Ang &
1
Francesco, 2002), emerging electronic-based channels such as chat-boards, discussion
boards, email lists, and others have yet to be extensively tested in empirical studies
(Hutchins, 2001). Among the range of electronic medium, the most widely used
interactive channel today is the electronic discussion board. This new medium allows
students to post questions and obtain responses similar to the function of traditional
feedback-seeking channels (Mabrito, 2000). However, there are two unique
characteristics: first, it does not require the physical presence of anyone and second it
does not require the simultaneous presence of anyone for feedback-seeking activities to
occur. Daft & Lengel (1986) recognized this point and drew attention to how such rich
electronic medium allowed both synchronous and asynchronous communication that
facilitated creative input and interactions. These new electronic medium also have
capabilities to easily carry a variety of didactical elements (text, graphics, audio and
video messages) not normally available in a traditional classroom interactive environment
—a tremendous advantage when one considers the effort needed to duplicate such
capabilities in class.
Despite many positive arguments for the electronic discussion board (Alavi, et.al.,
1997), its effectiveness in helping students attain learning performance goals has
remained largely unexamined (Arbaugh, 2000). Students’ motivation for seeking
feedback and consequent choice of feedback channels--whether traditional or the
emerging discussion board, also has not been explored. The issue of student motivation
is important because while on the one hand, some researchers believe that a cooperative
attitude is key to successful learning when using an interactive electronic medium, on the
other, there are researchers who have found students to be quite competitive in the
learning environment (King, Simmons, Welch, & Shimezu, 1995). Thus, whether a
competitive or collaborative attitude would lead to successful usage of the discussion
board is something that has yet to be empirically examined in the research literature.
Objectives of this Study
The objectives of this study were to:
1. Determine the significance of relationships between participation on discussion
board and learning performance as measured by scores on multiple choice
questions;
2. Compare strengths of three traditional feedback-seeking channels (InAsk,
OutAsk and OutCheck) and participation on discussion board for their
differential effects on learning performance;
3. Examine the extent competitive versus collaborative attitudes influence
different feedback-seeking channels and discussion board usage.
Findings
The following are the findings of the paper:
1. The sample for this study was made up of students from seven undergraduate
classes spanning courses such as business strategy, principles of management,
human resource management, and training and development. The classes were
2
hybrid classes, with about half the time spent in traditional classrooms and the
other half in web-based interactions. Class sizes ranged from 22 to 39. The
web-based interactions in each class were carried out through Blackboard, a
discussion board with both synchronous and asynchronous interaction
capabilities. All participants were allowed to participate only once in this study.
Of the total of 232 students in the 7 classes, 215 participated in the study. Four
students were dropped from participating more than once, and another 15 were
eliminated for incomplete data. The final useable sample had 196 students. The
mean age was 22 (SD = 4.3) the lowest age was 19, the highest age was 51, and
the median age was 21. Thirty-one percent of the sample was male. The mean
age and standard deviation of the respondents were similar to those of students
in the business school. However, the proportion of female students in this
sample (69%) was somewhat higher than the proportion of female students in
the business school (62%).
2. A Lisrel path model was used to test the extent competitive or collaborative
attitudes influenced choice of feedback-seeking channels and their consequent
impact on grades. The cooperative and three competitive attitudes (Hwang,
Ang, & Francesco, 2002; King, Simmons, Welch, & Shimezu, 1995) were
tested for possible relationships to all the different channels of feedback-seeking
behaviors. The results showed that two competitive attitudes were marginally
related to discussion board participation. Participation on the electronic
discussion board was significantly related to performance on multiple-choice
tests. Significant relationships from the tested model are shown in figure 1.
Figure 1 Influence of Cooperation and Competitiveness on Normal and
Virtual Feedback-Seeking Behaviors and Consequent Learning Performance
INTENSITY
.12
COOPERATE
.21
.35
-.11
.13 FORUMNO
COMPETE
.24
.28
.-.28
OUTCHECK
.50
.15
KIASU-POSITIVE
.23
MULTICHOICE
.16
.20
INASK
.34
.21
KIASU-NEGATIVE
.26
.34
OUTASK
Bolded continuous lines: t> 1.96; Dotted lines: t > 1.67;
Chi-sq (9df) = 11.44 (P>0.25); GFI = .99;NFI = .98; RMR = .015; n=196
3
3. The first important finding is that the number of topical sessions a student
participated on the discussion board (FORUMNO) has significant impact on
multiple-choice performance. While the average number of times a student
participated in each topical session (INTENSITY) was also tested in this study,
it was not significantly related to learning performance. Although the results
showed that normal face-to-face feedback seeking mechanisms, such as seeking
feedback from peers (OUTCHECK), seeking feedback from professor outside
of class (OUTASK) and inside of class (INASK) did not have significant impact
on learning performance in this study, these variables have been shown to have
significant impact on learning performance in normal learning context that do
not have virtual channels as feedback-seeking options. (Hwang, Francesco, &
Kessler, 2003). Therefore, while the electronic discussion board was shown to
have an impact on learning performance in this hybrid-learning environment, it
is not the sole mechanism that should be considered to meet learning needs,
especially in normal classes.
4. The first competitive attitude, Kiasu-Positive, an attitude of putting in extra
effort so as to get ahead of others was significantly related to feedback activities
of INASK, OUTASK, and also marginally to INTENSITY. These relationships
are meaningful because they reflect the motive of the Kiasu-Positive attitude to
participate as much as possible in every electronic forum and also to seek
feedback from professors to gather the needed knowledge. The lack of
relationship to seeking feedback from peers (OUTCHECK) showed that the
Kiasu-Positive attitude may have a preference for feedback from professors
since they are deemed to have the most correct responses to questions.
5. The second competitive attitude, Kiasu-Negative, an attitude of withholding
information from others so that they do not get ahead of oneself, is marginally
related to FORUMNO. This could be viewed as a “hedge strategy” by KiasuNegative to cover as many topical forums as possible but not necessarily
intensely in any single forum. This “hedge strategy” is logical for the KiasuNegative attitude since by participating across as many topical forums as
possible, the individual is able to achieve maximum coverage of ground and
also “show” others that one is learning while at the same time, by not
participating as intensely, minimizes the risk of giving away information to
others in an electronic discussion forum.
6. The Compete attitude that is focused on desirability of success is significantly
related to INASK. This Compete to INASK relationship probably arise from
underlying beliefs that the professor in class is the one who has the best
knowledge and is likely to lead one to the desired successful learning outcome.
7. In contrast, the Cooperate attitude is significantly related to OUTCHECK and
INASK. The Cooperate to OUTCHECK relationship makes sense since a
cooperative attitude should enhance mutual feedback-seeking activities among
peers. Cooperate is also significantly related to INASK, or seeking feedback
4
from professors in class. This relationship again recognizes the important role
of the professor in providing information to students in class. A fuller
discussion of the findings and implications is attached in the paper for
presentation at the 2004 Academy of Management Conference in New Orleans.
Curricular Importance of Project
Based on the findings of this study, the following curricular implications should be
considered:
1.
2.
3.
4.
In a hybrid learning environment, all students should be encouraged to
participate on the electronic discussion board since it has a significant impact
on knowledge acquisition;
Face-to-face feedback seeking mechanisms with professors inside and outside
of class, and avenues for seeking feedback among peers should not be ignored
in the learning environment as this has been shown to be important in other
studies, especially when virtual channels such as electronic discussion boards
are not available in the learning environment. These face-to-face options
should be made available to students in a formal way so that they could use
them to seek feedback for their knowledge needs.
The influence of competitive and cooperative attitudes on feedback-seeking
behaviors should be considered in both normal and hybrid learning
environments. Both competitive and cooperative attitudes have stronger
influences on normal face-to-face feedback-seeking behaviors and to a lesser
extent on virtual discussion board behaviors. Clearly, competitive individuals
who put in extra effort (Kiasu-Positive) are more likely to use more feedbackseeking options than the other attitudes tested in this study. For such
competitive individuals, the more options given to them, the more avenues
they have to seek feedback and the more they learn. Therefore, there is a
good basis to provide as many feedback-seeking avenues as possible to
students in the learning environment.
For students whose have adopted an attitude of staying ahead by withholding
information from others (Kiasu-Negative), they are likely to “hedge” by
participating across topical sessions but not necessarily intensely in any single
topical session so as to minimize giving away information to others—the
withholding part of this attitude. The findings here shows that even for such
students who try to do the minimum in each session but ensure coverage
across all topical sessions, they will also gain the necessary knowledge. This
implies that participating across all topical sessions is very important for
knowledge acquisition and students should be made to do so, rather than
allowed to participate intensely but only in a few sessions.
Conclusion
This study has revealed the usefulness of the electronic discussion board on knowledge
acquisition across various business subject areas in a hybrid-learning environment. The
5
influence of competitive and cooperative attitudes was also shown to influence choice of
face-to-face and virtual feedback-seeking mechanisms. In light of these findings,
educators should consider how best to institute a range of feedback-seeking mechanisms
rather than limit to a single mechanism in the learning environment.
REFERENCES
Alavi, M., Yoo, Y. and Vogel, D. R. 1997. Using Information Technology to Add Value
to Management Education. Academy of Management Journal 40(6):1310-1333.
Arbaugh, J. B. 2000. Virtual classroom characteristics and student satisfaction in
Internet-based MBA courses. Journal of Management Education. 24(1),32-54.
Daft, R.L. and Lengel, R.H. 1986. A proposed integration among organizational
information requirements, media richness, and structural design. Management Science,
32:554-71.
Fassinger, P.A 1995. Understanding classroom interaction: Students’ and professors’
contribution to students’ silence. Journal of Higher Education, 66: 82-96
Hutchins, H.M. 2001. Enhancing the business communication course through WebCT.
Business Communication Quarterly, 64(3): 87-94.
Hwang,. A., Ang, S., and Francesco, A.M. 2002. The silent chinese: the influence of face
and kiasuism on student feedback-seeking behaviors. Journal of Management
Education,26(1): 70-98.
Joreskorg K.G., and Sorbom, D. 1989. LISREL 7: A Guide to the Program and
Applications. Illinois, Chicago: SPSS Publications.
King, C.S., Simmons, C.H., Welch, S.T., & Shimezu, H. 1995. Cooperative, competitive
and avoidance strategies: a comparison of Japanese and United States motivation,
Journal of Social Behavior and Personality, 10(3): 807-816.
Mabrito, M. 2000. Computer conversations and writing apprehension. Business
Communication Quarterly, 63(1), 39-49.
6