Effects of collaborative and individual learning
in a blended learning environment
Pierre J. van Eijl (p.j.vaneijl@ivlos.uu.nl), Albert Pilot (a.pilot@ivlos.uu.nl) and
Peter de Voogd (Peter.deVoogd@let.uu.nl)
Utrecht University
IVLOS, Institute of Education
Heidelberglaan 8
Post Box 80127
3508 TC Utrecht
The Netherlands
Reference:
Eijl, P.J. van, Pilot, A., & Voogd, P. de (2005). Effects of collaborative and individual learning in a
blended learning environment. Education and information technologies, 10, 49-63.
1
2
Effects of collaborative and individual learning in a
blended learning environment
Abstract
In courses using a virtual learning environment (VLE), some students like to work
together, and some do not. If we give students the opportunity to choose either
teamwork or individual study, how does this affect their marks and their appraisal and
assessment of the course?
This question has been investigated in the context of an English Literature course at
the University of Utrecht. In this course, students work intensively with a VLE, and
attend lectures: a blended learning environment. Previous research has shown that
the pedagogical design used provides a powerful learning environment. This time,
students had the choice of working on the course assignments in small teams (2 – 4
students), or individually. Both groups were compared based on their study results,
and the answers to a questionnaire. Students valued the choice. Mainly those
students with high marks for a previous course, which had a similar pedagogical
design, preferred collaboration. Statistical analysis showed that collaboration resulted
in significantly better marks.
Keywords: virtual learning environment, higher education, pedagogical design,
collaborative learning, blended learning,
3
1 Start of the research
In the department of English, during the past five years, second year (sophomore)
students studied 18th century English literature in a setting combining Face-to-Face
(F2F) with a virtual learning environment (VLE). This course turned out to be very
successful. Both students and teacher liked the course (Van Eijl et al. 2000).
Originally, the course had been composed of a series of lectures and tutorials, a
reading list, and a final exam. Many students had tended to postpone the reading,
and the completion rate at the exam had only been 60%.
In the new version of this 18th century literature course based on F2F and the VLE,
students in its first half worked in small teams of 3 students, and then in the second
half individually. The VLE provided the students with extra content to enrich their
learning, self control tests, weekly graded quizzes, and assignments. The grades for
the quizzes and the assignments replaced the final exam. Each week had a theme,
with a book to read, its author, a literary subject, and social, cultural and historical
background information. For the completion of the quizzes and the assignments, the
students had to study the books of their reading list. Each week, a session was
planned during which students could work in teams, with computers available, and a
possibility to pose questions to their teacher. Questions could also be placed on the
discussion forum (Bulletin Board) of the VLE, where fellow students and the teacher
could respond to them. Each week, the assignments had to be sent in before a
deadline. The evaluation results showed a great deal of appreciation and effort on
the part of the students (and the teacher), a rise of the completion rate from 60% to
more than 90% (also after 4 years).
The evaluation also showed that some students preferred to work in teams and other
students did not. In a following course, 19th century English Literature, with the same
4
pedagogical design, this was turned into a topic for the research described in this
paper: what is the effect on the learning results and the appreciation for the course of
being able to choose either team or individual study?
2 Factors of interest for collaborative learning
2.1 Factors for collaborative learning
Within learning psychology and pedagogy, different theories have been developed on
collaborative learning. This refers probably to the great interest of many people in the
learning potential of students working together. Collaborative learning can be
considered (Dillenbourg et al. 1996; Roschelle and Teasley, 1995) to involve the
“mutual engagement of participants in a coordinated effort to solve the problem
together”. Social-constructive theories (Woolfolk, 2001), emphasize the importance to
the learning process of social interaction. This interaction during the learning process
should promote the achievement of higher cognitive goals. The theory about cooperative learning of Johnson and Johnson (1994) emphasizes the importance of its
organisation in order to make it meaningful and effective. In a review about
collaborative learning, Van der Linden and Haenen (1999) mention that the cognitive
learning results in different subject areas and tasks are better than, or at least just as
good as in individual or competitive learning situations. Later on, also motivation, selfconfidence, and relations between students are promoted. The authors conclude,
however, that the results concerning the cognitive, social, and motivational level do
not always point to the same conclusions, and are sometimes contradictory. They
say that many researchers consider group goals, and the responsibility of the
individual team member, as necessary conditions to arrive at good learning results.
Individual responsibility in collaborative learning reduced the chances for a ‘free ride’.
5
Ros (1994) found an interesting phenomenon in relation to this: it turned out that
students learned more when giving an explanation to another than when receiving
one. It appears that explaining stimulates the organisation of the individual’s prior
knowledge, helps to discover gaps in it, and to establish connections within the
subject matter. Existing structures of knowledge are reinforced, and anchored in the
memory of the student. The same has been demonstrated by Moust (1993) in the
role of the tutor in problem-based learning.
In the case of relatively open tasks, where higher cognitive skills are required,
collaborative learning seems more appropriate than tasks requiring a more routine
approach. Van der Linden and Haenen (1999) conclude that when the task is
complex, group interaction can be a help to arrive at a higher form of thinking. If,
however, a task is very complex, then group interaction might be less productive, and
a form of expert guidance by a teacher is more suitable. The positive results of
collaborative learning are only attained if there is an open learning situation, which
requires that the collaborating students are verbally active, feel themselves safe, and
work on the task from a position of equal responsibility.
In their review on the importance of group size, Lou et al. (1996) point out that
small groups of three to four students appear more effective than larger groups.
Besides this, many educational research projects show the vast differences between
students with respect to study pace, interest, studying habits etc.
2.2 Factors influencing ICT ánd collaborative learning
The aforementioned different research angles offer indications for the pedagogical
design of collaborative learning, but do not prescribe how to combine ICT and
collaborative learning in a specific course to turn the latter into a success. Based on
6
the analysis of four examples of ‘good practice’, Van Eijl and Pilot (2003) formulated
fifteen success factors for the application of a VLE in collaborative learning. These four
examples showed a great variety in the ways the concept of collaborative learning
was being implemented, with variations in the duration of the collaboration using a
VLE, the entry level of the students, the group size, the feedback, and the way
students were being assessed. Given a course using collaborative learning, these
fifteen factors referred to starting up a VLE, consolidating its use, and how to
organize the feedback and assessment with the help of it. Remarkable in these
examples of ‘good practice’ was that only a limited part of their time the students did
study together in groups . Most of the time was spent on independent study and
lectures.
In their review, Sivin-Kachala and Bialo (1994) refer to the software
characteristics significantly influencing the learning process of students, such as: the
degree of control students have of the steps in the teaching-learning process, the
type of feedback, the embedding of a cognitive strategy, and the use of visual
animations. In their literature study about the effects of learning in small groups, Lou
et al. (2001) conclude that, generally speaking, these effects are positive. Yet, the
effects of learning with ICT are dependent on the type of software, the feedback, the
degree of student control of the learning process, previous experience in working with
the computer, and the capacities of the students.
Puntambekar and Young (2003) propose in this context their theory of
computer supported collaborative learning (CSCL), consisting of the main factors:
nature of students’ learning goals, articulation of shared goals, nature of the teacher
facilitation and nature of the context. They consider successful collaborative learning
as a function of shared goals leading to a shared understanding.
7
2.3 Some results of previous research of using a VLE and collaborative
learning by the English department.
On the results of the pedagogical design as described in chapter 1 has already been
reported elsewhere (Van Eijl et al. 2000). The students’ completion rate increased
from 60% to 90%, students and teacher appreciated this pedagogical design very
highly, and the time spent on the course had been more or less the estimated time.
Students liked working in small teams of three students during the first half of the
course. It helped them to get accustomed to a learning environment in which face-to
face working was combined with working in a VLE. However, in a course evaluation
midway, some students expressed doubts about the collaborative learning: it turned
out that some students preferred to work individually, and some wondered what
would happen if the team did not function well, because their marks were dependent
on the team product. Some students also said that collaborative learning requires
extra planning. In addition, some students said that because of the collective team
product, their personal share in the content and their contribution were sometimes
not sufficiently recognized.
A question in the final questionnaire for this course checked students' preferences for
collaborative learning: 59% of the students preferred collaborative learning. This was
motivated as follows: ‘two know more than one’, ‘discussing with other students was
pleasant’, ‘then we did the assignments in time’, 'you get more out of the content’, 'a
free choice of team members is necessary’. 30% of the students preferred individual
leaning, which was motivated with: ‘that gives me a better chance to work at home,
‘then I don’t have to compromise’, and also ‘sometimes it is nice to discuss and
exchange ideas’. 11% of the students preferred a combination of collaborative and
8
individual learning with sayings such as: ‘individual learning during the preparation,
and collaborative learning during the assignments leads to a better formulation of the
answers’ and ‘before the course started, I had expected to prefer independent
learning, but it turned out that I also liked working in a team’.
The teamwork was for the greater part face-to-face, and for a small part virtual, by email and telephone. The tendency in these answers, and from what we recognise in
other courses in this faculty, is that quite a few students like to study collaboratively,
as long as the teams are not too big, they can decide on the choice of their team
members, and the assignments fit collaborative learning. A number of students value
a combination of individual and collaborative learning as something that can be
functional.
After this course, as described here, had been offered with great success for a
few years, the teacher involved decided to implement the same pedagogical design
in a follow-up course on 19th century English Literature, but now with a free choice by
the students to work collaboratively or not. In the research project of this article,
specifically the choice of the students is the central issue, and the results of this
choice in the context of a course using a combined VLE and Face-to-Face (F2F)
environment, so-called ‘blended learning’. This choice reflects the individual
preferences of the students for a specific way of studying. For instance, for some
students collaborative learning can have an extra value, and for others, the
opportunities for individual work, independent of time and place, which a VLE offers
can have an extra value. These lead to the following research question: what effects
does the choice of collaborative learning or of individual learning entail? Or more
precisely: are the learning results and the appreciation of the students who work
collaboratively different when compared with those of the learners working
9
individually? The most important aspects here are the pedagogical design of the
collaborative learning with a VLE, and the assignments (see section 4, figure 1). The
dependent variables are the learning results, and the appreciation of the students.
3 Methods
After the development of this course on 19th Century English Literature, a general
evaluation was carried out to check if this design functioned well, and as intended.
Because of the research questions, the data of the students’ marks were collected in
such a way that collaborating students could be compared with the students working
individually.
In addition, a questionnaire was placed in the VLE posing two general questions to
the students for evaluation, and a pencil and paper questionnaire was used for a
series of closed and open answer questions. And finally, the evaluator conducted
interviews with the teacher during and after the course.
A quantitative, statistical analysis was carried out on the numerical data (T-test and
co-variance-analysis with collaborative/individual learners as independent variables,
the assessment results as dependent variables, and the previous results as a covariants). To be able to correct for the differences in entry level, the marks from the
previous course (18th century English Literature) were used. In this analysis, only the
numerical data of those students, who did both courses, were used. In all other
analyses data on all students were used.
Also a qualitative analysis on the results of the open questions was conducted.
4 The pedagogical design
10
Figure 1 shows the schematic representation of the pedagogical design of the course
on English Literature of the 19th century. Compared with the previous course on 18th
century Literature, there were some important differences:

Students were allowed to choose if they wanted to study as a team or
individually. For many students who were part-time workers or had to travel a
long way to arrive at the institute, this would be a good opportunity to make an
optimal planning for their study.

Students were allowed to choose their team-mates. The maximum team size
was four students.

Besides this, there was a limited possibility for students to choose their
assignments, or even to propose an assignment themselves.
11
•
Input of
information
•
•
•
Instructional
activity
Explanation of subject
Explanation of method
Asking questions
•
•
•
•
•
Informative
meeting
Mini-instruction (ICT)
Written study materials
Hyperlinks (ICT)
Assignments (ICT)
Discussion (ICT)
•
•
•
•
Independant/
Group study
Teacher guidance
(F2F)
Selftest (ICT)
Test (ICT)
Feedback on previous
assignment (ICT)
Assignment individually
and groups (ICT)
•
Session (with ICT)
Deadline
assignment
E-mail
Weekly
sequence
Result
•
•
Acquainted with
new (a.o. ICT)
information and
with each other
Arrangement of
teams
•
•
Contribution to
discussion
Preparation for
session
•
•
•
Assessment of
selftest (ICT)
Contribution to
discussion (a.o. ICT)
Products of
assignments (ICT)
•
Assessment of
assignment
Figure 1. The instructional model of the course
Explanation of the schematic representation of the design
This schematic representation of the design in figure 1 presents an overview of the
planned weekly teaching and learning activities that took place during the eight
weeks of the course. A week started with an informative lecture on the authors that
were the focus of that week. The lecture did not explain the content or books of the
authors, but painted a picture of the period when an author’s books were written, and
how they lived. The teacher also handed the students tips and instructions about
reading the books in a focussed way, and to help in making a literary technical
analysis of it. This lecture aimed at motivating students. During the first lecture,
students formed teams and told the teacher their choices.
12
After this lecture, the individual and collaborative studies followed during which
students read the books, used the VLE, individually completed a quiz posted in the
VLE, and worked collaboratively or individually on an assignment.
Next was a computer session, during which students continued work on their
assignments. These assignments were formulated in such a way that both teams and
individuals could do them. Each week in these courses, students had the option to
choose two out of three writers. Before a specified deadline, the assignments had to
be mailed to the teacher using the VLE. The teacher graded the assignments, and
typed the results into the student records of the VLE. Team members received a
collective mark.
Each student could check the results in the VLE. Each week, on the discussion forum
of the VLE, the teacher published a general feedback on the assignments. During the
computer sessions or through the VLE discussion forum (Bulletin Board), students
could also pose questions.
5 Results
5.1 Delivery of the course
The general picture of the results resembled that of the previous course on 18th
century English Literature. Nearly everybody kept up with the course, as became
visible in the weekly quizzes, and the assignments. Three fourths of the group of 61
students consisted of students in their second year of studying English. Other
students came from different disciplines, most of them Language and Culture studies,
and some Erasmus and foreign exchange students. They were about 20 years of
age, with some exceptions of older students. The ratio female-to-male students was
13
70:30. The collaborating students did not differ as a group from the individual
learners.
The course was carried out in accordance with the planning. The teacher
estimated that ca. 60% of the students participated in the lectures. The computer
sessions (with the VLE) were well attended during the first weeks of the course. After
that, students failed to show up in the sessions, and logged in from other places and
at other times. At the start of the course, teams of two to four students were formed
spontaneously, who then discussed the assignments, and carried them out. The
teacher stimulated the collaboration: 56% of the students joined a team. A
considerable number of students (44%), however, chose to work individually. A few
students chose to propose an assignment of their own. In some cases, the teacher
accepted this proposal, in other cases he did not.
This time, too, a high completion rate was attained (95%), and the students
and the teacher greatly appreciated the pedagogical design. A summary of the
reactions and opinions of 90% of the (61) students on two open questions in the VLE
regarding plus and minus points illustrates this, see Box 1.
Question:
Please, enter below (not for a mark, but because I am curious) those elements of this
course that you found
(a) most interesting, stimulating or worthwhile
(b) least so – in other words: pros and cons of this course, in your personal opinion.
14
In the questionnaire, respondents mentioned a large number (90) of positively valued
aspects of the course, 24 critical remarks, 5 times different learning results were
mentioned, and 13 suggestions were given for the next course. The most positive
remarks (20) were about the content of the course that they found interesting, about
the books, and about the assignments. In 17 remarks, the appraisal of the course as
a whole was mentioned, frequently mentioned as one of the best they had had until
now. 16 times the integration of the course with the Internet was mentioned as a
positive aspect. The efficiency of this course was mentioned 13 times including
deadlines, assignments that could be done during the course and counted for the
final mark, useful suggestions for reading, independent planning of study time, and
working in teams. There were 24 critical remarks. The heavy workload was
mentioned most frequently (15x): too much, and sometimes too large a number of
pages to read (not enough time to read the books completely), and sometimes the
combination with another simultaneously programmed course. Three students
mentioned technical problems with WebCT. Five other critical remarks were about
the relation between the lectures and the assignments: not always parallel, did not
give suggestions for the quiz, and did not add anything to the online information.
Eight times the lectures were mentioned as a positive element: stimulating, a nice
style of teaching and the enthusiasm of the teacher.
Students valued the course in the questionnaire with the mean score of ‘good’.
Box 1. Impressions of students about the course.
15
5.2 Analysis of the assessment results
Comparison of the marks of collaborating students and students working individually
produced the following picture:
Great differences in marks between collaborating and individual workers.
Of the 61 students that participated in the course and were assessed, 58 passed and
3 failed. 34 Students worked collaboratively (56%), and 27 students individually
(44%). The mean mark of the whole group was 7.4 on a ten-point scale, which is
quite well in the Dutch educational situation.
If we compare collaborators with individual workers, a remarkable difference
becomes apparent: the mean mark of the collaborators was 7.69, that of the
individual workers 6.74. The difference between the mean marks of these groups
was 0.95. This difference has been examined in more detail.
Especially high performing students did choose to work collaboratively.
The question is whether the collaboration did cause the high marks. Or is it that the
group of students who had chosen to work collaboratively consisted of (already)
highly performing students, and that this selection had caused the effect we
described? To analyse this, the results of the students in the previous course, with a
similar pedagogical design (English Literature of the 18th century), were collected. 43
of our 61 students had participated in this course. The other students had done a resit of last year, or were Erasmus- and Socrates students, and students of other
disciplines. The 43 students were nearly all regular second year students of English.
These data were used in the further analysis of the results in this project. The
comparison of the marks showed that the collaborating students as a group were the
highly performing students (p=0.001) of the previous course. It turned out that
collaborating was obviously attractive to them.
16
Working collaboratively leads to significantly higher marks.
A next question was if collaboration resulted in a higher mark for these previously
high-scoring students. A co-variance analysis was performed in which, besides
collaboration or working individually, the mark of the previous course (18 th Century
English Literature) was used as a co-variant. The results showed that collaboration
had a significant effect (Bèta= 0.510; t=4.065***), after correction for the entry level.
This shows that collaboration lends an added value to the learning process.
5.3 Analysis of the questionnaire results
The response to this questionnaire was low. The reason for this was that students
took the questionnaire during the final lecture of the course, which took place outside
the regular period, and was not obligatory. All the students in this lecture (18 of the
61 that attended the course) gave their answers to the questions. The question
whether students collaborated with their fellow students on the assignments showed
the following results: 56% of the students had been working together on all
assignments, 44% had only occasionally collaborated, or not at all. This is in
agreement with the ratio of collaborating/non-collaborating students in the whole
group of 61 students.
In the statistical analysis of the data of the pre-structured questions, we limited
ourselves as much as possible to the issue of collaboration or working individually.
We checked to what degree the collaborating and non-collaborating students differed
in their answers to the questionnaire. We applied T-tests with collaboration as the
‘grouping variable’. In table 1, we present the significant relations with collaboration.
Respondents that had collaborated with their fellow students were of the opinion that
they had learned more (t= -3.12; df= 14; p=0.008). They also valued the ICT practice
17
sessions more highly than the students that had not collaborated (t= -2.94; df=15; p=
0.010); and their satisfaction with the assessment system was higher than that of the
non-collaborators (t= -2.39; df= 15; p=0.030). The opportunity to repeat the quiz
(including getting feedback on their answers) had stimulated them to reflect on the
subject (t= -2.21; df 15; p=0.043). Respondents who had not collaborated were of the
opinion that it takes more time to work collaboratively (t=2.69; df=14; p=0.018).
On other aspects of the course that came up in the questionnaire, collaborators did
not answer significantly differently from the non-collaborators. This involved e.g.
keeping up with the course schedule, opinion about difficulty of quizzes and
assignments, appreciation of opportunities to make their own choices, transparency
of assessment, evaluation of lectures, time spent on course work, coherency and
study load of the course. In the questionnaire, two questions were posed about the
time spent on course work: how much time did you spend on reading the literature,
and how much time did you work at the computer. On both questions, there was no
difference between these groups of students.
Table 1
Results of the analysis with collaboration as the ‘grouping variable’
Question in the
Mean mark of
Mean mark of
T-score and
questionnaire
collaboratively
individually
significance
working
working
students
students
(standard
(standard
deviation)
deviation)
18
To what extent did
4.30 (0.67)
3.17 (0.75)
-3.12**
How useful were in 4.30 (0.67)
3.00 (1.15)
-2.94*
4.20 (0.63)
3.29 (0.95)
-2.39*
3.60 (1.07)
2.29 (1.38)
-2.21*
3.60 (0.70)
4.50 (0.55)
2.69*
you learn more
when
collaborating?
your opinion the
meetings to
practice at the
computer?
What was your
opinion about the
assessment
system in this
course?
Did the option to
repeat the quiz
stimulate you to
reflect more?
To what extent is
collaborative
learning more time
consuming for
you?
19
The electronic forum for discussion (for all) was indeed used by the students
and their teacher. Students mainly asked questions on organisational issues, and
sometimes questions on content issues. Some students demonstrated more activity
than others, two thirds of the respondents had been involved in the discussion forum
one or more times. More than three fourths of the students were of the opinion that
the discussion forum was useful or very useful. At some moments, a discussion on
content matter took place. The teacher reacted very alertly in those instances. He
also posted texts containing general feedback and announcements on this forum.
Two thirds of the respondents additionally used their own e-mail system for their
communication.
The analysis of the open answer questions showed the following results:
Extra learning. Students mention as advantages of collaboration ‘extra
learning’ (13 respondents), and saving time to study (4). Examples of ‘extra learning’
are: ‘you get new insights, which you would not have achieved otherwise’; ‘you hear
different points of view and others draw your attention to things you would otherwise
not have noticed’; ‘you can contribute different ideas and arrive at a really good
answer’; ‘you also hear now and then how other students view the subject’, and
‘discussing issues makes you work more actively with the reading materials’.
Disadvantages because of time and planning problems. As disadvantages of
collaboration, students mention the extra time that is needed (5 respondents), the
problems with planning (4), and reaching one collective answer or result (3).
Especially, the individual workers mention these time and planning problems.
Other learning results. As important issues learned by the respondents other
than the course content, students mentioned the way of reading literature (12
20
respondents); a new understanding of their own manner of working (8); literature and
culture of the 19th century; further experiences with the instructional method used in
this course (6); collaboration as such (4); and the use of the internet (2).
Suggestions for improvement of the course. The suggestions students gave
for the further improvement of the course mostly involved a reduction of the large
amount of reading assignments, and the need for more personal feedback. A large
number of students had no suggestions, or wrote a note of encouragement to go on
with this type of instruction (“keep up the good work!”, a student wrote).
6 Conclusions
The instructional design of collaborative learning in combination with ICT proved to
be more than feasible, and produced good learning results. The opportunity given to
the students in this course to choose between working collaboratively and working
individually, in combination with face-to-face instruction, virtual instruction, and
studying from written materials proved to be highly appreciated by the students.
Another conclusion was that especially the group of students with high marks liked to
collaborate. For these students, collaboration provides perhaps an extra stimulus,
just as we have seen in the honours programmes (Van Eijl et al. 1999) taken by
those students who want to do something extra in their curriculum.
Collaboration paid off for those students who had chosen that option: they achieved
higher marks. The analysis of the data further showed students who had worked
collaboratively also found that they learned more.
Students who had worked individually also found that their way of working has
advantages. In particular, they mentioned problems of time and planning connected
with collaborative learning.
21
7 Discussion
In many respects, the instructional model the teacher applied in this course has
proved to function very well. Using this model produced many advantages when
learning results are considered. The lectures in the traditional course were largely
replaced by activating assignments and quizzes, which asked for more reflection on
the content. Because of the formative assessment during the course, the student is
kept on track much better (less procrastination) than in the traditional course with
only one final, summative assessment. Looking at the success ratio, 90% of the
students were successful at the first examination, while previously the success ratio
was not higher than 70% after two re-sits. For the teacher, giving the course was
much more pleasant, and in the end less time consuming than the previous situation.
But this instructional model also brought with it issues for discussion.
Collaborative learning, learning individually, or both? The opportunity for
students to choose between learning collaboratively or individually proved to be
successful: the same high success ratio and high dedication of students, comparable
with the course on 18th century English literature, were realised in this course. Not a
single student mentioned regretting this choice. On the contrary, both the
collaboratively working as well as the individually working students proved to be
satisfied with their choices. Collaborating students experienced important
advantages, and the individually working students, too, proved to value the flexibility
that the Internet gives them in the planning of their time schedule, and in their time
investment. In the questionnaire, not a single student complained about possibly
negative aspects of the collaborative learning, as had been the case in a previous
evaluation of the course on 18th century English Literature (see section 2.2). For that
22
matter, both collaborating as well as individually working students think evenhandedly about collaborative learning, and in the questionnaire students of both
groups mention advantages as well as disadvantages on this issue.
The good learning results of the collaborators can be an argument to oblige the
students to work together. The problem is whether this leads to new problems,
because some students see this obligation as undesirable, or are anxious that bad
collaboration leads to a lower mark.
Differences in time investment. Individually working students, more than
collaborators, found that collaborative learning requires them to invest more time (see
Table 1). Questions in the questionnaire about time spent on the different tasks in the
course, however, do not show this. But this does not exclude the possibility that coworkers used somewhat more time, e.g. for coming to meetings of their team.
Improvement of reading abilities? The teacher did pay extra attention to
reading abilities in this and the previous course (18th century English Literature).
Inquiry about the effect of this attention was not an issue in this research, but in an
open question in the questionnaire (see section 5.3). Asking about learning results
other than content results, 12 (67%) respondents mentioned ‘learned to read books
differently’. The new instructional model that requires students to read much literature
within a short time span, in combination with the teacher’s instructions and the
assignments, did probably promote the other way of reading the literature.
ICT supports now the ‘F2F’-collaboration, but can it replace it? In a previous
study (Van Eijl et al. 2000) with a course scheduled before this one, it appeared that
students who meet during a lecture or computer session show not much need for
digital chat sessions, or to collaborate otherwise in a fully digital manner. F2Fcontacts are much faster and are stimulating for a team of co-workers. However, they
23
found the discussion forum in the digital learning environment useful, and more than
75% of the students also used their own e-mail system.
Remarks on the design of this study. A question not completely answered by
this study involves the question how the individual mark of the students in the
previous course is related to the effect of collaborative learning: is the positive effect
for the high-scoring students the same as for the low scoring students? More detailed
research is needed to come to a more definitive conclusion. As mentioned before, a
drawback in this study was the circumstances that led to a low response to the
questionnaire (30%). That means that the conclusions in section 5.3 should be
considered with caution.
Too much time investment by the teacher? The instructional model used is
intensive not only for the students, but also for the teacher. When all students work
individually, the teacher has to assess in principle 120 essay questions each week. In
collaborative learning, this number is reduced, which is an argument for the teacher
to stimulate collaborative learning. This by the way explains the complaints of some
students about not getting enough feedback – that is a problem of teaching time. The
teacher comments are that he himself can influence to a certain extent the amount of
assessment work. For example, by concentrating on short answer questions, and by
checking the content related aspects. And the collaboration of students greatly
reduced the assessment load.
24
Are online tests vulnerable to fraud? To what extent can the teacher be sure
that, in online testing, the student himself gives the answers and not someone else?
In testing using a VLE, fraud cannot be excluded completely. There are however a
number of methods to reduce chances of fraud:
 Using the student tracking option of WebCT, the teacher can to a certain
extent check whether students really do their work. For teamwork, however, this
is difficult because sometimes only one student is connected to the VLE on
behalf of the whole team.
 The quiz is an obligatory unit. To be successful in the quiz, students have to
study the subject matter. Passing the quiz is necessary to be allowed to do the
assignment in the VLE that counts for the final mark .
 Formulating the assignments in such a way that reflection about the subject
matter is needed activates students. For example: “what passage is the best in
your opinion, and why?”, and not: “what is the key issue in that passage?”.
 Within the teams, a certain degree of social control prevents the phenomenon
of 'free riding'. In the new instructional model, it appeared that within one team
the team members were dissatisfied with one of their fellow students (‘he did not
do enough’). The other members expelled this student from the team.
To the teacher, however, free riding will be a cause for concern, and different
methods of coping with this problem will have to be considered (VisschersPleijers et al. 2001).
Can the effect of collaborative learning be extended to other courses using
ICT? The success factors mentioned above are relevant to the use of ICT in a
course, but not so much to the use of collaborative learning. To what extent can the
25
results of this study be generalised towards other courses with ICT? In our opinion,
not without further discussion. A number of issues should be considered:
- Collaborative learning should pay off in the experience of the student;
otherwise, the method does not work or has a counterproductive result. In this
respect, some elements of the instructional design of the course are important:
not merely a final exam, which was the assessment method in the traditional
course, but assignments during the course that count towards the final mark,
quizzes, and a weekly deadline. For the assignments that were assessed,
collaboration was functional and stimulating, as was shown in the appraisal by
the students, and the remarks they ventured about collaborative learning.
- The number of students in the team was also an important element in the
design, as might be expected from other research (Lou et al. 1996). The teacher
limited the number to a maximum of 4, because in his opinion the assignments
were not feasible for larger teams. The limited size of the team in this course
meant that the students could work intensively and in a task oriented manner,
and not much time was needed for the internal organisation of the team.
- The instructional model of the new course proved to be highly feasible for the
subject of the course. Generalisation of this model to courses in other
disciplines that also demand many reading activities is possible, but sometimes
other achievements will be expected from students, and sometimes teachers
will have other instructional preferences. A number of alternatives are
conceivable using the same instructional model, but with some changes in their
elaboration. Some alternatives that were considered in discussions with the
teachers about the model and the results were:
26
 Production of a paper. It seems to be possible to ask students to produce
some kind of a larger paper, individually or in teams, e.g. by a combination of a
number of assignments. Students can then work on a more substantial,
complex, and possibly more authentic learning task to realise other learning
goals. The same instructional design can be used for other products, which
should be submitted to the teacher through e-mail and/or to fellow students to
get (peer) feedback on an interim version (Van den Berg, Admiraal and Pilot,
2001).
 Next to digital, also F2F-presentations of the results. In a large group of
students (in the current course more than 60), it can be too time consuming to
organise oral presentations for all students. For a somewhat smaller group of
students, however, this might be a good alternative. The students might choose
to give an individual presentation, or to prepare it in their team. For a number of
students, in their preparation, verbalisation and explanation of the subject to
fellow students can be important learning activities. The instructional design
described above can be used for this purpose, scheduling oral presentations in
certain weeks, next to submitting results in a digital form.
Investment in an instructional design with ICT as described presumably only pays off
when the course content does not change too fast: but if so, a VLE is a good
instrument to present the assignments, (self-)tests with feedback, and parts of the
subject documentation (next to books) to the students. This instructional model
proved to be very suitable for courses where subject matter and learning activities
give rise to procrastination of the students: because of the design of the instruction,
they are motivated to study, individually or in a team, a large amount of content with a
27
great chance of success during the full period of the course. Also, the type of learning
objectives is important. The particular objective of this course was that the students
by reflection activities would get a good understanding of the coherence of the many
literary aspects of a period. The instructional model presented in this article proved to
be very successful for this objective.
We thank Wilfried Admiraal, Perry den Brok, Rick de Graaff, and Jacques Haenen for
their support with the interpretation of the data, and for their comments.
Literature
Berg, B.A.M. van den, Admiraal, W. F. and Pilot, A. (2003) Peer assessment in
university teaching. An exploration of useful designs. In Book of Abstracts, ECER
conference. Hamburg, Germany.
Dillenbourg, P., Blaker, M., Blaye, A. and O’Malley, C. (1996) The evolution of
research on collaborating learning. In E. Spada and P.Reimand (Eds) Learning in
humans and machine: towards an interdisciplinary learning science. Elsevier, Oxford,
pp 189-211.
Eijl, P.J. van, Pilot, A., Jong, Y.C. de, Voogd, P.J. de and Janssens, T. (2000) A model
for a website supported course. In Book of 24th International Conference Improving
University Teaching and Learning IUT2000. Frankfurt, Germany.
Eijl, P.J. van, Pilot, A. and Quaak, M. (2001) Collaborative learning and ICT: models
and design principles. In Book of BITE 2001 conference. Eindhoven, The
Netherlands.
28
Eijl, P.J. van and Pilot, A. (2003) Using a virtual learning environment (VLE) in
collaborative learning: criteria for success. Educational Technology 43, 2, 54-56.
Eijl, P.J. van, Faber, G., Jorissen, M.G.A. and Pilot, A. (1999) Improving academic
learning by an honors program. In Book of 24th International Conference Improving
University Teaching and Learning IUT'99. Brisbane, Australia.
Johnson, D.W. and Johnson, R.T. (1994) Leading the cooperative school. Interaction
Book Company, Edina, Minnesota, USA.
Linden, J. van der, and Haenen, J. (1999) Samenwerkend leren; van theorie via
onderzoek naar onderwijspraktijk. In N. Deen et al. (red.) Handboek
Leerlingbegeleiding. Samson, Alphen aan den Rijn, The Netherlands.
Lou, Y., Abrami, C.A., Spence, J.C., Poulson, C., Chambers, B., and d’Apollonia, S.
(1996) With-inclass grouping: A meta-analysis. Review of Educational Research, 66,
423-458.
Lou, Y., Abrami, P.C., and d’Apollonia, S. (2001) Small group and individual learning
with technology: a meta-analysis. Review of Educational Research, 71, 449-521.
Moust, J. (1993) De rol van tutoren in probleemgestuurd onderwijs. Universitaire Pers
Maastricht, Maastricht, The Netherlands.
Puntambekar, S. and Young, M.F. (2003) Moving toward a theory of CSCL. In B.
Wasson, S. Ludvigsen and U. Hoppe (Eds) Designing for Change in Networked
Learning Environments. Dordrecht, Kluwer Academic Publishers, pp 503-512.
Ros, A. (1994) Samenwerking tussen leerlingen en effectief onderwijs. De invloed
van de leerkracht. Rijksuniversiteit Groningen, Groningen, The Netherlands.
Roschelle, J. and Teasley, S.D. (1995) The construction of shared knowledge in
collaborative problem solving. In C.E. Malley (Ed) Computer supported collaborative
learning. Springer-Verlag, Berlin, pp 69-97.
29
Sivin-Kachala, J., and Bialo, E.R. (1994) Report on the effectiveness of technology in
schools, 19990-1994. (ERIC Document Reproduction Service No. ED 371 726.)
Visschers-Pleijers, A.J.S.F., Mulders, D.J.W.M. and Wouw, H.M.W.J. van de (2001)
Meeliftgedrag bij samenwerkend leren. Onderzoek van Onderwijs, 30 (3), 38 – 42.
Weert, T.J. van, and Pilot, A. (2003) Task-based team learning with ICT, design and
development of new learning. Education and Information Technologies 8, 2, 195 214.
Woolfolk, A.E. (2001) Educational Psychology. Allyn and Bacon, Boston, USA.
Authors
Pierre van Eijl is a senior educational consultant of higher education at IVLOS,
Institute of Education, at Utrecht University, The Netherlands
Albert Pilot is professor of Curriculum Development at IVLOS, Institute of Education
and professor of Education of Chemistry, Faculty of Chemistry at Utrecht University,
The Netherlands.
Peter de Voogd is a professor of English Literature at the Faculty of Arts, at Utrecht
University, The Netherlands.
Address for correspondence: Drs. P. J. van Eijl, Utrecht University, IVLOS, P.O. Box
80127, 3508 TC Utrecht, The Netherlands, tel. +31-(0)30-2531968, e-mail:
p.j.vaneijl@ivlos.uu.nl
30