Computers & Education 40 (2003) 377–391
www.elsevier.com/locate/compedu
Assessing online collaborative learning: process and product
Janet Macdonald*
Open University in Scotland, 10 Drumsheugh Gardens, Edinburgh EH3 7QJ, UK
Received 7 February 2001; accepted 16 July 2002
Abstract
The assessment of online collaborative study presents new opportunities and challenges, both in terms of
separating the process and product of collaboration, and in the support of skills development. The purpose
of this paper is to explore the role of assessment with respect to the processes and products of online collaborative study. It describes a qualitative case study of staff and students perspectives on two UK Open
University courses which have used a variety of models of online collaborative assessment. The findings
underline the importance of assessment in ensuring online participation, and in supporting the practice and
development of online collaborative learning. They have led to a number of recommendations for the
assessment of online collaborative learning.
# 2003 Elsevier Science Ltd. All rights reserved.
Keywords: Collaborative learning; Computer-mediated communication; Pedagogical issues; Teaching/learning strategies; Distributed learning environments
1. Introduction
The use of communication and information technologies for course delivery and support offers
new possibilities for study which were previously not possible through the traditional model of
ODL course. For campus based universities, it can free staff and learners to be more flexible in
the time when they work. For distance courses like those described here, it represents a new, and
greatly valued opportunity not only in terms of access to fellow students, but also in the chance to
work together.
The pedagogical advantages of online collaborative learning are well known (see for example
Kaye, 1992; McConnell, 1994). But it is equally clear that for a variety of reasons the implementation of this approach is more successful in some cases than in others, and Mason and
* Tel.: +44-131-226-3851; fax: +44-131-220-6730.
E-mail address: j.r.macdonald@open.ac.uk (J. Macdonald).
0360-1315/03/$ - see front matter # 2003 Elsevier Science Ltd. All rights reserved.
doi:10.1016/S0360-1315(02)00168-9
378
J. Macdonald / Computers & Education 40 (2003) 377–391
Bacsich (1998) hold the view that the degree of integration of online collaborative learning within
the course will radically influence its uptake with students.
The integration of a particular teaching and learning strategy within a course must impact on
the design of assessment, because we know that assessment plays a major role in driving student
learning appropriately (Knight, 1995). In fact it is common knowledge that assessment related
tasks attract student attention at the expense of non-assessed tasks. In the light of the new
emphasis on a skills agenda in Higher Education, not only must the assessment be appropriate
to the subject content of the course, it must also have an important role in supporting course
pedagogy.
If we can acquire some understanding of how online collaboration takes place, then it becomes
easier to plan ways of supporting students to achieve competence. At the most basic level, a
knowledge of the tools and the software environment speeds up effective participation in online
conferences, although in recent years this hurdle has become less significant, with increasingly
intuitive interfaces and a greater general knowledge of email amongst students.
However, students still need to learn how to interact online with their peers, and inevitably the
extent to which their interaction contributes to their learning and understanding will vary with
their competency. Indeed Salmon (2000) suggests that there may be a number of progressive
stages involved in online learning, which include access and motivation, socialisation, information exchange, knowledge construction and development. These stages illustrate the interplay
between competence and affective factors such as growing confidence, motivation and group
dynamics. And of course if students are to communicate effectively within an academic discipline,
then they need to become familiar with the language of a discipline and the academic genre. Lea
and Street (1998) maintain that this familiarity with the discourse is a defining factor in students’
abilities to read and write appropriately within a discipline. In fact, this familiarity grows as they
practice writing conference messages on course topics, and reading, and eventually responding to
messages from others.
Finally, if students are required to collaborate to undertake a common task, as opposed to
making optional contributions to a conference, then additional skills must be acquired, including
team working and negotiation skills, group decision making and task management (see for
example, Schrage, 1990). Again, affective issues may be significant here, for example the importance of group cohesion, and the evolution of mutual trust. It follows that the whole process of
online collaboration certainly requires practice, and will take some time to develop.
All these stages in the development of competence in online collaboration may need to be
reflected in the design of the assessment, and how this is done will depend on initial assumptions
on the competence of students at the start of the course. And of course students must develop
their abilities at a basic level, before they can be expected to take on more complex tasks.
The use of computer conferencing for online collaborative work means that the assessment
has a conspicuous advantage over the assessment of face to face collaboration, because the
medium provides a written record of the interactions between students. Although in a face to
face collaborative group members may interact by smiling, or using appreciative or rather less
encouraging grunts, in an online medium they have to use text messages to communicate. This
makes the process of collaboration more transparent, because a transcript of these conference
messages can be used to judge both the group collaborative process, and the contribution of the
individual to that process, thereby overcoming one of the traditional difficulties in implementing
J. Macdonald / Computers & Education 40 (2003) 377–391
379
collaborative work fairly. Of course, this is a description of the procedural aspects of collaboration, and on a more sophisticated level the transcript can be used to judge the extent to which
students were able to engage with each other in negotiating meaning in relation to the course
material.
The other evidence that collaboration has taken place is the product, which may take the form
of an essay or report, or perhaps a website. This can again reflect the individual contribution, for
example each student may provide an individual critique of an online debate, perhaps assessed
individually for the quality of reflection. Alternatively it can be a collaborative product, in which
all students in the group are given the same mark, or it might have both individually and collaboratively assessed components.
This paper evaluates the role of assessment with respect to the processes and products of online
collaborative study, by describing a study of two networked courses at the UK Open University
which employ various forms of online collaborative assessment in support of course aims and
objectives. It then discusses the factors influencing appropriate forms of assessment for online
collaborative study.
2. Networking and assessment. The courses included in this study
The two courses described here both use networking for collaborative learning, but vary in the
degree of its integration with the course.
The first of these, a second level undergraduate course, THD204 ‘‘IT and Society’’ explores
social and technological issues associated with IT developments. It was presented for the first time
in 1995 and is delivered annually to approximately 1500 students.
This course uses First Class to link students with staff and fellow students for tutorial support,
typically in groups of 15–25, in which participation is optional. In addition there are course-wide
conferences to which several hundred students may be joined. These conferences are related to the
content of each block of the course, and course concepts and readings are debated there,
throughout the course. Again, participation for these larger conferences is optional. The assessment of collaborative learning takes place at two distinct points in the course, when students are
organised into groups of four to six, under the supervision of tutorial staff. In this way, online
collaboration at certain stages in the course is required of all students who wish to pass the
course.
The second course H802 ‘‘Applications of IT in Open and Distance Learning’’ forms part of
the Open University’s MA in Open and Distance Education. The course offers practical
experience in online activities and interaction, collaborative work and Internet searching and
was presented for the first time in 1998 to 60 postgraduate students, world-wide. It attracts
students who are either planning, or already concerned with, the design and delivery of online
teaching materials.
Online collaboration forms a central part of course pedagogy, and in the first two blocks of the
course students are involved in activity based collaborative work linked to the assessment, for
which they are placed in groups of 4–6, under tutorial supervision. For the rest of the course they
engage in online discussion with peers, and use conferences for more general course support, but
this is not linked to the assessment and participation is optional.
380
J. Macdonald / Computers & Education 40 (2003) 377–391
3. Methods
The results described here are taken from a larger qualitative case study of perspectives on the
assessment strategy of the first course described above, which was undertaken over a three year
period, with three cohorts of students and their tutors. The results of initial evaluation work were
used to refine the course assessment strategy in subsequent presentations (Macdonald, Mason &
Heap, 1999). Data collection is summarised in Figs. 1 and 2. The findings were compared with a
short study of student perspectives on the second, postgraduate course (Macdonald, 1999).
For the undergraduate course, computer conferences were used to collect feedback from students
as they progressed through the course. Some of these conferences were organised by the researcher,
specifically for the purpose of gathering data. The researcher provided regular summaries of the
views expressed in these conferences, which were posted for comment, thus providing a form of
respondent validation. These conferences were supplemented by data from other conferences which
were established as a normal part of the course. All conferences provided a rich source of
unprompted feedback on issues which students felt were of topical interest, and 800 students
participated over 2 years, which was approximately 50% of the course population in each year.
These data were supplemented at significant points in the assessment process, using flexibly
structured telephone interviews. The interviews were conducted along the lines of a guided discussion, using main headings. The interviewer was free to interact and explore issues with the
students, in order to further define questions, or check the interpretation of remarks by paraphrasing the responses made by respondents. A total of 120 interviews was conducted with students over three years. In addition, feedback from 50 course tutors was collected, again using
computer conferences, and e-mail.
Given the open-ended nature of the investigation, respondents frequently introduced an issue
which had previously not been discussed, or raised interesting points which required further
Fig. 1. Evaluation: Perspectives on Assessment, THD204.
J. Macdonald / Computers & Education 40 (2003) 377–391
381
Fig. 2. Evaluation: Refinements in Assessment THD204.
clarification. It was therefore decided to email supplementary questions after the interview, in
order to provide a channel for further reflection. The replies were generally full and informative,
and the method is recommended for networked courses, as an extremely successful way of collecting well considered responses.
For the postgraduate course, flexible open ended telephone interviews were used to collect data
from 11 students after the end of the course. This was supplemented with comments and reflections recorded by students on the course Bulletin Board System. E-mail questions were also used
in some cases as a precursor to telephone interviewing, and proved particularly helpful for interviewing students for whom English was a second language.
Inductive data analysis for both courses started during participation in the conferences and
interviews, in the sense that common themes emerged. By repeated reading of the conference and
interview notes, together with the follow-up e-mail messages, it was possible to identify a number
of common trends in student behaviour and attitudes. The aim was to derive as full a picture as
possible of the variations in perspectives of respondents, based on the principles of constant
comparison (Glaser & Strauss, 1967). These trends are illustrated with verbatim extracts from
staff and student comments.
Since both courses required students to engage in reflection on the processes of online collaboration, this would certainly have heightened their awareness and may also have influenced the
results.
4. Assessing the collaborative process
Assessment makes a conspicuous impact on online collaborative study, and can be used to support students in their gradual acquisition of various skills; growing familiarity with the discourse of
382
J. Macdonald / Computers & Education 40 (2003) 377–391
the discipline and a parallel growth in confidence. At the most basic level it is effective in
encouraging the participation of students. Even the more reluctant students can be encouraged to
engage in online tasks or structured debate if their participation is part of an impending assignment, and the wording of the assignment also serves to focus tutorial support in an appropriate
direction.
For example, on the undergraduate course described here, students are awarded ten percent in
the first assignment for including a transcript of a message uploaded to an online course conference. This has been successful in helping to motivate students to get online at an early stage in
the course, builds their confidence and rewards them for overcoming the trauma of setting up the
software and inputting the first message. It accords with observations on other courses (see for
example Warren & Rada, 1998).
At the next layer of complexity in online collaboration, assignment four (Fig. 3) has been
employed to help students to develop further in trying to express themselves on a course topic,
and then to practise the art of engaging with, and responding to other students’ contributions as
well as concentrating on their own line of thought. In doing this exercise, in which online activities are linked to assessment, students can learn from their success and failures in working
asynchronously with fellow students whom they rarely, if ever, meet face to face; gain a first hand
understanding of the issues; and reflect on their experiences in the assignment.
The assignment requires students to participate in a structured online debate on a course topic.
They are required to submit five transcripts of their own contributions to the debate, in order to
demonstrate the extent of their participation, together with transcripts of related contributions
from fellow students.
Although students are judged on their individual contributions, the structure of the assignment
also encourages the interactive group process because they are expected to demonstrate an ability
to respond to each other within the framework of the course topic. We know from work on other
courses (Mason & Bacsich, 1998), that this model of assignment has a positive effect not only in
encouraging students to participate, but also on the quality of contributions, as well as being
crucial in developing on-line group working. This observation is supported by the findings here:
The conferencing itself was really stimulating, a positive experience, and the sources were
actually very good, I thought. I tried to read almost everything on the CD-ROM which was
relevant and although it was very heavy going, each time I went back to the conference I had
developed my thinking further. (THD204 student)
Fig. 3. Assessing structured online debate on THD204.
J. Macdonald / Computers & Education 40 (2003) 377–391
383
It’s different to other courses. They were tutor-oriented. This is a lot more student-oriented.
And we are moving forwards as a group. (THD204 student)
The wider conference thing [optional national course conferences] didn’t really do the job in
terms of really taking the subject forward and pulling out salient points. There was too much
chat. At a local level [conferences formed for assignments] they were more useful, but then we
were being marked on it. (THD204 student)
With respect to conferencing/debating, in [assignment 4, Fig. 3] we HAD to use the conference to discuss the various themes and issues. This was because it was an important and
specific part of the assignment, we had to produce five quality messages and they were worth
50% of the marks. Some of us spent quite a bit of time producing our messages to ensure
they were of the right quality. (THD204 student)
So not only does the assignment reward the contribution of the individual to the debate, it also
has a positive effect on the academic content of the debate as a whole, although it has to be said
that participation is fairly transient, and tails off after assignment submission.
It was clear from initial evaluation work in 1997 that students needed time and practice if they
were to develop as effective online collaborators. As a result, the Course Team devised a reflective
exercise in the first collaborative assignment (Fig. 3) for 1998 and subsequent years, to encourage
students to reflect on the process of online collaboration and then to produce a strategy for future
collaboration. In this way, skills for online collaboration could be developed in an incremental
way, using the assessment.
Many students had found the forward planning in this assignment helpful as an opportunity to
reflect on their experiences of online collaboration and as a forum for discussing procedures and
working practices, although many of the problems envisaged did not materialise, and not all
strategies proved appropriate. Their experiences had a positive impact on the outcome of collaboration in a later collaborative assignment (described in more detail in Fig. 5).
We recognised from our earlier exercise that we lost a lot of time in the initial overhead
associated with organising the task allocation, etc, so we tackled this very early and got
ahead. (THD204 student)
Because of our work together on [assignment 4] we had the confidence to contribute, comment and constructively criticise. Knowing that the five of us were actively contributing and
shaping our thoughts and developing ideas gave me, and I think the others in the group, a
great deal of support. Above all, I really enjoyed this aspect of the course, because I felt I
could contribute. (THD204 student)
It quickly became clear to me that the ‘dry run’ of [assignment 4] helped them become very
comfortable about the way that they wanted to interact for [assignment 6]. I would also say
that the part about planning future conferencing. . .may have allowed them time to consider
the errors they made as a conferencing team—errors which they dealt with for [assignment 6].
(THD204 tutor)
384
J. Macdonald / Computers & Education 40 (2003) 377–391
I also think that by the time [assignment 6] came along, each group had been able to get to
know each other. . .. My groups seemed to learn that they could depend on each other.
(THD204 tutor)
The initial collaborative assignment was also important in supporting a growth in group confidence and cohesion, as the groups worked together and learnt to rely on each other, and this is
illustrative of the importance of the affective aspects of collaborative working referred to above.
In this context some groups found that a face to face meeting, audio conference or online chat
discussion early on in the project period helped them to reach organisational decisions, and to
establish a working relationship, prior to working in the asynchronous medium.
The integration of online collaborative activities with the assessment is a central part of course
philosophy on the postgraduate course, and the assignments go a step further than those described above, in that students are required to write their assignment based on source material and
experiences drawn from online activities, rather than simply reporting on the online debate (see
examples in Fig. 4). Again students are given individual marks for their work, although this
implicitly reflects on the collaborative process, as it does in the assignment shown in Fig. 3.
The evaluation showed that most students found this experiential approach successful as a way
of offering practice and reflection on online collaboration, and for some it had immediate practical application within their own online teaching.
They were a really important focus and the activities were a great idea. A tremendous way of
building into the learning experience. (H802 student)
Fig. 4. Two examples of activity linked assessment on H802.
J. Macdonald / Computers & Education 40 (2003) 377–391
385
Fig. 5. Final online collaborative assessment on THD204.
As a result of this experience I have given my own students more practical, reflective tasks. . .
(H802 student)
. . .they were very appropriate, some directly relevant to my work at present: we are experimenting with collaborative on-line learning (H802 student)
In common with the undergraduate students, the fact that the activities were an essential part of
requirements for the assignments was appreciated, because it guaranteed the involvement of all
students. Many students found that it also imposed more exacting time schedules than for conventional distance learning. This observation on the pacing and control effects of online collaborative learning has often been made (see for example Thorpe, 1998). However, the effects must
have been compounded here because the online collaboration was linked to assessment, and so
the strict timetable imposed by assessment regulations was extended to include the timeframe
allocated to online activities.
I agree that the sort of group work we have been doing has removed some of the ‘flexibility’
element that we have all come to expect from DE [Distance Education]. (H802 student)
The ability to keep discussion dynamic are very dependent on all members of a group logging
on and contributing frequently. . .you have to do the theory reading in advance of the activity
starting. (H802 student)
Just by agreeing to do tasks by a certain time suddenly removes you from the cocooned
environment of the independent learner and forces you to engage more closely with the
resources and other people. (H802 student)
Apparently students encountered a number of problems when asked to use their contributions,
and those of their fellow students, as source material for an essay (see the first assignment in
Fig. 4). Goodfellow (2001) describes an online debate from the 1999/2000 cohort, in which both
students and tutors discussed the goals of the assignment and what might count as ‘‘quality’’
386
J. Macdonald / Computers & Education 40 (2003) 377–391
input. Lea (2000) suggests that this was because the task involved the transfer of material between
two academic genres, from an informal debate into a formal academic style of paper. There may
also have been difficulties in defining the relative importance of the individual learning and the
group process. Whatever the reason, it is clear that where students are expected to engage in an
unfamiliar task for an assessment, then they will need very specific guidance in the assessment
criteria in order to judge for themselves whether they have satisfied the requirements. McConnell
(1999) takes this process a step further, by advocating the involvement of students in self and peer
assessment, and also in criteria negotiation, so that students are encouraged to engage in self
judgement, and generally to take control over their learning. Whilst this certainly aligns well with
constructivist teaching and learning approaches, it is difficult to see how it could be implemented
successfully over large population courses, because of institutional requirements for consistency
and replicability.
5. Assessing the collaborative product
Collaborative work is traditionally assessed by means of a collaborative product, and online
collaborative assessment can also include a final product, although it need not necessarily do so.
There are two assessable elements to this, the overall quality of the collaborative product and the
contribution of the individual to the collaborative endeavour. Both elements are included in the
final assignment on the undergraduate course, where students are required to work in small
online groups of 4–6 to produce a report which is a critique of a fictional newspaper article
(Fig. 5).
The written work consists of an individually produced report, attracting individual marks,
together with a summary and conclusion for which everybody in the group receives the same
mark. The online debate is partly procedural, in other words it is a record of agreements negotiated on who is to do what in the group, and when. There is also a course theme related element,
where students are required to contribute their ideas on the scope and content of the collaborative
parts of the report, and may if they wish comment on the individual written work of their peers.
The transcript is assessed for each individual’s contribution, as well as for the group’s efforts.
There are a variety of ways in which individual contributions can be realised and acknowledged
by the assessment, for example, students may contribute in terms of group moderation, or in
responding encouragingly to fellow students, or in locating sources of information. There are
also various dimensions to the quality of overall debate, in terms of the group’s ability to
manage online collaboration and achieve some measure of group cohesion, and also the breadth
of discussion of course themes. In this way the assignment can reward the process of collaboration, including the group effort and the contribution of the individual, as well as the final
product.
Many students found this collaborative assignment interesting and rewarding:
. . .generally the group project and working with the group gave me (a beginner) a real insight
into how IT can work for distance learning, conferences etc, which has been invaluable. It
was also the most enjoyable part of the course and the most interesting as you are working
with others rather than reading and reading and reading! (THD204 student)
J. Macdonald / Computers & Education 40 (2003) 377–391
387
Others found the assignment frustrating, time consuming and inflexible, in line with the comments received from the postgraduate students. Even though participation was a required part of
the course, some students still joined in more enthusiastically than others.
Collaboration in any field is only as good as the willingness of all to collaborate, and their
own sense of responsibility. (THD204 student)
We had a good bunch, but human nature is such that some people get their inspiration at the
last minute and I like to be ahead. How much could you rely on them? It was all left to the
last minute. If we could have got an answer sooner, we could have learnt from each other.
(THD204 student)
At least partially this is due to the ‘leave to the last minute’ kind of attitude prevalent among
our group. At least three members were still writing their final draft 24 hours before the
deadline. . .! (THD204 student)
. . .to be honest, I think we have a kind of schizophrenia about collaborative working: we like
the sharing, the sense of pooling ideas and pick up ideas from others, but we are less sold on
the idea of relying on others. . . (THD204 student)
Of course, if marks are involved for collaborative work, this is the ultimate test of mutual trust,
and underlines the distinction, for some students, between the pleasure of online collaborative
study and the pain of collaborative assessment. Indeed, in an evaluation at the end of the first
presentation year of this course (Kear & Heap, 1999) many students said they would have preferred more marks for the individual contribution to collaborative work, and correspondingly
fewer for the co-operative endeavour. These tensions between individual endeavour and the
requirements of the group process are reflected in observations on collaborative assessment in
campus based environments (McDowell, 1995), and they may be exacerbated in an online context
(see for example Gee, Yamashiro & Lee, 2000).
Although most groups did achieve a consensus in terms of task management and timing, the
extent of further collaboration in these project groups was fairly limited, according to comments
from tutors and some students. Commonly the collaborative sections of the report were the
responsibility of self-appointed co-ordinators within the group, who edited a ‘‘collaborative’’
conclusion from the contributions of individuals within the group. Many students appeared
unable, or unwilling to comment constructively on each other’s drafts, and would have benefited
from more practice and support in this area.
There was pressure to fit tasks to a deadline: collaboration was really more on an organisational level. (THD204 student)
. . .the other approach which we adopted was to use one editor to synthesise the group conclusion and summary. The reason for this was essentially that four separate texts stitched
together would produce an incoherent piece. . .a editor is definitely required for group contributions. (THD204 student)
388
J. Macdonald / Computers & Education 40 (2003) 377–391
. . .the part. . .that worked was the co-production of materials. The need to do this forced the
students to work collaboratively and they did produce results. . .the least successful part was
however constructive criticism of each other’s work. . .they did little to help each other to
prepare good quality contributions to the group work. Several students submitted their
individual components to their conference, but nobody gave any constructive response to any
of this. The replies were generally along the lines of ‘that all looks very good’. (THD204
tutor)
There were times when I felt I was on an uphill task, trying to solicit comment from other
members. I felt I could have done with a lot more criticism of my ideas. (THD204 student)
There are two points to make with regard to these observations. The first relates to the comments on the nature of collaboration, which must reflect the wording, and underlying expectations of the assignment. In this case the assignment encouraged students to organise themselves
sufficiently to produce a piece of collaborative work, but did not necessarily require them to collaborate in a particular way. Students did not comment extensively on each other’s work, but
then this was not an explicit part of the assignment.
The second point underlines the fact that in addition to group management skills there are
other aspects implicit in collaborative work, notably that of peer review. This may relate to
comments earlier on the importance of familiarity with a discourse for competence in effective
writing. There may also have been a lack of confidence on the part of students for whom peer
review was an unfamiliar task. Whatever the reasons there is plenty of evidence to suggest that
students need effective facilitation in order to become competent when learning how to assess
work for themselves (see for example Boyd & Cowan, 1985; Orsmond et al., 1996) or indeed to
judge the work of peers. If collaboration involves peer review, then this may need to be supported
overtly in the assessment.
6. Discussion
These findings reaffirm the importance of assessment in underpinning course pedagogy.
Assessment should play a crucial role in helping students to develop as effective online collaborators. The evidence from both courses described here confirms that more students will participate in online collaborative activities if they are linked to assessment, and that such assignments
may also have a positive effect on the quality and coverage of the online debate itself. Assignments can be used to support the practice and development of online collaborative learning,
whilst at the same time providing a focus for course content. They can be linked with online
collaborative activities, or alternatively used in an incremental way to support the gradual
development of skills.
However, there will still be considerable variation in the extent of commitment of students to
collaboration, and enthusiasm may wane after assignment submission if a collaborative assignment is an isolated incident in an otherwise individually assessed course. In addition, the deadlines imposed by most assessment systems can exacerbate the loss in flexibility imposed by online
collaborative work. Students may benefit greatly from collaborative work, in terms of knowledge
J. Macdonald / Computers & Education 40 (2003) 377–391
389
construction and development, and also appreciate the opportunity for contact with their peers.
However, many are less enthusiastic about collaborative assessment, where it requires them to
rely on fellow students for marks.
How successful had the assessment been in helping students to learn and practice online collaboration? Well, the undergraduates in this study had mastered the necessary IT skills, and were
all confident in inputting messages to a conference on a course topic, although inevitably the
extent to which they were adept at discourse within the discipline was variable. They had learnt
many of the practical lessons on the management of online collaboration, and most groups had
built up a rapport. However some lessons take longer to learn than others: they had just begun to
learn how to build and criticise each other’s contributions, and to engage in peer review, and this
must be a reflection of their stage of academic development, or their relative lack of familiarity
with the discipline.
Arguably the most significant aspect of the assessment of online collaborative working is the
transcript, which opens up new avenues not feasible in a face to face situation. The transcript can
be relevant and useful to assessment in a variety of ways. For example, it can be used by students
as source material for written work. This reflects a move towards the greater use of non-conventional sources on these networked courses, and a greater dependence on constructivist approaches and peer-peer interactions, for negotiating an understanding of course issues. The transcript
can also be used as evidence of participation, which means that it is no longer essential to assess a
collaborative product, indeed it can be side-stepped in favour of rewarding the process of arriving
at, and reflection on the success of collaboration, as well as the extent of the individual’s contribution. In this way the process and product of collaborative assessment can effectively be
uncoupled.
7. Recommendations
These findings illustrate a number of models for the assessment of collaborative learning, which
rely to varying extents on product and process, individual and group marks. Clearly there are
advantages and disbenefits with all models and the priority remains to ensure fairness and consistency whilst recognising that not all students appreciate collaborative work.
The following are suggested directions for the assessment of online collaborative learning.
If course designers wish to give all students an opportunity to experience online collaborative learning, then it is recommended that such activities are tied to the course
assessment.
Online collaborative learning is a complex process, which may include basic IT skills, online
interaction skills; a familiarity with the discourse and a growth in confidence. Course
developers need to address the question of whether skills are to be developed during the
course, or whether it can be assumed that they have already been acquired. If they are to be
developed during the course, their practice must be written into the assessment.
The inclusion of skills development as an integral part of the course and its assessment has
implications for the course content which can be covered, particularly since collaborative
working may impose additional time burdens on students.
390
J. Macdonald / Computers & Education 40 (2003) 377–391
Activity linked assessment can be a useful approach in providing a link between assessment
and online work. The use of a series of linked assessments can also be effective in providing
for the gradual development of skills over the course.
The product of online collaborative work need not necessarily be assessed, but if it is this
may demand the use of additional skills such as peer review, time management and task
negotiation which students need to acquire, and which may again need to be articulated and
rewarded through the assessment.
The design of online collaborative assessment needs to take cognisance of the aims and
goals of the task, and the relative weight to be attributed to the individual learning and the
group processes. These aims need to be clearly set out in the assessment criteria, so that
both students and tutors are able to judge the extent to which they have been met.
This discussion leaves various uncomfortable questions in the air. Clearly, assessment has to be
a vital part of practising and participation in online collaborative study, and activity linked
assessment appears to offer a useful strategy. At the same time there needs to be a balance
between ensuring that students participate, and develop their skills, and encouraging a strategic
or mechanistic approach to study. In addition, online collaborative assessment could lead to a
lack of flexibility created by dependence on the group. If so this will be a poor reflection of the
pedagogy of constructivist courses claiming to offer greater flexibility and opportunities for more
open ended study.
Acknowledgements
I would like to thank Professor Robin Mason and Dr. Nick Heap for assistance in the supervision of this research project and for helpful comments in the preparation of this paper. Thanks
are also due to Dr. Robin Goodfellow and Dr. Bob Glaister both of whom provided much support and critical comment on successive drafts.
References
Boyd, H., & Cowan, J. (1985). A case for self-assessment based on recent studies of student learning. Assessment and
Evaluation in Higher Education, 10(3), 225–235.
Gee, X., Yamashiro, A., Lee, J. (2000). Pre-class planning to scaffold students for online collaborative learning activities. Educational Technology & Society 3(3).
Glaser, B. & Strauss, A. (1967). The discovery of grounded theory: strategies for qualitative research. (Aldine).
Goodfellow, R. (2001). Credit where it’s due. In D. Murphy, R. Walker, & G. Webb (Eds.), Online Learning and
Teaching with Technology: Case Studies, Experience and Practice (pp. 73–80). Kogan Page.
Kaye, A. R. (Ed.) (1992). Collaborative learning through computer conferencing. The Najaden Papers. NATO ASI Series F.
Kear, K., & Heap, N. (1999). Technology supported group work in distance learning. Active Learning, 10, 21–26.
Knight, P. (1995). Assessment for learning in Higher Education. London: Kogan Page.
Lea, M. (2000). Computer conferencing: new possibilities for writing and learning in Higher Education. In: M. Lea &
B. Stierer (Eds.) Student writing in Higher Education: new contexts. Open University Press.
Lea, M., & Street, B. (1998). Student writing in Higher Education: an acedemic literacies approach. Studies in Higher
Education, 23(2), 157–172.
J. Macdonald / Computers & Education 40 (2003) 377–391
391
McConnell, D. (1994). Implementing computer supported co-operative learning. London: Kogan Page.
McConnell, D. (1999). Examining a collaborative assessment process in networked lifelong learning. Journal of Computer Assisted Learning, 15, 232–243.
Macdonald, J. (1999). Appropriate assessment for resource based learning in networked environments. Unpublished
PhD thesis, Open University.
Macdonald, J., Mason, R., & Heap, N. (1999). Refining assessment for resource based learning. Assessment & Evaluation in Higher Education, 24(3), 345–354.
McDowell, L. (1995). The impact of innovative assessment on student learning. Innovations in Education and Training
International, 32(4), 302–313.
Mason, R., & Bacsich, P. (1998). Embedding computer conferencing into University teaching. Computers in Education,
30(3/4), 249–258.
Orsmond, P., Merry, S., & Reiling, K. (1996). The importance of marking criteria in the use of peer assessment.
Assessment & Evaluation in Higher Education, 21(3), 239–249.
Salmon, G. (2000). E-moderating: the key to teaching and learning online. London: Kogan Page.
Schrage, M. (1990). Shared minds: the new technologies of collaboration. New York: Random House.
Thorpe, M. (1998). Assessment and third generation distance education. Distance Education, 19(2), 265–286.
Warren, K. J., & Rada, R. (1998). Sustaining computer mediated communication in university courses. Journal of
Computer Assisted Learning, 14, 71–80.
Janet Macdonald is E-Learning Coordinator for the UK Open University in Scotland. She has taught the Open University course THD204: ‘‘IT and Society’’, and has also worked as a national conference moderator for the course. She
undertook her PhD, which is based on this course, with the Open University’s Institute of Educational Technology.
More recently she has been involved a variety of studies of the pedagogy of networked and web based courses.