Active learning in multimedia
Empowering students with soft skills and peer assessment
Carlos Cardoso Oliveira
Faculdade de Engenharia da Universidade do Porto, Portugal
colive@fe.up.pt
Abstract
In our course in multimedia systems, students come from a
technical, artistic or educational background. Given the
nature of multimedia and the diversity of the profiles of the
students we decided to use a project-based and active learning
approach.
An e-Learning system provides community support to the
course, the learning of soft skills is promoted and peer
assessment tools are used to help evaluate students’
performance.
The results of the use of these methodologies are a strong
motivation of students to cooperate with team-mates and to
acquire and apply new knowledge during their course and in
their professional lives.
Keywords: active learning, soft skills, peer assessment.
1 Active learning
Nowadays, lifelong learning is a need in most areas, in order
to update and refresh knowledge and skills and keep
competitiveness. Students are also becoming more
demanding on the quality of the teaching and on the
participation in the learning process.
Since 1996 that our University runs Master Courses (MSc.) in
Multimedia, joining teaching staff with knowledge in
engineering, communication design and science education.
The offered courses follow a common path in the first
semester, with further specialization in Technology,
Education or Arts on the second semester and in the thesis
work.
and that was the main reason to use an hybrid learning
format.
1.1 e-Learning System
Projects start with presential sessions covering the
presentation of relevant technologies and applications and
then the media laboratory is intensively used for skills
development, while all the team activities, support and
discussion is supported online using a Learning Management
System – Luvit LMS [6].
The web based learning system give teachers the opportunity
to become facilitators in the learning process and is also the
best way to develop construtivist activities exploring the web
resources [10], as recognized in the following student
statement:
“I think that this pedagogic model demands more from the
teacher, although it is more rewarding as he can focus on his
role of facilitator in the knowledge construction, act as a peer
in the learning process and, in a certain way, also as a
learning therapyst.” MV
In the e-Learning system, each project is provided with a
mission statement, detailing the objectives, a discussion
forum and relevant learning resources. Students are
distributed in groups, which have to publish online the project
results. Figure 1 presents the online environment with the
contents of one of the proposed projects.
In the first semester, an introductory course in Multimedia
Systems has the goal of achieving homogeneous knowledge
and skills among the students. Given the inter-disciplinar
nature of multimedia and the diversity of profiles of the
students we decided to use a project-based and active learning
approach for this course and to promote the building of a true
learning community.
Students in this master course are mainly stressed workers
and some come each week from far away. Being a projectbased learning course, there is a strong need for teamworking
SEFI 2003 Conference - Global Engineer: Education and Training for Mobility
Figure 1: Online Project Support.
302
1.2 Project-based Learning
Proposed project-based learning activities deal with specific
application domain problems where the integration of
different knowledge areas are the key for success [5].
In this semester course, students have to participate in three
different team activities, where technological issues are only
a part of the problem.
The proposed projects are similar for all the groups and ask
for multimedia technical skills, creativity and usability issues.
The resulting effort is large but rewarding, as stated in the
following paragraph by one of the students:
“The way that proposed projects were presented contributed
to a much more significant learning effort, but also to a more
complete and less volatile learning than usually happens.”
AQ
In the 2003 course the proposed projects were the following,
each lasting for three weeks.
- Develop interactive panoramic images using Apple QTVR,
- Create public spots using Flash animation,
- Explore Digital TV webcasting.
1.3 Teamwork
Teamwork performance has a strong relation with the
preferred learning style [1] and so all students were initially
invited to fulfill a quiz that gives them some insight on this
issues (see Fig. 2), although this information was not directly
used to choose group members.
Student groups for each project are choosed by the teacher,
based in the prior knowledge and profile of each student.
Groups have from three to five members and are as much as
possible different in each proposed project.
This forced rotativity of students among groups doesn’t easily
get the initial agreement of the class, but gives recognized
results, as detailed in the following student statements:
“Thanks to the rotativity of the teams, I had the opportunity
to experience different sensibilities and knowledge of my
colleagues, allowing us to produce richer works both in
technical complexity and in its meaning.” JL
“It is relevant to mention that rotativity in the groups allowed
me to better know some colleagues and helped to develop
adapting capabilities to different working procedures and
rythms.” CF
A learning community sense has also developed resulting in
several inter-group collaborations and peer-to-peer learning
procedures, as is illustrated in the following statement:
“I believe that groups are forced to work as they are under
continuous pressure from their peers, but in some situations
an empathy among team-mates has also developed.” LC
Figure 2: Learning Style Quiz.
2 Soft skills
To give students the needed tools for a successful lifelong
learning and teamwork and become the basis for an active
learning approach to our educational subject, we promoted
the learning of soft skills, like teamwork, communication or
critical thinking, and particularly of FITness skills
FITness (Fluency with Information Technology) [7] requires
that persons understand information technology broadly
enough to be able to apply it productively at work and in their
everyday lives, to recognize when information technology
would assist or impede the achievement of a goal, and to
continually adapt to the changes in and advancement of
information technology.
FITness involves three types of knowledge: intellectual
capabilities, fundamental concepts and contemporary skills,
all being essential to master the multimedia information and
communication technologies.
The proposed projects not only require the learning of
technical skills in different applications for media editing and
authoring, but also imply a deep understanding on the use and
implications of technology for the creation of innovative
services and applications.
3 Assessment
Assessment is problematic, as most of the work evolves
inside the team and these are very heterogeneous in nature
and prior knowledge in each of the proposed activities. We
were also aware that assessment had a potential disturbing
effect in students’ performance:
“In my view, assessment and its understanding conditioned
from the start the performance of each one and consequently
the team agreement. With time, students instintively
understood how to work in the group and harmonized the
ideas and effort of each one.” CB
SEFI 2003 Conference - Global Engineer: Education and Training for Mobility
303
Since the 2001 course edition, we decided to use self and peer
assessment tools [3] in the project work to overcome the
problem of a fair judgement from the outside of the group.
Peer assessment results on the three proposed projects are
detailed in Figure 3, where is clearly visible a distinctive
difference to the average in three of the students.
We also believe that students' awareness of the learning
process and involvement in the self and peer assessment give
them the needed responsibility for effective personal growth,
as becomes clear in the following student statement:
“When I was told how this course was supposed to develop I
was sceptic about the eficiency of the methodology and of the
assessment. Soon after the first projects, I realised that the
key issue was to improve our willing to team work, to share
and find common solutions to the faced problems and to use
the potential of each of us. In the end, I felt that I was better
prepared to assess my colleagues than the teacher.” FC
3.1 Peer Assessment
Assessment of group work is based in the criteria defined by
Strom et al. [9], grouped in five clusters of questions: whether
a student attends to teamwork; seeks to share information;
communicates with team-mates; thinks critically and
creatively; and gets along with others.
We soon discovered that we where asking too much detail to
get reasoned answers, and so we adapted and simplified those
criteria to the four questions in Table 1.
Figure 3: Peer Assessment results.
Self assessment was clearly not a concern of the students,
resulting in some large gaps with their peers’ assessment.
Figure 4 shows the difference between the resulting sums of
self and peer evaluation in the three proposed projects.
Student 12 presents the worst results and quitted the course.
Students 3 and 5 kept the bad performance during the course
and received the lowest final classification. Only a few
students were awarded larger classifications by their peers
than their own self assessment.
For each question, students must classify themselves and their
peers in the group as: “good”, “regular” or “bad”, using the
smiles “:-)”, “:-|” and “:-(“ .
In the online form to be filled, the default value for each
question is “regular”, forcing an active change to award a
better or worst performance in the project to the peers.
Communication Competences
Has clear and objective expression, focus discussion in key
aspects, shares information with team-mates.
Behaviour to the group
Doesn't blame or judge the colleagues, ears and considers
others’ opinion, admits ignorance or uncertainty.
Participation in group work
Participates in group meetings and activities, fulfills well all
roles in the group, accepts and promotes the negotiation.
Personal performance
Makes construtive comments, explores polemic visions and
suggestions, is perseverant even in difficulties.
Table 1: Personal Assessment Criteria.
Figure 4: Self (-) versus Peer (+) Assessment results.
We intended to offer a clear feedback to the students about
their rating relative to the average and improve their
perception on their self assessment related with the group
rating [8]. Students received for each project a status sheet,
comparing their self evaluation with the average of their peers
evaluation.
Some students were concerned with the results of this
assessment methodology, as seen in the following statements:
“I think that it is difficult to have a true peer assessment, as
people try to protect themselves and their group.” RB
“In the peer assessment process, students with a technical
background don't feel comfortable with the sudden power of
their team-mates, artist students consider it too much complex
to be worth and students with a profile in education give it
too much importance.” NA
SEFI 2003 Conference - Global Engineer: Education and Training for Mobility
304
3.2 Project Assessment
Students must also classify each group project results using
the three criteria on Table 2. Figure 5 shows the group
assessment results for the three proposed projects.
There are no distinctive differences to the average, as every
group results were unanimously considered good or very
good.
The final classification for each student in the project is
awarded by 40% of peer assessment classification and 60% of
project assessment classification.
The objectives of the project were fulfilled
Mandatory objectives of the proposed project have been
accomplished.
Optional secondary goals were explored.
Project results are friendly and innovative
Figure 6: Performing arts event.
Individual work and presentation are classified using an
online voting system (eVal), as presented in Figure 4.
The team explored innovative navigational, narrative or
interactive ways of doing the project.
The communicational design has quality and is efficient and
concerned with usability.
Technical and content quality
The project is technically advanced, well beyond the
minimum set objectives.
Contents are of superior quality, both in its selection and in
the media processing.
Table 2: Project Assessment Criteria.
Figure 7: Public Presentation voting results.
Each of the three proposed projects accounts for 25% of the
final classification, the individual work is awarded 20% and
the public presentation only 5%. Final grading is awarded by
the teacher based in these assessments filtered by his
subjective perception of each students’ performance.
Figure 5: Group Assessment results.
3.3 Individual Work
We also ask for an individual work to be presented in a public
session, enriching all the community through the share of
knowledge and experience.
The individual work may concern the technology (state of the
art portal), the arts (multimedia performing arts event) or the
educational field (eLearning content development).
3.4 Personal Learning Review
The purpose of the Personal Learning Review [4] is to enable
the student to review and summarise his experience as a
learner on the course and should include reflections on what
the participation in the course has meant to him in terms of
developing the understanding of learning and the learning
skills and critically assess the nature of his participation.
It should present a critical review of the development of
learning and groupwork skills, of the contribution to
collaborative work and of the learning in terms of
engagement with the topics, issues and multimedia
SEFI 2003 Conference - Global Engineer: Education and Training for Mobility
305
information and communication technologies and skills
covered in the course
We asked for this final assessment after gradings were
published and tried to focus in the reflection about the
learning process issues and not on the resulting quality of the
personal learning.
[2] Carlos Oliveira, “Project-based learning: online teamwork
and peer assessment tools”, EUNIS2003 Conference
Proceedings, Amsterdam, (2003).
[3]
J.Brown-Parker, I.Thomas, P.Wellington, “Peer
assessment of student performance: measuring
congruency of perceptions in a multidisciplinary team”,
Research and Development in Higher Education”, 20.
Abstracts of 23rd Annual Conference, Higher Education
Research & Development Society of Australasia,
Adelaide, (1997).
[4]
J. M. Baptista Nunes, Maggie McPherson, M. Rico,
“Instructional Design of a Networked Learning Skills
Module for Web-based Collaborative Distance
Learning”, WBLE2000 Conference Proceedings, Porto.,
(2000).
[5]
Jörg R.J. Schirra, "Computer game design: How to
motivate engineering students to integrate technology
with reflection”, UICEE Conference Proceedings,
Bangkok, (2001).
We therefore expected honest self-assessments, focusing on
both positive and negative aspects of the learning experience:
“In this Master Programme, this was the most stimulant and
also the most stressing course, the one where I made the
biggest investment, mainly giving up of sleeping hours.
Today, I feel there are no impossible things in this field. I can
imagine what I want to do and how to do it, or at least where
to find out how to.” EC
“The development of interpersonal relations and the
possibility to exchange knowledge and opinions with people
with different perspectives of the same problem, due to
different backgrounds and personal and profissional paths,
was for me the added value of this course.” AS
”I feel that I’ve learned a lot and with personal effort, so I
really acquired the knowledge and I'm proud and satisfied
with what we produced.” JL
4 Discussion and Future Work
The results of the use of these methodologies are a strong
motivation of the students to cooperate with team-mates and
to acquire and apply new knowledge during their course and
in their professional lives.
We intend to award less weight in the final grading to intragroup peer evaluation, refine project assessment in order to
better award resulting products and reward honesty in self
evaluation.
Individual presentations will become more relevant and will
be splitted in separate sessions in each area, with different
assessment criteria.
[6] LUVIT Learning Management System LMS.
Available: http://www.luvit.com
[7]
National Research Council report – “Being Fluent with
Information Technology.” (1999).
Available: http://stills.nap.edu/html/beingfluent
[8] Paul Wellington, “Developing core skills – Engineering
students in multidisciplinary projects”, 4th European
Forum for Continuing Engineering Education,
Trondheim, (1999).
[9] Strom, P. S., Strom, R. D., E. G. Moore. “Peer and Selfevaluation of teamwork skills”, Journal of adolescence
[online] 22, pp. 539-553, (1999).
[10] Tom March, “Working the Web for Education - Theory
and Practice on Integrating the Web for Learning”,
(1997).
Available: http://www.ozline.com/learning/theory.html
Online participation statistics available in the e-Learning
System, give an insight of the contribution to the learning
community and will also be weighted in the final grading.
We expect to complete the integration of the assessment tools
with the e-Learning system, allowing the automatic grading
calculation and shortening the time for feedback.
We also intend to automate the group generation, based on
student profile information and in the learning style quiz
results.
References
[1] Carl Eneroth, Cecilia Katzeff, Rasmus Larsson,
“Designing for Individual Learning Styles in Net-based
Education”, WBLE2001 Conference Proceedings, Lund,
(2001).
Carlos Cardoso Oliveira
Carlos Cardoso Oliveira was born in 1968.
He has graduated in Electrotechnical
Engineering (1991) and has an MSc. in
Telecommunications (1995) from the
Faculty of Engineering of the University of
Porto (FEUP).
He is the Chief Executive Officer of the
Information Technology Support Office at
FEUP and an invited teacher in multimedia
systems since 1996.
SEFI 2003 Conference - Global Engineer: Education and Training for Mobility
306