1
Interaction analysis in IBL courses: How can we help to
improve the educative process?
Santos-Fernández, R.1, Rodríguez-Triana, M.J. 1, Gómez-Sánchez, E. 1, MartínezMonés, A. 1, Carramolino-Arranz, B 1.
1
GSIC/EMIC Group, Campus Miguel Delibes. University of Valladolid, 47011
Valladolid, Spain
1. Introduction
The Computer Supported Collaborative Learning field is concerned with how
students construct knowledge in collaboration: the analysis of interactions among
students is a key issue in understanding how learning occurs. (Koschmann, 1996). The
more we know about how students interact, the better we can assess the learning
process.
We, the members of the GSIC-EMIC transdisciplinary research team, have spent
the last decade in promotion of active collaborative learning methods in higher
education mediated by technology. (Koschmann, 1996).
In these processes it is critical to give assistance to both teachers and students,
providing methods for reflecting on, monitoring and evaluating the learning processes.
This relates to one of our main research focuses, centered on computer-supported
interaction analysis (IA) methods. We have worked on different aspects related to this
area, and have produced tools, such as SAMSA (System for Adjacency Matrix and
Sociogram-based Analysis) (Martínez, Dimitriadis, Rubia, Gómez-Sánchez, 2003). This
tool was designed and developed with the aim of automating social network analysis
(SNA) as part of a more comprehensive approach for the formative evaluation of
participatory aspects of learning in CSCL (Martínez et al, 2003). The data stems from
two sources: direct observations and computer logs representing the interactions among
participants in a CSCL experience. This tool, SAMSA produces several SNA indexes
and enables the visualization of the resulting networks as sociograms.
In the past we have had many issues integrating data coming from the face-2face classroom observation reports with automatic data coming from Learning
Management Systems. It was incredibly time consuming to manually introduce this
data into SAMSA. Because of this, it was near impossible to integrate both types of
data for instructors to use for assessment purposes.
2
Last year, to overcome this situation, we developed ILOCA (Interactive
Learning Observer for Computer Analysis). This application allows an easy recording
of interactions taking place in a face-to-face setting by using a Tablet PC. The
application then generates a log report that can be easily integrated with SAMSA.
This paper describes the process followed to provide feedback to instructors and
students in an undergraduate course. We will address some of the advantages and
disadvantages found with combining the use of SAMSA and ILOCA for assessment
purposes; at the same time are working to deepen our understanding of how to better
serve participants. (no creo que quepa en el paper)
2. Hands on
The focus of this experience is a mandatory undergraduate ICT (Information and
Communication Technologies applied to Education) course at the College of Education
in the University of Valladolid (Spain). Thirty six students participated in the course
which was taught by one instructor.
Two different but complementary approaches were put in practice in the design
of the course. First, it is based on principles of the Computer Supported Collaborative
Learning (CSCL) (Koschman, 1996) field, using technologies to support the
collaborative learning process. Second, it follows an Inquiry based Learning
methodology (Bruce, 2000). The course is divided into four cycles, each spanning two
to six weeks. During these cycles, the students collaboratively create artifacts and then
reflect on how to integrate them into the curriculum of Spanish education schools. Each
cycle is divided into five phases, according to Bruce’s model of IBL: Ask, investigate,
create, discuss and reflect. (Incluir transparencia con el cuadro de Alejandra). Although
students were asked to work in groups of 2 or four, they were encouraged to interact
with other groups during lab sessions.
The most relevant tool used to support this whole process was a Wiki-based
environment customized for the needs of the course. The system included a pdf
visualizer, a concept mapping tool, a chat, etc (Incluir transparencia)
The following describes the research that took place within this context.
In this scenario an observer tracked student-student and student-teacher
interactions using Iloca over the course of 15 different sessions (8 one hour sessions,
3
and 7 two hour ones). Then, the gathered data was introduced into Samsa with the aim
of generating sociograms to represent the interactions. These sociograms were analyzed
by the instructor and then used as a platform for discussion with the learning pairs.
The whole cycle was repeated after the pair discussions in order to identify
possible changes among participant interactions. The last step of the process was a
focus group with 7 volunteer students to discuss the new sociograms generated. The
initial pair discussion sessions as well as the final focus group were recorded and
analyzed.
(hacer transparencia clara explicando el flow)
3. Findings (Learning from our own practice)
After the analysis of the interactions gathered during the observation of the
eleven first sessions of the course, we obtained a set of discouraging sociograms. This is
an example from the third observed session:
Figure 1: Sociogram from the 3rd observed session (03/11/2009). Face-to-Face
interactions
4
We were absolutely stunned for a number of reasons:
a) First of all, the instructor was in the middle of the net playing a central role
within the group, when the intention was that he serve as mediator.
b) We also noticed that there were only a few interactions among students
participating in the course.
c) And also that many students were not interacting with others at all. (as seen in
the left side of the sociogram)
With the aim of deeper understanding in what was happening in the course, we decided
to remodel the initial design of the experience. To do so, we analyzed the work done by
students in the wiki platform. This analysis allowed us to understand that collaboration
was occuring, but virtually rather than in the classroom. We decided then to
complement sociograms generated from the observations with interactions among
students automatically gathered by the Wiki platform. (pantallazo… wiki)
The following example shows the face-to-face interactions --in the same session-- as
well as the virtual interactions among students. In this session students were asked to
generate a set of recommendations for the integration of ICT´s in a particular school.
Red arrows show the comments and reflections done by 4 pairs to the work generated
by their classmates.
5
Figure 2: Sociogram from the 3rd observed session (03/11/2009). Face-to-Face
interactions and comments done by 4 pairs to the work done by their classmates.
It is clear that the density of interactions grew in a considerable way. Students were
engaged in a collaborative process. The analysis of this “enhanced” version of the
sociogram led the instructor to ask the students why they were not interacting with other
peers in class.
A number of reasons emerged from the pair sessions:
a) Some students told him that they were not used to work in collaboration in the
rest of the courses they were taking.
“We are not used to talking with our classmates in class… Teachers usually
promote traditional lectures, and we are supposed to sit down and take
notes. It is not the same in this course. We have options to interact, but we
don´t do it“
b) They also stated that the pedagogical design of the course was generating a
heavy workload, and they felt they were not well prepared to assume it. In this
sense, they mentioned that their preference was to work with their pairs during
lab hours rather than with other pairs, since it was less time consuming.
6
“We are asked to talk to each other in this course, and that´s good since you
can find out what you´ve been doing right and wrong… But to be able to
discuss we have to spend a lot of time searching information, reading and
writing… I think we don´t have enough skills to do it and it takes us a lot of
time…”
c) They also explained that they prefer asking the instructor the doubts they
might have rather than asking their classmates.
“In my case, I trust the teacher. He is the teacher and he is the one who is
going to evaluate my work.. I prefer asking him… If I ask my classmates I
would end up with more doubts than the ones I had at the beginning.”
d) The instructor also found a particular issue relating to the ubiquity of
learning. Student conversations were not confined only to the classroom.
“I usually don´t talk to my classmates in class, but we talk about the work
we are doing in many other places... at the cafeteria, in between classes….”
The instructor of the course used this reflection period to make students aware of
the neccessity of interacting with others to better accomplish the goals of the
course.
As it was mentioned before, a set of 4 more sessions were observed after pair
discussions, to find possible changes in the interactions among students.
The density of interactions grew some, but what it is relevant is that students
noticed the importance of interacting more.
“I think I misunderstood what I was supposed to do in this course... Before having
the pair discussion I spent most of my time working with my pair, without asking
barely anything of other classmates...I just sent comments on their final projects...
now I feel that sharing the process I am following with others can help me”
4. Future work
Several issues emerged during the experience regarding the investigation process.
Some of the lessons learned:
7
We realized that the use of this kind of sociogram helped the instructor to dinamize the
course pair discussions. At the same time this helped students to better make sense of
their own learning processes.
The integration of Samsa and Iloca, was quite succesful as it gave us the chance to
provide timely feedback to the instructor prior to the scheduled pair session discussions.
This experience led us to consider improvements that could be made to Iloca, the
Interactive Learning Observer for Computer Analysis used to gather interactions from
direct observations. This sort of analysis has some limitations—for example, the level
of detail a human observer can reach while observing big groups of students; flexibility
of the tool to create new observation categories; and flexibility of the system to identify
various types of interactions (one on one, group discussions, etc).
We are currently
working to improve the tool in a variety of ways: 1. We have come to believe that it is
necessary for the observer to use the tool to engage in more naturalistic observations. 2.
To help the observer in the tricky task of monitoring interactions in big groups, we are
building capacity for the tool to record both video and audio files. As we work toward
being responsive and attentive to the needs of both students and instructors, it is our
goal to continue enhancing this tool to allow greater understanding of the lights and
shadows in the learning process.