DIGITAL LEARNING OBJECTS: TOWARDS AN UNDERSTANDING OF THEIR VALUE IN
SUPPORTING KEY COMPETENCIES WITHIN THE NEW ZEALAND CURRICULUM
FRAMEWORK (2007).
Dr. Garry Falloon
The University of Waikato
Hamilton
New Zealand
Drs. Annick Janson and Robin Janson
EGL Ltd.
30 Aurora Tce.
Hamilton
New Zealand
INTRODUCTION
A new national curriculum for New Zealand schools (NZCF) was officially launched by the Minister
of Education, Hon. Steve Maharey, in October 2007. According to the foreword, the document ‘set
clear direction for teaching and learning, taking into account leading national and international
research, and the innovative work schools are already doing’ (Maharey, 2007, foreword to NZCF).
The framework is based on work completed by Rychen and Salganik (2005) during the OECD’s
DeSeCo project which sought to identify the type of competencies students in today’s schools will
need to acquire in order to meet the demands of a rapidly changing and increasingly diverse future
world. These competencies were viewed more broadly than discrete knowledge and skills alone, but
rather they “involve the ability to meet complex demands, by drawing on and mobilising psychosocial
resources (including skills and attitudes) in a particular context” (OECD, 2005, p.4). Rychen and
Salganik (2005) developed three main competency categories underwhich a range of contributing
competencies were identified. These three primary categories were: using tools interactively, acting
autonomously, and functioning in heterogeneous groups1.
The DeSeCo work provided a strong foundation for the development of the five key competencies
included in the NZCF. These five competencies are listed as: Thinking, Using language, symbols and
texts, Managing self, Relating to Others and Participating and contributing (NZCF, 2007, p.12). The
strengthening of these competencies was viewed as being integral to all learning areas and were vital
in ensuring students in New Zealand schools were able to draw on “knowledge, attitudes and values in
ways that lead to action” (NZCF, 2007, p.12). Furthermore, they were not seen as existing in isolation,
but rather were dependent upon and integrated with each other, drawing upon a wide range of other
‘resources’ such as “personal goals, other people, community knowledge and values, cultural tools
(language, symbols and texts) and the knowledge and skills found in different learning areas” (NZCF,
2007, p.12). Development of capability in the key competencies was seen to take place over time, and
was supported by exposure to a range of different contexts in which they could be utilised. The
following study sought to explore the role that Digital Learning Objects2 could play in supporting the
development of two of these competencies - Relating to others and Thinking – within units of learning
undertaken in three classes of year 7/8 students at a New Zealand Intermediate school .
1
Due to word limitations, there is not the opportunity to elaborate further on this foundational work and its relationship to
the NZCF. Further information on the DeSeCo project can be found at http://www.oecd.org/dataoecd/47/61/35070367.pdf
For the purposes of this study, a Digital Learning Object is defined as “any entity, digital or non-digital, which can be used,
re-used or referenced during technology supported learning” (Wiley, 2000, p.4)
2
1
THE KEY COMPETENCIES AND DIGITAL LEARNING OBJECTS (DLOS)
As mentioned above, the two key competencies targeted for this study were ‘Thinking’ and ‘Relating
to others’ (RTO). According to the NZCF, Thinking is defined as “using creative, critical and
metacognitive processes to make sense of information, experiences and ideas” (NZCF, 2007, p.12)
while Relating to others “is about interacting effectively with a diverse range of people in a variety of
contexts – and includes the ability to listen actively, recognize different points of view, negotiate, and
share ideas” (NZCF, 2007, p. 12). The primary aim of this study was to explore the potential for DLOs
to support the development of these two competencies, when incorporated as specific learning
resources within units of learning undertaken by three year 7/8 classes at Peachgrove Intermediate
School in Hamilton, New Zealand. The broad focuses of these units of learning related to developing
critical thinking skills, decision making, and evaluating options and opinions in reaching conclusions.
This study also aimed to build on the work of Australian researchers such as Booth (2007), Freebody
and Muspratt (2007) and Freebody, Muspratt and McRae (2007) who have undertaken trials with
learning objects in Australian classrooms, by attempting to make explicit and provide illustrations of
how students work with these objects, and the impact using them has on their thinking skill
development.
The learning objects were selected from a repository of objects produced by the Le@rning Federation3
as part of a joint Australian/New Zealand initiative to develop a range of digital resources to support
national curriculum objectives. The specific learning objects used in this trial were simulations from
the Community enterprise, people, economy and environment section of the repository, and related to
making a decision about the use of an old factory site for community purposes4.
Figure 1: The Scenario
The study was one in a series of a wider research initiatives being supported by Microsoft (N.Z) under
the worldwide Partners in Learning programme.5
3
The Le@rning Federation develops online curriculum content for Australian and New Zealand schools. The project is a
collaborative initiative of Australian and New Zealand education ministers.
4 The Learning Objects used can be found at http://www.tki.org.nz/r/digistore/protected/objects/?id=1002&vers=1.0
5 For more information on Partners in Learning in New Zealand, please refer to
http://www.microsoft.com/nz/education/pil/default.mspx
2
RESEARCH METHOD AND DATA ANALYSIS
A range of data collection tools were used in this interpretive study. In addition to more traditional
qualitative methods such as teacher interviews and researcher observations and field notes,
Camtasia™6 screen capture software was used to record the students working with the learning
objects. The videos produced by Camtasia™ were saved onto video disks and later analysed for the
frequency of occurrence of thinking at the different levels in the cognitive domain as described by the
revision of Bloom’s Taxonomy of Educational Objectives (Anderson and Krathwohl, 2001).
Although the original taxonomy (Bloom, 1956) may be viewed as somewhat dated, it still provides us
with a useful descriptive tool to assist in identifying the characteristics of the students’ thinking when
interacting with the learning objects, and as captured by Camtasia™. While it is accepted that using
the revised Bloom’s Taxonomy as an analytical tool in this way is could be viewed as somewhat
arbitrary, it did provide a good starting point and very useful descriptors which assisted greatly in
analyzing the discourse. It is anticipated that as further data are gathered and analysed, this framework
will be further developed and refined to reflect any ‘uniquenesses’ in the thinking processes of
students using such digital learning resources.
The categories and associated descriptors which were used are summarised in Table 1 below.
Category
Remember
Understanding
Apply
Analyse
Evaluate
Create
Description
Recognising and recalling knowledge.
Interpreting, exemplifying, classifying, summarizing, inferring,
comparing and explaining using knowledge.
Using knowledge to execute or implement a process or procedure.
Differentiating, organizing, and attributing. Ability to analyse
component parts and consider how these work together to contribute
to an overall goal or purpose.
Checking and critiquing. Making judgments and decisions using
criteria or standards.
Generating, planning and producing. Combining knowledge
elements to create a new, novel, or original outcome.
Table 1: From Anderson and Krathwohl, (2001). A Taxonomy for Learning, Teaching, and Assessing: A
Revision of Bloom's Taxonomy of Educational Objectives.
To further enhance the accuracy of these interpretations, three researchers were involved in the
coding. Each researcher independently coded a video file and then met to compare and moderate their
interpretations to ensure consistency, validity, and reach ‘agreed’ interpretations. In total 32
Camtasia™ samples were collected and a representative sample of six were selected and developed
into transcripts then analysed, representing analysis of 180 minutes of Camtasia™ recording.
Additional coding was undertaken from the Camtasia™ videos which focused on identifying examples
of students exercising the key competency of Relating to others. Once again all three researchers
independently coded video data for occurrences of the Relating to others competencies listed in the
NZCF, namely “students’ abilities to listen actively, recognize different points of view, negotiate, and
share ideas” (NZCF, 2007, p.12). As above, moderation of the interpretations then occurred to ensure
a level of internal validity.
6
www.techsmith.com/camtasia.asp
3
RESEARCH DESCRIPTION AND CONTEXT
Digital Learning Object Description
The Community enterprise: People, economy and the environment Digital Learning Object (DLO)
guides students through the investigation of proposals for redeveloping an old factory site into either a
swimming pool, a toy factory or a park. Students gather, compare, evaluate and analyse facts and
opinions, rating the economic, social and environmental impact of each proposal. They use an
interactive notebook to record their analysis and recommend their choice in a printable letter to the
Mayor with an explanation of their reasons. The educational value lies in building up students’ critical
thinking and discriminating acts from opinions.
The DLO's attractive graphics provide interactive instructions and information. For example –
students go into the city council to watch a council session, listen to speakers, gather facts from the
graphs relating to the council budget, and choose who they want to interview. Students are guided
through an interactive site map where they can access different parts of the DLO (see Figure 10), a
notepad on which to record their thoughts (see Figures 7 and 11), an analysis chart to fill in (see
Figure 3), and a letter of recommendation to the Mayor (see Figures 8 and 11).
Context
The researchers worked with three teachers and a subgroup of 15 students in a suburban Intermediate
school of 525 students and 25 teachers. Research participants represented a normative sample in this
school. Data were recorded on five dedicated laptops. Teachers wanted to investigate the value of
cooperative learning in the creation of understanding, and chose a DLO that reinforces their school’s
emphasis on inquiry learning7.
FINDINGS
Table 2 shows percentages of Thinking occurrences per student (as explained in the previous section);
each Thinking category is also expressed as an average.
Student
J
A
S&O
Average
Remembering
10%
13%
13%
12%
Understanding
10%
27%
25%
21%
Applying
10%
13%
6%
10%
Analysing
30%
27%
13%
23%
Evaluating
30%
13%
31%
25%
Creating
10%
7%
13%
10%
100%
100%
100%
100%
Total thinking
categories
Table 2: Summary percentages of Thinking skills
This data is unique in its genre as the real time data recording enabled by Camtasia™ allowed the
researchers to get to descriptive levels not previously possible per learning episode. Naturally, we will
7
The Inquiry Learning model builds on students’ natural curiosity and the development of questioning skills. This model
assists students in developing strategies and processes for collating and evaluating information, formulating a set of questions
at the core of their investigation. Students then formulate hypotheses, plan and carry out their research, reach a conclusion,
and decide what the implications of their findings are.
4
need to collect large data sets before generalising about how Thinking is used to create knowledge
from information through DLOs – but as far as learning through this DLO is concerned, additional
data collected allowed us to literally draw ‘learning timelines’ to accompany narratives of each
‘Learning Journey’. Three case studies below elaborate in further detail the data from Table 2, and
describe students’ Learning Journeys and related Key Competency development.
Student J. – Peer Interaction and Journey Turning Point
Figure 2 depicts the two clearly distinguishable phases of J’s journey– before and after the ‘Relating to
others’ occurrences, which appear as the four grey bars.
Creating
6
Evaluating
5
Analysing
4
Applying
3
Timeline showing Order of Thinking and the clustering effect of students'
Relating to Others (RTO) Student J
Understanding 2
Remembering 1
99%
96%
94%
91%
89%
86%
84%
81%
79%
77%
74%
72%
69%
67%
64%
62%
59%
57%
54%
52%
49%
47%
44%
42%
40%
37%
35%
32%
30%
27%
25%
22%
20%
17%
15%
12%
7%
10%
5%
2%
0%
0
`
Relative Timeline %
KEY:
Appearance of Relating to Others
Appearance of thinking relative to duration of learning Object
Figure 2: Timeline graph Student J’s learning journey
For the first 67% of her learning, J worked alone interacting ‘functionally’ with the DLO (as though to
get it done quickly as opposed to well). She skimmed through written but listened to verbal material,
disregarding instructions or questions and contributing partial comments e.g. not correlating the option
ranking to her recommendation to the Mayor.
5
Figure 3: Student J's First Ranking - Prior to RTO (J: 9.32)
Figure 4: J Cannot Continue - The Only Place in the DLO That Needs Completion (J: 9.35)
Initiating a question (“…is this letter being sent to the mayor?” RTO; J:12.51), she began a journey of
self-monitoring. Her motivation to share information with peers (“…I am going to look around town
again”; J:17.18) helped her ‘own’ her learning. The realisation that her recommendation was going to
‘count’ prompted her to listen to peers’ discussions for the first time. Questions about her choice (J:
13.23) jolted her previous false sense of accomplishment, and prompted her to reconsider her choice
6
by backtracking to check budget information (Figure 5, J:13.38) which she had previously glossed
over.
Figure 5: For the First Time J. Studies the Budget
A peer who chose the pool appeared to sway her opinion back to this option (J:14.32). She engaged
with the extra knowledge brought about by peer interaction, and asked “…Do I have to put all the
reasons on?”( J:18.33): RTO helped her reach higher order thinking (see following section). She
shared information with her peers (“…The factory will cost $2 million, so there won’t be any left over
to do other work…”; J:14.00) even going over to peers’ workstations to see their letters (J: 19.17).
J’s thinking was intimately related to the RTO Key Competency. Until the aforementioned turning
point, J. only used lower levels of thinking ( ie: 1 & 2). Coinciding with the first RTO episode, she
began using higher order thinking skills (4-5-5-4-5). The late burst of higher order thinking
occurrences, however, did not improve her final performance, as her recommendation letter was not
fully supported by the facts and opinions she had collected.
7
Student A. – Thoroughly Going for His Best
Determined to produce quality work, A. had a clear learning strategy, methodically going from source
to source, sometimes reading and listening simultaneously (A: 2.17) to understand the DLO
requirements and checking ahead to future tasks (A: 8.29). He modified his notepad recording (A:
11.31), displaying a spirit of inquiry and searching for ‘bigger picture’ explanations, constructing
knowledge by ‘filtering’ contextual data.
`
Timeline showing Order of Thinking and the clustering effect of students'
Relating to Others (RTO) Student A
Creating
6
Evaluating
5
Analysing
4
Applying
3
Understanding 2
Remembering 1
0
Relative Timeline %
KEY:
Appearance of Relating to Others
Appearance of thinking relative to duration of learning Object
Figure 6: Timeline Graph of Student A.'s Learning Journey
At the start of this information-gathering journey, A. worked quietly alone, without relating to
surrounding peers. His RTO began with J’s question about the letter and his asking about her choice
(A: 14.49). From then, they entered into discussion with other peers about choices, (A: 16.03)
reasoning (A: 16.30-17.50), opinions (A: 17.50-19.59), and related topics – like the water cycle (A:
17.15-17.44). A. seemed motivated by this interaction. This RTO interaction added to A’s learning
journey but in more subtle yet effective ways than for J., because A. had already started using, albeit
moderately, higher order thinking skills (see his learning timeline graph). Through RTO, however, he
changed his choice from the toy factory (A: 14.49) (Figure 7) to the pool (A: 16.03).
8
Figure 7: A. Changed his Recommendation to Swimming Pool - Without Changing his Ranking to Match
When A. modified his analysis (A: 16.23), though oddly not his ranking, the DLO did not alert him to
this inconsistency and he was able to continue his work. A. invested effort in writing the best letter he
could – rather than just completing the DLO. He wrote confidently once he has made his decision,
constantly checking his writing and correcting mistakes. His reasons (Figure 8) were well thought out,
accurate, relevant, and coherent (A: 23.04).
Figure 8: A. Completed his Letter - Aiming for His Best Work
9
A. embraced the collaborative atmosphere, writing “I think that we should…..” as opposed to others
using ‘I’/‘you’, including himself and his workmate in the outcome of his decision. He also added:
“…Please consider this letter containing my strongest, utmost respected opinion that I have written
ever. With all due respect…” (A: 34.48). The development of A’s Thinking Key Competency was also
related to the RTO Key Competency (Figure 6).
A. self-managed by thoroughly going through the DLO parts, using sporadic higher levels of Thinking
(3 out of 9 times, up to 88% into the DLO) until his first interaction with peers. He used lower order
thinking (rankings 1s, 2s and 3s) up to 55% of the time spent on the DLO, at which point he began
using higher order thinking skills (rankings 5-6). Unlike J., however, the thinking foundations he laid
out at the start of the DLO were strong enough by then to support his learning journey, culminating in
a well articulated, quality output (Figure 8).
S. and O. – A Collaborative Exercise
S. and O. worked together on the DLO from the start. Their strategy was consistent and they managed
their learning journey to analyse and reflect on data, record the information, and target additional
information. They also extrapolated data and discussed future scenarios. It seemed their curiosity
drove their learning journey, and they appeared highly committed to the outcome.
S. and O. effectively interacted between themselves, and with their peers. Without ever negotiating
how to work together, they created a collaborative environment, simply sharing tasks equally: one
began reading and the other took over (S&O: 2.27), one took action, informing the other “…I’ve filled
in our names…” (S&O: 00.25). They shared the reading and listening, exchanging feedback. They
confidently made decisions together, showing trust and respect for each other’s part and the quality of
their input. Their DLO experience was very different to that of students working individually, their
collaboration having an early consolidating effect on their opinions and ideas. They also related to
their teacher’s praise about getting “rich data” (S&O: 7.12) and followed his suggestion to include the
word ‘utilise’ instead of ‘use’. They even related to and thanked DLO ‘characters’ for their
contribution - “Thank you, Fatima…” (S&O: 5.47). S. and O.’s learning journey displayed 16
occurrences of Thinking skills, seven of a lower and nine of a higher order (see Figure 9).
Timeline showing Order of Thinking and the clustering effect of students'
Relating to Others (RTO) Students S&O
Creating
6
Evaluating
5
Analysing
4
Applying
3
2
Understanding
Remembering1
0
`
Relative Timeline %
KEY:
Appearance of Relating to Others
Appearance of thinking relative to duration of learning Object
Figure 9: Timeline Graph of Students S. and O.'s Learning Journeys
10
They transcended the content displayed and speculated about its impact in creative ways. They used
the city map as an artefact (Figure 10) to discuss the placing of the old factory site and how the
different options available to them would impact on the area (S&O: 1.01-2.24).
Figure 10: S. and O. Open and Discuss the Placing of the Old Factory (S&O: 1.06)
This collaboration in refining the best available option is illustrated by oral exchanges such as this,
captured by Camtasia™,
…having the library and city council near suggest it’s in the centre of town, so you
wouldn’t usually have a factory. If you had a toy factory on the old site, the job
centre people could recruit people. (S&O: 1.45)
They also discussed values (eg: eating vs. not eating meat; S&O: 14.45) and consistently integrated
their DLO experience and their thinking to improve their reasoning and judgement, approaching the
task with an open mind. They related to the DLO goal through comments such as “…he’s made a
good point there”(S&O: 6.37); and “…That is a good idea. They could do cafés as well…” (S&O:
14.30). They actively listened to information such as the opinions of the players, which they discussed.
Their critical appraisal was self-guided and self-disciplined, and they were responsive to the
information presented. While writing in the note pad about the pool, they commented, “…large public
events could be held there (S&O: 8.42), so if you’ve made a swimming pool or park, you could assign
the rest of the $500,000 to other projects” (S&O: 9.50).
Our observation from the other two case studies that RTO encouraged use of higher thinking skills is
corroborated by data from S. and O.’s case study with one difference – the time dimension. That is,
they used higher order thinking skills much earlier in their learning journey (eg: Evaluating at 11% of
11
the journey) and were able to sustain this level throughout. They also produced a high quality
recommendation letter as an output (Figure 11).
Figure 11: S. and O. Work in Progress and Final Result (S&O: 24.28)
DISCUSSION
Whilst acknowledging the limited scope of this trial and the highly tentative nature of any conclusions
which may be drawn, in terms of the overall goals of this research, early indications are that these
digital learning objects provided the students with a motivating and engaging learning ‘microworld’,
in which they were able to exercise and effectively develop their thinking skills. The extent to which
this was possible and the quality of thinking which was evident, was tied in closely to several factors,
including whether they were working individually, or in variable or more established peer groupings.
As a general rule, S&O achieved higher levels of thinking earlier and more frequently than the two
students working individually, their regular oral interactions appearing to stimulate greater clarity and
refinement in decision-making, and justification of conclusions which were reached. While those
working individually also on occasions were seen to be reaching the upper levels of the Thinking
hierarchy, it is interesting to note that most of these occasions occurred at times where they were
interacting in some way with a peer, albeit, in most instances, informally. The impact of such
interactions on the quality of the resulting output (ie: the proposal selection and letter) was also
generally positive, with students who more actively engaged with each other arriving more quickly at
sounder and more reasoned decisions.
However, it needs to be acknowledged that social interaction and its role in decision-making and the
creation of understanding (Table 2), by themselves cannot totally account for individual performance
differences. Knowing when in the learning episode each of the Thinking categories is activated, how it
contributes to the overall learning process (as described by timeline/graphs) and how it is triggered,
12
supported, or motivated by the learning object and/or students interactions with it, is fundamental to
furthering our understanding of how such resources can be designed and used effectively. Our findings
indicate that interactions helped students formulate their analyses, and the earlier students interacted,
the more likely they proposed an appropriate recommendation. Those assigned to work individually
started using more high order thinking after relating to others, which sometimes caused them to
modify earlier judgments, a possible indication of learning through social interaction. Further analysis
of data from this and future trials will attempt to identify any patterns or consistencies in the features
or attributes of the learning objects which may have assisted by acting as a ‘foil’ for these interactions.
From this preliminary study, further data relating to the design of elements within learning objects
which either facilitate or disrupt the process of Thinking skill development, or impact upon their
general utility in the classroom, has been gained. At present, these features fall under three
categorisations –the accuracy and ‘intelligence’ of scaffolding functions, features which support
information gathering and recording related to the content and purpose of the DLO, and methods of
outputting data from the DLOs for assessment purposes. These three areas will be used as focuses for
future studies in this series.
REFERENCES
Anderson, L. W., and Krathwohl, D.R., (Eds). (2001). A Taxonomy for Learning, Teaching,
and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York:
Longman.
Bloom, B.S., (Ed.), (1956). Taxonomy of Educational Objectives: The Classification of
Educational Goals; Handbook I, Cognitive Domain. New York: David McKay.
Booth, S. (2007). The Integration of Learning Objects in a Multiliteracies Framework.
University of Tasmania. Retrieved January 23, 2008, from:
www.thelearningfederation.edu.au/verve/_resources/sara_booth.pdf
Freebody, P. & Muspratt, S. (2007). Uses and effects of the Le@rning Federations learning
objects: An experimental an observational study. The Le@rning Federation. Retrieved
January 23, 2008, from www.tki.org.nz/e/community/ict/pdf/tlf-report-final.pdf
Freebody, P., Muspratt, S., and McRae, D. (2007). Evaluating The Le@rning Federation’s
online curriculum content initiative. The Le@rning Federation. Retrieved January 23, 2008,
from www.tki.org.nz/e/community/ict/pdf/tlf-report-final.pdf
Maharey, S. (2007). Foreword to The New Zealand Curriculum Framework. Wellington:
Ministry of Education.
Ministry of Education, (2007). The New Zealand Curriculum Framework. Wellington.
Ministry of Education.
Rychen & Salganik, (Eds.), (2005). Defining and Selecting Key Competencies. Germany.
Hogrefe and Huber.
Wiley, D.A. (2000). Connecting learning objects to instructional design theory: A definition,
metaphor and taxonomy. In D.A. Wiley, (Ed.), Instructional Use of Learning Objects.
Association of Educational Communications and Technology. Retrieved, January 13, 2008,
from eBooks: http://www.reusability.org/read/
13
Acknowledgement:
The research team would like to acknowledge the contributions of the staff and students of
Peachgrove Intermediate School in allowing us access to their classrooms for the duration of
this project. Your help, support, and willingness to ‘give it a go’ is greatly appreciated.
14