E-portfolios Exemplar Projects
ePortfolios Exemplar Trial
Box Hill Institute
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
National VET E-learning Strategy 2012 – 2015
Acknowledgements
Project Manager: Pauline Farrell
Researchers: Julianne Seaman and Barbara Macfarlan
Writers: Pauline Farrell, Julianne Seaman and Barbara Macfarlan
Data collection: Andrew King
With the exception of the Commonwealth Coat of Arms, the Department’s logo, any material protected by
a trade mark and where otherwise noted all material presented in this document is provided under a
Creative Commons Attribution 3.0 Australia (http://creativecommons.org/licenses/by/3.0/au/) licence.
National VET E-learning Strategy 2012 – 2015
Table of Contents
1.
Introduction .................................................................................................................. 2
1.1 Background ............................................................................................................................... 2
1.2 Purpose of this project .............................................................................................................. 2
2.
Organisational overview .............................................................................................. 4
3.
Literature Review ......................................................................................................... 6
3.1 Personalising Education ........................................................................................................... 7
3.2 What is learning analytics ....................................................................................................... 14
3.3 ePortfolios - Background......................................................................................................... 20
4.
Methodology and Data Analysis ............................................................................... 26
4.1 Process ................................................................................................................................... 27
4.2 Procedure ............................................................................................................................... 28
4.3 Summary ................................................................................................................................. 29
5.
Data Collation and Analysis ...................................................................................... 30
5.1 Quantitative Data Analysis ...................................................................................................... 31
5.2 Quantitative data analysis: ePortfolios - Student Survey........................................................ 35
5.3 Quantitative data analysis: ePortfolios - Teacher Survey ....................................................... 43
5.4 Qualitative Data Analysis: E-portfolio implementation including case studies ....................... 52
6.
Recommendation and Outcomes from Data Findings and Case Studies .............. 81
6.1 Longitudinal study ................................................................................................................... 81
6.2 Qualitative Data Analysis: Surveys and one to one interviews............................................... 83
6.3 Qualitative Data: Case studies ............................................................................................... 85
7.
Conclusions................................................................................................................ 87
8.
Resources and references......................................................................................... 92
Contact Information .......................................................................................................... 97
National VET E-learning Strategy 2012 – 2015
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
1. Introduction
1.1 Background
The 2012-2015 National VET E-learning Strategy (‘the Strategy’) is aimed at strengthening
the Australian training sector’s use of new learning technologies, stimulating innovative
approaches to increasing participation in training and employment, and improving the skill
levels of the Australian workforce
Supporting learner pathways into and across the education and training system is a key
focus for the Strategy. Many learners new to the VET system have problems identifying and
mapping their learning to gain recognition for course entry or articulation, and can be
assisted by an electronic portfolio (e-portfolio) as an access and equity tool enabling
individuals to manage their learning evidence.
1.2 Purpose of this project
The Learner Pathways Business Activity’s E-portfolios Exemplar Projects initiative provides
co-investment funding to targeted RTOs and community organisations with the objective of
demonstrating the ways e-portfolio tools assist learners to identify and manage their learning
evidence for course entry, recognition of prior learning (RPL) and/or articulation.
This project will explore how three critical strategic initiatives have worked in tandem to
create the environment for the project implementation of ePortfolios:
Strategic Initiatives
What
Program Name
Creating a meaningful
Institute wide student induction
GPS Learning Pathways
context for the use of
and orientation program – over
Individual Learning Plans
ePortfolios:
6,000 students have completed
this program
Creating an appropriate
System integration and support:
Student Web
environment:
Creating an appropriate
Institute wide teacher professional BeLS Professional
skill set:
development program including:



Development program
Assessment led learning
Strategic Blended
Learning Plans
Professional development
program
National VET E-learning Strategy 2012 – 2015
Page 2
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Our belief is that ePortfolios are successful due their integration into a broader project
context that provided access and meaning. This research paper will review the broader
educational context as the pedagogical world changes rapidly with changing educational
technologies. We will review emerging education trends including the use of learner and
learning analytics. We will review the use of learner analytics internationally including how
Box Hill Institute is currently using learning analytics and developing expertise in learner
analytics; explore how this program has driven change at the organisation and strategic
levels; while our professional development staff continues to drive change at the operational
level with teachers and learners. We will also review future trends in the field of analytics.
The paper will then review the history of ePortfolios and define what they actually are.
Exemplars in the field of ePortfolios will also be addressed and a review of future trends in
the field of ePortfolio usage. The paper will analyse the data gathered to explore the use of
ePortfolios over the past 16 months and lessons we have learnt as a result of this large scale
project. Four case studies will be used to deepen the exploration.
The final section of this research paper will review the future application of learner analytics
for the VET sector and how we believe that ePortfolios should be developed into the future to
make this system an embedded educational technology. We will include future dreaming as
to where we perceive it needs to go, by exploring a one and three year window in strategic
planning. Do ePortfolios as they currently exist meet the needs of the teachers, employers
and students or do we need to have a rapid re-think around the system design as opposed to
the educational concept that delivered so much promise?
National VET E-learning Strategy 2012 – 2015
Page 3
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
2. Organisational overview
Box Hill Institute is a leading Victorian vocational and higher education provider known for
our collaborative and creative approach to education in Australia and overseas. The Institute
has a long and diverse history and over the years has been the recipient of many awards
and achievements demonstrating Australia-wide recognition for our efforts.
Box Hill Institute offers a wide variety of courses to both local and international students,
from apprenticeships to degrees, short courses to diplomas, and in a range of delivery
modes including full time, part time and off campus.
Box Hill Institute provides education and training in a range of local and international
locations with a total enrolment of approximately 40,000 over its campuses and licensed
partners in Melbourne and in Singapore, Macau, Kuwait, Tehran, Dubai, Abu Dhabi, Fiji,
Samoa, PNG, Vanuatu, Vietnam (4 sites), Malaysia and China (14 approved sites). The
recent partnership with CAE has added a further 38,000 students who are predominately
short course participants and expand Box Hill Institute’s geographic reach into the CBD,
North and West of Melbourne.
Our Teaching and Service Centres work in partnership to enable students to achieve their
career goals. We add value to business and deliver innovation and technological change to
industry. Our high quality strategic relationships, facilities and programs make us recognised
leaders in a number of vocational areas. Six key planks underpin our approach to education:






A culture of community involvement
A focus on relevance and currency in our courses
Innovation in all aspects of our business
Responsiveness to clients
Equitable access for all
A commitment to quality
Our corporate values recognise the importance of contributing to our community. The
Institute is well known for its community corporate giving program and is involved in many
projects that give back to the community including building houses for homeless youth, the
Burgess Foundation for disadvantage, Special Olympics, Too Young to Work: India, KOTO:
Vietnam, Reach Out for Kids, Eastern Relief, Jamie Oliver Foundation - Melbourne 15
program, Oxfam, Relay for Life
Box Hill Institute is building our reputation to ensure our graduates gain vocationally relevant
skills and knowledge, and experience a global dimension to their learning and are
successful. Box Hill Institute operates in a global environment of intense competition,
National VET E-learning Strategy 2012 – 2015
Page 4
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
technological change and continuous innovation. With our international partners in the Global
Education Network, we are expanding global learning opportunities for our students to
enhance their employability in the global market.
Box Hill Institute industry partnerships give students access to internationally recognised
training and practical experience with current technology and processes. Box Hill Institute
connects our people with best practice and new environments through developing industry
partnerships and national and international activity. Students graduate ready for work. Box
Hill Institute’s program design and delivery equips students with the skills, knowledge and
attitudes to apply their learning to new situations, to solve problems, to work creatively and
cooperatively and to sustain lifelong learning.
National VET E-learning Strategy 2012 – 2015
Page 5
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
3. Literature Review
Box Hill Institute has embarked on a new era of personalised learning. It is student-driven
and student-focussed and aims to provide control of the student’s destiny within their own
grasp.
“We know that personalized learning is not new; it’s as old as learning itself. But what
is new is that the factory model that we’ve used to meet the needs of the average
student in a mass production way for years is no longer meeting the needs of each
student as our student body diversifies. What is new is that our expectations have
grown of what students need to know and understand. What has changed is that our
students...are surrounded by a personalized and engaging world outside of the
school, but they’re unplugging not only their technology, but their minds and their
passions too often, when they enter into our schools. And what is new is that
technologies are poised to provide tools and supports to scale and enable
personalized learning.” (Software & Information Industry Association 2010)
Our students resist being constrained by the technology they are able to use to assist them
to successfully complete their study. They are also committed to finding the quickest, most
engaging and path to a successful career. Box Hill Institute would ultimately like to be able to
create a truly constructivist approach to learning for students but one where the students
construct the best program to create the optimal outcome for them. Our teachers would
increasingly become concierges of learning. “We need to think about shifting from controlling
what’s happening with students to coordinating it.” (Software & Information Industry
Association 2010)
Defining personalisation of learning has multiple definitions:
Personalization is often confused with the related terms individualization and
differentiation, which are frequently employed in education, but sometimes represent
tweaks within the long-standing, mass-production approach. True personalization
goes further and requires a major shift in focus from an institution/teacher-centred
approach to an authentic, student-centred approach. True personalization provides a
learning program and approach specifically tailored to the abilities, interests,
preferences, and other needs of the individual student. (Software & Information
Industry Association 2010)
Box Hill Institute’s teachers are passionate about finding the best way forward for their
students that is realistic within our current resources. As adult learners the best skill we can
provide our students is the knowledge, skills and self-belief to become successful lifelong
National VET E-learning Strategy 2012 – 2015
Page 6
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
learners. Success is a journey…not a destination - Your greatest asset for the journey is your
mind. Learning how to use it is the secret. (Egle, C 2009).
3.1 Personalising Education
Personalising education has been a significant trend in education since 2004 where
technology appeared to be evolving to the point it was possible. “We can personalize
learning on the basis of academic needs. Why should students be held hostage because of
the pace of the class? Rose, J. 2010, School of One, New York City Department of
Education, Personalized Learning Symposium, August, Software & Information Industry
Association
According to Pearson (2007) four broad areas have emerged as having common elements
pertaining to the personalisation of education. These common themes can be summarised
as
•
‘Learners are central
•
Information and communications technology (ICT) is a key enabler
•
Lifelong learning
•
Communities of collaboration’. (Pearson 2007)
From BHI’s point of view, our next steps in the evolution of personalised learning would
require both a methodological and organisational response to prepare us for stage 2 of this
evolution. The launch of the GPS Learning pathways program, has further evolved with the
launch of the individual learning plans for all students and the inclusion of this data into the
quarterly traffic light reports. This data is the voice of our students and allows us to plan and
respond to their emerging needs in a just-in-time manner from teacher to organisational
level.
In her article titled “Innovate to Educate: System [Re]Design for Personalized Learning”
(Wolf, Software & Information Industry Association 2010) suggests that “Today’s industrialage, assembly-line educational model—based on fixed time, place, curriculum and pace—is
insufficient in today’s society and knowledge-based economy.” (Wolf, 2010, p.6) Wolf further
suggests that
Our education system must be fundamentally reengineered from a mass production,
teaching model to a student-centred, customized learning model to address both the
diversity of students’ backgrounds and needs as well as our higher expectations for
all students. (p.6)
National VET E-learning Strategy 2012 – 2015
Page 7
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
If we wish to attain educational equity we need to be able to design a meaningful learnercentred educational path, that includes curriculum, instruction, and schedule to be
personalised to meet unique learning needs, inside and outside of the institution. Educational
equity should be aligned to meet each student helping them to achieve their potential through
a wide range of resources and strategies appropriate for their learning style, abilities, and
interests, as well as social, emotional, and physical situation. (Software & Information
Industry Association 2010)
Your choice is not to be about perfection. We're not going to be perfect. But we need
to shift from being ordinary to becoming extraordinary . . . Our options are to be about
bold action or status quo. To be about focusing on evidence – the research – or being
speculative. And finally, being happy with criticisms from our cynics or criticism from
future generations. Those are the options that we face. Carter, G. 2010, in Software
& Information Industry Association 2010)
How the ‘Net Gen’ Learns
This generation is demanding changes to the way they learn. It is suggested “that for the Net
Generation there is no separation between learning and e-learning, where technology is
embedded within their daily activities.” (Oblinger, D. and Obligner, J. 2010)
McNeely 2005) advocates that these learners want to be able to:



Learn by doing using the latest technology whether cell phones, computers, PDS’s,
audio players, and digital cameras.
learn through social interaction and collaboration with others, where online social
tools such as FaceBook are seen to assist in augmenting learning opportunities
without replacing the need for human interaction
Participate in a ‘cut-and-paste culture’ that challenges traditional educational
practices and ethics such as the definition of ‘cheating’, what constitutes acceptable
practice in assessment tasks, and how teachers assess.
Into this environment the Blended eLearning Solutions have aimed to bring together the
student satisfaction results, identify learner needs and ultimately address the disconnect
between the teachers and learners and facilitate fast-rate change to bring the two closer
together. The Box Hill Institute GPS Learning Pathways student induction and orientation
program was designed and developed to meet each of the aforementioned needs to better
engage, support and retain out learners and provide them with a more relevant, authentic,
and holistic learning experience focussed on present and future learning objectives. It was
also designed to act as a model for best practices in teaching learners into the future.
National VET E-learning Strategy 2012 – 2015
Page 8
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
So how are the Generations different? How does this impact the divide between teachers
and learners?
White et al. (2011) discuss how the generations perceive technologies differently:
Most “emergent” media are actually mainstream and fairly widely penetrated. But adoption of
use is not the same as embracing the technology. Gen Y is much more passionate about
emergent media where older consumers are more reluctant adopters.
–
Boomers use new media as extensions of traditional media behaviors, while Gen
X are more likely to have adopted new media behaviors
–
The most hyped media (social networks, video sharing) are heavily skewed to
Gen Y and Gen X have limited Boomer adoption however this is the fastest
growing demographic
–
Boomer groups use consumer generated content, however they are more likely to
be passive users, choosing to view but not create content.
–
GenX and Boomers see media as the domain of the “specialists” while Gen Y do
not feel constrained by traditional media creators; as evidenced by their interest in
blogs, social media, personalized web, RSS
The gap between the generations is broad but what they do have in common is a passion to
connect. In Australia, Google attracts the highest percentage of internet traffic (10.11%)
followed by Facebook (6.50%) and YouTube (3.04%) retrieved week ending 2 June, 2012
http://www.experian.com.au/hitwise/data-centre.html
It is a strong biological urge to educate the successive generations and make a meaningful
difference. At Box Hill Institute we are working through giving the students a voice to
communicate clearly what they need and want to be success within the educational
parameters that BHI can offer.
The top internet activities for 14- to 17-year-olds were chatting to friends (87%),
listening to music (82%) and school-related research (79%). Src:
http://www.acys.info/ysa/issues/v.30_n.4_2011/papers/sociality_online
This growing difference evolved into the GPS Learning Plans project which is stage two of
the GPS Learning Pathways project.
National VET E-learning Strategy 2012 – 2015
Page 9
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
How Box Hill Institute is Personalising Education
Box Hill Institute teachers are passionate about meeting the needs of the learners. The
Individual Learning Plan project creates objective data that provides factual information for
them to ruminate on and action.
In his article “Personalising Education: from research to policy and practice No. 11, No. 11,
September 2007 the author articulates that personalised learning is intrinsic to the Next
Practice program in Victoria. The Innovation and Next Practice Division identifies the
elements of personalised learning in practice that are relevant to the VET sector as:
Elements of personalised learning
Box Hill Institute’s response
• tailoring education to individual need
Individual Learning Plan program
GPS Learning Pathways
• ICT rich learning environments
iPad program, mobility in education and
Flipped classroom project which includes
the use of 360 Collaborate program.
• student voice
Individual Learning Plan program
StudentWeb including Moodle and
Mahara ePortfolios
The GPS Learning Pathways program also acknowledges the link between career
development and VET, as highlighted in the Economic Co-operation and Development
(OECD) report (2007)
Well-organised career development services are particularly important in postcompulsory education. Here, wider curriculum choice results in more diverse and
complex routes into later stages of education, into employment, or into both. Where
choices are more complex and their consequences are more costly, effective advice
and guidance on educational options, and on links between these options and later
occupational destinations, can help to better match individuals’ learning choices to
their interests, talents and intended destinations (p. 3).
Overall, the design and development of the GPS Learning Pathways Program and Individual
Learning Plan Program that the ePortfolios project is embedded within is centred solidly in
educational theory and pedagogy to ensure we are not only meeting the communicated need
National VET E-learning Strategy 2012 – 2015
Page 10
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
of the learners but also emerging educational and technological trends as they continue to
evolve.
Learning Plans @ BHI
There are numerous definitions of what a learning plan is. The Canadian and UK versions
are similar to what we have built and will continue to develop at Box Hill Institute. The Box
Hill Institute Individual Learning Plans project enables all students to collate an individual
learning plan that records personal, learning and career targets and achievements. This
provides a method for students to keep on track of where they want to be in their life and
ultimately their career. The Learning Plans has nine keys sections which include short and
long term goals, a S.W.O.T. analysis and digital literacy quiz. The paper will focus on the
Learning Styles analysis. Despite the controversy that surrounds this field of study we
believe this theory assists teachers to think through how to improve their delivery approaches
and style of teaching to better meet the diverse learning needs of our students.
Multiple Intelligences
Howard Gardner's multiple intelligences theory identifies at least eight kinds of intelligences,
including verbal linguistic, visual spatial, mathematical logical, bodily-kinaesthetic,
intrapersonal, musical rhythmic, naturalist, and interpersonal. The theory implies that each
person possesses a different combination of intelligences, thereby impacting on the way in
which a student learns. The implication is that any topic can and should be learned and
delivered using a range of teaching approaches. These may include visual depictions, a
kinaesthetic or collaborative experience, or a video clip demonstrating a real-life example
(Gardner, 2010).
"Personalization refers to instruction that is paced to learning needs [i.e. individualized],
tailored to learning preferences [i.e. differentiated], and tailored to the specific interests
of different learners. In an environment that is fully personalized, the learning objectives
and content as well as the method and pace may all vary."
U.S. Department of Education, 2010, P. 12
Kratzig and Arbuthnott, (2010) conclude that learning styles “may indicate preferences and
motivations, rather than inherent efficiency at taking in and recalling information through
specific sensory modalities.” They suggest that educators adapting their instruction should
not expect improved learning by their students, except “as it influences students’ motivation
for voluntary effort.” Therefore, they concluded the “practical utility” of learning styles
research is “limited” and recommend instructors focus on “ensuring that the most effective
instructional methods are used for a given learning objective.” (Gruber 2011)
National VET E-learning Strategy 2012 – 2015
Page 11
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
“[F]ocusing on learning styles as defined by sensory modalities [learning styles]
may be a wasted effort … [M]ost people are likely multimodal and multisituational
learners, changing learning strategies depending on the context of the to-belearned material. “[P]resenting material to students in multiple sensory modalities
is undoubtedly beneficial to learning and interest.” (Kratzig & Arbuthnott, 2006)
The research team concluded that ‘in contrast to learning style theory, it appears that people
are able to learn effectively using all three sensory modalities’
(Kratzig & Arbuthnott 2006, p.241)
Whenever an assessment instrument is developed, we must be concerned with its
psychometric properties. Learning style inventories are known to have problems with both
reliability and validity (Kratzig & Arbuthnott, 2006). More critically, although it is easy to find
studies suggesting that students differ in their preferred learning style, it is difficult to find
studies showing that teaching to individual learning styles actually makes in difference in
student learning outcomes
A review of such studies into programs that have been tailored to meet individual learning
styles has concluded that matching instruction to meet an individual’s sensory strengths
appears no more effective than designing content-appropriate forms of education and
instruction (Coffield et al, 2004). Perhaps the best known inventory of learning styles within
education is the one categorising individuals in terms of their preferred sense of modality for
receiving, processing and communicating information: visual, auditory or kinaesthetic (VAK).
In a laboratory study of memory performance however it was found that participants “memory
scores in different modalities appeared unrelated to any measure of dominant learning style”
(Kratzig and Arbuthnott, 2006and instead, evidenced that participants ‟were self-rating their
learning styles in ways possibly promoted by the inventory itself. As a result, the study
concluded that educators’ ‟attempts to focus on learning styles were “wasted effort”.
Kornhaber, 2001,p276) asserts that “the theory validates educator’s everyday experience:
students think and learn in different ways. It also provides educators with a conceptual
framework for pedagogical practices. In turn, this reflection has led many educators to
develop new approaches that might better meet the needs of the range of learners in their
classrooms’
Box Hill Institute believes that using learning styles establishes a shared set of language
which can assist in establishing a broader dialogue around changing teaching practices. By
opening the discussion around the divide between preferred learning styles of Baby Boomers
compared to those of Generation Y and Generation Z, the challenges in remaining relevant
to our learners and industry can be addressed.
National VET E-learning Strategy 2012 – 2015
Page 12
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Box Hill Institute Student Learning Styles Analysis
Data 01/12 to 03/12
1968 first year Box Hill Institute students completed
%
Section
Score
Verbal/Linguistic
55.8
Logical/Mathematical
58.5
Visual/Spatial
64.7
Bodily/Kinaesthetic
73.1
Musical/Rhythmic
64.9
Figure 1: Graphic illustration of preferred learning styles
It is interesting to note that Generation X have traditionally been Verbal Linguistic learners.
The advent of the Internet has irrevocably changed the way our children think. Cisco has
predicted that 90% of all web content in 2013 will be video. Our children think in images,
consume music like we once consumed text, and border on having generational ADD with
the need to move. If Generation Y and Z could write their version of learning “Hell” it would
be to be constrained in a desk in a classroom with a teacher who talks at them and uses
print-based hand-outs.... and what do we deliver in the many of our current classrooms? We
teach the way we were taught... this concept needs to be changed as our students’ brains
have physiologically changed, as evidenced in studies using MRIs.
Our students at BHI are kinaesthetic, visual /spatial and musical (see Figure 1)How does this
change the way we design learning programs, teach learning programs and ultimately track
what our students respond to?
While some of the research is also clearly challenging the use of Gardner’s theory, it is
defendable within an academic context to use it to drive change
National VET E-learning Strategy 2012 – 2015
Page 13
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
3.2 What is learning analytics
Learning analytics is the collection; analysis and reporting of data about learners and their
contexts, in order to better understand the learning process. The results of this analysis can
highlight the successes and short-falls of the teaching and learning systems and create
“actionable intelligence” (Campbell, et al. 2007) that has the potential to serve as a model for
systemic change.
In an era of increased pressure on funding and accountability in the TAFE system, there is
an imperative to bring a business intelligence into the academic setting. This is referred to as
academic analytics and institutions are realising the potential of the data that is collected at
every stage of student engagement from the first enquiry about a course through enrolment
to graduation. Academic analytics combines large data sets, statistics and predictive
modelling to influence policies and decisions about marketing to prospective students or
likely funding sources; informing decisions about change in teaching and learning culture; or
alerting teachers to students who might be struggling and in danger of dropping out.
The ubiquitous use of the internet and Web 2.0 means that we now have digital footprints or
what Long & Siemens (2011) call “data trails” of every online interaction: every library
access, every click in the Learning Management System (LMS), every social media action,
every swipe of a student card; all of these give an indication of student effort and activity
within the institute’s systems.
These learner-produced data trails provide valuable insight into what is
actually happening in the learning process and suggest ways in which
educators can make improvements. ...Indeed, some in higher education
have recently begun to consider how to apply analytics to better understand
the learning process.
(Long & Siemens, 2011, p32)
Learning analytics, as distinct from academic analytics, refers to the study and evaluation of
data gathered to measure the impact and effectiveness of the organisation’s decisions and
actions across the learning environment. It focuses on the learning process by analysing the
relationship of the learner with the content, the teachers and the support services. The
captured data creates an imperative that cannot be ignored. Student digital activity across
the institute’s systems is tracked and recorded; activity, or lack of it, is mapped along the
way. These early initiatives “are seeking to predict which students are in academic difficulty,
allowing faculty and advisors to customize learning paths or provide instruction tailored to
specific learning needs.” (Campbell, et al. 2007, p44) Institutions that have this data now
National VET E-learning Strategy 2012 – 2015
Page 14
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
have a mandate to effect large-scale systemic change and improve organisational efficiency,
particularly as it pertains to teaching and learning.
Sample analytics implementations
Sinclair Community College, Ohio USA, and Purdue University, Indiana USA, have
developed systems to gather and analyse data that is used to support students in their
academic endeavours.
Sinclair Community College has developed a Student Success Plan (SSP) and has won
national awards for its innovative and effective application of learning analytics to student
support systems and retention strategies. The system gathers data via an Structured Query
Language (SQL) database which integrates demographic and admissions information with
course registration, grades, financial aid status, counsellor risk-assessment notes and
faculty-initiated early alerts.
Analytics generates a system alert for advisors to initiate an Individual Learning Plan
(ILP) whenever any one of the following four criteria appears in a new student’s
profile:
 Placement-test referrals into two or more developmental courses numbered
below “100”
 Individual or family income level below the federal poverty level
 Full-time work
 Undecided major
(Campbell, et al. 2007, p46)
The student and a counsellor then work together to develop an ILP which gathers additional
data including a Myers-Briggs type indicator; study strategies; personal challenges; student
course satisfaction levels, progress markers; and so on. Success is measured against
factors such as, achieving a Grade Point Average (GPA) of 2.0 or higher; making decisions
about a major area of study and a career; resolving personal issues which may pose a threat
to successful study and regular class attendance. The analysis of the data demonstrates that
for those new students who completed an ILP retention was 93.3%, while those who did not
participate in the ILP program had a 65.7% retention rate.
Purdue University has similarly developed a strategy for supporting students in their first year
that might be in danger of dropping out called Course Signals (CS). This is a student success
system that allows faculty to provide meaningful feedback to students based on predictive
models. The signals are in the form of traffic lights that appear on the student’s course
National VET E-learning Strategy 2012 – 2015
Page 15
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
homepage and indicate via red, yellow or green a student’s current likelihood of success.
The idea behind Course Signals is fairly simple: “utilize the wealth of data found at an
educational institution, including the data collected by instructional tools, to determine in real
time which students might be at risk, partially indicated by their effort within a course.”
(Arnold and Pistilli, 2012, p2)
It works by using a Student Success Algorithm (SSA) which gathers data from a multitude of
sources and generates a risk analysis for each student based on their performance (course
grades), effort (interaction in the LMS), and student socio-cultural background. Course
teachers can access this information from the faculty dashboard and initiate intervention. The
goal is to predict which students are at risk of failing and guiding them to the appropriate
support services. “Using learner analytics, faculty can provide action-oriented and helpful
feedback much earlier in the semester, which students appreciate.” (Arnold and Pistilli, 2012,
p.3)
These student intervention programs that harness the power of learning analytics have
demonstrated again and again that effective data management and its implementation in
just-in-time intervention can greatly benefit the student, whose success reflects a concerted
improvement in support across the institution.
Learning Analytics versus Learner Analytics
At Box Hill Institute we are using statistics in a range of ways. We have commenced tracking
learning analytics and through the use of the Individual Learning Plans really evolving this
area into a key strength and diagnostic tool for identifying potential retention risks. We still
need to complete more work in this area. However across the past two years we have
collected and collated a significant body of work around an area we are calling “Learner
Analytics”.
We are defining Learner Analytics as:
The collection, collation and analysis of organisation wide data on a range of
strategically important measurables that identify the level of organisation wide
eLearning adoption and continue to track this data on a longitudinal basis.
Box Hill Institute is collecting analysing data at:




Organisation wide level
Teaching Centre wide level
Course Level
Individual student level
National VET E-learning Strategy 2012 – 2015
Page 16
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
We have identified four quadrants that we collate data against:
1.
2.
3.
4.
System Use – Moodle and Mahara and Helpdesk usage
Teacher Professional Development attendance
Student Induction and orientation attendance and data results
Resource design and development engagement
This data is compiled into a quarterly Traffic Light Report, which details percentage growth or
decline in a key statistical area and designates a corresponding colour. In 2012 we have
further developed this report to include a course by course analysis providing Teaching
Centre Managers and Directors that ability to assess which courses are using eLearning
systems and also identify those with minimal or no engagement. This report then triggers
further training and support for those courses that have no online profile.
Educational Outcomes
This data has the following benefits:





It provides an Institute-wide picture of what is “really” happening
It provides Teaching Centre Managers the capacity to get a snapshot across the
system usage and professional development rates and then negotiate further
assistance as required.
It allows the Blended eLearning Solutions Team the ability to focus resources,
support and training in areas that require specific assistance and continue to
customise our training as required.
It provides Senior Educators with prior knowledge of traction gained and informs them
of activity occurring within the courses they manage.
It also provides BeLS with a view as to which strategies are being successfully
implemented and what impact these initiatives have had on student and teacher
usage.
National VET E-learning Strategy 2012 – 2015
Page 17
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 2: Sample of the Quarterly Traffic Light Report
– PD (hrs)
desk requests
Elearning help
– staff (hrs)
desk requests
Elearning help
logins
Moodle teacher
logins
Moodle student
Centre
resources
Teaching
Moodle active
System Use Sample
1
Centre for ATE
686
350
35
2.5
2.75
2
Centre for BAS
2438
539
46
12.25
1.25
3
Centre for BFS
1050
683
33
7
4
Centre for BP
1350
371
45
13.18
5
Centre for CCI
1243
986
71
9.61
6
Centre for ERT
470
319
21
3.66
7
Centre for HBF
478
330
12
2.2
8
Centre for HACS
1393
732
37
43.05
1.75
9
Centre for CHATS
1065
326
34
35.22
1.5
10
Centre for IET
103
1577
29
3.75
0.25
11
Centre for ICT
1988
704
36
6.85
2.5
12
Centre for VAE
160
756
31
13
Non-Teaching Centres
1151
59
2.33
0.75
1.5
Individual Learning Plan data
Each student has their own Individual Learning Plan on their Student Web profile. However
the real value is in the self service website designed that allows Directors, General
Managers, Teaching Centre Managers, Senior Educators and Teachers the ability to view
data at different levels depending on their permission level. A director can view all data from
all Teaching Centres they manage. A teacher can view data from the courses that they teach
in.
Teachers are now being trained through the professional development team working with the
Box Hill Institute eLearning advocates and teacher networks how to use the data produced
from the Individual Learning Plans to customise their teaching strategy, resource design and
development and most importantly the assessment strategy to meet the learning styles,
digital literacy levels, literacy and numeracy skills and preferred modes of learning of their
current students.
National VET E-learning Strategy 2012 – 2015
Page 18
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Business Outcomes and strategic planning
The Individual Learning Plan data has been used as part of the Institute long term strategic
planning. It has provided the students with a voice communicating their goals and perceived
strengths and weaknesses and also their skill sets that we can successfully plan future
initiatives to meet their communicated needs.
Retention/customer satisfaction and contestability
The Individual Learning plan data and Traffic Light reports are also providing a snap shot into
potential “ students at risk” as we begin to integrate electronic roll and continue to evolve our
programs to meet those areas identified within the student satisfaction surveys where
dissatisfaction is identified. In the past two years we have, as an Institute been able to impact
on a range of areas that were assessed as being an area of extreme dissatisfaction. These
areas no longer appear as an area of dissatisfaction – the Student Web is an excellent
example of this.
Change management
The data gathered is becoming a brilliant change management tool as all staff are able to
share the same information, develop a shared set of benchmarks and enjoy a level of
transparency that we have never had before. Previously we have had Teaching Centre
Mangers suggest that their Teaching Centre has limited adoption of education technologies
and have been pleasantly surprised at the difference between their perceived and actual
levels of activity being reported within their centre. It also allows BeLS to communicate
objectively and in measurable terms. It provides a level of honesty that requires BeLS to
become increasingly accountable with how we use our time and resources to meet Institute
required outcomes.
National VET E-learning Strategy 2012 – 2015
Page 19
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Future trends in learner and learning analytics
As institutions have to manage greater financial restraints, learning and “learning” analytics
will facilitate a strategic and targeted deployment of resources to maximise the return on
investment. New paradigms using data gathered from multiple data sources of student
interactivity must consider the issues around privacy and security, and legal and ethical
considerations; they must recognise that a student’s activity in the LMS and other online
systems doesn’t represent actual learning. However, as Campbell (2007) puts it: “Analytics
tools might facilitate better communication between faculty and students while empowering
students to monitor their coursework and take greater responsibility for their learning.”
Three characteristics of successful analytics programs have been identified by Campbell, et
al.(2007). The first is leaders who are committed to evidence-based learning; secondly, an
institution needs administrative staff who are skilled at data analysis and lastly, an institution
needs a flexible technology platform that is able to collect, mine and analyse data. The
institution must commit to not only gathering the data in an ethical and secure way, but to act
on the analysis of that information in order to develop a customised learning environment for
each student so that the most effective instructional approaches are employed. “Eventually,
institutions may be able to provide unique learning paths, matching instructional activities to
a student’s learning needs.” (Campbell, et al. 2007, p.56)
Economic pressures will necessitate a higher level of accountability by decision-makers in
educational institutions and the emerging field of learning academics has demonstrated a
responsiveness to learners’ needs. Survival has become an imperative in the increasingly
competitive global education market.
Learning analytics can penetrate the fog of uncertainty around how to allocate
resources, develop competitive advantages, and most important, improve the quality
and the value of the learning experience. (Long and Siemens, 2011, p.40)
3.3 ePortfolios - Background
“When compared to paper based portfolios, ePortfolios or ‘electronic portfolios’ have been
recognized as enabling learners to digitally collect, select, reflect, publish and connect formal
and informal learning experiences in a private or shared space, thereby providing a strong
and flexible medium for personal, professional, and educational growth across one’s
lifetime.” (Hallam, et al. 2008, p.9).
With the advantages of containing a variety of media, linking and background, and sharing
and publishing (Khan 2004), ePortfolios have generated a wide level of interest across the
education and employment sectors with economic opportunities to ‘enhance labour market
National VET E-learning Strategy 2012 – 2015
Page 20
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
participation and mobility’ (Janssen, et al. 2011, p.1) As a result, the last decade has seen
peak educational bodies including the Joint Information Systems Committee (UK), The
Centre for Recording Achievement (UK), European Institute for eLearning (EIfeL), the
Association for Authentic Experiential and Evidence-Based Learning (AAEEBL) (USA), and
the Australian Flexible Learning Framework (AFLF), establish standards and provisions of
best practice to steer the development of software tools and services that will support these
philosophies.
In 2007, the Australian Learning and Teaching Council was commissioned by the
government to undertake research that would help progress ePortfolio practice in Australian
higher education (Hallam, G. et.al (2008). ePortfolio use by university students in Australia:
Informing excellence in policy and practice., (p 134). This study identified that there are four
interrelated contexts to foster ePortfolio practice including government policy, technical
standards, academic policy, and learning and teaching (Hallam, G. et.al (2008, p135 ).
ePortfolio use by university students in Australia: Informing excellence in policy and practice.
Separate efforts to support learner pathways leading to higher qualifications, opportunities for
Recognition of Prior Learning (RPL), or articulation from the Vocational Education and
Training (VET) sector, led to the formation of the VET ePortfolio Roadmap 2008-2011,
supported by the Australian Flexible Learning Framework (Galatic, et al. 2009).
Despite state-based initiatives in Australia providing Registered Training Organisations with a
hosted ePortfolio tool, a nationwide service seen to connect learners across Institutes and
organisations such as “My Portfolio” established in New Zealand (MyPortfolio, 2012) does
not exist, thereby creating a challenge for both educational providers and students in
selecting the most appropriate e-portfolio tool and implementation strategy.
While ePortfolio standards including those set out by the IMS Global Learning Consortium
(2012) are critical in enabling interoperability across different systems and institutions it has
been suggested that educational providers should consider hosting their ePortfolio externally
due to the ‘temporary’ role an institute plays in a students life, and the contradictions that
evolve from the philosophies of lifelong, life-wide and self-directed learning and institutional
ownership and control (Heinrich, 2008, p. 409)
ePortfolios – Definition and purpose
For the purposes of this study, and to further understand the considerations required when
implementing ePortfolios, it is necessary to establish a shared understanding of the term
‘ePortfolio’ due to variances in both the term used to describe an ePortfolio, and the definition
itself.
National VET E-learning Strategy 2012 – 2015
Page 21
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Across levels of education, business, and internationally, the term ePortfolio isn’t common
language and instead may be referred to as a ‘webfolio’, ‘efolio’, ‘digital learning portfolio’,
‘performance management tool’ or ‘career management tool’ (Learning Communities, 2009,
p.1)
The lack of common language and descriptors for ePortfolios can present a challenge in
implementing e-Portfolios (Hallam, G. Et.al, (2008) p.2) and can also lead to the
understanding of ePorfolios as a product (the end result) rather than a process (a series of
activities) (Barker, K., 2006, p.8). Just as with paper-based systems, ePortfolios should align
to pedagogical principles and curriculum practices (Ward, 2008, p.1)
Rather than having the technology as the key driver. Within an education and training
context, ePortfolios are seen to be learner-centred and outcomes-based providing
opportunities for learners to take ownership of their learning pathway. In alignment to
national e-learning standards set out by the Australian Flexible Learning Framework (2012),
this study adopts the following definition:
“An e-portfolio is a learner-driven collection of digital objects demonstrating
experiences, achievements and evidence of learning.”
This definition closely reflects the nature of ePortfolios in the VET sector as an evidencegathering tool, where learners can build a collection of artefacts including video, audio,
documents, blogs, and plans for Recognition of Prior Learning (RPL) and competency based
learning and assessment, including apprenticeships and traineeships.
The scope of ePortfolios in VET however spans outside of course delivery supporting student
led, lifelong and life-wide learning objectives through informal, non-linear and chaotic
learning processes (Miller, A 2009). Subsequently, it would be fair to extend upon our initial
definition with an emphasis on process whereby:
Behind any product, or presentation, lie rich and complex processes of planning,
synthesising, sharing, discussing, reflecting, giving, receiving and responding to
feedback. These processes – referred to here as ‘e-portfolio-based learning’” “…since
the process of learning can be as important as the end product.’ (JISC, 2008b
The specific purposes for ePortfolios can vary depending on the context, the degree of
maturity in using ePortfolios, and the differing levels of understanding within and across an
institution. It has been suggested that these complexities are evidenced by threshold
concepts, which can make the landscape for ePortfolio implementation at both an Institute
and course level complex (Joyes, G., et al. 2010, p.15)
National VET E-learning Strategy 2012 – 2015
Page 22
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The IMS Global Learning Consortium (2005), identifies six major types of ePortfolios
including assessment ePortfolios, presentation ePortfolios, learning ePortfolios, personal
development ePortfolios, multiple-owner ePortfolios and working ePortfolios. In consideration
of a learner’s journey from a personal, learning, and career perspective, ePortfolios can also
be seen to assist learners as they transition into, within, and from VET (Curyer, et al. 2007,
p.1) helping them to both develop soft skills in writing, information and communications
technology literacy, critical thinking, and communication, (see Figure 3) while undertaking the
processes of reflection, collaboration, review, workplace learning, social networking,
presentation and planning. (Lorenzo, G and Ittelson, J (2005)
Figure 3. ePortfolios supporting learner transitions
(Src: http://eresources.tafe.tas.edu.au/epw/introduction/index.html )
ePortfolios can be a “vehicle for:






supporting transitions and student mobility
planning and reflecting on learning and career development
recognising skills and learning styles
verification of qualifications
security and control of private information
recording evidence of employability skills.”
(Galatic, H., et al. 2009)
National VET E-learning Strategy 2012 – 2015
Page 23
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
While the purpose of an ePortfolio may significantly vary, commonalities exist between the
ePortfolio tool itself, the types of content which can be added, and the process in which to
develop the ePortfolio.
At Box Hill Institute learners use the New Zealand based Mahara ePortfolio system to
capture evidence of learning through video, audio, documents, blogs, and plans. Mahara
combines a range of social collaboration tools including blogs, comments, groups, forums,
profile pages, and options to share parts of the ePortfolio with others. By integrating the use
of mobile devices with ePortfolios and customised delivery and assessment strategies,
teachers have opportunities to enable individualised and industry contextualised assessment
opportunities for their learners.
Box Hill Institute teachers adopted the philosophy that learners can use their ePortfolio to
support both lifelong and life-wide learning concepts as evidenced in Figure 4. Each aspect
of a student life is interrelated, and in the context of ePortfolios, consideration of a student’s
personal, learning and career objectives can assist to support the development of a student
centred holistic ePortfolio that identifies past experiences and assists in realising future
progression.
Learning
•Set educational goals
•Identify gaps in learning
•Create and monitor a learning plan
•Undertake learning and assessment
•Reflection
•Track educational attributes
•Capture authentic learning
•Support transitions between education and
employment
•Recognition of learning, skills and experience
Personal
•Lifelong, lifewide learning
•Holistic view of learning
•Personal development, recognise
strengths, weaknesses, and identity
•Undertake self organisation and
time management
•Financial tracking
•Share experiences with others
•Well being
National VET E-learning Strategy 2012 – 2015
Career
•Career planning
•Identify professional competencies
•Employability skills
•Build resume/CV
•Track and reflect career
development
•Aids in promotion
Page 24
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 4: BHI ePortfolios student-centred model to support lifelong and life-wide
learning.
ePortfolios Future
At Box Hill Institute we gather data about preferred learning styles, digital literacy, and so on
as well as interaction in the LMS and the ePortfolio system and disseminate this information
to the teaching centres so that they have the information that they need to better cater for
their learning cohort. It acts as a persuasive argument against the “stand and deliver” lecture
style when many of the students are kinaesthetic learners who need to be actively involved in
their learning.
These indicators are what we at Box Hill Institute call Traffic Light indicators and these will
continue to evolve. In 2012 we now report at Course level and by the end of 2012 we will
report on unit of competency level. We have the capability to do this now but are very
cognizant of the need to introduce change carefully and have all key stakeholders on board
understanding the motivation is to provide greater support and not to used in a punitive
manner.
Each change to level of reporting requires an adaptation to the professional development
program to meet the emerging trends and areas that require further development. The
Individual Training Plans for teachers was an evolution that developed out of this program as
we identified pockets of resistant behaviour. Once we analysed the data and interviewed the
key stakeholders, the issues were identified as being a lack of skill – not lack of want.
Conclusion
Leaner and Learning analytics will continue to evolve within the Box Hill Institute
environment. Ultimately, we aim to track all data from Organisation level through to individual
level that is relevant and statistical and does not breech and privacy or security guidelines
set-up nationally and internally with the Box Hill Institute environment. ePortfolio data will
continue to be reported as part of holistic programs but also when embedded as a key
component of assessment strategies for courses and units of competence.
National VET E-learning Strategy 2012 – 2015
Page 25
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
4. Methodology and Data Analysis
This section provides a description of the research methodology employed in this research
project. This section examines the use of qualitative methodology. This discussion will
explore the usage of a post positivist interpretive approach (a research theory that combines
both qualitative and quantitative research), which was undertaken as a means to use both
qualitative methods and purposive sampling that is characteristic of the post positivist
interpretive approach. The rationale for utilising this approach in this study is that it may
provide a means to potentially develop a ‘richer and more extensive’ research pool (Zikmund,
1997) than may be possible adopting a purely quantitative (positivist) approach which aims to
numerically estimate the market (Kotler, 2004).
Post positivist interpretive inquiry as a research methodology is
…not contrived in contrast to the controlled, artificial, laboratory-type experiment or
quasi-experiment which decontextualises the phenomenon being studied. In post
positivist interpretive inquiry phenomena are studied in context. The experiment
usually cannot be generalised to the natural setting, whereas there is likely to be
some degree of transferability from one natural setting to another for post positivist
interpretive inquiry.
(Caulley, Latrobe University, 1992)
The methodology of post-positivist interpretive inquiry requires opportunity for clarification
and summarisation.
This process …fits within an interpretive (qualitative) approach due to the open-ended
approach adopted. (Zikmund, 1997). The focus groups and interviews aim to
augment ‘the process of making pertinent inferences and drawing conclusions
concerning the meaning and implications of a research investigation’ (Zikmund,
1997).
This study seeks to utilise purposive sampling :”to help manage the trade-off between the
desire for in-depth, detailed information about cases and the desire to be able to generalize.”
(Caulley, 1992, p.3).
National VET E-learning Strategy 2012 – 2015
Page 26
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
4.1 Process
The post positivist theory of research analysis requires the creation of key drivers that inform
the research process from the participants as opposed to the researcher determining solely
what is worth researching. The initial one on one surveys will assist us to create a
quantitative survey based on the findings determined by their peers. This provides a more
relevant and appropriate research tool.
The Research Process of Determining the Drivers
Research has determined the key requirements of ePortfolio usage within a
blended learning approach.
Survey is collated based on the initial research questions designed based on key
drivers being identified from the analysis of the case studies
Research project commences with one on one interview with four key ePortfolio
users who discuss their interest and current or likely uptake of Mahara ePortfolios.
The research data is collated and the key drivers and barriers confirmed.
Questionnaire designed and trialled as a result of the analysis.
Questionnaire disseminated and results collated
Final analysis conducted and report written to discuss results found.
Exhibit 1 – Flow Chart of the research process for determining ‘drivers’
National VET E-learning Strategy 2012 – 2015
Page 27
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
4.2 Procedure
The following is a brief overview of how the research was conducted.
Research method
Who was
When
used
questioned
conducted
Secondary research
Nil
April, 2012
literature review
Why conducted
To assess current trends in personalised learning, learning
analytics and ePortfolios
Qualitative Research: Purposive Sampling
1-on- 1 interview with
4 Box Hill Institute
Teachers
Teachers
May, 2012
To determine what their perception/interest in ePortfolios
Assess their capacity to continue to use this Technology
Assess key driver to collate the surveys
Focus Group x 1 with
11 Box Hill Institute
Students
Students
May, 2012
To determine what their perception/interest in ePortfolios
Assess their capacity to continue to use this Technology
Assess key driver to collate the surveys
Quantitative Research: Survey
Longitudinal study:
Traffic Light Report
Nil but over 5,000
January 2011
Track real usage of ePortfolios and how linked the usage is
student and teacher
- May, 2012
to the GPS Learning Pathways project
May, 2012
To determine what their perception/interest in ePortfolios
data collated
data Analysis
Teacher Survey
12 Box Hill Institute
Teachers
Student Survey
20 Box Hill Institute
Assess their capacity to continue to use this Technology
May, 2012
Students targeted
To determine what their perception/interest in ePortfolios
Assess their capacity to continue to use this Technology
The analysis technique used was arithmetic. We do not have a significant enough sample to
merit the use of statistics. All questionnaires were disseminated to 20 participants as email
addresses can be potentially unreliable and the return rate was relatively low. In order to
achieve a valid sample we needed to send out more questionnaires.
National VET E-learning Strategy 2012 – 2015
Page 28
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
4.3 Summary
This section outlines the adoption of the post positivist interpretive methodology as the
research method and the implementation of this process. This method enables the use of
purposive sampling. The use of this research method has enabled an in-depth exploration of
individual viewpoints garnered from the use of qualitative methodology and the consequence
and external orientated collection and collation of information gathered as a result of
exposure to specific stimuli that purposive sampling methods have gained. The drivers
enable the researcher to pinpoint specific areas of strengths and weakness.
National VET E-learning Strategy 2012 – 2015
Page 29
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
5. Data Collation and Analysis
Longitudinal Study: Box Hill Institute Traffic Light Report
The Box Hill Institute traffic light reports commenced collation in 2009. This report
commenced as a simple collation of student and teacher log-ins on a monthly basis. This
report has since evolved into the collation of over forty data sets. This forty data sets have
been consistently used since the beginning of 2011 and is now allowing us to track growth or
slowing adoption rates. This report also includes data on what type of data each Teaching
Centre is uploading allowing us to analyse active versus passive eLearning and further
promote appropriate use of assessments, media and resources in the online environment.
Box Hill Institute analyses the data at:

Organisation wide level

Teaching Centre wide level

Course Level

Individual student level

Individual teacher level
We have identified four quadrants that we collate data against:

System Use – Moodle and Mahara and helpdesk usage

Teacher Professional development attendance

Student Induction and orientation attendance and data results

Resource design and development engagement
This data is compiled into a quarterly Traffic Light Report, that details percentage growth or
decline in the key statistical area and designates a colour. In 2012 we have further
developed this report to include a course by course analysis providing Teaching Centre
Managers and Directors that ability to assess which courses are using eLearning systems
and identify those with minimal or no engagement. This report then triggers further training
and support for those courses that have no online profile. The following is an analysis of the
data collated over the past fifteen months that are particularly pertinent to ePortfolio usage
and correlating relevant professional development data with a Teaching Centre by Teaching
Centre Learning style analysis.
National VET E-learning Strategy 2012 – 2015
Page 30
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
5.1 Quantitative Data Analysis
Longitudinal Study
Preferred
Learning
GPS
Mahara
Styles ePortfolio
Unique
Number
Mahara
ePortfolio
GPS
Number
Mahara
ePortfolio
GPS
Number
Mahara
ePortfolio
GPS
Number
Mahara
ePortfolio
Unique
of Q2
Unique
of Q3
Unique
of Q4
Unique
of Q1
2011
User
2011
User
2011
User
2011
User
2012
Number
Number
Number
Number
Centre
Number
of Q2
of Q3
of Q4
of Q1
Name
Q1 2011
2011
2011
2011
2012
BAS
ATE
407
183
V/S: 65.2
B/K: 75
49
V/S: 64.6
B/K: 77.3
(BFS)
1644 170
3387 2902
V/S: 63.6
B/K: 73.5
1953 565
0
65
97
62
660
304
Number
of Q1
User
Null
GPS EPortfolio
20
81
16
116
100
94
981 46
0
56
116
64
133
95
796
National VET E-learning Strategy 2012 – 2015
2011
1916
188
7 staff
138
1 staff
250
3 staff
274
0
197
2
29
Training
355
Page 31
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
B/K: 71.9
INTER:
203
(BPS)
63.6
35
(CCI)
B/K: 71.2
MUS: 77.6
340
(ICT)
L/M: 65.3
B/K: 67.9
210
(ERT)
V/S: 67.2
B/K: 78.1
124
(HBF)
B/K: 75.3
MUS: 66.5
72
V/S: 63.5
B/K: 72.9
(HACS)
(CHATS)
(IET)
(VAE)
28
411
41
145
117
V/S: 64.3
B/K: 75.1
V/S: 67.6
B/K: 71.9
MUS: 70.6
137
148
49
39
420
189
V/S: 64.1
B/K: 73.6
9
0
19
26
178
71
54
109
29
12
50
93
112
63
29
131
231
0
26
20
493
361
148
165
28
90
328
45 staff
120
5 staff
77
5 staff
126
0 staff
335
14 staff
86
14 staff
27
4 staff
39
3 staff
173
0
7
0
69
13staff
427
0
0
122
0
89
0
150
228
0
0
0
107
8
60
0
0
40
139
33
37
147
0
122
198
0
0
202
357
56
70
79
66
430
Figure 5: Longitudinal study of data
National VET E-learning Strategy 2012 – 2015
Page 32
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Key findings
There is a direct correlation between professional development activity and Mahara
ePortfolio usage.
The greatest users of ePortfolios are the paraprofessionals with a compelling
workplace based context
Teaching Centre
 Biotechnology
 Creative Industries


Business Programs
Hospitality

Health and Community Services
Context
 Work placements
 Industry portfolios for
fashion and music
 Work placements
 Work placements and
industry based RPL
 Work placements and
industry based RPL
There are key champions in the Teaching Centres that lead highly contextualised
examples of apprenticeship based usage. The ePortfolio usage student numbers in
the trades teaching centres are growing in numbers unrelated to the enrolments in
the GPS Learning Pathways programs. Mahara ePortfolios is being used as an
alternative to YouTube to deliver a flipped classroom methodology that is
pedagogically appropriate to the trades areas, Steve Dalton is filming instructional
videos and uploading for his students and asking them to complete assessable tasks
using the same methodology.
In all Teaching Centres we are seeing sustained log-in numbers above the GPS
Learning Pathways numbers for new first year students. This is indicative of second
year students continuing to use the system beyond the lifecycle of the GPS Learning
Pathways program.
The greatest users of ePortfolios are kinaesthetic/bodily learners. However, there is
no direct correlation between learning styles and level of ePortfolio adoption. The
compelling usage is where teachers have led the charge with relevant and
contextualised delivery programs. Please refer to the case studies for further
evidence.
National VET E-learning Strategy 2012 – 2015
Page 33
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The only exception to this is the Centre for Creative Industries which was student-led.
The teachers within Creative Industries were reluctant to use the system but
hundreds of their students were using the system in a student-led model
collaboratively working on projects and uploading their videos and photographs. All
credit to this teaching centre who are now the most trained, skilled, and biggest users
in the Institute.
There is a direct correlation between Moodle Learning Management usage and
Mahara ePortfolio use. The most productive and engaged Centres are using both
systems.
We noticed a distinct drop-off of usage rates in Mahara ePortfolios new projects
when the Professional Development team were gainfully employed in a range of
other programs. This lessening of training had a direct impact on sustained uptake
rates for both teachers and students.
Mahara
ePortfolio
Mahara
ePortfolio
Mahara
ePortfolio
Mahara
ePortfolio
Mahara
ePortfolio
Unique User
Number Q1
Q1 Q12011
Unique User
Number of
Q2 Q22011
Unique User
Number of
Q3 Q32011
Unique User
Number of
Q4 Q42011
Unique User
Number of
Q1 Q12012
CCI
340
137
147
93
493
1210
ICT
210
148
178
112
361
1009
VAE
198
107
122
69
430
926
Centre
Name
Total activity
914
HACS
189
70
139
89
427
BFS
304
94
95
29
355
877
IET
357
109
231
90
7
794
BAS
183
65
81
56
274
659
ATE
49
62
116
64
197
488
ERT
124
49
71
63
148
455
BPS
35
28
56
40
228
387
CHATS
79
33
60
28
173
373
HBF
72
39
54
29
165
359
2140
941
1350
762
3258
8449
Figure 6: Mahara usage by teaching centre
National VET E-learning Strategy 2012 – 2015
Page 34
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
5.2 Quantitative data analysis: ePortfolios - Student Survey
Figure 7.1
Question 2 aimed to ascertain “What level of study are your undertaking?” The
students who responded to this survey were predominately enrolled in a Certificate III
course followed closely by students who classified themselves as Apprentices.
Diploma and Advanced Diploma students consisted of 21.1% of the survey
respondents.
Figure 7.2
National VET E-learning Strategy 2012 – 2015
Page 35
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Question 3 asked “How is your course delivered?” Students’ response to this
question was interesting. This data also reflects that students attending Box Hill
Institute on-site were the most likely respondents. 100% of the students indicated
that their course was delivered in the Classroom with only 15% indicating use of
online or workplace based learning. This data does not correspond to Institute wide
data but is accurate for the self selected respondents.
Figure 7.3
Question 4 asked the students to select their age group. Over 65% of respondents
were in the 20 to 29 age group. This is not representative of the broader age
demographic of Box Hill Institute students or those undertaking the GPS where
approximately 50% of students fall into this age range. However, this is the most
vocal of our student body and the group most willing based on past surveys to
express their opinion.
Figure 7.4
National VET E-learning Strategy 2012 – 2015
Page 36
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Question 5 asked “How would you describe your level of skills/knowledge in using
ePortfolios”? 55% of the respondents self ranked as “Beginners” and 40% assessed
themselves as Intermediate level of skill and knowledge using ePortfolios. 5% of the
students self assessed themselves as being of an “Advanced” level of
skills/knowledge using ePortfolios.
Figure 7.5
National VET E-learning Strategy 2012 – 2015
Page 37
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The question “How do use ePortfolios and why?” elicited a very interesting spread of
answers from our students. This question aimed to assess what students perceived
as features or items with ePortfolios that were Teacher Initiated versus features or
items that would be used for student initiated activities within the use of ePortfolios.
The top teacher initiated feature was the submission of assessment with 78.9% of
the students selecting this response. The second highest item ascertained to be used
by teachers was present work to others. The third highest item selected with 44.45%
of the response was undertake work on my own if teacher initiated.
The self-initiated responses were very different from the teacher initiated responses
and were far more collaborative in nature underscoring the previously discussed
issue that our teachers and students have very different learning styles and see our
systems being used quite differently. The student initiated activities were ascertained
as being learn from other students (31.6%), communicate/collaborate with fellow
students (26.3%) and both responses on 21.1% were get feedback from my teacher
and keep track of my goals. These responses reflect the way that ePortfolios have
been used within Box Hill Institute with students creating their own groups to work
collaboratively with their fellow students without teacher input. A high proportion of
students also identified that they utilised the ePortfolio to maintain a journal,
undertake reflection, and networking as self-initiated activities. These responses
have significant lessons for the future as we work to lessen the gap between teacher
and student initiated use of ePortfolios. A closer alignment around the use of
ePortfolios could thereby potentially improve the sustained uptake of ePortfolios
within Box Hill Institute.
National VET E-learning Strategy 2012 – 2015
Page 38
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.6
Question 7 asked “How often to do you use the following features?” The responses
around the features used align very closely with the features introduced through the
GPS Learning Pathways project further underscoring the need for students to be
introduced to the features within a system in order to provide knowledge and skills for
ongoing use. The top features used were equally ranked at 73.7% for both the goal
planning tool and use of the journal/blogs. The use of the resume builder was the
third highest feature used. The contact information was updated monthly by 21.1% of
the students surveyed and pages/views checked monthly by 15.8% of our students.
The only feature used weekly was update of their contact information.
National VET E-learning Strategy 2012 – 2015
Page 39
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.7
Question 9 asked “How do you use groups?” Reflecting our students being
predominately Generation Y the groups major use was for the download/share of
files/photos and videos with 64.3% of students selecting this option. The second
highest selection with 57.1% was submitting pages for assessment followed with
50% selecting receive updates from my teacher.
Figure 7.8
Question 10 “Would you still use ePortfolios if your teacher didn’t expect it?” This
response was perhaps the most illuminating of all with only 21.1% responding that
they would continue to use ePortfolios. The features that the students value within
ePortfolios can be found in a range of social media tools so once again this
reinforces the need for a compelling learning context and teacher engagement to
ensure further embedding of ePortfolios within the Box Hill Institute context.
National VET E-learning Strategy 2012 – 2015
Page 40
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.9
Question 13 “Have you accessed any training or support services to assist in using
ePortfolios? 66.7% of the students responded that teacher support was the main
form of training or support accessed. Library support was ranked at 33.3% with the
GPS Learning Pathways program ranking at 16.7%. The role of teacher is once
again reinforced in this question result.
Figure 7.010
Question 14 asked “Were you satisfied with the support provided?” 57.9% reported a
positive response.
National VET E-learning Strategy 2012 – 2015
Page 41
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.011
Question 16 asked whether the student would like to keep using ePortfolios? 85.7%
indicated that they would with 21.4% being prepared to continue the use of
ePortfolios after they leave Box Hill Institute.
Key findings
The student survey results provided very interesting data for analysis. The key
findings are listed below:
1.
The teacher is critical in providing context and support and the data analysis
supports this conjecture
2.
Students want to use ePortfolios differently to the teachers. Students want to use
ePortfolios collaboratively. Teachers tend to use ePortfolios in one way
communication – to post messages or access submitted assessment tasks.
Teachers provide the context would provide the motivation
3.
We, as educators need to change the way we write our assessment tasks to
better embed student preferences so we embed social learning and design a
collaboration context.
4.
We need to focus ongoing efforts to upskill teachers to understand the broader
potential of ePortfolios. This may take the guise of designing ePortfolio exemplar
assessment tasks to role model potential use to broader the use of ePortfolios
within a meaningful educational context.
5.
Make significant system changes to reflect the growing needs of our teachers
and students specifically for the VET sector and to improve pedagogical
outcomes. We need to redevelop the rigidity of the frame structure and make
ePortfolios much more user friendly. Alternatively we need to create or adapt a
new system to meet the needs of our students and teachers.
6.
We need to consider either re-developing the interface to be more like Facebook
or actually use Facebook for key aspects of portfolio builds. Facebook dominates
the social networking and forums industry with 56.4% of all visits in May 2010.
http://www.experian.com.au/news/hitwise-awards-jun2010.html
National VET E-learning Strategy 2012 – 2015
Page 42
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
7.
We need to educate learners in lifelong/life-wide learning concepts. Instill the
broader context and usage of ePortfolios early in the course the students have
enrolled in and guide learners to continue to use ePortfolios throughout their
course and ultimately career and life progression.
8.
Improve access to areas that students like to use such as the career planning
and tracking tools. At present there are too many layer and levels for students to
traverse – this needs to be simplified to improve usage that is not teacher led.
5.3 Quantitative data analysis: ePortfolios - Teacher Survey
Figure 7.012
Question 2 asks “What is the main age group/s of your learners?” The teachers
indicated that 60% of their learners were between 15 and 20 years old. 40% were
between 20 and 30 years old with a further 20% of the learners being between 30
and 40 years old.
National VET E-learning Strategy 2012 – 2015
Page 43
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.013
Question 3 asks the teacher to “rate your learners top three learning styles?” Our
teachers ranked their learners in numerical order as the following: Linguistic (100%),
Musical (100%) and Logical and Mathematical / Bodily/ kinaesthetic (66.7%). This
data does not accurately reflect the BHI student results, which are
Bodily/Kinaesthetic (73.1), Musical (64.9%) and Visual/Spatial (64.7%). Linguistic is
our lowest preferred learning style.
Figure 7.014
National VET E-learning Strategy 2012 – 2015
Page 44
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Question 5 looks at what level of study are your learners undertaking? Of the
teachers surveyed they indicated that 50% of their students were Diploma/ Advanced
Diploma students, 40% were Certificate 4 and 30% were Certificate III.
Figure 7.015
Question 7 asked how the course was delivered. Interestingly the teachers’
responses were very different from the students responses. 90% believed that their
course was delivered in a classroom. However, 60% believed it was workplace
based, 50% thought there was online learning and 40$ indicated there was offcampus/ distance education. Is this a perception difference where the teachers
integrated aspects of the above different from how the students defined it? It could
also be a sampling issue where the sample is so small that is has provided us with
samples of teachers that are distinctly different from the students surveyed.
Figure 7.016
National VET E-learning Strategy 2012 – 2015
Page 45
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Question 9 assessed the perceived skills and knowledge level of the teacher. The
most interesting aspect of this answer was that the teachers assessed their level of
skills and knowledge to be lower than the students assessed themselves. This has
significant implications for how we proceed with further implementation of ePortfolios.
Do we endeavour to build in assessments that are student-led so the students can
continue to evolve unencumbered by their “lesser” skilled teachers? Or do we upskill
the teachers to ensure their confidence levels improve especially in light of the
students believing that teacher led will ensure a contextual outcome? Or do we
identify students who are exceptionally skilled and willing to take on a student
mentoring role for other students?
Figure 7.017
90% of teachers surveyed had attended professional development in the area of
ePortfolios hence an indicator as to why they were using the system. 30% had
undertaken further training in creating authentic learning using video and ePortfolios,
and 20% in group based learning using ePortfolios. In addition to class based
sessions, two staff also received specialist support and mentoring in restructuring
their delivery and assessment strategy to better incorporate ePortfolios. The
relationship between training, implementation in delivery and assessment, and
National VET E-learning Strategy 2012 – 2015
Page 46
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
ongoing use reinforces the necessity for the provision of ePortfolios training and
support for both staff and students.
Figure 7.018
Question 11 asks teacher to identify whether any other support has been accessed
to assist in using ePortfolios. The results further support the assertion that we need
to provide ongoing support to assist with embedding ePortfolios within Box Hill
Institute.
Figure 7.019
Question 12 queried “Have you included ePortfolios in your delivery?” Only 70% of
teachers surveyed had embedded ePortfolios in their delivery despite completing
training and having ongoing support. Further research should be considered to
ascertain the compelling factors where ePortfolio inclusion in unit delivery is most
applicable, which would thereby provide guidance for teachers interested in
embedding eLearning in a meaningful way into their delivery strategy
National VET E-learning Strategy 2012 – 2015
Page 47
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 7.020
Question 15 asks if the teachers consolidated with other teachers to embed
ePortfolios in a unit or course. Results showed that 42.95% of teachers said they had
consulted with others. Further investigation would assist in better understanding the
extent of this consolidation, including its purpose, effectiveness and impact in the
implementation of ePortfolios with students. More significantly however, 57.1% of
teachers worked in isolation or with a support area to implement this significant
change in pedagogical practice.
Figure 7.021
National VET E-learning Strategy 2012 – 2015
Page 48
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Questions 16 asks teachers to identify the main purpose for using ePortfolios with
their learners. The two most important purposes identified was to build learners
digital/literacy employability skills (57.1%) and capture evidence and experiences in
learning (57.1%). This is a gratifying response as BeLS has worked doggedly to
promote ePortfolios as a tool to integrate workplace based learning and capture
kinaesthetic and visual assessment tool responses.
Figure 7.022
Question 17 assessed “How were ePortfolio activities directed?” While the majority of
teachers utilised a teacher led approach (57.1%), it was also identified that teachers
felt a combination of both teacher and student led was important (28.6%). This did
not fully align with the student’s response where there was a strong indication that
ePortfolio use was coupled with both teacher and student directed activities.
Figure 7.023
Question 20 asked “How should ePortfolio activities be directed?” Of the three
teachers who responded all agreed it should be both student and teacher led. This
suggests that while teachers may be delivering using a teacher led approach, they
are aware of pedagogical benefits around a student-led learning and the role of the
teacher in guiding students’ use of ePortfolios.
National VET E-learning Strategy 2012 – 2015
Page 49
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Key findings
 The teacher results indicated that a combination of teacher and student-led
activities are preferred by both
 The teacher’s perception of their skill level was lower than the student assessed.
We attributed this to the difference between the students assessing their
knowledge as a user as opposed to a teacher needing to assess their skills
against being component enough to lead a group. However, it does require the
BeLS Team to provide more support and assistance to teachers but to also
encourage them to take on a “concierge” role where they facilitate the process and
are prepared to allow the students to lead initiatives and ideas for ePortfolio use
which could arguably further engage the students in the meaningful use of this
system. Teachers could negotiate the context that could map to SNR
requirements, industry outcomes and students interests.
 Teacher training is critical to ensure uptake and the embedding of ePortfolios
within and across units and courses
 The more Teacher training the teacher engaged in the greater the chance of
meaningful uptake, However once two to three sessions had been attended then
high quality support was required to assist the teacher with the implementation
strategy.
 Including ePortfolios in the assessment strategy provides a context for learners.
We need to further consider the design and development of exemplars and
templates to role model best practice within the Institute and have the resources
required achieving this. Students should be used to review these templates so
their thoughts and opinions are included in the design and development process.
 Once we have identified ePortfolio advocates we need to build a community of
practice to support them to further upskill and mentor teachers within their own
teaching centres. A BeLS led model has worked well for initial implementation
however we need to support teacher centre based contextual experts to work with
their colleagues to achieve a more wide spread adoption within an appropriate
and customized course based context.
 We need to adjust or broaden our current ePortfolios training program to reflect
the outcomes of this study specifically pertaining to embedding ePortfolio use in
pedagogical outcomes as opposed to system use. We need to include more
collaborative use and include teacher experts as part of the training. This can be
achieved through the use of video case studies filmed.
National VET E-learning Strategy 2012 – 2015
Page 50
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
 BeLS staff also need to promote and provide ongoing support to teaching staff as
required as often innovation in delivery done in isolation. Box Hill Institute studentled approach has assisted teachers to feel confident about continuing to use this
system but we have more work to do in creating a truly student/teacher-led
approach systematically across the Institute through careful planning using our
current “plant, water and harvest” innovation cycle
.
National VET E-learning Strategy 2012 – 2015
Page 51
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
5.4 Qualitative Data Analysis: E-portfolio implementation
including case studies
Case Study 1: Box Hill Institute: GPS Learning Pathways
Background
Course and Unit
Certificate 1 in Certificate of General Education
for Adults (CGEA): VBQU130 Develop and
document a learning plan and portfolio
Concept
Box Hill Institute students responses within the student satisfaction surveys
completed annually identified a range of on-going issues. These issues included lack
of familiarity with Institute systems, support services, greater integration with the
workplace, support with career planning and lack of knowledge around pathways.
Students often indicated they wanted a consistent induction to Box Hill Institute and a
louder voice in the future direction of teaching and assessment within the Institute.
Our students also wanted greater acknowledgment of them as individual students
with unique skills, knowledge and learning needs.
The GPS Learning Pathways project evolved to meet Student identified needs. The
program included a personal plan, education plan and career plan. Students were
asked to plan their short and long term goals, complete a S.W.O.T. analysis,
complete a learning styles analysis and commence building their own resume. Our
focus is on life long and life wide learning pathways providing linkages between
learning, support services, and personal, learning and career objectives
While designing this project, BeLS was also designing and developing the Institute
wide implementation of the new Student Web which was a system that integrated:

A customised Moodle Learning Management System

A customised Mahara ePortfolio System

The integration of social media including Facebook, Twitter and YouTube
National VET E-learning Strategy 2012 – 2015
Page 52
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The GPS Learning Pathways project was designed to induct and orientate all our
new students into this new system and as a result the assessment strategy of this
program was designed to integrate both Moodle and Mahara into the students
program.
Delivery mode
The program commenced in 2010, training over six hundred teachers in the use of
the Moodle Learning Management System (LMS) and Mahara ePortfolios. A second
project ensured that all teachers reviewed their existing eLearning resources and
assessed whether they should be deleted, archived or migrated into the new system.
In 2011, 3400 students were provided a four hour face to face GPS Learning
Pathways training session. Three assessments were embedded into the program
with the critical final assessment task being embedded within Mahara ePortfolios.
The students completed the assessments using a combination of in our student
Learning Management System and the Mahara ePortfolio. Assessments were
undertaken in the student orientation and induction class, at home, and in some
cases by using student iPads.
Participants
Age range 15 – 60
All new students within the Institute were enrolled into this program. The specific
demographics to date are :
Age group
Percentage completing GPS Learning Pathways
15-19
47%
20-24
29%
25-34
13%
35-44
6%
45+
5%
National VET E-learning Strategy 2012 – 2015
Page 53
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Rationale (Reasoning for choosing ePortfolios)
Goals
The rationale for introducing ePortfolios into the student Induction and Orientation
program was to:

induct learners into ePortfolios as part of a broader strategy

educate students into how ePortfolios can be used to support current and future
personal and professional development endeavours

emphasise that while the Moodle LMS is Institute owned, the provision of
Mahara ePortfolios is serviced by a student led, student owned philosophy.

provide a means for students to capture authentic learning experiences that
could be used for assessment and recreational purposes,

facilitate transition into further study and employment through the use of the
resume builder and ultimately the capacity to share resources including
assessment tasks with their employers
Implementation considerations
Prior to establishing an Institute wide ePortfolio, Box Hill Institute were cognisant of
selecting a system which best meet the following criteria:

Transferrable
To support a student-owned, lifelong learning approach, it was integral that
student ePortfolios could be transferred to other ePortfolio systems where VET
students typically transition from their current studies into further education or
business. For this reason, compliance with Leap 2A standards set out by the
IMS Global Learning Consortium was imperative.

Interoperable and accessible
As technology advances the range of devices being used by students is
constantly growing and changing. The ePortfolio chosen needed to work across
a range of devices and Institute learning systems, including the student records
system (SMART) and enable single-sign-on (SSO) for students with the Moodle
LMS. Furthermore, due to the large number of enrolments BHI receive each year
the ePortfolio would need to enable automatic account creations for students by
supporting LDAP.

Support and scalability
The capability for the system to integrate with current and future internal
National VET E-learning Strategy 2012 – 2015
Page 54
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
infrastructure and resources was a key determinate, along with the level of
resources required to provide internal support including storage, staffing and
maintenance.

Longevity
Due to the ongoing level of demand placed on the internal IT resources and
support team, consideration into the degree of external support for the provision
of updated ePortfolio versions is required. While open source systems have
many advantages there is a large associated risk if the open source community
dissipates.

Customisable
The degree to which the ePortfolio can be internally customised to better meet
Institute, including teacher, student, infrastructure, and support needs.

Usable
The ease of use of the system including system navigation and compliance with
W3C design and accessibility standards was a factor in determining whether the
system would be appropriate for our users.

Tracking
To extend enable our traffic light reporting to consider the uptake of ePortfolios,
the ability to collect and track its usage across teaching centres was preferred.
The implementation strategy also required:

A strategy to train and support teachers and students, that fostered an
organisational approach, consultation with teaching and service centres so the
approach could be customised to meet each centres unique needs, and ensured
a consistent induction into Institute systems for students, and

Internal business rules to be established including guidelines around security,
privacy, and copyright.
Discussion
Critical success factors
We needed a system that would allow Box Hill Institute students to:

Gather evidence of a broad nature including documents, video, photographs

Integrate quizzes that could be fed into the students resume builder for future
use

Allow students to identify and communicate their learning needs
National VET E-learning Strategy 2012 – 2015
Page 55
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios

Enable teachers and students to review and provide feedback to others

Develop employability skills including digital literacy, communication,
collaboration, and presentation skills

Develop the capacity for our students to reflect and articulate these learnings
into future actions.

Participate in collaborative learning by sharing and building knowledge and
experiences with others

Collect and reflect on informal learning experiences

Capture evidence of work undertaken in work placements, a critical means for
students to apply their learning in an authentic context, as well as develop new
skills and understanding. ePortfolios will ultimately provide our students a closer
link between the workplace and their studies enabling them to review and reflect
on their experiences and provide a better integration between the workplace and
educational Institution.
Challenges
Mahara ePortfolios proved to be an unwieldy system that sought to work as a system
in isolation. The build parameters were not conducive to achieving what we wished to
achieve – both a push and pull strategy. Our ultimate aim was for students to submit
their assessment tasks into either system and we could automatically back populate
into the other system.
The frame structure within Mahara has also proven to be limiting and not always
teacher and student friendly. The non-intuitive upload and download features tends
to be step heavy and needs to be further simplified for quicker and more seamless
use.
We also found the reporting capability of Mahara was extremely limited and did not
allow us to create longitudinal data that could be tracked. For this report we had to
ultimately access all archived data and hand enter this data to achieve a longitudinal
study. Greater work needs to be completed in improving the reporting depth of
Mahara for BHI to continue to be able to track long-term usage within our quarterly
Traffic Light Reports. We are working to find a hard coded option for improving a
range of the outputs and inputs to further be able to use the Mahara ePortfolios
system to meet the emerging personalised learning pedagogy being implemented
within BHI.
National VET E-learning Strategy 2012 – 2015
Page 56
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Lessons learnt
We made significant assumptions when scoping both the Student Web and GPS
Learning Pathways project. We assumed that Mahara ePortfolios would have similar
capability that we found in Moodle.
Assumptions included:

Reporting needs work so that cached data at the end of the day remains so we
can analyze real data

Frame structure – very hard to adapt and change to customize – not
unachievable

GUI Interface – need greater capability to brand to the organization

Ease of use – the process to upload documents and navigate around the system
proved to be challenging and not intuitive.

Interoperability – was not simple to integrate with other systems, including the
LMS where there is still only limited functionality for transferring assessments
between the two systems.
Future objectives

Greater linkages between classroom based learning, online learning, the
workplace and industry

Promote a higher level of involvement from the teaching centres through
assisting them create tailored delivery models (potential team teaching)

Promote better assessment tool design and development that ensures that an
ePortfolio is an integral formative or summative assessment tasks in the majority
of our VET and Higher Education courses.

Assist in the skills assessment and RPL Process so that skills and knowledge
can be documented over a lifetime

Integrate with an Alumni mode and provide this system as an on-selling option
for students that enroll in our Alumni

Greater integration with student support to improve student engagement and
retention of at risk students, and to increase the number of referrals to learning
support and careers advice when flagged appropriately within the ePortfolio or
quizzes in Moodle.

Better support for a range of media including photos, video and audio which are
key for our learners

Ongoing professional development and support for teachers and students with a
focus on both pedagogy and technology
National VET E-learning Strategy 2012 – 2015
Page 57
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios

Further research to better understand what is needed from the teacher and
students point of view.
There is no simple passage in learning from ‘easy’ to ‘difficult’; mastery of a
threshold concept often involves messy journeys back, forth and across
conceptual terrain. (Cousin, 2006)
National VET E-learning Strategy 2012 – 2015
Page 58
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Case study 2: ePortfolios to support workplace internships in the
Bachelor of Fashion, 2011
Debbie Pratt: Teacher, Bachelor of Fashion
Background
Course name: BAF01 Bachelor of Fashion
The Bachelor of Fashion is a 3 year practical degree which is focused on teaching
students both fashion design and the fundamentals of operating within business and
industry.
Unit: ISTU321 Workplace Internship, a 13 week unit that is undertaken in the
students final semester of study designed to consolidate knowledge and skills
learners across the course.
Delivery mode:
The unit is delivered using a blended learning approach combining classroom-based,
work-based, and online delivery modes. Students attend a face to face classe run
every 3 weeks across the 13 week unit, and all other communications are facilitated
through the ePortfolio and Moodle.
Participants
This class consists of 9 female students aged between18-22.
Learning styles
The group is identified as being highly visual, kinesthetic and musical. Their bottom 3
learning styles are logical/mathematical, intrapersonal, and verbal/linguistic.
Rationale
Goals
The unit requires students to gather and present evidence to demonstrate personal
and professional development within their workplace internship. Learners are
required to consolidate knowledge built throughout their course in a journal that
provides a reflection of their workplace internship learnings, and assesses personal
and professional development, goals and performance. By the end of this unit,
learners are expected to have developed the skills required to undertake critical
analysis and reflection, while developing a portfolio that demonstrates their
National VET E-learning Strategy 2012 – 2015
Page 59
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
capabilities. As secondary objectives students have the opportunity to establish a
network of industry contacts to assist them in securing employment prior to
graduation, and; develop a portfolio that they can share with future employers
Delivery and assessment strategy
The Mahara ePortfolio was used extensively to enable students to collect, reflect,
and share work samples with others. It also allowed learners to operate as part of an
online community where peer learning and feedback occurred.
Across the unit students undertook a weekly practical placement that required them
to create an ePortfolio page that including the following components:

Professional resume and cover letter for application to workplace internship
position.

A database of network contacts established during the Internship.

A workplace supervisor reference

A work action plan listing goals the student would like to achieve in their
Internship

Photos evidencing work undertaken during their internship

A weekly blog posting describing experiences, key learnings and reflections in a
personal and professional context.
Each week, learners answered a different question in a Moodle discussion forum
around the work placement and were required to contribute feedback and advice to
other student’s comments.
Every three weeks learners participated in a face-to-face class to discuss the industry
placement, and further explore the learning’s and experiences documented in their
ePortfolio and Moodle forum.
In the last week of the unit, students contributed a summative reflection to their
ePortfolio considering their personal and professional development. They were also
required to share their learning with other students in a class presentation.
Figure 8 outlines the blended learning delivery and assessment strategy utilised for
this 13 week unit.
National VET E-learning Strategy 2012 – 2015
Page 60
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Figure 8: Bachelor of Fashion workplace internship delivery and assessment
strategy.
Discussion
Key findings
The nature of the workplace internship unit leant itself well to the delivery and
assessment strategy used where the ePortfolio had “all the features that [students]
needed to document their learning and experience, undertake critical analysis and
reflection, and share and present their work to each other” (Pratt, D. 2012)
While it was acknowledged that “…initially students didn’t really understand the
purpose, and like many new things said comments such as ‘why do I have to do
this’….As they began to use their portfolio, they started to realise the benefits.” (Pratt,
D., 2012)
From a student perspective, the ePortfolio provided a meaningful experience an easy
way to submit work without paper wastage; a platform to communicate with others; a
basis for reflection; a showcase of work for job applications and assisted in improving
organisational and digital literacy skills.
National VET E-learning Strategy 2012 – 2015
Page 61
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Critical success factors
The following factors contributed to the success of this implementation:

Use of planning tools to create goals and track progress
Goal setting early in the course paired with regular updates using the Mahara
planning tool allowed students to track their progress and identify the degree to
which they had completed their goals. This can be seen to assist in reflection
and in building a sense of purpose and motivation for students “emphasising
students' responsibility for their own learning.” (Src:
http://eresources.tafe.tas.edu.au/epw/setting-goals/index.html ). For example, a
student who had undertaken a placement at Loreal Melbourne Fashion Festival
reached her goal to create a professional network of up to ten people by week
eight and felt a large sense of achievement out of this.

Assessment led learning approach
“If assessment wasn’t surrounding its use I don’t think it would have been
used”(Pratt, D). As with a progress portfolio, this approach provides structures
and opportunities for learners to “organize, reflect on, and revise project artefacts
at various phases” (Land et al. 2003, p.1810 and can also help to foster student
agency in a collaborative learning environment.

Encourage peer learning by establishing an online community through linkages
between the face-to-face and online environment
The ePortfolio group began to establish a sense of community after the first faceto-face session where discussions were facilitated by the teacher around
progress and points of interest in each student’s ePortfolio. This discussion
generated a level of curiosity and fear of ‘missing out’ in the online learning
experience by students who hadn’t yet participated. Students encountered
learning and personal benefits from the ePortfolio community, and the feedback
from peers provided a way to reinforce learning.

Regular feedback to improve motivation, participation, and learning outcomes.
Weekly online feedback teamed with a classroom discussion around reflective
practices helped shift learners’ skills from documenting events and activities into
undertaking critical analysis and reflection. As students began to participate in
the peer review process, they started to understand the breadth of potential
experiences from one workplace to the next and a greater depth of
understanding was established.

Professional development for staff
The teacher had undertaken group and individual professional development to
National VET E-learning Strategy 2012 – 2015
Page 62
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
establish an effective delivery and assessment strategy and build ideas in how
ePortfolios could fit specifically into the unit.

Mix of teacher-led and student-led
The assessment strategy empowered learners to take control of their learning
experience, where they were responsible for tracking progress, and providing
feedback to others thereby creating a constructive community of learners that
could compare progress, advise one another, and engage in online social
collaboration making the time they had away from class more enjoyable. The
teacher also deliberately excluded presentation guidelines which resulted in a
diverse mix of individualised ePortfolios.
Challenges
The teacher encountered the following challenges:

Student ownership beyond the life of the unit
While lifelong learning and career benefits were recognised by the teacher,
students didn’t continue to use their ePortfolio beyond the duration of the unit. “In
terms of ongoing use, maybe I didn’t sell it well enough…I don’t think any of the
students have used it since. They really just look at it as a tool that they will use
to help them in their studies and that’s it” (Pratt, D).

Familiarity in using the ePortfolio system
While of benefit to the learners, the lack of assessment presentation standards
presented challenges for the teacher when reviewing blog submissions which
can be displayed in Mahara in three different ways. The teacher expressed
however that she was not deterred by this and that it simply meant that more
time was needed to get used to the ePortfolio environment and that further
training would improve her familiarity with the system.

Not aware of support available to students
“Initially, it was the students and I plunging into the unknown together” (Pratt, D).
The teacher trained the students however feels as though they would have also
benefited from an initial training session by staff who were familiar with the
system.
In addition, the students encountered the following challenges:

Lack of customisation
Students had commented that the page layout didn’t provide a satisfactory level
of personalisation, flexiblity, or creativity for presenting work outside of their
curriculum and to potential employers. The column format in particular was
National VET E-learning Strategy 2012 – 2015
Page 63
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
described as limiting. This feedback from students was an influencing factor for
other teachers across the fashion degree and diploma courses to opt against
including the Mahara ePortfolio in their delivery (Debbie P).

Confusing navigation
Students felt that navigating around the ePortfolio was initially highly confusing
with ‘too many sections’.

Confusing usability
The students felt as though the functionality wasn’t highly intuitive or simplistic
where it took a considerable amount of time outside class for learners to identify
how to add documents, create pages, and then display the content in a neat,
presentable, and individualised layout.

Lack of training and support
Students were able to access support through Library staff however felt that a
dedicated follow up face to face training session outside of the GPS Learning
Pathways program in how to use the ePortfolio would have been of most benefit.
Lessons learnt
Educate learners and teachers
Spend more time in educating learners and teachers in what an ePortfolio is, how to
operate within the ePortfolio, and provide strategies for use as a lifelong learning and
career planning tool, which can be seen to promote richer and more complex
illustrations of learning (Gabriella, M.B. et al, 2008, p1)
Adopt a course wide, life wide approach

The student and teacher both saw benefits of utilising the ePortfolio across the
course, to track fashion styles and influences, integrate knowledge and skills
acquired within their course, and display work from their design showcase which
is also undertaken the in the final semester of study. The student perceived that
the ePortfolio could act as a ‘one-stop-shop’ to display and upload folio work and
assignments and identified that the benefits would not only entail ease of
submission, but would better facilitate sharing and presenting designs to others,
and minimize the environmental and financial impact of resources and money
required for paper based submissions.

Integrate digital elements into assessment
Designing delivery and assessment strategies that allow students to build a
collection of experiences, achievements, and evidence of learning in the format
National VET E-learning Strategy 2012 – 2015
Page 64
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
of photos, videos, and audio will enable students to commence their ePortfolio
with minimal additional effort during their course, and by having a rich foundation
of consolidated learnings, students would be more likely to feel a sense of
ownership and commitment to extend their ePortfolio to fulfill other purposes end
of their studies.
One size doesn’t fit all
Students introduced teachers to the Web 2.0 tool ‘Pintrest’ which they indicated
provided a more personalised, professional, and accessible layout of the display
of photos evidencing class work and designs. There is reason to investigate
other ePortfolio services that the students are already engaged in so that class
time is maximised by exploration of the unit rather than the ePortfolio tool itself.

Recognition of industry and organisational environment
The student indicated that she didn’t plan on using ePortfolios to present to
others such as potential employers, or to assist in career advancement due to
the lack of personalisation and professionalism of Mahara and the preferred
online job application processes as set out by organisations and recruitment
agencies. Consideration needs to be made as to the feasibility of suggesting the
ePortfolio is a suitable means for presenting to employers when the potential
employers themselves may not online resumes in the form of an ePortfolio.
National VET E-learning Strategy 2012 – 2015
Page 65
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Case Study 3: Certificate III Diploma of Children’s Services - Carol
Ong: Teacher in Health and Community Services
Background
Unit: Cert III Diploma of Children’s Services
Carol teaches a few groups in this course and has created an ePortfolio group for
each of them.
Delivery mode
This year Carol is teaching almost all the units in the whole course a one of the
groups, so she can build a closer relationship with the students and follow up on their
learning pathway. Carol also finds that this gives her time to follow up on the
ePortfolio introduction from the GPS. The course is delivered entirely face-to-face
although Carol puts materials in their Moodle course as well.
Participants
Age range 17 – 60
The computer skills are quite varied due to life experiences. The younger students
are familiar with social networking sites like Facebook which they use on a daily
basis but they don’t have much experience using PowerPoint. These students prefer
to sit around a table and collate shared information which they cut and paste onto a
poster. They find this less stressful than PowerPoint - they like the interaction of
sitting together, the manipulation of materials, and sharing face-to-face. They are
less keen to work on the computer because they would then work in isolation at
home.
Some of the other learners who are from professional backgrounds are used to
PowerPoint and comfortable with it. These students prefer to work at home when it is
more convenient and can fit around family and other commitments.
Rationale (Reasoning for choosing ePortfolios)
Goals
To create an area where students can access the files created by classmates.
File repository is the main group activity. The class presentations can have very large
PowerPoint files and sharing them through the ePortfolios file area is very convenient
and saves filling up the inbox with large emails. . It is also good for ESL students who
National VET E-learning Strategy 2012 – 2015
Page 66
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
may not have understood everything the first time; they really appreciate the
opportunity to revise the material.
Delivery and assessment strategy
Carol has created a class group in the ePortfolio and has urged her students to use it
to develop social groups and submit their work. Her students have been asking for a
Facebook page and she wants to use the ePortfolio in a similar way for collaboration,
social learning and sharing. Carol has encouraged the use of forums but the students
just don’t seem interested in using them.
An ePortfolio group is created for each group Carol teaches. She encourages
younger students to explore the potential and selects one to act as an eFacilitator to
manage the ePortfolio group. In another group a student has been very active in
uploading files, adding weblinks, and so on. However, the main use for their class
ePortfolio is file-sharing and they all use it for that purpose.
Carol puts a lot of the learning materials up in the Moodle course and has asked the
other teachers to do the same but they don’t seem to get to this task. They email the
materials to their students which sets up an expectation in the students. Carol tries to
explain to the other teachers that it is ultimately more efficient to have all the
materials accessible to the learners online in the Moodle course rather than email
them to individual students.
All students in Certificate III Diploma of Children’s Services are enrolled in the same
Moodle shell so now they have access to all the materials, which Carol uploads. It
only benefits the students to have the teachers share the resources and materials.
Once a course is set up then most of the work is done and it saves a lot of time from
year to year; minor tweaks are all that are necessary. Carol seems to work alone
without much support from the Centre. She accesses the help from the eLearning
Helpdesk and has very good computer skills so she has taught herself the skills that
she needs.
National VET E-learning Strategy 2012 – 2015
Page 67
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Discussion
Critical success factors
Carol has been the main driver of this resource and her implementation of other
aspects of ePortfolios is limited by her lack of skill. While she has been very active in
accessing the support offered for Moodle through training sessions in the PD
program and the eLearning Helpdesk, Carol has not sought the same assistance for
ePortfolios. However, she is very keen to learn because she sees blended learning
as a delivery model that offers flexibility and variety to a diverse group of students
who need to be actively engaged in interactive, social learning.
Challenges
When Carol organises a session in a computer lab she finds that the computers are
inadequate for the group. The younger learners do not have the patience to wait for a
page to load and often slip into Facebook while they wait for StudentWeb to load.
When they do get some work done they don’t have memory sticks with them so
cannot save anything and then ask if they can just do it at home. All of which is very
frustrating for the teacher.
Carol has recommended using ePortfolios to other teachers but they don’t take it up.
She would like there to be a Centre-wide uptake by the teachers so that there is
consistency across the course for all students. As Carol is the only one promoting
their use, the students question why they should have to use ePortfolios when the
other teachers don’t use them. They resent the differing approach by different
teachers. The email is an example of inconsistency. Carol tries to insist that the
students check their BHI email but the other teachers will accept the student’s
personal emails so, again, there is inconsistency.
Lessons learnt
Carol would like every course to have an online presence because the students won’t
bother if only one of their teachers uses it. It is consistency of approach across the
Centre that will drive the message home that a blended learning delivery model has
advantages for everyone.
Carol has not accessed any training or support for the ePortfolios. Anything she does
has been picked up from attending the GPS induction with her students.
It would be useful for all the students to have a follow up ePortfolio training session
so that they can see the potential of them. If GPS ePortfolio training is followed up in
National VET E-learning Strategy 2012 – 2015
Page 68
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
a class with the teacher and the students, then everyone can share in development
of this valuable tool. If students were given more opportunities to develop ePortfolio
knowledge and skills then more activities would be student-led as they discovered
different formats for assignments, reflective journals, assessments and collaborative
learning.
There’s a lot more to explore and Carol would like to expand her knowledge and
skills to further develop this mode of delivery.
National VET E-learning Strategy 2012 – 2015
Page 69
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Case study 4: Stephen Dalton: Teacher in the Electrical and
Refrigeration Trades centre.
Background
Course name: UEE099 Cert III in Electrotechnology (Electrician)
This is a three year course and this group of students was in the 2nd year.
Delivery mode
The students are full-time apprentices. They attend TAFE one day a week from 8.00
to 4.30 during the teaching term for the duration of their course. This amounts to
1,060 hours of face-to-face classroom and practical workshop delivery.
The units are studied in blocks. The students would have a practical block or a theory
block which runs over several weeks. Most of the time at TAFE is spent on
knowledge-based learning in a classroom. Some units of competence are 40 hours
(5 weeks) and other units are 120 hours of classroom-based theory (15 weeks) which
doesn’t suit the kinaesthetic learners. Depending on the subject, bench-type wiring
exercises during the theory classes are possible but it depends on classroom and
equipment availability.
Participants
This class is a second year group of 14 males and 1 female, with ages ranging
between 18 and 25 years old.
Learning styles:
The group is identified as being largely kinaesthetic and this practical unit which gave
them the opportunity to demonstrate their competence in testing procedures was
very popular with the students.
Rationale (Reasoning for choosing ePortfolios)
Goals
The students had to demonstrate that they could work through a range of practical
tasks of mandatory testing. The mandatory testing procedures consist of 6 tests –
two power available and four power off. The students were to demonstrate four
power-off tests.
National VET E-learning Strategy 2012 – 2015
Page 70
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Delivery and assessment strategy
This was an 80 hr subject (10 weeks) and students needed to show that they could
wire up certain equipment types including domestic switchboards, motors, etc. The
class was divided into groups of four and each group had the set of four mandatory
power-off tests to perform. Each student had one test that they had to film
themselves performing with their phones, but they were required to do all four tests
as part of their assessment. As they took turns performing the tasks and went
through the process of filming each other, they could observe, learn and improve
their understanding of each procedure.
Each person in the group did the four mandatory power-off tests and the group had
to ensure that they had one video of each test that they needed to learn. They had to
ensure that they had one video of each of them to upload to the YouTube account
created by the teacher. Each of the four groups had a page created in the class
ePortfolio and the links to their videos were inserted.
Later in class they watched the videos and observed the same test performed by
each group over four different pieces of equipment. They could compare tests and
watch for correct procedures and make note of errors that needed improving. There
was discussion in these sessions which critically evaluated the testing procedures
highlighting correct procedures; the validity of the tests; and how to correctly interpret
the results.
This project was initiated and maintained by the teacher. Stephen set up a class
group in Box Hill Institute’s ePortfolio system and ensured that everyone joined.
Stephen also created a YouTube account into which the students uploaded the
videos directly from their phones. He then made the undiscoverable links onto each
group’s page. Videos were made undiscoverable which means that they are private
and cannot be found by searching.
Stephen had this group again the next year, their third and final year, and they had to
perform the same test that they had performed the previous year but on a piece of
equipment wired by someone else which more closely replicates real-life situations.
In the licensing exam they have 20 minutes to perform all required tests. They
needed to practice this and Stephen used to videos as a refresher and also as
evidence that they could do the tests.
National VET E-learning Strategy 2012 – 2015
Page 71
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Discussion
Critical success factors
All students were happy to participate in this project. They were enthusiastic about
having the opportunity to use their phones in a meaningful context to enhance their
learning; to facilitate opportunities to develop their skills; and to gather evidence of
competency.
The group that did the initial testing was not comfortable with watching videos of
themselves in class but they trusted the process and really appreciated the
opportunity the videos gave them for revision when they had to perform the same
tests for their exams the following year.
Permission was given by this video group to show their videos to other groups to
review testing procedures. These other groups really benefitted from the opportunity
to observe the tests with which were unfamiliar. A forum was also created for these
groups to discuss the procedures and observe a variety of tests over different pieces
of equipment. They were better able to critically evaluate the procedures being
demonstrated and learn from them.
Using ePortfolios in this way was very effective for this unit about testing circuits and
Stephen has put a description of the task into the student handbook and promoted it
to other teachers in the centre. The students really responded well to this strategy
and enjoyed the opportunity to use their phones to assist in developing their skills by
watching their performances and comparing them to others.
Challenges

Accessibility to Mahara
Making sure that everyone was logged in and able to find their way to the
ePortfolio and their Moodle page was a real challenge as there is limited
computer access in the classroom. Stephen had to take the students to the
library to get assistance with the logins and sometimes to give the students an
opportunity to participate in the forums.

No full screen option for video
The videos couldn’t be displayed in full screen size from the ePortfolio page. The
ePortfolio page doesn’t have that capability and the videos are displayed in a
quarter of the page. This is not really suitable when you want to watch them and
generate discussion in the classroom. Stephen now puts the videos into the
National VET E-learning Strategy 2012 – 2015
Page 72
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
class page in Moodle because it can display videos in full screen mode. He has
obtained permission from the group to show the videos to other classes for
instruction and review.

Digital literacy of learners
The digital literacy of the students was not as high as expected. There were
gaps in the knowledge and skills and time was spent up-skilling the students in
the use of the technology. Many of the students had never connected their
phones to their computers to synch photos or videos and neither had they
uploaded videos to YouTube.
Lessons learnt
Using ePortfolios in this way has allowed Stephen to access the different learning
styles of his students. It took them out of a routine task and elevated it into a
collaborative, meaningful learning activity that has had benefits far beyond those
originally planned. The students really enjoyed using their phones to gather the
evidence needed for assessment of the mandatory tests. The added bonus was the
reviewing and the critical evaluation of their performances as well as those of others.
Learning to give and receive feedback is a very mature skill and difficult to develop in
isolation. With the participants’ permission, Stephen has shown the videos to other
groups who have not yet had the opportunity to perform these tests. They also apply
a critical eye to the tasks and look for procedures and practices that they need to
learn. It applies the theory that they need to understand and gives a practical
reference for testing procedures.
National VET E-learning Strategy 2012 – 2015
Page 73
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Case study 5: ePortfolios for collaboration in Certificate IV in
Laboratory Techniques, 2012: Heidi Wenk
Teacher, Animal Studies
eLearning Advocate, Centre for Biotechnology and Animal Sciences
Background
Course name: MSL40109 Certificate IV in Laboratory Techniques
Unit: MSAENV272B Participate in Environmentally Sustainable Work Practices
This competency covers the outcomes required to effectively measure current
resource use and carry out improvements including those reducing negative
environmental impacts of work practices.
Delivery mode
Blended learning approach for on campus students using the student Learning
Management System, and Mahara ePortfolios.
Participants
Sixteen students, mostly female, aged 18-22 years.
Learning styles
Learners were predominantly linguistic, logical/mathematical and visual.
Rationale
Goals
This unit requires students to come up with their own project around sustainable
practice. The purpose of using e-portfolios was “not so much to share learning, but to
share resources” (Wenk, H).
With a key part of the unit centred around communication and consultation, it was
anticipated that the ePortfolio would:

Allow students to gather, upload and share a collection of photos of samples of
waste and ways that waste can be managed. This was intended to help learners
compile their individual reports.

Act as a collaborative space to assist students in project planning and
implementation, and thereby facilitate student-student discussions in addition to
teacher-student discussions.
National VET E-learning Strategy 2012 – 2015
Page 74
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Delivery and assessment strategy
While the assessment strategy is consistent for all classes, the teacher indicated that
the delivery strategy in this unit is often dictated to best suit the needs of the learners
project objectives. For this group, the following delivery strategy was used:

Face to face classes – students attended weekly classes that were delivered
using an approach the teacher termed ‘interactive whiteboards. This method a
PowerPoint being created in class with the students through discussion linking
back to the learning.

StudentWeb (Moodle) was used:
o
to deliver the required learning resources
o
deliver online assessments including quizzes and an online discussion
forum

The Mahara ePortfolio was an informal space to enable students to collect and
share photos needed for their report.
Discussion
Key findings
This implementation strategy using ePortfolios was considered a trial to help
determine future practice, however was observed to present many challenges that
resulted in the recommendation to use a different strategy in the future to meet the
intended purpose of photo sharing and collaboration between learners.
Critical success factors
For this implementation strategy to be a success the teacher acknowledged that
factors presented in the list of challenges below would need to be addressed in order
to aid the approach or improve the level of participation in ePortfolios within a unit.
Challenges
The teacher encountered the following challenges:

Time to build community, and encourage communication and consultation
The teacher attempted to create linkages between the class and online
environment to encourage uptake, however acknowledged that beyond the early
adopters there was no further interest.

Lack of student engagement and participation
The level of activity within the ePortfolio group was low with only a few students
National VET E-learning Strategy 2012 – 2015
Page 75
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
joining the group, one student uploading a video around the subject of
sustainability, and no students uploading a photo. It was acknowledged that the
students who thought they course get personal or educational benefits out of
joining the group were the most likely to participate.

Separating the need to do from the nice to do
With students being expected to participate in class, in StudentWeb, and in their
ePortfolio the purpose of using the ePortfolio became a lower priority. As it
wasn’t a part of the unit requirements the teacher acknowledged the “students
may not have seen the relevance of it and the way it could have assisted them”
where “the need to do’s happen and the nice to do’s don’t” (Wenk, H).

Assumption that students had the appropriate skills set
“It may not have been the tool…it could have been the students not having the
confidence to do it.” Beyond the initial student induction, no additional training in
how to use the system was provided and suggested by the teacher that a more
effective approach could have been to ask students to upload their photos in
class time.

Navigation of system
It was highlighted that the navigation in Mahara wasn’t intuitive, and that the
navigation adds another layer of complexity for both first time users, and for
teachers and students who don’t actively use the system.

Time for students to develop skills in using ePortfolios
While a greater level of adoption was noted amongst the ‘tech savvy’ students,
the teacher acknowledged that the time that would be required to establish entry
level skills in all students using ePortfolios isn’t available for courses lasting less
than one year whereby: “In what is an ever decreasing number of contact hours
with students, to lose time by teaching them how to use the technology is just not
an option anymore, and I can see that’s just going to get worse” (Wenk, H)..
Lessons learnt

Better integrate at course level to add relevance and a depth for learners.
To integrate ePortfolios across all units in a targeted course as a focused
initiative with an agreed student objective, and initial as well as follow up training
sessions. This means that skills in using the ePortfolio could be reinforced
across the whole course for learners, and they would thereby establish a greater
depth of understanding in its potential benefits and could generate a teaching
community focused around cross skilling and holistic delivery and assessment.
National VET E-learning Strategy 2012 – 2015
Page 76
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios

Better linkages between GPS Student Induction and classroom delivery.
A greater level of involvement in the student induction from the teaching centre
would enable:
1.
the student induction to be more closely aligned to classroom delivery
where “maybe I was asking them to do something that was completely
unfamiliar and foreign to them, so that immediately created the barrier”
(Wenk, H).
2.
Teachers to understand what aspects of the ePortfolio students have
been introduced to.

Assessment led approaches create relevance and instill motivation within the
learners to participate
Students engaged with assessed online components in StudentWeb, including
the forums, and quizzes, suggested that the online environment wasn’t the
blocker and “I think the only way that I will be able to get learners to use it is to
link it to an assessment” (Wenk, H).
Face-to-face collaboration challenges reflect online challenges
“We can’t expect the technology to deliver something that probably doesn’t come
naturally in the classroom either...If they’re active contributors in class, they’ll
tend to be active contributors in the technology as well” (Wenk, H). The teacher
acknowledged that the students generally didn’t enjoy group activities online or
in the face to face environment whereby:
o
only about 5% of the students engaged with each other in the
discussion forums
o
the face-to-face interactive whiteboards approach didn’t always work
and was depending on the students willingness to contribute.
Keep it simple for teachers
“There are a tsunami of things that we are meant to be doing, and I’m crawling
back into my comfort zone” (Wenk, H)
While the teacher has undertaken training in ePortfolios and is aware of the
available support, she acknowledged that the challenges encountered in terms
of navigation, understanding best practices for use, identifying its key point of
difference from the learning management system, and finding the necessary
time to become “immersed” in the system have led to “spreading ourselves really
thinly” and thereby opting against further exploration and integration into future
training and assessment strategies.
National VET E-learning Strategy 2012 – 2015
Page 77
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios

Keep it simple for students with a student-owned system
“Unless the students come to me and are fully functional in it I would probably
not choose to use it with them again as it adds another layer of complexity to
what they are already trying to learn in the unit, and has a real potential to put
the barrier up for the whole unit” (Wenk, H).
It was suggested that students would respond better to using just one system
such as StudentWeb only, or a system they are already familiar with such as
FaceBook, where the learners already have the skills and motivation to re-visit
the online space.
Key Findings
Box Hill Institute needs to adopt a more strategic centre wide adoption strategy for
ePortfolios with the long term goal to have:

an online presence for every course that has been customised and
contextualised

work with the ePortfolio advocates to achieve better integration at course level
with the design and development of a holistic assessment strategy co-designed
by teachers and students that also has a lifelong focus
The program would benefit from the implementation of a Professional Development
program for students as an adjunct program to be implemented if we have
appropriate resources in 2013. This program will include:

The creation of student mentor roles that are provided with extra training and
support to fulfill these roles

the realignment of the GPS Learning Pathways student induction program to
current classroom delivery and integration of the students’ preferred ways of
using ePortfolios within their course.

Development of a communication strategy to better share what support is
currently available for both staff and students

The further development of a follow-up student-orientated training session in
how to use ePortfolios from a BeLS professional development staff member
which could include (depending on the student needs):
o
System training
o
How to undertake critical analysis and reflection
o
How to create achievable plans, etc.
o
How to integrate video and other multimedia artefacts
National VET E-learning Strategy 2012 – 2015
Page 78
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
There will be a re-design of the teacher professional development program that
develops skills and knowledge in teachers around online delivery, and helps them
understand key blended learning issues, including :

The value to classroom teachers in attending the GPS induction process

Strategies for embeding within the future ePortfolio strategy, a lifelong and lifewide learning philosophy

How to design and develop assessment strategies for the holistic use of
ePortfolios across a course

Embedding strategies for online facilitation

The design and development of social collaboration strategies and the creation a
project for building a community within ePorfolios

Setting expectations for learners so that what we expect is achievable within
their enrolment timeframe at Box Hill Institute but also to include what is possible
into the future

Better embedding our current assessment-led learning approach that we use
within our Moodle Learning Management System and its realisation in the next
stage of ePortfolio implementation

Developing an implementation mode that is a mix of teacher-led and student-led
methodology

The design of a suite of ePortfolio exemplars

Multimodal delivery integrating online, workplace, mobile, and class delivery that
integrate ePortfolios as a critical system within our teaching and learning
portfolio.
Teachers to instil student ownership to improve motivation, participation, and
individual outcomes for our students through:

Use of planning tools to allow students to track their own goals and progress

Educating learners so they understand that their ePortfolio as a lifelong learning
tool beyond the life of the unit.

Developing groups to enable students to provide feedback to each other, crosscorrelate their work and contributions etc.

Training teachers to guide students by:
o
Providing regular feedback to learners
o
Creating links between the face-to-face and online environment to
encourage peer learning and review
o
Identifying areas of relevance in using ePortfolios and its relation to the
National VET E-learning Strategy 2012 – 2015
Page 79
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
unit being undertaken

Better integrating industry/workplace context with video, mobiles etc.
The teachers suggested the following system changes:

Design the capability to allow students to customise look and feel of pages

Improve navigation in Mahara

Investigate options to integrate Web 2.0 tools such as Pintrest

Investigate the network speed of StudentWeb which is built within a Moodle
Learning Management System

Investigate problems in Mahara further and prepare a report for actions to
address the issues identified

Enable a larger storage capacity so that videos can be incorporated in Mahara.
National VET E-learning Strategy 2012 – 2015
Page 80
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
6. Recommendation and Outcomes from Data
Findings and Case Studies
6.1 Longitudinal study
The analysis of the longitudinal data has shown that there is evidence of a direct
correlation between professional development activity and Mahara ePortfolio usage.
The greatest users of ePortfolios are the paraprofessionals with a compelling
workplace-based context usually through the use of workplace-based portfolios that
have lifelong learning applications, that is, they can be used for job applications and
to build a resume:
Teaching Centre
 Biotechnology
 Creative Industries


Business Programs
Hospitality

Health and Community
Services
Context
 Work placements
 Industry portfolios for fashion
and music
 Work placements
 Work placements and industry
based RPL
 Work placements and industry
based RPL
The ePortfolio advocates within the above centres designed assessment strategies
that integrated ePortfolios a summative assessment tasks ensuring the ePortfolio
became an integral component of the course structure.
The model designed and implemented by Steve Dalton using instructional videos
developed by him and then expecting students to film a visual diary of their tasks has
now become an exemplar for customised apprenticeship implementation and Steve
is a strong ePortfolio advocate across the Institute.
As GPS Learning Pathways matures into the second year of implementation, we are
seeing sustained student log-in numbers is excess of the GPS Learning Pathways
numbers for new first year students. This is indicative of second year students
continuing to use the system beyond the lifecycle of the GPS Learning Pathways
program. However we are also seeing a clear pattern of behaviour that although
some students will continue to use the system autonomously, sustained and
meaningful use requires a teacher’s development of an integral assessment tool to
ensure a broader use of the system occurs.
National VET E-learning Strategy 2012 – 2015
Page 81
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The greatest users of ePortfolios are kinaesthetic learners. However, there is no
direct correlation between learning styles and level of ePortfolio adoption. The
compelling usage is where teachers have led the charge with relevant and
contextualised delivery programs that are firmly integrated into a workplace-based
strategy.
The Centre for Creative Industries had its genesis with a student-led approach. The
teachers within Creative Industries had been reluctant to use the ePortfolios system
at first. However, when faced with hundreds of their students using the system
autonomously in a student-led model, they have followed their students into this
system designing ePortfolios assessment strategies and creating a more
collaborative approach. This approach has included uploading videos, photographs
and other multimedia artefacts, all of which are integral aspects of the Creative
Industry. This Teaching Centre has had forty-six staff complete a range of
professional development in 2011.
We also identified that there is a direct correlation between Moodle LMS usage and
Mahara ePortfolio use. The most productive and engaged Centres are using both
systems.
We did analyse that there was a distinct drop-off of Mahara ePortfolios usage when
the Professional Development team were not available to provide further training due
to sheer volume of demand for their services. Access to less training had a direct
impact on the uptake rates for both teachers and students.
National VET E-learning Strategy 2012 – 2015
Page 82
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
6.2 Qualitative Data Analysis: Surveys and one to one
interviews
Two surveys were conducted over the research project timeframe. One was for
teachers and the other for students. The student survey results provided some
interesting data for future planning consideration. The key findings are listed below:
The teacher is critical in providing context and support and the data analysis supports
this conjecture
Students want to see ePortfolios used differently from teachers. Students want to use
ePortfolios collaboratively; teachers tend to use ePortfolios for assessment
submission or to send messages. Since teachers determine the context and students
follow, a critical Institute-wide strategy needs to be designed and developed to adapt
our current assessment-led learning strategy to include the integration of ePortfolios
as an integral tool - specifically as a summative assessment tool. We also need to
look at a broader collaborative option within ePortfolios to meet students’ demands.
The student surveys also further underscored the need to continue with our current
professional development program but broaden this to include better workplacebased assessment tasks and introducing a collaborative context. This also requires
the design and development of exemplars and templates to assist with development
parameters to provide the teachers with greater autonomy and to also embed student
feedback in our future strategy.
Critical system changes are required to meet the emerging needs of our teachers
and students. If these cannot be achieved within current ePortfolio system
requirements then an alternative system(s) would be required to achieve the
outcomes identified. If Mahara continues to design the system with the current rigid
development structure that does not include an open source development community
or make significant user-friendly changes, then Box Hill Institute would need to
review a range of options to meet the student and teacher system requirements.
Facebook could be adapted to create a more collaborative approach to information
sharing including the design and development of templates within a Facebook Group.
http://www.experian.com.au/news/hitwise-awards-jun2010.html
We need to educate learners in lifelong/life-wide learning concepts. This professional
development program would need to instil a broader context for all students earlier in
their courses and guide learners to understand a lifelong and life-wide context.
National VET E-learning Strategy 2012 – 2015
Page 83
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The ePortfolios currently has too many layer and levels for students to traverse – this
needs to be simplified to improve usage that is not teacher-led. Both teachers and
students have indicated that a combined teacher and student-led approach is
preferred.
The teacher’s perception of their own skill level was lower than the students
assessed themselves. We attributed this to the difference between the students
assessing their knowledge as a user as opposed to a teacher assessing their skills
as an instructor. ths highlights a role for the BeLS Team to provide more support and
assistance to teachers as well as encouraging the teachers to take on a “concierge”
role where they facilitate the process and are prepared to allow the students to lead
initiatives and ideas for ePortfolio use which could arguably further engage the
students in the meaningful use of this system. The more teacher training the teacher
engaged in, the greater the chance of meaningful uptake, however once two to three
sessions had been attended then high quality support was the “x” factor needed to
support a teacher to engage in significant ePortfolio implementation.
BHI – BeLS needs to identify Teaching Centre based ePortfolio advocates and we
need to build a community of practice to support them to further upskill and mentor
teachers within their own teaching centres. A BeLS led model has worked well for
initial implementation, however we need to support teaching centre based experts to
work with their colleagues to achieve a more wide-spread adoption within an
appropriate and customized course-based context.
We need to adjust or broaden our current ePortfolios training program to reflect the
outcomes of this study specifically pertaining to embedding ePortfolio use in
pedagogical outcomes as opposed to system use. We need to include more
collaborative use and include teacher experts as part of the training. This can be
achieved through the use of video case studies filmed.
BeLS staff also needs to promote and provide ongoing support to teaching staff as
often teaching based ePortfolio innovation in delivery done in isolation. The Box Hill
Institute student led approach has assisted teachers to feel confident continuing to
use this system but we have more work to do in creating a truly student/teacher led
approach that systematically spreads across the Institute through careful planning
using our current “plant, water and harvest” innovation cycle.
National VET E-learning Strategy 2012 – 2015
Page 84
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
6.3 Qualitative Data: Case studies
At Box Hill Institute, we have ensured that all first year students are exposed to
Mahara ePortfolios. Over 5,800 students have already completed the program and
have as a result:

Commenced creating a resume in Mahara

Understood how to import an “asset”

Learnt how to create a folder and work with fellow students
This program also ensured that every Teaching Centre has a presence within
ePortfolios that is student-led. Many teaching centres have now implemented a
Teacher-led program to complement the initial work done. Our ultimate aim in our
second year of the GPS Learning Pathways project is to design and develop a
Centre-wide customised program to promote further adoption of ePortfolios and so
creating a dual level program.
Our student program will also need to deepen and broaden to include the creation of
student mentors; a realignment of the GPS Learning Pathways program to better
integrate student feedback; and better communication of the support services
available to assist them. We also need to work through options for creating a second
semester program for students to ensure that knowledge of the ePortfolio system and
tool options is further explored specifically around the themes of lifelong and life-wide
learning. Other features of this program should include teaching students how to use
ePortfolios as an analysis and reflection tool, a planning tool and how to upload
appropriate videos and other media into pages to use for job interviews, collections of
evidence of competency, etc.
A review and redevelopment of our current professional development program
should include the promotion of all teachers becoming actively involved in the student
induction program in entirety. We should also work through new offerings including a
more holistic approach to ePortfolios across a course, use of virtual classrooms and
higher levels of social collaboration. We should also train teachers in how to embed
ePortfolios as an integral tool in their assessment-led learning toolkit. This toolkit
should include exemplars and templates for ease of use and implementation.
As an Institute we should also be designing system solutions that should include
templates embedded within Portfolios and clear learner pathways. This should
include help tools to design groups, feedback options, and group reflection options
including course-based reflection processes embedded within students’ workflow.
National VET E-learning Strategy 2012 – 2015
Page 85
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
These templates should include a teacher-only section to support them to further
develop delivery options which could include a teacher’s guide, better system
linkages and exemplars for teachers to leverage off.
The biggest area of future impact is the design and development of the workplace
and industry links within ePortfolios capability. Employers need to be able to sign off
portfolios and action-based learning projects as their employees progress. BHI- BeLS
needs to invest development time in designing and developing clear processes and
procedures to ensure that this occurs in the near future.
We need our students to be able to customize the look and feel and have input into
improved navigation. From a system point of view we need to research how to
improve the speed of the network, improving future storage options and how to better
integrate Web 2.0 tools. Ultimately we need to improve the current problems being
experienced specifically around the volume of steps to upload and download assets
which is inordinately complex at present.
National VET E-learning Strategy 2012 – 2015
Page 86
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
7. Conclusions
Conclusion recommendations
As we focus on the gathered data and work towards the implementation of
sustainable strategies, we need to keep in mind our long-term goals. We must not
lose sight of the learner in all the reporting that we are gathering. The learner is
central to everything we do and our success hinges on his or her success. Our traffic
lights report has highlighted a direct correspondence between teacher activity and
learner activity. Where the teachers are actively engaged in creating an online
learning environment that implements a blended learning methodology, the students
follow and are actively using these resources and making the most of opportunities
for collaborative, constructivist learning.
Our data also shows a direct link between teacher training and support in using the
systems and their success in implementing them. This was also clarified by the
results of the online surveys and the face-to-face focus group interviews.
The recommendation would be that training in the use of ePortfolios is rolled out to
teachers and students at the same time, if possible. The students have an
introduction to their ePortfolio during induction; they create a page and add artefacts
but this is not followed up by the teachers in their course. We should harness the
enthusiasm of the learners to use digital resources for learning and assessment and
train them in the use of our systems. They in turn could mentor the teachers in their
use and collaborate on creating a meaningful, authentic learning experience that
incorporates the best of pedagogic practice whose momentum is maintained by all
users – teachers and students. The students thus become active participants in their
learning community and the teachers have the benefit of a shared responsibility in
the online environment.
On reflection, we would have spent more time designing a learning pathway GUI
interface and spent more time building a community of practice for the teachers that
included assessment tool templates and exemplars. This work was completed for the
Moodle Learning Management System implementation but less time was spent on
designing and developing the same level of online support, role-modelling and
assistance for ePortfolios.
National VET E-learning Strategy 2012 – 2015
Page 87
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
In BHI – BeLS we now need to design a Stage Two project plan that further extends
our quarterly reporting requirements down to the course and unit level – ultimately
extending to usage data on each student. We also need to meet with the leadership
team for each Teaching Centre to design a customised approach that extends on the
first eighteen months of implementation within Box Hill Institute.
We also need to broaden the action that results from the data analysis and ensure
that these action plans are communicated through an Institute and Teaching Centre
specific change management plan
National VET E-learning Strategy 2012 – 2015
Page 88
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The future
Our findings from the report include the following:
1.
To design a learning strategy that better integrates between the workplace and
Institution
2.
To develop broader and skill-specific training and development programs to
meet the needs of both students and teachers
3.
To design and develop better tools to action the data analysis from the traffic
light reports. This should include programs to support the development of
ePortfolios specific assessment tasks that meet needs of both teachers and
students and ultimately the employer.
4.
The re-design of ePortfolios or a new system that amalgamates key social
collaboration aspects within a simpler interface. The interface needs to be redesigned to be more Facebook-like
5.
Create step-by-step instructions for both teachers and students
6.
Create better integration between Moodle and Mahara so we can achieve both a
push and pull strategy for asset sharing across the two systems.
We need to further research the removal of institutional ownership from education
providers to advocate ePortfolios as a process as the tool will keep changing
dependent on the platform/student choice/etc. The longevity of ePortfolios lies in the
design of a global process that is interdependent but able to be integrated with any
LMS. The system needs to be able to gather information for the end user – its
strength is in the power of the user not the system itself.
We also need to focus on the how - not the what. Our role should be to act as a
guide for the portfolio development “The challenge in many institutions is not to build
an e-portfolio culture, but to integrate it into existing activities and practices.” Src:
A critical next step is support the development of student advocates. ‘Importance of
students as advocates for provision rather than simply being recipients of it’ Src. This
encourages the students to be active learners and empowers them to lead in the
integration of ePortfolios into their coursework and assessment.
The system also needs to be highly customizable by learners and if Mahara cannot
achieve this then Institutions using this system needs to have the freedom to
customise the system to meet their learner’s needs.
Box Hill Institute will also need to further research how to effect platform changes that
will include automating exchange of ePortfolios (i.e., Web Services), better
National VET E-learning Strategy 2012 – 2015
Page 89
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
integration with Web 2.0 technologies with a student-centred learning approach while
considering how cloud storage could be used to better support mobility in education,
accessibility and broader usability. We also need a decision on whether the system
will be Nation, Institute or course-owned or ownership ceded on behalf of the
Nation/Institute to our students.
We would also like to see a Moodle-type open source community nurtured which
would improve development cycles and ideas. Ultimately the system needs to
improve in terms of interoperability and transferability and offer better options for
students who already have an ePortfolio so existing eportfolios can integrate more
seamlessly.
Ideally, templates could be designed to showcase best practice around skills
assessment and RPL as these systems work seamlessly within this system but need
better options into the future.
The concept of life-wide learning needs further grounded research that investigated
the incorporation into corporate and human resources standards and activities, and
better integrates with Institutional services including career and learning support. We
want the ability to save resume pages as a PDF so that learners don’t need to have
their resume in 2 places when applying for jobs
Informal learning experiences need to be linked to assessment, and education
provided needs to help facilitate learner pathways with authentic learning
experiences - informal learning, work internships - and provide graduate capabilities
and employability/digital literacy skills. We wish to see this occur so we can better
support the broader implementation of the assessment-led learning approach and
deliver on the pressure to reduce delivery hours and better motivate learners.
Education and training organisations now recognise that individuals no longer gain all
of their employability skills through formal learning paths.
http://kt.flexiblelearning.net.au/tkt2009/?page_id=18
Box Hill Institute would also like to see major work done on both a system and
Institute level in the area of social collaboration. We see a major area of opportunity
in supporting collaborative inquiry (learning) and building on the expertise,
knowledge, and experiences of the group.
National VET E-learning Strategy 2012 – 2015
Page 90
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
The future of ePortfolios within the Box Hill Institute learning environment has been
significantly impacted as a result of the opportunity to analyse the longitudinal data,
case studies and surveys. Listening to the voice of our key users has proven
illuminating and actually enabled to design a blueprint for the future. However are
now extremely focused on engaging industry and employers in this dialogue so the
plan for the future embeds lifelong and lifewide strategies for learning.
National VET E-learning Strategy 2012 – 2015
Page 91
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
8. Resources and references
Arnold, K. E. & Pistilli, M. D. (2012). Course Signals at Purdue: Using learning
analytics to increase student success. Proceedings of the 2nd International
Conference on Learning Analytics & Knowledge. New York: ACM, accessed 28 May
2012,< http:// <http://www.itap.purdue.edu/learning/research/pdf/Arnold_PistilliPurdue_University_Course_Signals-2012.pdf
Australian Learning and Teaching Council 2009, ‘Learning Communities:
International Journal of Learning in Social Contexts’ e-Portfolio edition, Issue 2
accessed 10 May 2012,
<http://www.cdu.edu.au/centres/spill//journal/IJLSC_Dec_2009_eportfolio.pdf?q=cent
res/spil/journal/IJLSC_Dec_2009_eportfolio.pdf>
Campbell, J.P., DeBlois, P.B. & Oblinger, D.G 2007, ‘Academic Analytics: A New
Tool for a New Era’ Educase Review Vol.42, No4 pp. 40-57 accessed 28 May 2012
Carter, G 2010, “Personalization in Context: Student Equity and the Whole Child”
presented at the [Re]Design for Personalized Learning Symposium, Harvard Club,
Boston (MA), accessed 28 May 2012
<http://www.siia.net/pli/presentations/PerLearnPaper.pdf>
Curyer, S., Leeson, J., Mason & Williams, A., (2007), in Miller, and A., O’Neill, O
“Supporting successful learning pathways using e-portfolios and mobile devices”
Retrieved from
http://pre2012.flexiblelearning.net.au/files/AVETRA_2011_Aust_Flexible_Learning_F
ramework_full_paper_Final.pdf
Department of Education and Early Childhood Development 2007 Personalising
Education:from research to policy and practice accessed 28 May 2012,
<http://www.eduweb.vic.gov.au/edulibrary/public/publ/research/publ/personalisingeducation-report.pdf>
Egle, C 2007, A guide to facilitating adult learning, Rural Health Education
Foundation, accessed 28 May 2012,
< http://www.rhef.com.au/wp-ontent/uploads/userfiles/716_alp_lr.pdf>
ePortfolio for skilled immigrants and employers: LIfIA project phase one final report.
Retrieved March 15, 2012 from http://www.futured.com/documents/ReportePforSkilledImmigrants-LIfIA.pdf
National VET E-learning Strategy 2012 – 2015
Page 92
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
http://eresources.tafe.tas.edu.au/epw/undertaking-work-placements/index.html
Experian Information Solutions Inc. 2012, Experian Asia Pacific Pty Ltd, accessed 12
June 2012, http://www.experian.com.au/hitwise/data-centre.html
Galatic, H., Leeson, J., Mason, J., Miller, A., ONeill, O. (2009). The VET ePortfolio
Roadmap: A strategic roadmap for e-portfolios to support lifelong learning. Retrieved
March 15, 2012, from
http://flexiblelearning.net.au/content/our_work/docs/VETePortfolioRoadmap_web.pdf
Gruber, K 2011 Assessing and Accommodating Learning Styles: Is it Really a Myth?,
accessed 28 May 2012
http://www.ocwtp.net/PDFs/Trainer%20Resources/CG%200211.pdf
Hallam, G., Harper, W., McCowan, C., Hauville, K., Hauville, K., McAllister, L.,
Creagh, T., Lee, J., Lambert, S., Brooks, C. (2008) ePortfolio use by university
students in Australia: Informing excellence in policy and practice, e-book, accessed
28 May 2012,
http://www.eportfoliopractice.qut.edu.au/docs/Aep_Final_Report/AeP_Report_ebook.
pdf
Heinrich, E. (2008). “Contrasting approaches: Institutional or individual ownership in
ePortfolio systems”. In Hello! Where are you in the landscape of educational
technology? Proceedings Ascilite Melbourne, 2008
IMS Global Learning Consortium (2005), retrieved from:
http://www.imsglobal.org/ep/epv1p0/imsep_bestv1p0.html#1663600 .
IMS Global Learning Consortium http://www.imsglobal.org/ep/index.html Accessed
11 April 2012
Janssen, J., Brouns, F., Vaessen, H., Weijnen, G., Huveneers, E., and Hodzelmans,
R. (2011). “A Nation-wide ePortfolio: Reflections on Ambitions and Experiences in
the Netherlands.” Proceedings of the ePortfolio and Identify Conference (ePIC) 2011,
London, July 10-12, 2011 <http://hdl.handle.net/1820/3436>.
Joyes, G., Gray, L., Hartnell-Young, E. (2010) “Effective practice with e-portfolios:
How can the UK experience inform implementation?” Australian Journal of
National VET E-learning Strategy 2012 – 2015
Page 93
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Educational Technology, Volume 26, CH1, pp 15-27 Retrieved 15 March from
http://www.ascilite.org.au/ajet/ajet26/joyes.pdf
JISC, (2008b) Retrieved 20 April from http://www.jisc.ac.uk/eportfolio
Kornhaber, M. L. (2001) 'Howard Gardner' in J. A. Palmer (ed.) Fifty Modern Thinkers
on Education. From Piaget to the present, London: Routledge. Accessed from:
http://www.infed.org/thinkers/gardner.htm
Krätzig, G. & Arbuthnott, K. (2006) Perceptual learning style and learning proficiency:
a test of the hypothesis, Journal of Educational Psychology, 98(1), pp. 238 – 246.
Land & Zembal-Saul, 2003 from Aalst, J & Chan, K 2007 “Student-Directed
Assessment of Knowledge Building Using Electronic Portfolios” The Journal Of The
Learning Sciences, 16(2), 175–220, Lawrence Erlbaum Associates, Inc. Cited 15
April 2012, http://hdl.handle.net/10722/57352
Learning Communities International Journal of Learning in Social Contexts
December 2009, Issue 2: e-Portfolio Edition Retrieved May 10 2012 from
http://www.cdu.edu.au/centres/spill//journal/IJLSC_Dec_2009_eportfolio.pdf?q=centr
es/spil/journal/IJLSC_Dec_2009_eportfolio.pdf,
Learning Symposium, Harvard Club, Boston (MA) accessed 28 May 2012
<http://www.siia.net/pli/presentations/PerLearnPaper.pdf>
Long, P. and Siemens, G., 2011 Penetrating the Fog: analytics in Learning and
Education. Educause Review, September/October 2011, pp.31-40., accessed 28
May 2012
http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume
46/PenetratingtheFogAnalyticsinLe/235017
(Lorenzo, G and Ittelson, J (2005) “An Overview of E-Portfolios” Educause Learning
Initiative, http://net.educause.edu/ir/library/pdf/ELI3001.pdf
McNeely, B. (2005) Using Technology as a Learning Tool, Not Just the Cool New
Thing. In:Oblinger, D. G., Oblinger J. L eds Educating the Net Generation, Educause,
pp 40-49 accessed 28 May 2012 <http://net.educause.edu/ir/library/pdf/pub7101.pdf>
Miller, A. (2009) “The World of Eportfolios”. The Knowledge Tree, Edition 18,
Retrieved April 20 from http://kt.flexiblelearning.net.au/tkt2009/?page_id=18)
National VET E-learning Strategy 2012 – 2015
Page 94
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Minnes Brandes, G., and Boskic, N, Eportfolios: From description to analysis The
University of British Columbia Canada)
MyPortfolio http://myportfolio.ac.nz Accessed 11 April 2012
National VET E-earning strategy 2012., Commonwealth of Australia (Department of
Industry, Innovation, Science, Research, Technology and Teritary Education),
accessed.28 May , <http://flexiblelearning.net.au/about.htm>
Nguyen, S. 2010 Workplace Psychology, accessed 28 May 2012
<http://workplacepsychology.net/2010/04/10/evidence-based-training-debunking-themyth-of-learning-styles/>
Organisation for Economic Co-operation and Development 2002, Why Career
Information, Guidance and Counselling Matter For Public Policy (working draft #1),
accessed 28 May 2012, <http://www.oecd.org/dataoecd/25/30/2736729.pdf>
Hallam, G., Harper, W., McCowan, C., Hauville, K., McAllister, L., & Creagh, T.
(2008). ePortfolio use by university students in Australia: Informing excellence in
policy and practice. Retrieved March 15, 2012, from
http://www.eportfoliopractice.qut.edu.au/docs/Aep_Final_Report/AeP_Report_ebook.
pdf
Pittman,K (Software & Information Industry Association) 2010, “Personalization in
context: Student Equity and the whole child” presented at the [Re]Design for
Personalized Learning Symposium, Harvard Club, Boston (MA), accessed 28 May
2012 <http://www.siia.net/pli/presentations/PerLearnPaper.pdf>
Rose, J (Software & Information Industry Association) 2010, “Grand Challenge &
Opportunity: Defining Personalization” presented at the [Re]Design for Personalized
Schneiderman, M 2010, “Education Policy” presented at the [Re]Design for
Personalized Learning Symposium, Harvard Club, Boston (MA), accessed 28 May
2012 <http://www.siia.net/pli/presentations/PerLearnPaper.pdf>
U.S. Department of Education (2010). Transforming American Education: Learning
Powered by Technology. <http://www.ed.gov/technology/netp-2010>
National VET E-learning Strategy 2012 – 2015
Page 95
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Waite, C., (20110 An exploratory study into the online habits of young Australians,
Australian Clearinghouse For Youth Studies, Department of Education, Employment
and Workplace Relations, accessed 12 June 2012,
http://www.acys.info/ysa/issues/v.30_n.4_2011/papers/sociality_online
Ward, R (2008) PDP and e-Portfolio UK Newsletter Issue 15, December 2008.
Retrieved April 20 from http://www.recordingachievement.org/news-andevents/publications/pdpuk/item/pdp-uk-newsletters/pdp-issue15.html
Wolf M.A. (Software & Information Industry Association) 2010, Innovate to Educate:
System [Re]Design for Personalized Learning; A Report from the 2010 Symposium.
In collaboration with ASCD and the Council of Chief State School Officers,
Washington, DC, accessed 28 May 2012
<http://www.siia.net/pli/presentations/PerLearnPaper.pdf>
National VET E-learning Strategy 2012 – 2015
Page 96
E-portfolio Exemplar Project: Box Hill Institute Learner Analytics and ePortfolios
Contact Information
For further information regarding the Learner Pathways business activity, please
contact:
Name:
Phone:
TBA
Email
TBA@flexiblelearning.net.au
Website: http://flexiblelearning.net.au
National VET E-learning Strategy 2012 – 2015
Page 97