2015 International Conference on Communication, Information & Computing Technology (ICCICT), Jan. 16-17, Mumbai, India
Augmented Reality In E-Learning
Review Of Prototype Designs For Usability
Evaluation
Trupti Satpute*, Subhash Pingale, Vijaysinh Chavan
Department of ComputerScience and Engineering
SKN Sinhgad College of Engineering Pandharpur-4133004, India
E-mail:*satpute.trupti@gmail.com
Abstract—The concept of e-learning has reached beyond use of
presentations, videos. Augmented reality (AR) based systems are
preferred to support teaching and learning activities along with
these e-learning systems. In this paper we present a systematic
review of such prototypes developed for educational purposes and
compare their usability for finding benefits of AR. It also focuses
on use of web 2.0 tools in e- learning and combined use of both
technologies.
Keywords:Augmented Reality; e-learning; usability; web 2.0.
I.
I NTRODUCTION
Augmented Reality (AR) was initially a technologically
challenging topic and an interaction paradigm alternative (or
complementary) to Virtual Reality though both are born within
graphics. The wide adoption of smart phones and availability
of AR software has made AR widespread. Novel applications
propose forms of interactions that superimpose layers of digital
contextualized information over physical settings offering new
opportunities for learning experiences[3]. For instance, the
iPhone application, Starmap, allows learning about the constellations on a map displayed on its screen superimposed onto
the real stars being viewed in the sky. Within this context, we
refer to AR as the technologies that enable the superimposing
of layers of digitally contextualized information over physical
settings for enriching or augmenting the real world.
AR can be applied for learning, entertainment, or edutainment by enhancing a user’s perception of and interaction with the real world. User can move around the threedimensional virtual image and view it from any vantage
point, just like a real object[2]. The information conveyed
by the virtual objects helps users perform real world tasks.
Tangible Interface Metaphor is one of the important ways
to improve learning. This property enables manipulation of
three-dimensional virtual objects simply by moving real cards
without mouse or keyboard. Augmented Reality can also be
used to enhance collaborative tasks. It is possible to develop
innovative computer interfaces that merge virtual and real
worlds to enhance face-to-face and remote collaboration. These
augmented reality applications are more similar to natural faceto-face collaboration than to screen based collaboration.
With the advent of web 2.0, users residing worldwide can
contribute and shape the content of web E.g.blogs, wikis,
etc.Use of this technology in technology enhanced Learning
(TEL) has been practiced as a part of learning activities. It is
used for harnessing collective intelligence and making learning
immersive.The use of Information and Communication Technologies in education has been fostered during the last decade
by the widespread adoption of Virtual Learning Environments
(VLEs), also called Leaning Management Systems. Moodle,
LAMS, .LRN, Sakai or Blackboard are only a few well-known
examples of successful VLEs that stand out among educators
and institutions preferences. VLEs normally provide a shared
customizable workspace in which educators can design and
instantiate individual and collaborative learning activities that
students enact afterward.
As far as TEL is concerned, these two trends, VLE’s
and Web 2.0 might seem contrasting.On the one hand, Virtual Learning Environments VLEs are employed in order to
centralize the access to the learning activities. On the other
hand, Web 2.0 tools like online collaborative tools such as
GoogleDocs are also becoming widespread in technologyenhanced education. The clash of these two trends has recently
prompted an interest in research and development regarding the
integration of VLEs/PLEs and external Web 2.0 tools.
Concerning to the advantages in learning setting, both
AR and Web 2.0 have been proved to improve engagement
with learning activity and support collaborative activities. The
purpose of this review is to find how much of ARs benefits
are applied with e-learning systems. The primary objectives of
this survey are:
•
Usability evaluation of AR based systems and benefits
they achieved
•
Finding usability of AR along with web 2.0 tools in
educational systems
II.
BACKGROUND
A. Augmented reality
Azuma et. al. [1] define AR as 3-D virtual objects are
integrated into a 3-D real environment in real time. So the
basic requirements of AR are the combination of virtual elements and real environment. Also it requires three-dimensional
registration such that the virtual elements are aligned to the
real environment and it requires real-time interactivity with
the virtual elements. Thus, the virtual elements must behave
like a real element in the real environment. This may mean,
but is not limited to, the AR system responding to changes
978-1-4799-5522-0/15/$31.00 c 2015 IEEE
Authorized licensed use limited to: Qatar University. Downloaded on September 26,2020 at 18:11:39 UTC from IEEE Xplore. Restrictions apply.
are reviewed for deciding the effectiveness of AR in education.
The technology is mostly used along with e-learning systems
and handheld devices. Articles which describe use of AR and
web2.0 in are surveyed focusing their use in e-learning.
in the perspective of the user, changes in lighting conditions,
occlusion and other physical laws.
Augmented Reality Learning Experiences: The Augmented
Reality, as a next-generation interface, affords a different way
of interaction with information. This interaction can be used
to design better learning experiences. We define the term
Augmented Reality Learning Experiences (ARLEs) to refer
to learning experiences facilitated by AR technology. Some
examples of ARLEs for various subjects are like physics,
chemistry, geography and mathematics, as well as, educational
games for primary education. Aside from these contents,
ARLEs can be used for astronomy, biology, geometry and
cultural heritage. These kinds of content depend on the abilities
of AR to:
1)
2)
3)
4)
IV.
A. Meta-analysis
We conducted a systematic literature review based on the
work of Ericson et al. [4]. Their meta-analysis aimed to
measure the impact of ARLEs in K-12 (pre-school, grade
school, and high school) educational settings. Their analysis of
503 articles show that the effects associated with technology
have not changed substantially over the years. The mean effect
size of technology applied to education observed is 0.56 or low
to moderate effect.
Illustrate spatial and temporal concepts.
emphasize contextual relationships between real and
virtual objects
provide intuitive interaction
visualize and interact in 3D
The methodology for the systematic literature review is as
follows:
1) Search for prototypes: : A literature search was conducted in year 2013-2014, published in IEEE Xplore Digital
Library, ScienceDirect, Springer, and Elseveir etc. The search
is limited to journal articles and conference proceedings that
are written in English.
B. AR affordances and benefits
The researchers designed their ARLE to take advantage
of the affordances of AR technology. These affordances are
derived from the very nature of AR: the real-time integration
of virtual elements to a real environment. By the definition,
augmented reality affords:
1)
2)
3)
2) Inclusion criteria: : The content relevant to educational
applications and mostly focusing on higher education was
considered for this survey. Thus, for the research paper to be
included, the following criteria must be met:
Real world annotation - to display text and other
symbols on real world objects
Contextual visualization - to display virtual content
in a specific context
Vision-haptic visualization - to enabled embodied
interactions with virtual content.
Aside from the natural affordances of AR, design strategies
have been applied to the creation of more effective ARLEs. In
ARLEs, researchers have used the following strategies:
1)
2)
3)
Enable exploration - designing AR content that is
non-linear and encourages further study
Promote collaboration - designing AR content that
requires students to exchange ideas
Ensure immersion - designing AR content that allows
students to concentrate more and be engaged at a
constant level.
III.
M ETHODS
•
The research paper must have at least a preliminary
working ARLE prototype.
•
The prototype should be applied to learning a new
concept or skill.
•
The content should be relevant to secondary or higher
education.
•
The paper reports an effect size or provided a means
to calculate the effect size (reports both mean and
standard deviation).
Applying these criteria resulted in 4 articles.
3) Data gathering: : We computed the effect size (d) using
formula
d = (xe − xc)/s
(1)
Where, xe is the mean of experimental treatment that using
AR and xc is the mean of control and s is pooled standard
deviation obtained as:
LITERATURE REVIEW
A. About AR
s = (se + sc)/2
According to Azuma et.al. [1] Augmented Reality technology is not a new issue. It has been used in fields such
as: military, medicine, engineering design, robotic, telerobotic,
manufacturing, maintenance and repair applications, consumer
design, psychological treatments, etc. The authors have studied
the benefits of technology in various fields.
(2)
Where se is standard deviation of experimental treatment and
sc is the standard deviation of the control. We interpret the
calculated effect size based on Cohens recommendation, that
is, an effect size of 0.8 or higher is considered large, around
0.5 is considered moderate, and around 0.2 is considered small.
B. Qualitative analysis
B. About AR and web2.0 in education
1) Search for prototypes:: Search for prototypes was carried out in the same way as in meta analysis. The articles
considered for meta analysis are also included for qualitative
analysis.
As AR can be used for education, a large number of prototypes have been developed for educational settings. Various
such articles published in journals and conference proceedings
2
Authorized licensed use limited to: Qatar University. Downloaded on September 26,2020 at 18:11:39 UTC from IEEE Xplore. Restrictions apply.
TABLE I.
SUMMARY OF PROTOTYPE EXPERIMENTAL
EVALUATION AND EFFECT SIZE
Ref
[6]
content
Game of
Go
Interactive
agent
Participant
18
volunteers
(age:21-32)
15 students
(age:8-13)
Control group
PC based
Go game
learning
Collaborative
e learning
environment
[10]
[11]
OOP
course
200
students
[12]
English
language
course
6 classes
e-learning
approach
using LMS
Text based
audio-visua
data
Exp. group
AR based
ARGo
game
AR based
interactive
agent to
speak
AR based
course
content
AR based
l immersive
e-learning
TABLE II.
effect
0.232
Ref.
[5]
Year
2011
[6]
2011
0.8
[7]
2011
0.486
[8]
2012
[9]
2012
[10]
2012
[11]
2013
[12]
2011
[13]
2011
[14]
2009
0.525
2) Inclusion criteria:: For inclusion in qualitative analysis,
we focused on the evaluation technique and benefits achieved.
So, the criterion for effect size was relaxed.
3) Data gathering:: A survey questionnaire was drafted to
facilitate the gathering of data from the 10 included articles.
The questionnaire has four main parts namely:
•
publication details,
•
prototype description,
•
use of AR,
•
design and results of the user
TABLE III.
Ref
[5]
V.
SUMMARY OF PRELIMINARY STUDIES USING
SURVEY TOOLS
RESULTS AND DISCUSSION
[6]
A. Meta analysis results
[7]
There were four articles which reported their values of experimental evaluation. The AR applications used for education
and their effect sizes are summarized in Table I.The mean
effect size obtained is 0.511 which is moderate.
[8]
[9]
[10]
B. Qualitative analysis results
[11]
We selected ten articles having educational prototypes
according to given inclusion criteria. The usability of AR
in education is mostly evaluated using survey questionnaires
and personal or group interview methods. Some researchers
have used combination of such data collection methods. The
prototype descriptions and tools described in each article are
summarized in Table II.
[12]
[13]
[14]
Most articles reported have used questionnaires and evaluated results based on Lickert scale. [2] has used questionnaire
based on IMI that is the researchers used a part of the
Intrinsic Motivation Inventory (IMI) to analyze four subscales:
enjoyment, competence, usefulness, and tension.
There are various ways of using AR technology like marker
based AR, AR for projection, AR for GPS based system. For
educational setting mostly used is marker AR. The affordances
of AR technology implied in the prototype and the other
pedagogical benefits achieved are summarized in following
Table III.
Prototype Description
AR based 3D digital
media teaching materials
for teaching Physical
Science
A game of GO, for self
learning
2D and 3D models for
learning computer
graphics
AR for architectural
visualization
Cultural and natural
heritage
Interactive flower garden
with interactive agent in
augmented picture
Mobile software to
support learning
experience for OOP
Authoring tool for elearning applications
Game base introduction
to programming
Interactive learning
system for conservation
of fish
Tools
Triangulation-combined
evaluation using interview,
observation, questionnaire
Questionnaire based on
IMI, result based on
Lickert scale
Group interview
Own questionnaire, result
based on Lickert scale
Feedback Survey
Observation, result based on
Lickert scale
Own questionnaire, result
based on Lickert scale
Experimental evaluation
using observation
Feedback Survey
SUS questionnaire result
based on Lickert scale
SUMMARY OF AR AFFORDANCES AND BENEFITS
ACHIEVED
Type of AR
technique used
Marker AR for 3D
material and objects
Augmentation of
Go game board
3D models using
3DMax and AR
Plug-in
Marker AR for
architectural project
AR using Hoppala
and Layar tools
Marker AR for
augmentation in
picture
Marker AR to play
video and enable
AR sessions
Marker based tool
for creating elearning content
QRcodes for
augmentation of
game board
Marker AR for 3D
virtual models for
fish
AR Affordances
Real world
annotations
Contextual
visualization
Contextual
visualization
Benefits
achieved
Ubiquitous learning
environment
Self learning support
Collaborative and
Interactive learning
Real world
annotation
Contextual
visualization
Contextual
visualization
Student satisfaction
Collaborative and
immersive learning
Increased engagement
in learning
Real world
annotation
Increased engagement
and enjoyment
Vision haptic
visualization
Increased academic
achievement
Real world
annotation
Increased engagement
in learning
Real world
annotation
Increased enjoyment in
learning, positive
usability
reported the positive usability of AR technology. But experimental evaluation of systems developed is also necessary for
effect size calculations.
C. Web 2.0 along with AR in e-learning
Web 2.0 technology has support for collaboration, social
interaction, creative thinking of students in e-learning environments.
Most researches have proved that their prototypes using
AR have three inherent affordances as Real world annotations,
Contextual visualization, and Vision haptic visualization. So
the benefits achieved include collaboration support, increased
enjoyment, engagement in learning etc. The articles have
Anastasios K. et. al. have done evaluation with a goal
to identify how to design and deploy adaptive Computer
Supported Collaborative Learning (CSCL).[17] The authors
3
Authorized licensed use limited to: Qatar University. Downloaded on September 26,2020 at 18:11:39 UTC from IEEE Xplore. Restrictions apply.
report about the tools like GLUE! for integration of web 2.0
and LMS for adaptive collaborative support.
[3]
Prez-Sanagustn, M., Hernndez-Leo, D., Santos, P., Delgado Kloos, C. and
Blat J., Augmenting Reality and Formality of Informal and Non-formal
Settings to Enhance Blended Learning , IEEE Transactions on Learning
Technologies, Volume 7,Issue 2, pp.118-131, 2014
[4] Marc Ericson C. Santos, Angie Chen, Takafumi Taketomi, Goshiro
Yamamoto, Jun Miyazaki, Hirokazu Kato, Member, IEEE,Augmented
Reality Learning Experiences:Survey of Prototype Design and Evaluation,
IEEE Transactions On Learning Technologies, VOL. PP, Issue:99,2014(11
December 2013 )
[5] Min-Chai Hsieh*, Hao-Chiang Koong Lin, A Conceptual Study for
Augmented Reality E-learning System Based on Usability Evaluation,
CISME, Vol.1, No.8 , PP.5-7, 2011
[6] Takahiro Iwata, Tetsuo Yamabe, Tatsuo Nakajima, Augmented Reality
Go: Extending Traditional Game Play with Interactive Self-Learning
Support, 17th IEEE International Conference on Embedded and Real-Time
Computing Systems and Applications, pp-105-114, 2011
[7] Malinka Ivanova, Georgi Ivanov, Enhancement of Learning and Teaching
in Computer Graphics Through Marker Augmented Reality Technology,
International Journal on New Computer Architectures and Their Applications (IJNCAA) 1(1), pp. 176-184, 2011
[8] David Fonseca, Isidro Navarro, Ernest Redondo, Albert Snchez,Using
Augmented Reality and Education Platform in Architectural VisualizationEvaluation of Usability and Students level of Sastisfaction, International
Symposiam on Computers in Education, 2012
[9] Francesc Gonzlez, Luis Villarejo1, Oriol Miralbell, Joan Miquel Gomis,
How to use mobile technology and augmented reality to enhance collaborative learning on cultural and natural heritage- An e-learning Experience,
New Trends on Global Education Conference 2012 (GEC 2012) North
Cyprus, 24-26 September, 2012
[10] Sejin Oh, Yungcheul Byun, The design and implementation of elearning systems, 11th International Conference on Computer and Information Science, pp-651-654, 2012
[11] Utku Kose, Durmus Koc, Suleyman Anil Yucesoy, An Augmented Reality Based Mobile Software to Support Learning Experiences in Computer
Science Courses, International Conference on Virtual and Augmented
Reality in Education, pp.370-374, 2013
[12] Hyung-Keun Jee & Sukhyun Lim & Jinyoung Youn & Junsuk Lee, An
Augmented Reality-Based Authoring Tool For E-Learning Applications,
Multimed Tools Appl Vol.68, pp-225235, 2014
[13] Nicholas Masso and Lindsay Grace, Shapemaker:A Game-based Introduction to Programming, The 16 th International Conference on Computer
Games, CGAMES, pp-168-172, 2011
[14] Hao-Chiang Koong Lin, 1Min-Chai Hsieh, 1Cheng-Hung Wang, 1ZongYuan Sie, 2Shei-Hsi Chang, Establishment And Usability Evaluation Of
An Interactive Ar Learning System On Conservation Of Fish, TOJET:
The Turkish Online Journal of Educational Technology, volume 10 Issue
4,2011,pp-181-187, October 2011
[15] Thuong N. Hoang, Shane R. Porter, Benjamin Close, and Bruce H.
Thomas, Web 2.0 Meets Wearable Augmented Reality, International
Symposium on Wearable Computers, pp-151-152, 2009
[16] Paolo Montuschi and Alfredo Benso,”Augmented Reading: The Present
and Future of Electronic Scientific Publications ”,Computer, Volume:47 ,
Issue: 1 ,pp-64-74,2014
[17] Anastasios Karakostas, Luis P. Prieto, Yannis Dimitriadis, Opportunities
and Challenges for Adaptive Collaborative Support in Distributed Learning
Environments: Evaluating the GLUE! Suite of Tools, 12th IEEE International Conference on Advanced Learning Technologies, 2012
[18] Marijana S. Despotov, Aleksandra B. Labus and Aleksandar R. Mili
Fostering Enginering E-learning Courses with Social Network Services,
19th Telecommunications forum TELFOR 2011
Along with collaborative suport, web 2.0 can be used to
motivate students by displaying their test results on social
media like Facebook. Marizana S. et. al have created and
used a facebook social services in e-education. They created
pages for declaring events, creating fan page etc. and examined
the web site traffic. So the authors say that introduction of
social networks ineducation process improve student- teacher
communication, interaction which affects quality of education
process[18].
Some researchers have developed to use benefits of both
AR and web 2.0. Hoang et.al.[15] have developed a system
which uses wearable device enabled with AR interface to
share social network information which can capture location
information using GPS co-ordinates. Such kind of efforts are
extende for combinig GPS based augmented information web
2.0 support by introducing ARML2.0. ARML 2.0 is an XML
grammar thal lets developers describe an augmented scene, the
objects in the scene, and some essential objects in the scene.
Paolo M. et.al.[16] say that Journal articles now can
increasingly include a variety of supplemental multimedia and
interactive materials for augmented reading that will impact
both the nature and presentation of scientific research.
The main findings from review are AR and Web 2.0
both have been studied for their benefits in e-learning. But
combination like ARML 2.0 has more scope in e-learning
application in study tours or the outdoor activities. Also marker
based AR and web 2.0 based tools can be combined with
LMS for initiating various learning activities. Such prototype
designs can have benefits like increased engagement, motivation, creative thinking, harnessing collective intelligence etc.
Such prototypes must be evaluated using specifically designed
questionnaire. Also students’ performance before and after
using prototype must be recorded.
VI.
C ONCLUSION
We have studied the educational prototypes for their effect
size and benefits achieved by them. Combination of AR and
Web 2.0 technologies can give better results in educational
achievements. AR increases engagement, immersion and web
2.0 supports collaboration, social interaction. These technologies have been studied separately for benefits in learning
systems. Therefore both technologies should be studied for
experimental evaluation of their combined use in educational
settings.
R EFERENCES
[1]
Azuma, R. T., A Survey of Augmented Reality, Teleoperators and Virtual
Environments, pp-355-385, 1997
[2] Mehmet Kesima, Yasin Ozarslanb, Augmented Reality In Education:
Current Technologies And The Potential For Education, Procedia - Social
and Behavioral Sciences 47, pp 297- 302, 2012
4
Authorized licensed use limited to: Qatar University. Downloaded on September 26,2020 at 18:11:39 UTC from IEEE Xplore. Restrictions apply.