Iberoamerican Journal of Applied Computing
ISSN 2237-4523
USING AUGMENTED REALITY TO CONSTRUCT
PEDAGOGICAL MATERIALS FOR ELEMENTARY SCHOOL
STUDENTS
Diolete Marcante Cerutti (UEPG) – E-mail: diolete@cd.inf.br
Heder Luiz Martins Junior (UEPG) – E-mail: hederlmjr@gmail.com
Lucas Vieira Werner (UEPG) – E-mail: lucas.werner02@gmail.com
Maysa Lovatto Lopes (UEPG) – E-mail: maysalovatto@hotmail.com
Rennan Rodrigues da Silva (UEPG) – E-mail: rennan_r@hotmail.com
Denise do Rocio Maciel (UEPG) – E-mail: dnise_maciel@hotmail.com
Tatiana Montes Celinski (UEPG) E-mail: tmontesc@uepg.br
Abstract: Computing Museum of State University of Ponta Grossa is an extension
program and has been developing activities which relates teaching-learning process to
new technologies. Through augmented reality kit which was developed by computing
students as part of their final essay in 2010, the Museum presents this technology in
workshops to the general community in Ponta Grossa city. The project presented here
aims at constructing new pedagogical materials by using augmented reality and
improving materials which were already constructed.
Keywords: Computing Museum of UEPG, Augmented Reality, Extension Activities.
1. INTRODUCTION
As an extension program, the Computing Museum of Sate University of Ponta Grossa
(UEPG) provides lecturers, students and general community with a space of reflection
where it is possible to think about computing and its impacts on society. Moreover, this
space makes possible for students involved in the projects interact with community in
order to rebuild the time line of computing technologies and discuss ethical issues on
the theme.
Students are fostered to construct pedagogical materials in a visual and
interactive way. These materials present the evolution of computers and technology
involved with as well as provide opportunities to discuss impacts of such technology in
different social contexts. Through seminars and workshops, teachers and students from
private and public schools are invited to know the history of computers and to discuss
the use of information and communication technology and the future.
The Computing Museum of UEPG is divided into physical and virtual spaces.
The former is the show of antique technological artefacts. Workshops about how
computer equipment works are presented to community and students. In this part of the
project problems with electronic waste and sustainability issues are treated and
discussed with students and teachers. Awareness of environmental questions is taken
into great consideration. The virtual space is a site available in the Internet which
emphasizes the content about technologies related to computer and its relationship with
scientific knowledge. It allows reflecting on impacts of those technologies cause on
current social processes (CERUTTI, 2011).
V. 3, N.2, Ago/2013
Page 53
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
The Virtual Museum presents content about the informatics history to users.
However, it is important to make the explorative activity more interactive in order to
foster reading the content. In this way, the concept of interaction design was used where
Preece et al. (2005) argues that the design of interactive products must provide support
to quotidian activities in both work and home.
One of the aims of the Museum is to engage students into development activities
of interactive pedagogical materials for the Museum. So, this article presents a study
developed by Bachelor in Informatics students as part of their final essay. The material
was composed by using augmented reality as technology to produce that material for
elementary school students.
Initially, the material had students as focus. However, through questionnaires
and talking with teachers, it was realised the need to concentrate efforts on the
improvement of this material to teacher to use it with flexibility. The material would
comprise every kind of content in order to provide teachers with flexibility to change
the content as they wanted. In this sense, this article also presents the development of a
new graphical interface to teachers (or less expert users) in order to they can change or
adapt the content of the class to their own needs.
2. THEORETICAL BASIS: AUGMENTED REALITY
Augmented reality (AR) is a term to define a different way of interacting with
computers and it has become present in everyday life. This can join the real and virtual
worlds in order to open up different ways of interaction and expand this interaction
between these two worlds by looking at the reality in a digital way. By following a
model based on overlapping of virtual objects with objects from the real life this
technology can be used, for example, for entertainment, marketing, medical processes,
in the industry and for improving the teaching and learning processes of content in the
classroom.
AR can be defined as mix of three basic components which are a combination of
real and virtual reality, interactivity in a real time and to be registered in three
dimensions (AZUMA, 1997, p. 356). There is also the possibility to use a real object as
reference which can be interpreted to create a virtual one. Thus, a video camera can be
used to capture images and the software that can mixture those images and interpret the
code.
One of the first software based on augmented reality was known as HeadMounted Display. In the year of 1968 it was able to make virtual information allocation
over physical objects and it was created by Ivan Sutherland Hautsch (2009). After the
emergence of the term Augmented Reality one project was developed for mobile
equipment based on GPS (Global Positioning System). The main aim of this project was
to help blind people (or with low degree of vision) and it used as methodology the
overlapping of captured images with sound information.
According to Houtsch (2009), knowledge about augmented reality has been
spread since few time with the constant use of two dimensions codes (2D) which was
developed to overcome the limitations of the bar code and optimize the tasks.
Augmented reality has higher capacity of storage of information than bar code. In this
case, it combines two dimensions code with software which can read them.
Augmented reality is based on gathering field marks, tags, images or something
else pattern which can have its movement captured as reference. Software based on
augmented reality should be programmed to identify those references and thus to know
V. 3, N.2, Ago/2013
Page 54
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
what to do from pre-defined actions. In general terms, Hauscht (2009) explains that it is
possible to define steps to use software which is based on augmented reality as:
- Capturing images with a camera;
- Images are transmitted to the software in real time;
- Software recognizes pre-programmed images in a data base;
- Software identifies the actions to be taken through the image sent – the actions
are pre-defined;
- Output device builds up the virtual object on the actual image and displays this
set.
 Development tools for AR
In order to develop software based on augmented reality there is several tools in the
market which can help developers such as ArtoolKit , ARToolKit Plus, ARTag, Dart,
OSGART. In this work, two tools were studied but only one was selected to construct
the software based on AR.
The first was ArToolKit, it consists in library in C Language with which is
possible to create interfaces of augmented reality. By using the method of direct vision
by video it detects images of markers.
The second was Ezflar, it is a library which was developed by developers of
Newspaper O Estado de Sao Paulo. Ezflar allows developing application of AR without
a need of programming. Its code in ActionScrip3 was created based on libraries as
FLARManager, FLARToolkit and Papervision3D.
Thus, the fact that ArToolKit is open code (CONSULARO, 2004) can be
considered as a differential point for those developers with technical knowledge to
optimize the tool. Therefore, such characteristic makes difficult for novice users to use
this set of tools. ArToolKit was used in this study because developers were able to deal
with technical issues and then to build the software which was used for demonstrations
in workshops at schools.
 Limitations of AR based on computer vision
ArToolKit calculates the viewpoint of the camera relative to a marker to perform the
overlap of virtual objects on real world. However, this task has some limitations which
directly influence the success of AR applications (Consularo, 2004). Objects can be
only displayed whether the markers were completely read. In this case, it can impact on
the size of the marker and consequently on the size of the virtual object and its
movements.
There is also limit on the slope of the marker in respect to optical axis of the
camera. The larger the size of the marker the largest the distance in which it can be
detected, but it is difficult to perform tracing it. Moreover, full of detail standards have
reduced efficacy. Another aspect is the lighting, which can create reflections and bright
regions in the marks of the paper making it difficult to detect by software.
3. RESULTS: HOW THE FIRST PEDAGOGICAL MATERIAL WAS
CONSTRUCTED
The ARToolKit was chosen because it is a set of free tools for non-commercial
purposes as well as having open source enabling adapting applications of the tool.
To minimize some limitations of augmented reality it was decided to construct the
markers in the format of cubes (Figure 1), as they keep the plane figure which facilitates
V. 3, N.2, Ago/2013
Page 55
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
reading by the camera, and prevent children put your fingers on the marker making it
impossible to read.
Figure 1 – Cubes used in the workshops
Source: http://www.uepg.br/proex/anais/trabalhos/281.pdf
 Workshops with children
This extension activity was developed by four students: two students of the course of
Computing Engineering and two students of the course of Bachelor of Computing. The
workshops began in March 2012 and used the first version of the material with
augmented reality (using the ARToolkit and the cubes as shown in Figure 1). Four
workshops with three public schools for students in fourth and fifth grade of elementary
school, a workshop with students of the third year of high school and three shows at
Extension events were held so far (May 2013). The workshops amounted participation
of 86 elementary school students and Extension events approximately 150 participants
(students, teachers and the community at large). Currently, workshops are held in public
and private schools in the region of Ponta Grossa.
During the activity, students are informed about the project and advised on
application usage. To carry out the activity are needed 16 students divided equally into
four teams. For each team member is given a cube with six markers to be positioned in
front of the camera until the figure is showed for analysis.
In this activity the student must determine whether the displayed figure is from
the area of computer science and is old equipment or not. Moreover, competition is
organized in the way that the team that set the figures more efficiently and effectively is
the winner.
Students are observed during interaction with the application. After that,
students are asked to answer a questionnaire about the workshop and the application.
One aspect to be considered is that the purpose of the activity will not only generate
knowledge, it also stimulates critical thinking, competitiveness and team collaboration.
Thus, in order to develop the new version of the augmented reality application,
allowing teachers to optimize the application according to your preference, a study of
the concepts involved and the tools that will be used was performed. The characteristics
of the proposal are described in the following topics.
3.1. The Concept of Graphical User Interface
The means by which the human part requests to the program the execution of tasks,
some answer, or any communication between the parties is made by an interface
V. 3, N.2, Ago/2013
Page 56
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
(PREECE ET AL, 2005). Often confused with the software itself, the interface is very
important in the acceptance of software, since, of course, customers are attracted by the
ease of handling, attractive screens with good design as well as the self-explanatory
visual components. These characteristics show the need for a good interface for
software.
Graphical environment is software designed to facilitate and make practical use
of the computer through visual representations of operant system. According Deitel
(2003) the user interface allows interaction with digital devices through graphical
elements such as icons, windows, menus, and pointers and other visual indicators. From
this point of view the basic and fundamental problem in the area is the transformation of
data into image.
Therefore, the purpose of the prototype of an interface for ARToolkit technology
is making a viable tool for teachers to assemble your own set of images without the aid
of a programmer to start the application. In this aspect, the first screen of the application
was constructed by considering simplicity, in order to facilitate usability. It was based
solely on descriptive buttons in contrary to initial design in which the programmer must
open the root folder of the file and explore for all folders to find the file which is
responsible for video playback and the capture of the markers. Therefore, the Java
development platform was chosen because its facilities to access the graphics library.
3.2. Creating Java Interface
The classes used for creating graphical user interfaces as well as to provide them
functionality are grouped into two major packages: java.awt (core package) and
javax.swing (extension package).
 Swing
Created in 1997, most components are written, manipulated and displayed completely
in Java, without assistance, they are considered simple and provide uniform appearance
and functionality in any platforms (DEITEL, 2003) (Figure 2).
Figure 2 – Swing interface.
 AWT - Abstract Windowing Toolkit
The GUI components of AWT (Figure 3) appeared in Java in version 1.0, features of the
user platform. Thus, the appearance of the components differs when run on different
operating systems, designed so that each virtual machine implements the interface
software. This may be desirable because it allows the software to use the GUI
components that are already familiar, but the layout can be changed by varying the
platform (FRIENDLY, 2003).
V. 3, N.2, Ago/2013
Page 57
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
Figure 3 – Comparison between AWT and Swing interfaces
 First layout
The whole layout was produced in Java (Figure 4), specifically in the javax.swing
library, due to its characteristic called appearance of metal. It means that the appearance
does not change regardless of the operating system in order to prevent any changes and
the user does not understand the operation of the program.
.
Figure 4 - First layout
The first button will be responsible for video playback, that is, implementation of mixed
reality. Its functionality is basically to open the "simpleVRML" application, and further
bring the user to the camera settings that are already defined by the application and only
need a simple confirmation.
To compile this project in Java, the library glut.h and openGL.lib package
(implemented by the library jogl.h) are required. The study was based on 2D pictures. In
case of 3D pictures and animations the openVRML.lib, libpng.lib, lipjpeg.lib, zlib.lib,
artln.lib libarvrmld.lib and libraries should be added.
V. 3, N.2, Ago/2013
Page 58
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
4. CONCLUSIONS
By analysing the workshops the Museum of Computing project developed it was noted
great interest on the part of both students and teachers in relation to the pedagogical
material which was based on augmented reality. The design and presents the technology
of augmented reality, was able to arouse the interest of computer interaction with the
educational environment. Among the schools in which the workshop was developed
many requested to keep informed about new technologies.
Despite the good results the applicability of the project is limited by the need for
computer equipment such as digital cameras. Materials that in some schools, are not
always available for students.
The university students involved in the Museum of Computing project aims that
the new interface is understandable to users in order to disseminate technology, to
simplify the learning of complex content and encourage interaction with the means of
computer based learning.
REFERENCES
AZUMA, R. T. et al. Recent Advances in Augmented Reality. IEEE Computer
Graphics and Applications, v .21, n.6, p. 34-47. 2001.
AZUMA, R. T. A Survey of Augmented Reality, UNC Chapel Hill, In Presence:
Teleoperators and Virtual Environments, 1997.
CERUTTI, D. M.L., CELINSKI, T.M., IELO, F. G. P. F., DUDA R., VILLALBA,
R. D. A., MELO, G. A.Museu da Computação da UEPG. Ensino-Pesquisa-Extensão e
o Museu Virtual In: CBEU – Congresso Brasileiro de Extensão Universitária, 2011,
Porto Alegre.
CONSULARO, L.A.; CALONEGO JR, N.; DAINESE, C.A.; GARBIN, T. R.;
KIRNER, C.; TRINDADE, J.; FIOLHAIS, C. - ARToolKit:Aspectos Técnicos e
Aplicações Educacionais. In: Cardoso, A.; Lamounier Jr, E. editores. Realidade Virtual:
Uma Abordagem Prática. Livro dos Minicursos do SVR2004, SBC, São Paulo, 2004, p.
141-183.
DEITEL, H. M., DEITEL P. J. Java Como Programar. Bookman, Quarta Edição,
2003.
FRIENDLY, L., CAMPIONE, M., WALRATH, K., HUML, A.The Java
Tutorial.Sun
Microsystems,
Terceira
Edição,
2003.
Available
in
http://java.sun.com/docs/books/tutorial/. Access 10 jun. 2013.
HAUTSCH, O. Como funciona a realidade aumentada, 2009. Available in
http://www.tecmundo.com.br/realidade-aumentada/2124-como-funciona-a-realidadeaumentada.htm. Access 10 jun. 2013.
V. 3, N.2, Ago/2013
Page 59
Iberoamerican Journal of Applied Computing
ISSN 2237-4523
PREECE, J. et al. Interaction Design. Porto Alegre: Bookman, 2005.
V. 3, N.2, Ago/2013
Page 60