2011 Emerging Technologies Grant Scheme
Application Form
Application for:
Stage 1
Stage 2
Stage 3 (Circle one stage)
1. Title of Project
Manipulating Molecules: Using Kinect™ for Immersive Learning in Chemistry
2. Project Leader and contact details
Dr Ian Jamie, ian.jamie@mq.edu.au, ph x8293
3. Participants Details
o
Faculty/Office:
o Department:
Faculty of Science
Department of Chemistry and Biomolecular Sciences
o LTC Contact:
4. Names of other participants and contact details
Dr Christopher McRae, christopher.mcrae@mq.edu.au, ph x8288
5. Total funding sought
$3820
PROJECT DETAILS
6. Project rationale
An element of “play” is an essential condition for deep learning to occur. Play extends beyond
enthusiasm and a good time. It is about working out rules, understanding constraints, being
absorbed by novelty, using imagination and innovating. Play is also about extending, modifying
and augmenting activity. This project is about bringing an overt element of play to the
representation of fundamental chemical concepts and thus engaging students in the learning
process.
Chemistry has evolved into a science highly dependent on spatial-visual intelligence. More and
more, the abstract concepts of the sub-visual level, that of molecules and atoms, are being
explored through the aid of visualisation software. On Johnstone’s triangle1 of macro (what can
be seen), the sub-micro (atoms and molecules) and representation (the language that chemists
use – chemical symbols, formulae, etc), technology is narrowing the distance between the submicro (reality) and representation. There are many applications for visualisation of chemistry
concepts, with one of the most widely used being the “3D” presentation of molecular structures.
This involves the projection of a 3D structure onto a 2D surface and manipulation by use of the
mouse and keyboard (e.g., rotating, zooming in and out, and highlighting parts of the molecule).
There is a large number of molecular structures available in a range of databases and formats
(e.g., MDL/Elsevier/Symyx (MOL), Protein Data Bank (PDB), Crystallographic Information
File (CIF)).
Virtual reality (VR) environments provide an engaging educational context, able to support rich
and interactive experimental learning.2 They can provide a medium for experiential learning,
with the first-person experience enhancing the learning procedure.3 This type of experience is
central to a student’s interaction with the world and their construction of their interpretation of
the knowledge and skills that we as educators wish them to acquire. “Immersion” is the
subjective impression that one is participating in a comprehensive, realistic experience.4 With
engagement and motivation of learners recognised as a problem faced by many institutions5 the
use of games is a way to capture the attention of the video-game generation, thus games
technology is increasingly being used in the educational context. The widespread use of games
provides educators with opportunities to harness motivating factors such as fantasy, challenge,
immersion and communication to enrich the learning experiences of students.6
An illustration of the interest of games and immersive environments can be found in
Microsoft’s Kinect™ gesture recognition video game device. This device has broken the
Guinness World Records title as the world's fastest-selling consumer electronics device. The
hardware sold an average of 133,333 units per day, for a total of 8 million units in its first 60
1
Johnstone, A.H. (2006), Chemistry Education Research and Practice, 7, 49-63
Mantovani, F. (2003). “VR learning: potential and challenges for the use of 3D environments in education and
training”. In Giuseppe Riva & Carlo Galimberti (Eds.), Towards cyberpsychology: mind, cognitions and society in
the internet age (pp. 208–225). Amsterdam: IOS Press. Mayer, R. E.
3
Burner, J. (1996). Towards a theory of instruction. New York: WWNorton.
4
Dede, C. (2009), Science, 323, 66–69
5
Gee, J.P. (2005), Phi Kappa Phi Forum, 85, 33. Ref 6.
6
Reimann P, Freebody P, Hornibrook M and Howard S (2009), Immersive learning environments, Centre for
Research on Computer-supported Learning and Cognition (CoCo), Faculty of Education and Social Work, The
University of Sydney, p11-12
2
days on sale from 4 November 2010 to 3 January 2011.7 This device is a “controller-free
gaming and entertainment experience”. It uses a webcam-style peripheral to enable users to
control and interact with the “game” through a “natural user interface” using gestures and
spoken commands. While initially a tightly controlled proprietary system, the device was
quickly “hacked” and personalised applications were quickly developed. Microsoft has
announced that the company will release a software development kit (SDK) “to make it simpler
for the academic research and enthusiast communities to create rich natural user interfaces using
Kinect technology”.8
We intend in this project to bring together the play element of the Kinect technology with the
VR immersive environment of molecular structure by constructing a molecular manipulation
“game” for use in tutorials, lectures and the home environment. Our students are very
comfortable with visual imaging and the combination of natural gesture “games” has the
potential to invigorate chemistry education by means of the visual-spatial approach, an
approach in conformance with the way chemistry is thought about and practiced today.
7. Project description
The project will focus on developing the human gesture interface (HGI) for the manipulation of
molecular structures using the Microsoft Kinect™ Software Developers Kit (SDK) for
Windows and applications for translation of structure information from existing databases into a
form useful for the Kinect system. This will result in a PC-Kinect based system that can be used
with monitors, data projectors and smart whiteboards. It may be used in lectures, tutorials and
by students at home if they have access to a Kinect peripheral.
Dr Chris McRae has extensive experience in hardware/software development and will be
primarily responsible for developing the HGI. Dr Jamie is familiar with the file structures of the
commonly used molecular structure visualisation applications and will develop translation
software for conversion of these files to the Kinect format. Drs McRae and Jamie will jointly
supervise the research assistant in their task of converting the chosen files and testing the HGI
using these structures.
This project falls primarily into the Stage One category, with an emphasis on software
development, with testing performed by staff and research students (Honours and p/g). If the
opportunity arises some preliminary class-room trials will be performed within CBMS307
Physical and Environmental Chemistry II, where molecular symmetry is studied. The Kinect
system can be compared against existing molecular structure manipulation software and
traditional physical molecular models.
8. Project Development Plan (How the project will be developed)
This is a straight-forward development project. We anticipate that the HGI and database of
molecular structures will be completed within three months. The development of the HGI will
be an iterative one, in that we anticipate that as the database of molecular structures expands we
will receive feedback from our academic colleagues and experienced research students on the
best method of displaying the structures and using the system.
The HGI and file conversion software will be developed in parallel. Having developed the
software, the conversion of the files into the Kinect format will be routine and continue until the
research assistant funds are exhausted. The number of molecular structures available in existing
7
http://community.guinnessworldrecords.com/_Kinect-Confirmed-As-Fastest-Selling-Consumer-ElectronicsDevice/blog/3376939/7691.html accessed 25-Apr-2011.
8
http://research.microsoft.com/en-us/news/features/kinectforwindowssdk-022111.aspx accessed 25-Apr-2011
databases numbers in the tens of thousands, so it is unlikely that the research assistant will be
short of conversions to perform.
The main outcome of the project will be the hardware platform and software programs for use
in the development of further applications, and a database of molecular structures that can be
used in a variety of teaching applications immediately. We expect that full evaluation of this
system as a teaching tool will for the basis of a subsequent project.
9. Monitoring and Evaluation
As a Stage One project, evaluation of the pedagogical value of the system will not be
undertaken. Simple trials with academic colleagues and experienced research students will be
sufficient to prove the technical utility of the system.
Drs McRae and Jamie will be responsible for the achievement of the milestones of project.
They will develop the HGI and molecular structure file conversion program and supervise the
conversion of research assistant.
10. Risk Management
As a Stage One project, there are no apparent risks to staff or students.
11. Budget
Personnel (salaries + on-costs)
50 hrs level 4.1 casual research assistant for producing molecular
structure files and to assist with testing at $32.60/hr
Total personnel
1630
$ 1630
Project support
Microsoft SDK for Kinect
Hardware: Microsoft Kinect
Hardware: Workstation to host render server
No cost
190
2000
Total Project support
$ 2190
Grand Total
$ 3820
12. Consultation
There has been no consultation outside of the project proposers, Drs Jamie and McRae. The
proposed software and hardware does not require use of any existing University infrastructure
or system and Drs Jamie and McRae are competent in this level of systems development.
If the review committee feels that it appropriate to involve LTC or other bodies in this project
the project leaders are happy to undertake this.
13. Endorsement and statement of support by the Executive Dean and Head of
Department (or equivalent)
_________________________
Signature Applicant
_____________________
Date
__________________________
Name and Signature
Faculty Dean/Department Head/
Director
_____________________
Ranking by Faculty Dean/ Director
(If applicable*)
Date
__________________
(Rank of 1 is highest)
*Ranking is applied if there are multiple applications from a Faculty/Office for the one funding period.
Submit an electronic version of you application by Wednesday 27th April 2011*
to:
Katarina Poorova
Office of the Provost
Room E11A, 208
katarina.poorova@mq.edu.au
*This is the University closing date. Different Department and Faculty closing dates might apply. Please liaise with
your Department and Faculty well ahead this closing date to ensure an endorsement and signatures on the
application are obtained before the University closing date.