Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Do Mobile Device Applications Affect Learning?
Doug Vogel
Dept. Information Systems
City University of Hong Kong
isdoug@cityu.edu.hk
David M. Kennedy
Information & Tech Studies
Hong Kong University
dkennedy@hkucc.hku.hk
Ron Kwok
Department of Information Systems
City University of Hong Kong
isron@cityu.edu.hk
Abstract
Mobile devices (e.g., PDAs and smartphones) are
increasingly emerging as part of daily life, particularly
with university students. The City University of Hong
Kong has embarked on a long-term program to
develop and integrate mobile learning activities into
the context of undergraduate courses. This paper
reports on the development, introduction and
evaluation of a portfolio of collaborative mobile
learning applications. Results support convictions that
intrinsic and extrinsic motivation to embrace mobile
applications correlates with enhanced performance,
albeit with constructive alignment of student learning
interests as a moderator.
1. Introduction
Technology in education is not new but times are
changing as we move into more distributed contexts
with a variety of pedagogical motivations [11]. Early
efforts focused on collaboration within a more
traditional learning environment [1]. More recently
(e.g., [2]), attention has been given to institutional
direction and options. Aspects of successful integration
of web-based course management systems have been
explored (e.g., [15]). Increasingly, institutions
recognize that the learning environment extends
beyond the classroom and are thus exploring a variety
of technological support options.
Mobile devices are increasingly prevalent and are
seen as a “birthright” in some parts of the world (e.g.,
Hong Kong); accordingly, a variety of mobile learning
applications have been developed (e.g., [6]). However,
little exists to illustrate what difference (if any) they
Kevin Kuan
Dept. of Information Systems
City University of Hong Kong
kkykuan@cityu.edu.hk
Jean Lai
Department of Information Systems
City University of Hong Kong
isjlai@cityu.edu.hk
make, especially in terms of learning. This is curious
given the ubiquitous nature of these devices and
applications and the ability to potentially connect
anytime and anyplace for interactive learning. Digital
natives are at the center of their own personal learning
environment (e.g., smartphones, PDAs, iPods, etc.),
yet limited attention has been given to the impact on
learning of mobile devices and associated applications
in education [12]. As mobile device applications
become even more pervasive, it is incumbent that we
understand their (potential) impact and integrate them
into the portfolio of educational and learning options.
Further questions exist as to what degree and in
what ways institutions can become a part of this
emerging student-centered learning environment and
how they can maximize the impact of that
involvement. Students have a variety of alternative
uses for any mobile device. In particular, a question
exists as to the characteristics of learning applications
that will be accepted and meaningful to students in a
collaborative learning space with a variety of
interaction channels. To research this domain, the City
University of Hong Kong (CityU) has provided
wireless PDAs to all incoming business student over
the past two years (over 1600 in total). These devices
have been used extensively on and off-campus in
conjunction with a portfolio of applications to support
learning.
In this paper, we explore the impact on learning of
these applications designed for use outside the
classroom. Particular attention is given to a multiple
choice learning application with summative and
formative feedback running on mobile devices that
enables students to prepare remotely for their final
exam. Comparisons are made first between students
using the learning support versus those who chose not
to do so, and second between topics that are chosen.
1530-1605/07 $20.00 © 2007 IEEE
1
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Usage is further correlated with performance as
exhibited on exams. Particular attention is given to the
pattern of mobile learning application use, e.g., for
exploration of alternatives. Conclusions are drawn.
2. Literature
There is a variety of literature from different
disciplines regarding motivation (both intrinsic and
extrinsic), learning, student performance and
technology support. However, there is relatively little
in the intersection of these literatures, especially in
conjunction with mobile devices. Alavi and Gallupe
[2] recognize the high levels of cultural change at
individual and institutional levels, and note the
importance of performance assessment in conjunction
with technology-mediated learning.
Mobile devices, e.g., PDAs and “smartphones,” are
a categorically different form of technology with
different behavioral consequences. The ubiquitous
nature of these mobile technologies in terms of being
constantly within reach of the users and continuously
connected to a broader communications network give
them a unique status in the realm of technology
support for education and learning. In this sense, these
devices become a form of “wearable” technology that
places students in the center of their own unique
learning environment. They have the opportunity to
choose what to access, when and where, and with more
degrees of freedom than other forms of technology
support e.g., PCs.
Self-guided exploration has been found to be
conducive to the development of intrinsic motivation
in more traditional educational settings [7]. We can
enhance learning motivation by emphasizing the
importance and applicability of the material and by
trying to connect the material to students’ intrinsic
motives [5]. They particularly note that learning
motivation is likely to be greater if a student feels a
particular class is consistent with their interests and
with personally satisfying career goals. However,
learning motivation is malleable and can change over
time [13]. The early development of high learning
motivation is critical. Thus it is important to focus on
building learning motivation for education in general,
as well as for specific classes [5].
Mobile devices, e.g., PDAs and “smartphones,” are
increasingly pervasive, especially in student
populations. In some parts of the world (e.g., Hong
Kong) this exceeds 100%, i.e., each student may have
more than one such device. The pervasive nature of
these devices provides an unprecedented opportunity
for creation of learning applications. As with paper and
books, we can increasingly expect that our students
will (or can be mandated to) have personal access to a
mobile device to complement their traditional learning
activities.
Developing effective learning applications requires
a number of considerations. We need to consider both
the human dimension (students and instructors), as
well as the design dimension (content, technology,
interaction, etc.) [10]. There are additionally a wide
range of institutional considerations along the way
towards broad-based institutional use [16].
A key aspect of mobile device applications is
pedagogical integrity. The research of John Biggs [3]
[4] provides pedagogical direction that is particularly
relevant in terms of aspects of constructive alignment
and deep learning. Biggs points out that students who
are given learning materials consistent with their
interests and aspirations are likely to perform better; he
has provided a taxonomy (SOLO) to assist in
evaluating the degree to which students have gone
beyond surface learning towards deep learning that can
better enable transfer to other tasks. Wood et al., [17]
further note that enactive exploration on the part of
students also tends to enhance performance on transfer
tasks. The research of Biggs and Wood is especially
complementary to that of Cole and Feild [5] who
emphasize motivation.
3. Research environment
At the City University of Hong Kong (CityU), the
development of mobile applications has focused on
providing more flexible possibilities for student
learning. Students in Hong Kong are like most others,
juggling busy lives and moving in and out of wired
and wireless environments. Zheng & Ni [18] have
suggested a model for the manner in which one may
consider the elements of mobile computing which is
more congruent with current practice and
infrastructure. The Hong Kong students move between
the:
x connected mode (at the campus),
x nomadic mode (at home or connect to a desktop
computer on campus or at home), and
x disconnected mode (on public transport, away from
wired or wireless connections).
Creation of the mobile environment and associated
applications has proceeded along the lines proposed by
Nunamaker et al. [14] and Hevner et al. [8]. The
current project is a holistic approach that seeks to:
1. Develop and research the use of a range of
tools that are designed to more readily support
academic teachers in their quest to match the
2
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
student learning outcomes with appropriate
activities, assessment and feedback;
2. Develop the technical infrastructure that
enables academics and students to
collaboratively author activities to the web or
personal digital assistants (PDAs) or smart
phones;
3. Develop the technical infrastructure to allow
lecturers to monitor student activity, record
student learning outcomes about student
interactions on their PDAs to the lecturers
from within the university learning
management system (LMS), BlackBoard; and
4. Provide advice (mainly pedagogical) and
support (with examples) for developing
content suitable for mobile learning.
To create an environment for application and
research, CityU provided 150 wireless PDAs to first
year undergraduate students in the Fall of 2004.
Assessments of usefulness and acceptance were made
during the semester, while at the same time, existing
infrastructure, support, and adoption problems were
addressed. The experiences proved enough of a
success to enable a faculty wide roll-out one semester
later. By late January 2005, 800 freshman students had
successfully been equipped with PDAs, and were
using them every week in their core courses. Positive
feedback from instructors and students led to the
decision to repeat the initiative for a further year (with
plans for the year following). Hence, in August 2005,
the Faculty of Business provided a further 800 wireless
PDAs to equip the second entire freshman class with
interactive computing devices.
A series of mobile-oriented applications have been
developed and evaluated. These include:
x e-tips provides exam questions from the
previous year. No answers are supplied.
x Crossword Puzzle (as its name implies)
provides hints and clues for PDA users to
solve a disciplinary-centric puzzle.
x Mobile Exercise provides sets of multiple
choice questions that the students can
complete offline.
x Tatoes is a quiz application based on ‘Hot
Potatoes’ (freely available to non-profit
educational institutions;http://hotpot.uvic.ca)
that provides detailed instantaneous formative
feedback for incorrect (as well as correct)
choices in sets of multiple choice questions.
The project team has created a set of icons to
provide a consistent interface and usability experience
for students that crosses applications. Additionally, an
authoring environment provides support for creating
application instances.
An eToken system has also been created to help
motivate student participation. Using the e-token
system, students download an application and then are
rewarded with points based on the degree of
application completion success. Results are encrypted
to discourage inappropriate sharing of results, e.g.,
students are not able to simply upload results files to
multiple student accounts from a single PDA. As such,
there was reasonable confidence that students were
independently submitting results. Students (and their
scores) are recorded on the e-token server which is
subsequently linked back to the BlackBoard LMS for
selective incorporation into course grading. Top
ranking students are also recognized on the e-token
website.
Additional applications have been (and are
continuing to be) developed as more is learned. These
include:
x IGO (Interactive Graphing Object) was
originally developed for the web [9]. IGO
enabled students to sketch a graph on screen
in order to articulate their knowledge directly.
This is in contrast to watching an animation
or choosing the correct graph from a set of
static images. Repeated feedback is given as
students adjust the graph they create to meet
problem specifications.
x Phototate provides field trip support for
students to take a picture with their PDA,
annotate it and add a voice tag, all of which is
then uploaded to a BlackBoard portfolio.
x Rogaining is a collaborative scavenger hunt
in which student teams are given a budget and
urged to buy a set of products (using
Phototate) for a simulated department store
(e-organization).
These newer applications have not yet been
formally evaluated with large numbers of students but
initial trials have been conducted, for example,
Phototate has supported a group of students on a field
trip to Norway to examine plant life above the Arctic
Circle.
4. Assessment approach
The anytime/anyplace/any pace availability of
mobile devices is felt to provide an environment in
which barriers to participation are accordingly
lowered, noted by Deci and Ryan [7] as being a factor
in intrinsic motivation. There is also an element of
extrinsic motivation in conjunction with the e-token
3
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
approach and availability of support designed to help
students prepare for their final exam.
The theoretic foundations for PDA impact
assessment are based on two sources. The first is the
work of John Biggs [3] [4] who proposes that
“constructive alignment” between the desires of
students and learning materials is key to performance.
We additionally draw on the work of Wood et al. [17]
who note that those students who explore the range of
possibilities associated with a task tend to outperform
students who stop after arriving at a correct answer
when faced with the need to transfer learning to new
(related) tasks. This leads to the model illustrated in
Figure 1 and associated postulates (P).
Constructive
Alignment
Mobile-induced
Motivation
Performance
Figure 1. Research model
P1. Motivated use of mobile learning applications will
positively influence student performance.
P2. Mobile learning applications that align with
student learning interests will positively moderate
performance.
Both quantitative and qualitative data have been
gathered to evaluate these postulates. Surveys have
been conducted and focus groups run. eToken results
have enabled comparison of student activity with
grades to give indications of performance impact.
Specifically, analysis of covariance (ANCOVA) has
been used (with prior student performance as a
covariant) to recognize results of PDA use within the
normal variances in student performance as a function
of interest, motivation or natural ability. Additional
field experience with Phototate has also been reported
to shed further light on results from use of Tatoes.
5. Results
To date, perceptual data on a large scale have been
gathered from two courses of study: 1) from 416
students in multiple sections of an introductory
business course (of approximately 800 students) in
Semester 1, of the 2005-2006 year, and 2) an
introductory business class in Semester 2 of 2006. The
use of the PDA was not mandated in either course, but
was left a voluntary activity. Current feedback
(voluntary) indicates that students who engaged in the
use of mobile devices for learning found that the
experience of using a PDA enhanced their learning
experience. In particular, they noted that the mobile
environment made the course more interesting,
enjoyable and fun in addition to being useful for
understanding and remembering, as well as generally
helpful for motivation and increasing confidence.
However, they saw the environment questionable for
changing learning behavior and not especially useful in
stimulating students to explore new topics. We begin
by reporting data based directly on PDA use,
accompanied by focus group comments and followed
by ANCOVA results.
Recently, 186 students actively used the eToken
system from a total of 812 enrolled in the Semester 2
course, as summarized in Table 1. “Max” indicates the
maximum number of instances accomplished by an
individual student. “Sum” indicates the total number of
instances accomplished by all students participating in
the learning activity. “S.D.” is the standard deviation
around the mean student accomplishments.
Mobile Applications
E-Tips downloads
Crossword puzzle downloads
Crossword puzzle uploads
Mobile exercise downloads
Mobile exercise uploads
Tatoes downloads
Tatoes uploads
Total count
Max
11
7
8
8
9
10
2
31
Sum
288
116
49
201
72
305
36
1067
S.D.
1.15
.75
.57
1.07
.69
1.35
.29
3.22
Table 1. Descriptive statistics of applications
More applications were downloaded than uploaded
due, in part, to the relative small amount of extrinsic
motivation compared to intrinsic motivation and
individual curiosity. The use of the eToken system was
entirely voluntary. There is also higher variability of
downloads as opposed to uploads, i.e., students were in
a more exploratory mode when downloading and those
who chose to upload results were more internally
consistent and definitive as a group. The total count is
less that the sum of the individual application “Max”,
as one would expect as a result of students choosing
different applications. The students who downloaded
“e-tips” were also inclined to download “tatoes” (i.e., a
4
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
correlation of .57), indicative of feedback immediacy
impact in those two applications.
Each of the applications will be discussed
separately, based on demonstrated usage as well as
comments from two focus groups, comprising a total
of 26 students conducted using GroupSystems to
record and organize results. Student comments have
been reported in their own words, i.e., complete with
grammatical and spelling errors to retain authenticity.
e-tips had 288 download instances but the results
were mixed in terms of helping to achieve a higher
academic grade. Students noted:
“eTips is useful and a good tools to help us to prepare
for the examination”
“not really as it is quite short and not detailed. It can
tell the main key point, but not all key points, so, I also
need to revise other things”
“concise and precise, but sometimes, too short”
“sometimes the contents of the sentances are too
"wide" that i don't exactly know which materials is
important.”
“if there is more guildlines, we can follow it more
easily and more willing to do the exercises.”
“tooo difficult to play ,becasue no choice are given, we
need to very familiar with the topic first be4 i can
know one or two ans.”
“From the word "Crossword Puzzle", I think it is quite
silly and will not be helpful and useful enough for my
learning, so I don't use it.”
Mobile exercise experienced 201 download
instances with 72 uploads. Student comments included:
“it can help us to review the lesson and test our
understanding and it is easy and not troublesome in
inputting the answeres”
“It is good for us to test whether we are understand to
content or not”
“It can help me to achieve a better grade. It can test
my understanding on each chapters. If I find out that I
am not doing well on the MC questions, then I will
revise that chapters again.”
There were, however, some frustrations including:
Suggestions for improvement included:
“if possible, it should include a diagram to summarise
and present the linkage to other topic, becasuse
students like diagram more and easy to remember”
Crossword puzzle had 47 students downloading a
total of 116 crossword puzzle instances from the 17
different puzzles available. However, only 49 instances
were uploaded for e-token credit. Student focus group
comments included:
“After doing the crossword puzzle, I could understand
some words that I don't familiar with before.”
“if you can learn more words of this subjects, it is
more easier for you to revise and you are more willing
to do so. Therefore, after you "know" the which topics
you are not familar with, you can put more effort on
that topics. it can indirectly upper your grade.”
Some frustrations were also present, for example:
“some words are too long and difficult and some are
technical words and difficult for us”
“Sometimes, I only want to check whether I
understand a few questions but not all.”
“There is no explaination on the answers. I think that
it will be better if there is some explaination for each
choice of each question.”
“sometime the programm will shut down because of
hang computer, and i cannot check the answere
immediately and have to answere all the questions
again.”
Tatoes recorded 305 download instances from 87
students. However, only 28 students uploaded 36
results that could be analyzed. No students (of those
who uploaded data) actively explored alternative
answers after arriving at the correct answer. The
students stopped once they discerned the correct
answer, sometimes after 1, 2 or 3 wrong attempts.
Comments from the student focus groups were largely
positive. The students especially appreciated both the
immediacy and substance of the feedback, for
example:
“for sure is the immediate response function. also, i
can know the answer without connecting the internet.
5
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
So, I can downlaod it at school and then do it at home
when I have time.”
“Hints. This feature save our time to find the right
answer and related topics form textbook and notes.”
“It gives some hints and explanations to the users. It is
useful because the users can know the reasons for their
wrong answers”
“I can immediately know whether my answer is correct
or not.”
However students also noted some problems in that:
“Hints is useful sometimes but not always because
some hints made me more confuse.”
“I think the lecturers and tutors should promote this
during lectures or tutorials so that students may know
it. Also, the download speed should be fast enough so
that students do have to wait for a long time.”
With respect to the Tatoes supported ability to
continue to explore the wrong answers after arriving at
the right answer (to enable further exploration to better
understand the topic), the students noted:
“When I get the right answer which is just because I'm
lucky, I'll click all the other answers to see why not
them; but for those answer that I'm quite sure, I won't
as I think it is just a waste of time.”
“when i got the right answer, i won;'t waste time to
click on the other choices because i know well on that
question already.”
Comparison of student use of the mobile
applications correlated with their performance in a
number of ways. Students who made use of the mobile
applications performed significantly better on the midterm exam (p<0.01) and final exam (p < 0.01) and
received a better course final grade (p < 0.01)
compared to those students who chose not to use the
mobile applications. Among the applications, Tatoes
and e-tips were especially salient.
Tatoes is of special interest since it had the highest
instance usage and was also introduced only in the last
two weeks of the course, i.e., after the mid-term exam.
The content given to the students was exclusively
oriented to supporting material unrelated to the midterm but directly related to the final exam. Since the
mid-term exam was a good indicator of prior student
study performance (p < 0.01), we adopted it as an
ANCOVA covariant. Students who downloaded
Tatoes instances had significant higher scores on the
Final Exam (p < 0.05) and a higher Final Grade (p <
0.05) than students who did not download Tatoe
instances. Tatoes was also especially well received as
noted in focus group comments, for example:
“It is a good way, because it can arouse my awareness
if similar question occurred in mid-term or exam.”
“Even the questions are not exam-oriented, you can
still gain more knowledge on this subject. It is another
way to help us to achieve a higher academic grade.”
“tatoes help me most because it help me to understand
the content more.”
6. Discussion
In general, our results provide some support for our
research model (Figure 1) and associated postulates.
As noted by one focus group student, “i think all of
these can help us to achieve better academic
performanace.”
Empirically, those students who
were motivated to use the mobile applications tended
to achieve higher levels of performance as indicated on
the mid-term exam, final exam and ultimate course
grade. This is independent of whether the motivation
was intrinsic or extrinsic noting that fewer than half of
the students who downloaded application instances
bothered to upload to results to receive e-token credits.
The more general support for Tatoes particularly
provides an element of support for constructive
alignment as a moderator since the exercises were
more directly in line with students’ learning interests
and choice (based on higher levels of use and focus
group comments) relative to other applications.
However, we caution that these results should be
seen as preliminary and not necessarily demonstrative
of enhanced learning on the part of the students. For
example, there is no special way to discern from these
results that the pool of students who chose to use the
mobile applications were not just simply better
performing students compared to those who chose not
to use the mobile applications, or that their learning
style was more compatible with PDA use in
accordance with application characteristics. Further, it
is questionable whether exam performance has any
special correlation with learning.
Whether constructive alignment is confirmatively a
moderator on performance also remains a question.
There was no stratification in the sample that enabled
such level of detailed comparison. How to get students
6
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
to explore a domain beyond the scope of a particular
instance is also a remaining challenge. It is clear from
the log data that students generally ceased exploring
alternatives once they arrived at the right answer.
However, there may have been some confusion with
respect to trying to achieve the best possible mark.
Active use of the distracters to confirm or query why
these components of each question were not the correct
answer remains to be seen.
We also note that more can be done to influence
general use of the mobile applications. As focus group
students noted:
“tutors and lecturers should spend some time to
promote this system. let students know it is useful is
very important as it can encourage them to use it.”
“I dont know where to do/download these exercises.
Also i am not necessary to go the e-token website”
“There should some guildlines for students to teach
them how to use this system.”
Field trip use of Phototate amplified the importance of
instructor and administrative support to create an
unambiguous and seamless learning environment. As
the field trip group leader stated, “I look forward to
how we can maximise the PDA device to engage or
motivate students to better learning.” Unlike Tatoes,
Phototate use was preceded by in-depth pedagogical
instruction given to the students by the Education
Development Office (EDO) representatives, focusing
on a number of issues:
x Students were encouraged to use the PDAs to
capture their immediate reactions in their
native language (Cantonese) from which they
could subsequently create summarized notes
and conclusions in English supported by their
photos.
x Students were especially encouraged to
capture the essence of the context surrounding
the photos (and associated verbal comments)
to assure that they appreciated “the bigger
picture” and didn’t “miss the forest for the
trees.”
x Students were also encouraged to reflect on
the use of Phototate in conjunction with
support for their learning. To jumpstart the
process, they were given a presentation on
how to use the PDA’s to enhance both short
and long-term learning as well as recall.
On return from the field trip, Phototate usage data
was examined and a debriefing focus group was
conducted. The results were interesting, albeit
somewhat disturbing. The students did not use the
PDAs in the expected fashion; rather, they used them
casually, much as they would use their mobile phone
(e.g., to take some pictures of interest) but did not
make them an active part of their learning
environment. Upon investigation, the reasons were
both technological and behavioral, resulting in
unrealized pedagogical innovation.
x From a technological perspective, the cameras
on the PDAs were unable to capture the detail
of the samples desired by the students. There
was simply a lack of resolution at the focal
distances they desired. On a larger scale, there
were problems with the sampling and
analyzing technologies the students were
supposed to use that negated field analysis.
x From a behavioral perspective, the students
were not particularly encouraged to
personally explore their field environment. In
particular, there were a number of academic
advisors (none of whom were PDA users)
who not only immediately responded to any
student questions, but additionally directed
the students explicitly to collect samples for
later analysis.
In summary, learning interventions require a
prodigious amount of concerted effort with iterative
and prolonged student and staff involvement and
interaction as process and technology join in
pedagogical evolution. This simply did not exist in this
field trip. Lacking a concerted level of effort and
seamless integration of technology, the students did
not change habits and engage in new forms of
learning.
Overall, we feel that general use of the mobile
applications leaves much room for improvement, both
in terms of the existing applications and, especially,
the newer applications that have yet to be evaluated.
However, we note that currently fewer than 10% of the
students in these courses actually purchased a
textbook. From that perspective, mobile applications
can become an increasingly important segment of the
student learning time and space; we feel that we have
made a solid start in that direction.
We are not suggesting that PDAs will replace
books. In fact, expecting students to read a traditional
text on a PDA is even less feasible than expecting
them to read from a book. What it does suggest is a
pedagogical rethink of the way we educate “digital
natives” to include the kinds of learning activities that
excite them and, from which, they may indeed learn.
7
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Our purpose in creating a portfolio of PDA-based
applications is an attempt to explore the impact of
learning activities that continuously travel with the
students and can be accessed at the time and place of
their choosing.
We also increasingly expect that technological
barriers to mobile use will continue to decline. For
example, we expect that within a few years, the type of
“smartphones” that all students are likely to have will
eliminate the necessity to provide them with wireless
PDAs and further enable more personalized mobile
application development and support. The current need
to download and upload from wireless “hot-spots” will
simply disappear. Students will be continually
surrounded with a learning environment that can be
better customized to their interests.
There are, however, a number of institutional
challenges that remain to be addressed as the
proliferation of mobile applications and devices
increases. As Wagner and Vogel [16] have noted,
emphasis moves from individual to more systemic
initiatives and, ultimately, policy considerations over
time. Technology interventions without examination of
the broader learning systems and practices in place can
easily result in a failed innovation. A key consideration
is how to keep faculty effectively engaged (and
supported) as technology changes. Mobile application
use is a sustaining innovation to students but a
disruptive innovation to many faculty and institutions.
Opportunities for research are many, as we move
forward to better understanding the effect of mobile
device applications on learning. Some research
opportunities can address limitations of the current
research, e.g., populations characteristics, application
stability and use over longer periods of time. Many
opportunities exist for the creation and evaluation of
more sophisticated applications, e.g., in the area of
collaborative gaming. The current research is but a
start on a long and winding road.
7. Conclusion
This paper explores the impact on learning of a
portfolio of mobile applications designed for use
outside the classroom. Towards that end, particular
attention is given to a multiple choice learning
application with summative feedback running on
PDAs that enabled students to prepare remotely for
their final exam. Comparisons between students using
the learning support versus those who chose not to do
so illustrate positive support for performance
enhancement with some support for constructive
alignment as a moderator. However, a number of
caveats are in order, especially in terms of population
characteristics and consideration of the broader
relevant system issues. Thus, to answer our question
“do mobile device applications affect learning,” we
feel that the evidence is tantalizingly positive, albeit
weak and much in need of further study. Ultimately,
we conclude that the future is challenging but bright
with respect to mobile application effect on learning.
8. References
[1] Alavi, M.; Yoo, Y.; and D. Vogel, “Information
Technology to Add Value to Management Education,”
Academy of Management Journal, 40(6), 1997, pp. 13101333.
[2] Alavi, M. and B. Gallupe, “Using Information
Technology in Learning: Case Studies in Business and
Management
Education
Programs,”
Academy
of
Management Learning and Education, 2(2), 2003, pp. 139153.
[3] Biggs, J. “Approaches to the enhancement of tertiary
teaching,” Higher Education Research & Development, 8(1),
1989, .pp. 7-25.
[4] Biggs, J. Teaching for Quality Learning at University. 2nd
Edition, Maidenhead, Berkshire UK: Society for Research
into Higher Education and Open University Press, 2003.
[5] Cole, M.; Feild, H. and S. Harris, “Student Learning
Motivation and Psychological Hardiness: Interactive Effects
on Students’ Reactions to a Management Class,” Academy of
Management Learning and Education, 3(1), 2004, pp. 64-85.
[6] Csete, J., Wong, Y. and D. Vogel, “Mobile devices in and
out of the classroom,” In L. Cantoni & C. McLoughlin
(Eds.), ED-MEDIA, 2004, Proceedings of the 16th World
Conference on Educational Multimedia and Hypermedia &
World Conference on Educational Telecommunications,
Lugano, Switzerland: Norfolk VA: Association for the
Advancement of Computing in Education, 2004, pp. 47294736.
[7] Deci, E. and R. Ryan, “The Empirical Exploration of
Intrinsic Motivation Processes,” in Berkowitz (ed.),
Advances in Experimental Social Psychology, New York:
Academic Press, 13, 1980, .pp. 39-80.
[8] Hevner, A.; March, S; Park, J. and S. Ram, “Design
Science in Information Systems Research,” MIS Quarterly,
28(1), 2004, pp. 75-105.
[9] Kennedy, D. M. “Continuous Refinement of Reusable
Learning Objects: The Case of the Interactive Graphing
Object,” In L. Cantoni & C. McLoughlin (Eds.), ED-MEDIA
2004, Proceedings of the 16th World Conference on
Educational Multimedia and Hypermedia & World
8
Proceedings of the 40th Hawaii International Conference on System Sciences - 2007
Conference on Educational Telecommunications, Lugano,
Switzerland: Norfolk VA: Association for the Advancement
of Computing in Education, 2004, pp. 1398-1404.
[10] Kennedy, D. M., Vogel, D. and T. Xu, “Increasing
opportunities for learning: Mobile graphing,” Beyond the
Comfort Zone: ASCILITE, Proceedings of the 21st Annual
Conference of the Australian Society for Computers in
Learning in Tertiary Education, University of Western
Australia, Perth, Academic Computing Services, 2004, pp.
493-502.
[11] Leidner, D. and S. Jarvenpaa, "The Use of Information
Technology To Enhance Management School Education: A
Theoretical View," MIS Quarterly, 19(3), 1995, pp. 265291.
[12] Naismith, L., Lonsdale, P., Vavoula, G., & M. Sharples,
Literature review in mobile technologies and learning (No.
11). Bristol, UK: FutureLab, 2005.
[13] Noe, R. (1986). “Trainees’ Attributes and Attitudes:
Neglected Influences on Training Effectiveness,” Academy
of Management Review, 11, pp. 736-749.
[15] Piccoli, G.; Ahmed, R. and B. Ives, “Web-Based Virtual
Learning Environments: A research Framework and a
Preliminary Assessment of Effectiveness in Basic IT skills
Training,” MIS Quarterly, 24(4), 2001, pp. 401-426.
[16] Wagner, C. and D. Vogel, “Pathway to Massive
Mobility in Educational Computing,” Proceedings of the
2005 International Conference on Informatics Education
Research, Las Vegas, Nevada: International Academy for
Information Management, 2005, CD.
[17] Wood, R.; Kakebeeke, B.; Debowski, S. and M. Frese,
“The Impact of Enactive Exploration on Intrinsic Motivation,
Strategy, and Performance in Electronic Search,” Applied
Psychology: an International Review, 49(2), 2000, pp. 263283.
[18] Zheng, P. And L. Ni, “The Rise of the Smart Phone,”
IEEE Distributed Systems Online, 7(3), pp. 1541-4922,
retrieved on 12 June 2006, from
http://csdl2.computer.org/comp/mags/ds/2006/03/o3003.pdf
[14] Nunamaker, J.F., Chan, M. and T. Purdin, “Systems
Development in Information Systems Research,” Journal of
MIS, 7(3), Winter 1990/91, pp. 89-106.
9