USER AND DESIGN PERSPECTIVES OF MOBILE AUGMENTED REALITY A THESIS SUBMITTED TO THE GRADUATE SCHOOL IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE MASTER OF ARTS BY DANIEL COOPER BALL STATE UNIVERSITY ADVISOR: JENNIFER GEORGE-­‐PALILONIS MAY 2011 ii ABSTRACT Augmented Reality is one of the hottest trends in mobile media, yet there is a substantial lack of user studies within this field of research. The purpose of this study is to evaluate the value of mobile augmented reality as a vehicle for information delivery on the basis of a usability and design analysis. Using a multi-­‐
dimensional qualitative method, this study examines the impressions of users and designers regarding mobile augmented reality, along with a heuristic evaluation of select mobile applications. This analysis finds that (1) users believe mobile augmented reality offers great promise as a medium for visual communication, and (2) it’s current execution is limited by technical restraints, design flaws and a lack of compelling content. It is concluded that mobile augmented reality will one day revolutionize how consumers engage geographical and time-­‐based information. Yet, like all new technologies, mobile augmented reality needs time to mature in order to address these limitations. Finally, based on the findings of this study, this thesis offers suggested guidelines for future developments in this medium. iii ACKNOWLEDGEMENTS A great deal of thanks is due to my thesis committee. My advisor, Jennifer George-­‐Palilonis provided invaluable guidance and support toward the completion of this study. Her passion for excellence and unwavering support she shows toward her students has profoundly influenced my professional career. I would also like to thank James Chesebro and Michael Holmes who routinely challenged me to examine my work with acute criticism and curiosity. Their constructive feedback was both invaluable and enlightening. Special thanks is also due to my colleagues at the Center for Media Design, particularly Michelle Prieb and Jennifer Milks who graciously allowed me the opportunity to bounce methodological ideas off them when this study first took shape. Thanks are also due to Angie Faller and Shawna Pierson for assisting with the videography and documentation of the focus groups and field research. Finally, I thank my participants for taking the time to share their experiences.
iv TABLE OF CONTENTS Abstract ......................................................................................................................................................... ii Acknowledgements ................................................................................................................................. iii Chapter 1: Introduction .......................................................................................................................... 1 Defining Augmented Reality .................................................................................................. 5 Research Questions ................................................................................................................ 11 Significance to the Field ........................................................................................................ 11 Chapter 2: Review of the Literature ............................................................................................... 12 History .......................................................................................................................................... 13 Consumer Mobile Augmented Reality Applications .................................................. 15 Development Concerns and Limitations ........................................................................ 17 The current state of user research and evaluations ................................................. 20 A potential solution ................................................................................................................. 21 Chapter 3: Research Methods ........................................................................................................... 26 Data Collection and Procedures ......................................................................................... 31 Ethical Considerations .......................................................................................................... 34 Chapter 4: Results .................................................................................................................................. 35 First Impressions ...................................................................................................................... 36 User Experiences and Feedback ........................................................................................ 41 Heuristic Evaluations .............................................................................................................. 47 Chapter 5: Discussion ........................................................................................................................... 58 Guidelines for Future Mobile Augmented Reality Applications .......................... 63 Limitations ................................................................................................................................. 65 Recommendations for Future Research ........................................................................ 67 References ................................................................................................................................................. 71 Appendix A: Focus Group Discussion Guide Protocol ............................................................ 75 Appendix B: Designer Perceptions Questionnaire .................................................................. 77 Appendix C: Heuristic Evaluation Protocol ............................................................................... 78
CHAPTER 1: INTRODUCTION Mobile augmented reality is a disruptive technology that is redefining how people perceive data by transforming the visual environment into an immersive, information-­‐based ecosystem. By blending our world with virtuality, augmented reality can enhance the perception of one’s personal space by overlaying digital information in the form of text, video, audio and other forms of imagery. Geographical and time-­‐based information has been made relevant to the unique perspective of individual users, simply through the push of a button on one’s cell phone. However, in order to realize the full potential of mobile augmented reality, there is fundamental need for understanding how users engage and perceive augmented reality content. Most research within the field is currently focused on the technological state of the art, in contrast to substantial user studies. Furthermore, a valuable means for evaluating the effectiveness of mobile augmented reality applications ability to deliver information in a clear and concise way is also required. To that end, a concise understanding of how information should be designed for an augmented reality system is paramount for creating a desirable user experience. If augmented reality is to become a commercial success, designers and developers alike need to understand the needs of their user bases and establish 2 fundamental design principles in order to prove value for displaying information over more traditional means. This thesis posits that a usability and design analysis of augmented reality is both feasible and useful, and perhaps provides a conception of value for mobile augmented reality as a vehicle for the visual communication of personally relevant information. Statement of the Problem Mobile Augmented Reality is struggling to find its niche. Marketers are banking on its hype by fusing this new technology with trading cards, cereal boxes and movie posters while dozens of mobile developers are investing on their hopes that augmented reality app will become the “next big thing” (Hurley, 2009; Husson, 2010). Yet, mobile augmented reality has many problems to overcome if it is to become a commercial success. Technical issues such as slow network speeds and information display errors continue to plague users (Haller, Billinghurst & Thomas, p. 26). It also remains to be seen how augmented reality can be used to solve real-­‐
world problems rather than serve simply as a marketing gimmick. If solutions to technical constraints and practical benefits for consumers are not realized, augmented reality could meet the same fate as other overhyped technologies, such as virtual reality worlds like Second Life (Wen, 2010). While augmented reality may be “next big thing” just as virtual worlds once were, tomorrow it could fade into obscurity due to a lack of consumer interest. 3 Nevertheless, the commercial sector sees potential business in augmented reality. According to market projections by Juniper Research, revenues from mobile augmented reality applications will reach $732 million by 2014. However, the current user base consists primarily of “early adopters,” a demographic consisting of “trendsetters” who are eager to try out new technologies before their peers, despite existing limitations (Rogers, 1962). Additionally, this user pool is rather small considering the number of existing smartphone users. According to Mashable.com, of the 83% of Americans who own a cell phone, only 25% are smartphone users (Schroder, 2010). Furthermore, according to research by Nielson, a third of all smart phone users do not subscribe to a data plan, which is a necessary requirement for operating augmented reality based apps (Nielson, 2010). Furthermore, the most popular augmented reality apps have only realized limited is distribution. As of 2010, the most downloaded augmented reality app, Layar has a user base of one million active users. Meaio’s Junaio is a distant second, having been downloaded only half a million times (Bryne, 2010). Not only are there a limited number of users; we also know little about them. Most research within the industry is focused on the advancement of augmented reality technologies and software applications, rather than on the user experience. In academia, understanding the problem of how users perceive augmented reality content has been virtually ignored. As of 2010, only a handful of usability tests have been conducted, most of which have been informal. (Schmalstieg et al.) Nor have the 4 opinions of users on the desirability and viability of existing mobile augmented reality applications been addressed. This lack of user research, combined with technological limitations of current augmented reality systems, is providing unique challenges for content designers and developers alike. Peter Meier, CTO of Metaio explained in a 2010 interview with Veture Beat, that current user experience needs improvement and a greater emphasis must be placed on design. Claire Boonstra, co-­‐founder of Layar, also tells Venture Beat that mobile augmented reality is moving from “functional AR to experience AR.” If the mobile augmented reality industry is to move forward, developers must convert the existing hype into engaging user experiences for larger audiences (Bryne, 2010). Moving forward, mobile augmented reality’s greatest strength over prior digital technologies is its ability to localize geographic and time-­‐based information within the perspective of the user’s personal environment (Rutledge, 2010). Yet, most companies are struggling to make this work (Husson, 2010). In addition to a lack of user research, there are also few options for evaluating the ability of mobile augmented reality applications to display information effectively (Schmalstieg, et al.). Effective information design is increasingly important when users will navigate a complex information ecosystem (Baer, 2008). 5 Defining Augmented Reality Unlike virtual reality, augmented reality attempts to enhance one’s personal environment, rather than replace it. By employing a technological lens that allows a user to see and interact with the information that surrounds us, mobile augmented reality allows users to interact with data on a more personal level. New technologies and applications are rapidly emerging in the areas of information search, entertainment, gaming and location-­‐based services such as tourism and directional mapping. Defining augmented reality: Although many researchers have broadened the definition and scope of augmented reality, the most commonly accepted defining criteria were conceived by Ronald Azuma in 1997 (Zhou, 2008). Azuma states that augmented reality systems share the following three characteristics: 1. Combines real and virtual objects in the real world. 2. Possesses interactivity and is presented in real-­‐time. 3. Registers and aligns virtual and physical objects with each other in 3-­‐D. The rationale for these criteria is to avoid limiting augmented reality to specific technologies. Prior to Azuma, researchers had primarily defined augmented reality through the use of head-­‐mounted displays, an approach unable to distinguish some AR and VR applications. These new criteria expand the applications of augmented reality to handheld mobile devices such as smartphones and monitor-­‐based systems that read specialized coded tags that can present augmented reality information on a stationary computer screen (Azuma, 1997). 6 Augmented reality vs. virtual reality: Augmented reality is often looked upon as a “middle ground” between virtual reality and the physical environment (Milgram, 1994). In contrast to augmented reality and the physical world, the realm of virtual reality lies entirely within the synthetic. Virtual reality can be defined as, “a computer generated, interactive, three-­‐dimensional environment in which a person is immersed” (Aukstakalnis & Blatner, 1992). Virtual realities, such as those found in computer games like World of Warcraft and Second Life, are experienced by the player through control of an avatar in a computer-­‐generated environment presented on a computer screen. Users may also experience virtual environments by wearing head-­‐mounted displays such as those are commonly used by the military for training purposes. The fundamental difference between virtual and augmented realities lies in the user perspective and immersion methods employed by each system. Virtual reality systems strive to be a completely immersive experience, similar to the level of immersion users experience with everyday living. In contrast, augmented reality distinguishes itself as a “mixed reality” between two worlds by blending virtual elements within a real-­‐world environment (Milgram, 1994). Paul Milgram expanded upon this concept with his “Reality-­‐Virtuality continuum” (Figure 1). Milgram argues that augmented reality is a “mixed reality”, sharing characteristics of both real and virtual environments. Milgram’s “virtuality continuum” visualizes the degree of user immersion and classes of objects displayed within a specific reality. For example, a user in a virtual world is completely 1 Introduction
For many decades researchers have been trying to blend Reality and Virtual Reality in
interesting ways to create intuitive computer interfaces. For example, in the area of
Tangible Interfaces real objects are used as interface widgets [Ishii 97], researchers 7 in
Augmented Reality (AR) overlay three-dimensional virtual imagery onto the real world
[Feiner 93], while in Virtual Reality (VR) interfaces the real world is replaced entirely
immersed within an artificial environment consisting entirely of synthetic artifacts, with a computer-generated
environment.
As Milgram points out [Milgram 94], these types of computer interfaces can be placed
while an augmented reality environment mixes the real world with virtual elements. along a continuum according to how much of the users environment is computer
generated (figure 1). On this Reality-Virtuality line Tangible Interfaces lie far to the left,
In contrast, augmented virtuality is the merging of real world objects into virtual immersive Virtual Environments are placed at the rightmost extreme, while Augmented
Reality and Augmented Virtuality interfaces occupy the middle ground. To encompass
environments (Milgram, 1994). this broad range of interfaces Milgram coined the term “Mixed Reality”.
Real
Environment
Augmented
Reality (AR)
Augmented
Virtuality (AV)
Virtual
Environment
Figure 1. Milgram’s reality-­‐virtuality continuum model. Figure 1: Milgram’s Reality-Virtuality
Continuum
In thisMobile sectionaof
the tutorial
we describe
factors
that
should be rtaken
account when
ugmented reality: A typical mobile augmented eality into
system developing Mixed Reality usability studies and promising areas of research where
usabilityvstudies
could be tconducted.
previous
presenters
haveover shown
there has
presents irtual elements o the user bAs
y othe
verlaying data and visuals a real been a wide range of user studies conducted in immersive Virtual Reality. However,
there have been
fewer experiments
presented
the wider
context
Mixed or
environment by employing either a handheld or winearable display device of
equipped Augmented Reality. It is in the areas outside of immersive VR on the Reality-Virtuality
continuum
thatand we pare
interested in, and tparticularly
in Augmented
Reality
interfaces. In
with a camera osition/orientation racking system. The two m
ost common many ways AR interfaces may have more near term application, so it is important that
rigorous users
studies
are conducted
in this
area.
augmented reality display technologies are head-­‐worn devices and smart phones. 2 Types
of User Studies
A HWD provides the most immersive augmented reality experience, since the In general user studies in Augmented and Mixed Reality can fall into one or more of the
following categories:
visual information is directly tied to the view of the user. The device can be as small x Perception: How do users perceive virtual information overlaid on the real world?
perceptualwcues
used to distinguish
between
real and virtual
content?
as a pair What
of sunglasses, hich can
are be
available from established electronic companies x Interaction: How do users interact with virtual information overlaid on the real
world?
real
world
objectscan be dused
to interact
withraugmented
content?
such as M
inolta How
and Scan
ony. HWD devices isplay augmented eality visuals in two x Collaboration: How can MR / AR interfaces be used to enhance face-to-face and
remote acollaboration?
ways: through mounted video display, or through an opaque AR overlay while the user can still see his surroundings through a transparent display device (Azuma 2001). Smart phones work along the same principles as HWDs but they are held instead of worn. Typically, the user points the device in the direction of an item of 8 interest, and the camera output will augment the display with additional information about the environment (Gotow, 2010). Due to the portable nature of smart phones, mobile augmented reality devices have become the most widely deployed consumer augmented reality display device and show promise for becoming the first commercial success for augmented reality technologies. Additionally, augmented reality graphical information can be presented through the use of wearable spatial-­‐display devices that project visuals on top of physical objects. At a TED conference in February 2009, Pattie Maes and Pranav Mistry presented SixthSense, a gestural AR interface that is worn around the neck. The device itself consists of a camera, projector, smart phone and mirror, which is able to project an image on any nearby surface such as a hand or a wall. The user also wears four colored caps on the fingers, which are used by the AR unit to detect hand gestures made by the wearer to interact with the projected images (TED, 2009). An augmented reality system must be able to recognize what users are looking at, acquire the appropriate virtual graphics, and register them in the correct position on the display device: 1. Recognition: Mobile AR applications are able to recognize what a user is looking at by triangulating its location by means of information provided by GPS, cell-­‐phone towers, or Wi-­‐Fi hotspots. Another approach used by augmented reality developers is to use physical markers, or ARTags, which can appear alone or appear on objects such as an advertisement in a 9 magazine. A mobile phone equipped with augmented reality software and a camera will recognize the marker and accurately overlay a 3D graphic image in the real world on screen. 2. Registration: In addition to recognizing what a user is looking at, an augmented reality device must be able to accurately align and register virtual objects over the existing ones in physical surroundings. Augmented reality display technologies are able to accomplish this through landmark recognition by utilizing images of the surrounding area stored on websites such as Google, Bing, and Flickr (Vaughan-­‐Nichols, 2009). The system is able to accurately pinpoint the exact position of the artifact through the camera’s level of focus and image-­‐recognition software (Saite, 1996). 3. Image acquisition: Once the augmented reality system accurately recognizes and registers a fixed location in the real world, it will pull in the correct data and images to overlay on screen from the Internet (Vaughan-­‐
Nichols, 2009). Uses: Mobile augmented Reality is used in a variety of disciplines ranging from military training programs and architectural surveys, to aiding surgeons during medical procedures or replacing engineering technical manuals in the manufacturing and repair sectors. Consumer-­‐based applications have recently emerged in the form of games, information browsers and immersive storytelling experiences. 10 Purpose of Study As augmented reality continues to mature as a communication medium for information design, there is a need to explore the advantages, potential uses and limitations of this immersive technology for smart phone users and content developers. This research will investigate (1) the perceptions held by first-­‐time mobile augmented reality users, (2) the opinions of digital content developers about potential applications of augmented reality in information design and visual storytelling. Another aim of this research is to establish a paradigm for effective information design when applied to mobile augmented reality. In this study, the researcher will explore users' impressions of this technology and how they interact with smart-­‐phone based augmented reality content. Additionally, this study will explore designers' and content developers’ ideas about how augmented reality can be used as a medium for information design and interactive storytelling experiences. The combination of user and designer perspectives, along with an evaluation of existing mobile applications will provide a comprehensive perspective on augmented reality design. This research will help us understand: •
Users' first impressions of mobile augmented reality applications. •
The desirability of augmented reality as a delivery model for interactive content. •
Perceived limitations of augmented reality design as seen by users and content developers. •
11 Potential applications of augmented reality when applied to digital storytelling and information design. Research Questions RQ1: What are mobile augmented reality users' impressions of the technology before and after using it for the first time? RQ2: What do users and content developers see as current advantages and limitations of mobile augmented reality? RQ3: Do current information design principles need to be revised in order to apply them to augmented reality experiences? Significance to the Field The power of augmented reality lies in its ability to place the world in context to the needs of the consumer. Combined with mobile devices, this technology can filter massive amounts of data in relation to the time and space on an individual user. The market has recognized mobile augmented reality’s potential for redefining information distribution and is banking heavily on its success. To that end, it is paramount that developers understand how users perceive augmented reality content and define the best practices for information design for this medium. Doing so will be the first-­‐step in ensuring mobile augmented reality’s commercial success
CHAPTER 2: REVIEW OF THE LITERATURE Consumer applications of mobile augmented reality are still in their infancy, yet the technology itself is decades old. Only recently has augmented reality gained popularity due to the convergence of smartphones, faster networks and cloud computing (Hurley, 2009). Yet many problems continue to plague mobile augmented reality as the field struggles to find its niche. Most research within the field is focused on developing better tools, while few researchers have looked at how users perceive this technology. This gap in empirical user research represents a need for understanding how to create a better user experience. Such knowledge would benefit developers as their struggle to move past mobile augmented reality’s initial “wow factor,” and avoid the stigma of being a victim of marketing hype (Bryne, 2010). This literature review will provide a brief overview of augmented reality advancements, including a survey of existing applications and development limitations. Additionally, the current state of user research and application evaluation methods will be addressed. Finally, the last section will discuss using prior research in the field of information design as a basis for evaluating augmented reality applications. 13 A brief history Although augmented reality is currently one of the hottest trends in mobile media, the technology itself is not new. The earliest example of rudimentary augmented reality can be traced back to the first heads-­‐up display employed by the British Royal Air Force during World War II. A radar screen superimposed on a pilot’s windshield would display information about nearby aircraft, including the identities of enemy targets (Vaughn-­‐Nichols, 2009). In 1968, Ivan Sutherland created the first modern example of an augmented reality system that fulfills the modern definition by Azuma and Milgram (see Chapter 1). Sutherland invented a head-­‐mounted display device capable of displaying wireframe 3D graphics, which moved as the wearer turned his head. By 1992, Tom Caudwell was the first to coin the term “augmented reality” when he designed a head-­‐mounted device for Boeing to assist engineers wire an airplane during the manufacturing process. His system displayed the aircraft’s schematics on the factory floor (Zhou, 2008). Hirokazu Kato and Mark Billinghurst introduced the first computer vision-­‐
tracking library, called ARToolKit (Kato & Billinghurst, 1999). Virtual imagery could now be overlaid in the real world through digital markers called ARTags that could be read by desktop computers equipped with cameras. When held in front of a camera, printed ARTags are recognized by the system and a 3D image is superimposed in its place on a computer screen. Today, advertisers and magazines commonly use ARTags as a way to bring interactivity to a printed product. 14 The December 2009 issue of Esquire magazine demonstrated how print and digital content could be merged through the employment of ARTags. After downloading augmented reality software from the Esquire website, readers could use their computer’s webcam to view the magazine’s digital features. This issue includes the cover featuring Robert Downey, Jr. sitting on an ARTag that displays a video promoting the film Sherlock Homes. More than half a dozen pages showcased augmented reality content, including two advertisements from Lexus (Esquire, 2009). Other marketers and publications have followed suit including Popular Mechanics and MINI. Recently, state of the art research in augmented reality has shifted toward mobile, hand-­‐held devices. Researchers are studying how portable devices can be used to enhance vision through AR tracking (Wagner, 2008), while industry developers are employing directional-­‐based augmented reality experiences through mobile phone applications. Early mobile AR during the 1990s relied on Head-­‐Worn Devices controlled by wearable “backpack systems” and later expanded to PDAs and cell phones (Azuma 2001, Schmalstieg et al., 2011). With the advent of powerful smartphones equipped with GPS systems, faster processing speeds and data transfer, and larger displays, today’s mobile augmented reality devices are capable of overlaying virtual information on a smartphone's camera output in real-­‐time. Combined with the popular iPhone and Android mobile platforms, mobile augmented reality has moved beyond its traditional military and industry applications and has begun to enter the hands of consumers. A variety of 15 applications have been introduced, such as: information browsers, educational tools, and games. Consumer mobile augmented reality applications Consumer-­‐based mobile augmented reality application development has seen tremendous growth over the past few years. According to Augmented Planet, as of June 2010 there were 396 applications listed in Apple’s iTunes store with descriptions listing augmented reality as a feature. 1 As of March 2011, there were 630 iPhone applications matching the same description3. The applications range from information services and navigational tools to games and entertainment. Table 2.1 Mobile Augmented Reality Categories Information Browsers Informational point of interest overlaid within a smartphone camera’s field of vision. Users can find local points of interest, restaurants, photos, and social media content. Social Networking Shares and displays geo-­‐tagged social networking content Tourism City guides and sightseeing applications Entertainment and Games 360-­‐degree shooters, virtual scavenger hunts, virtual pets and puzzle cheat overlays. Education Learning based applications such as satellite finders or constellation identifiers. Shopping Virtually try out products and or shop for merchandise by using image or bar code scanners Navigation Assist users with navigating to a fixed location such as a local restaurant or bus stop. Utilities Compasses, head up displays and overlays such as color identifiers, rulers or assembly instructions. Translation Dynamically translates text in images from a camera’s perspective into one’s native language. 1 Figure determined by an information search using the term “augmented reality” in iTunes App Store. 3 Updated figures based on search based on same terminology by the researcher on March 1, 2011 16 The most popular and largest application category to date is information browsers. Mobile augmented reality browsers display data points of interest within a mobile phone camera’s field of vision. They are used to find local points of interest such as landmarks, public transportation, restaurants or retail outlets. In 2009, SPRX mobile created Layar, the first mobile augmented reality browser. The application offers several options for overlaying digital information relating to brands, services and social networking. When a user searches for local restaurants or businesses, Layar will overlay navigational information directing the user toward the nearest location. There are over 1,500 “layers” to choose from including: Wikipedia entries, restaurant guides, games and social networking integration. Additionally, Layar is an open platform, which encourages other developers and content providers to create additional layers showcasing their information or services. Other competitors have surfaced in the browser category. Junaio, a general-­‐
purpose browser similar to Layar, is the first augmented reality application to employ LLA (latitude, longitude, altitude) markers. LAA markers are considered to be a more accurate location system when employed indoors, which tends to be problematic under GPS-­‐based systems (Junaio, 2011) due to the difficulty of indoor reception of GPS satellite signals. Other information browsers employ more targeted functions. These can include city guides, subway locators, or wikis. For example, Zagat To Go, based on 17 the popular Zagat Dining guide, directs diners to the nearest fine dining locations. Wikitude World Browser is a smart phone travel guide application that superimposes information about historical landmarks or other points of interest from Wikipedia. In addition to information-­‐based services, mobile augmented reality has become integrated into new forms of social networking, shopping and games. Integrating social networking has been a popular trend. It can appear either integrated into a larger application or as a separate mobile application. Social networking applications such as TwittAround allow augmented reality users to virtually geo-­‐tag their tweets, which can be visualized in real time on a mobile display for others to see. Shopping apps such as Red Laser by Occiptal can read labels and bar codes in order to compare prices and discover discounts and sale items. Mobile augmented reality is also delivering new gaming experiences. Smartphone gamers can participate in the virtual scavenger hunt game, Crimsonfox (Toto, 2010) or play a 360 degree shooter like Sky Siege, in which the user shoots down enemy aircraft flying around their living room or office. Development Concerns and Limitations Augmented Reality is far from the point at which consumers and developers universally accept it as a viable technology. While most people have seen examples of how augmented reality works in the news or other popular culture sources, few people have experienced AR first hand due to access barriers and limited adoption of AR-­‐ready smart phones. Significant advances have been made over the last 15 18 years in AR display devices, especially in mobile media. However, mobile augmented reality still faces significant hardware and software limitations. Additionally, developers must deal with a lack of open standards and a growing concern over user privacy and security. Tobias Höllerer, an associate professor of computer science at UC Santa Barbara, has said in a Wired.com interview: Augmented reality is stifled by limitations in software and hardware, he explained. Cell phones require superb battery life, computational power, cameras and tracking sensors. For software, augmented reality requires a much more sophisticated artificial intelligence and 3D modeling applications. And above all, this technology must become affordable to consumers. The best possible technology that is available today would nearly cost $100,000 for a solid augmented-­‐reality device. (Chen, 2009) Smart phones also provide limited screen space for displaying augmented information. Registration problems continue to plague augmented reality systems, since GPS is only accurate to within 30 feet and does not work well indoors (Metz, 2009). Registration errors are considered the most significant limitation of mobile augmented reality systems. They are often the result of tracking and alignment issues caused by sensor and/or camera calibration errors and network latency (Haller, Billinghurst & Thomas, p. 26). Registration errors are of particular concern in the scope of information design as misaligned visual representation of data leads 19 to inaccurate reporting of information. Furthermore, it hinders the viability of the application by diminishing user satisfaction. Interoperability and open standards will also be necessary to encourage content developers to adopt augmented reality. Currently, users are required to access content data through a specific application. For example, the only way to access Wikitude data is to use the Wikitude mobile application. Lack of cross-­‐
platform interoperability could result in “AR wars” similar to the web browser wars of the 1990s (Kirkpatrick, 2009). If augmented reality development is to become mainstream, standardized data formats, are necessary. Concurrently, application developers also lack easy to use authoring tools for creating mobile augmented reality content (Schmalstieg, et al., 2011). In addition to these technological limitations, augmented reality has several image and security concerns. Science-­‐fiction author and renowned augmented reality critic Bruce Sterling discussed the issues the augmented reality industry is facing in his keynote address, “At the Dawn of the Augmented Reality Industry.” According to Sterling, augmented reality is an exciting technology with tremendous promise, yet it still has a sleazy, gimmicky feel to it (2009). Sterling also warned that security and spam could be potential problems in the future as porn and pharmaceutical companies and hackers attempt to crowd the augmented view with unwanted spam. In addition, augmented reality adopters can potentially face information overload or dependence, similar to Internet addiction. 20 Moving forward, the mobile augmented reality industry must address the monetary and technical limitations of existing display systems if this technology is to achieve any significant market penetration. Furthermore, open development standards must be realized if the industry wishes to attract developers for creating new mobile augmented reality content. The current state of user research and evaluations The majority of published research in the field of augmented reality has focused on the development of new technologies and application prototypes. Few studies have addressed user experience and mobile application evaluation. Studies that have focused on users have been informal and under structured testing setups (Schmalstieg, et al.). According to a 2005 literature survey by Swann et al. (2005) reviewing all augmented reality literature published in leading journals and conferences, less than 8% had any reference to formal user evaluations. The HIT Lab NZ conducted a similar study in 2008 and found that only 29% of published augmented reality articles in major computer science journals included references to user research. As of 2011 according to scholarly reference searches conducted by this researcher, no published research studies address perceptions and experiences of consumers using mobile augmented reality applications. If mobile augmented reality is to become a viable medium for delivering visual information, empirical research must be conducted that seeks to discover how commercially available applications are perceived and employed. According to Schmalstieg, et al., if mobile augmented reality applications are to become 21 commonplace, attention must be devoted to user studies. There is a need for novel approaches to evaluating application interfaces and the consumers that use them. Research gaps in this area can be found in the evaluation of mobile applications that employ social networking and location-­‐based user collaborations (p. 33); These features are commonly employed by information-­‐ bowser application such as Layar. A potential solution Mobile augmented reality’s potential lies in its ability to make location-­‐based information contextually relevant to the perception of the individual user. The amount of available data is limitless. The Internet has brought worldwide access to data sources including news, information services, shopping and social networking. However, the problem lies in how the end-­‐user wades through the exorbitant amount of data. A way to make sense of this data from one’s personal point of view is needed. Paul Saffo, a technology forecaster and analyst states: “’Point of view’ is that quintessentially human solution to information overload, and intuitive process of reducing things to an essential relevant and manageable minimum… In a world of hyperabundant content, point of view will become the scarcest of resources.” (Baer, 18) The principles of information design, combined with the technological advantages offered by mobile augmented reality displays, can be a filtering lens that brings order to this chaos. Information design is a framework for clear communication of data. It is employed as an organization and presentation method 22 designed to bring meaning to unstructured data. Data alone is considered worthless without user context (Shedroff, 270). Through effective information design, one can create value by defining filtering constraints that communicate data in such a way that carries both meaning and purpose in context to the user's point of view (Dervin, 43; Shedroff, 270). The presentation of a tailored information scheme is important for mobile augmented reality since it relies on the placement of visual artifacts within the user's environment. Additionally, the aesthetics of information design are closely related to graphic design, where principles of visual communication are applied to the delivery of information. Foremost among information visualizers is Edward Tufte, the leading authority on the visual presentation of data (AIGA, 2004). Tufte is a pioneer in the field who established the general principles of information design, using visual language as a means to enhance the delivery and clarity of information (Horn, 20). In his three books, The Visual Display of Quantitative Information (1983), Envisioning Information (1990), and Visual Explanations (1997), Tufte describes his general principles of information design. These principles are a means for enabling users to understand and “envision” information. The foundational principle is avoidance of visual overload, a condition Tufte refers to as “chartjunk.” Tufte defines chartjunk as any frivolous visual elements not required for the comprehension of information, or distracting the viewer from said information (1983). This is significant for the design of mobile augmented reality applications. Designers must 23 take into account competing environmental elements within the users field of vision, which the designer has no control over. To counter this, designers and mobile application developers can look to Tufte’s information design principles: micro/macro design, layering and separation, small multiples, color and information, and the integration of words and images. Micro/Macro Design: Tufte describes Micro/Macro compositions as visualizations that contain large amounts of detail, yet within which patterns emerge. "Micro" refers to the smaller details that make up the greater whole (macro), thus creating structure (Tufte, 1990, p. 33). Tufte argues that macro/micro design is both a critical and valuable method for displaying information in ways that readers can understand the complexities of the “big picture,” yet also place the smaller details in context (1990, p. 50). Layering and Separation: By layering information, one can establish a visual hierarchy that emphasizes content important to the reader and deemphasizes what’s not. By creating salience through design, Tufte argues that the reader can easily navigate through a structured, organized construct for presenting information. The separation of data categories is equally as important as hierarchal presentation structures. According to Tufte, Designers should separate layers of information by means of weight, shape, size, value or color. Failure to do so results in “jumbled up, blurry, incoherent, [and] chaotic” designs (1990, p. 58). 24 Small Multiples: A small multiple is a single design element, repeated multiple times in order to show variations of a theme. It is a reliable method for allowing users to see changes and differences among similar objects (1990, p. 67). Through the principle of small multiples, Tufte argues one can create contrast in detail, yet maintain context without the need for changing the design (1983, p. 170). Designers often employ this method to create a sense of unity by using similar colors and typefaces, yet may use larger type for headlines or categories to visualize significance. Color and Information: The principle of applying color to information is a means to label categories, quantify intensity, imitate reality, and to enliven or beautify data (1990, p. 81). When using color, Tufte suggests that strong, darker tones should be used sparingly since they are both “loud” and “disruptive” and can distort one’s visual field, thus creating unpleasant effects. Secondly, using light and brighter colors next to each other can also cause visual distortions (1990, p. 83). Instead, Tufte suggests that designers use earth tones, and lighter shades of blue, yellow and grey (1990, p. 90). Integrating words and images: Words and images are the fundamental building blocks of information design. Designers employ this principle in order to present information in story form. Tufte argues that words and images should never be separated, as it would burden the reader with the task of having to cognitively associate the separate elements. Designers who effectively integrate text and images 25 can control the navigation and flow of the reader’s experience to facilitate understanding (1990, p. 116). Although these principles were originally conceptualized with print in mind, Tufte and information designers consider these principles to be “timeless” and applicable to any medium. Tufte argues his principles of information design “are universal-­‐like mathematics and are not tied to unique features of a particular language or culture” (1990, p. 10) Tufte’s influence has expanded into web design and other forms of information interfaces, including the iPhone (Tufte, 2008). This point has also been proven through academic research. Beverly Zimmerman of Brigham Young University conducted a study in 1997 to test if Tufte’s principles of information design could be applied to effective web design. The study used Tufte’s principles to evaluate the design effectiveness of two websites. Her results indicated Tufte’s principles are applicable to interactive media like the web, and can also serve as a measurement tool for evaluating the effectiveness of a web page in conveying information (Zimmerman, 1997). This research is relevant to mobile augmented reality since there is a need for effective measurement of the effectiveness of mobile augmented reality application designs. Edward Tufte’s principles could potentially fill this need. CHAPTER 3: RESEARCH METHODS Evaluating a new communication medium such as mobile augmented reality requires a multi-­‐dimensional qualitative method that takes into account both user and design perspectives, along with an evaluation of existing applications. This study implemented a qualitative thematic analysis and heuristic evaluations in order to answer the following: RQ1: What are mobile augmented reality users' impressions of the technology before and after using it for the first time? RQ2: What do users and content developers see as the advantages and limitations of mobile augmented reality? RQ3: Do information design principles need to be revised in order to apply them to augmented reality experiences? This study describes the initial perceptions of college students before and after they engaged with a mobile augmented reality application for the first time. Additionally, a small number of questionnaires were conducted to explore the problem from the perspective of augmented reality designers and content developers. Focus groups and questionnaires were used to collect data in the areas of usability and desirability as well as the advantages and disadvantages of the medium. The primary goal of this research is to evaluate the viability of mobile 27 augmented reality applications as a vehicle for information design. The narrative data was transcribed, coded and categorized into main themes related to RQ 1-­‐2. To answer RQ 3, the researcher conducted a heuristic evaluation of Junaio and Layar, the two most popular information-­‐based mobile augmented reality applications available at the time of this study. A heuristic evaluation is an effective method for identifying the strengths and limitations of an interactive system, based on accepted standards of practice. For this study, evaluative criteria were derived from established principles of information design. Methods overview Thematic analysis: This study relied on thematic analysis to identify user perceptions of mobile augmented reality application and the designers who create them. Braun and Clark (2006) define thematic analysis as “method for identifying, analyzing and reporting patterns (themes) within data” (p. 79). A themes, as defined by Boyatiz (1998), is: “...a pattern found in the information that at the minimum describes and organizes possible observations or at the maximum interprets aspects of the phenomenon. Themes can be generated inductively from raw data sets, or deducted through the use of theory and prior research.” (p. 161) A theme may be identified at the manifest level (directly observable in the information) or at the latent level (categorizing issues underlying the phenomenon) (p. 9). 28 The advantage to using thematic analysis is that it is a flexible methodology used to quickly highlight similarities and differences within a data set (Braun and Clark, p. 97). Thematic analysis is sound way to compare and contrast the strengths and weaknesses of mobile augmented reality when applied to information design from the perspective of users and designers. Furthermore, this method can be used to categorize problems found in usability tests, such as heuristic evaluations. Researchers often employ a thematic analysis when data focuses on experiences, practices or compliance-­‐oriented evaluations. Heuristic Evaluation: Heuristic Evaluation is a common tool for interaction design analysis . It is an easy and inexpensive way to evaluate the viability of a design, such as a mobile phone application (Unger & Chandler, 2009, p. 71). Rogers, et al. (2007) define heuristic evaluation as, “an approach to evaluation in which knowledge of typical users is applied, often guided by heuristics, to identify usability problems“ (p. 590). Heuristics are principles based on common-­‐sense knowledge, or rules of thumb and design usability guidelines. A key feature of this method is that does not require the presence of users and can be conducted by an expert reviewer (Nielsen & Tahir, 2002). For the purpose of this study, the heuristics used in the evaluation of smart phone applications are based on the principles of information design derived from the works of Edward Tufte. Described by The New York Times as “The Leonardo Da Vinci of data,” Tufte is considered the leading authority on the visual presentation of data. (AIGA, 2004) His principles are regarded as “universal” across all forms of 29 media. For this reason, Tufte’s guidelines are the ideal criterion for evaluating information design practices on mobile augmented reality platforms. Setting This study was conducted at Ball State University, a mid-­‐sized university located in Muncie, Indiana. Participants were selected from a population of 22,000 students. Additional field research was conducted in Indianapolis, for the purpose of heuristically evaluating mobile augmented reality applications usage within a metropolitan environment. Participants First-­‐time users: The researcher employed convenience sampling to collect subjects for Test Group 1. Selection of participants was restricted to undergraduate and graduate students currently enrolled at Ball State University. Participants were also selected because they currently owned an iPhone or android-­‐based smartphone and had no prior experience using mobile augmented reality applications. Six non-­‐
design students were chosen at random to participate. The participants in the study represented a variety of majors and departments within the University. Four students were male; two were female. Two of the four male students were graduate students while the rest of the participants were undergraduates. All student participants were in their early to mid-­‐twenties. Designers and content developers: Two participants were identified and selected to complete questionnaires by the researcher through Internet searches 30 and recommendations. Participants were selected based on their skills and experience in mobile augmented reality development, along with a visual communication background in one or more of the following areas: information design, visual storytelling, multimedia graphics, mobile platforms, and or art direction. Both participants have been recognized as leaders in their field, based on their publication and presentation records. Research Instruments Focus group: A focus group explored initial user perceptions of augmented reality, before and after using a mobile augmented reality application for the first time. The focus group met for two sessions, both lasting one and a half hours in length. Sessions were separated by a seven-­‐day period during which participants were asked to complete an assigned task with an AR application. The total time commitment for participants was five hours (see Appendix A). I.
Session One: Participants were presented two short videos introducing the concept of augmented reality and the Layar mobile application. Both of these videos were produced by Layar and demonstrate the application’s purpose, along with a preview of its information, social media and gaming features. Following the videos, subjects were asked to discuss their first impressions, how they might use the application, whether they found it desirable, and/or potential concerns. At the conclusion of the session, the participants were instructed to download the Layar application onto their personal smart 31 phone and to use it to complete an assigned task. The task was to try out and evaluate the Layar application for at least two hours over a seven-­‐day period. II.
Session Two: Students reconvened to discuss their experiences using the Layar application. Discussion probes covered usability, viability as a medium for information delivery, features, suggestions for future applications, problems and concerns, desirability, and willingness to use the application in the future. Questionnaires: Perceptions about mobile augmented design and its implications in communicating information were measured by questionnaires completed by two design professionals. Questions explored respondents opinions about how augmented reality can be used to advance information design and interactive storytelling, including its advantages and disadvantages relative to other communication mediums (see Appendix B). Heuristic Evaluations: An evaluation checklist was devised using judging protocols derived from the information design principles written by Edward Tufte in his books: The Visual Display of Quantitative Information (1983), Envisioning Information (1990) and Visual Explanations (1997). Problems and instances are noted for each applied principle on the basis of structure, presentation and dynamics in the application design (see Appendix C). Data Collection and Procedures Data was collected through moderated focus groups, surveys and heuristic evaluations: I.
32 Focus groups: Data was collected through interactive group discussion, moderated by the researcher. The focus groups were conducted under natural settings using a discussion guide protocol (see Appendix A). Discussions took place on campus at Ball State University, where all participants were currently registered students. The researcher guided the group’s discussion through predefined probes (see Appendix A) and asked follow-­‐up questions based on participant responses. The data collection process took place during two focus group sessions conducted over a two-­‐
week period, totaling three hours of discussion time. Each session was video-­‐
recorded for accuracy and lasted one-­‐and-­‐a-­‐half hours. II.
Surveys: An open-­‐ended questionnaire (see Appendix B) that consisted of unstructured and sentence completion type questions were sent to participating design experts, along with instructions for how to complete and submit the survey by e-­‐mail. Participants completed the questionnaires at home or work on their own time. III.
Heuristic Evaluation: The researcher evaluated two mobile augmented reality applications, Layar and Junaio by using an evaluation checklist based on heuristics derived from the following information design principles by Edward Tufte: micro/macro design, layering and separation, small multiples, effective use of color and integration of words and images. The evaluation was conducted by testing the mobile applications at Ball State University’s main campus and its surrounding neighborhoods in Muncie, Indiana. Every 33 function was tested twice for reliability, and all problems were documented for further analysis. Additional field research was also conducted in Indianapolis to provide contrast regarding how the applications preformed in a metropolitan environment versus rural area. Observations were recorded on the heuristics scorecard during a one-­‐week period, totaling six hours of evaluation time. Data Analysis The collected narrative data from the user focus groups and designer surveys were transcribed and categorized by thematic analysis. Subject responses were categorized into themes applicable to answering the research questions. The researcher used a coding method to organize collect data into themes that illuminated the key concepts of this study. Additionally, interview responses from designers were used to provide supplemental evidence in support of user responses The applications Layar and Junaio were analyzed in accordance with the information design principles defined on the heuristic evaluation scorecard (see Appendix C). A thematic analysis was applied to categorize the strengths and weaknesses of the applications in accordance with the design principles to establish a paradigm for information-­‐based augmented reality design. Emergent themes from the heuristic evaluations were cross-­‐referenced and validated by the narrative data sets. 34 Ethical Considerations As this study required the participation of human subjects, ethical issues were addressed to protect the privacy and safety of all participants. The most significant issues considered during this study were informed consent and confidentiality. Prior to participation, each subject was fully informed of all-­‐
important aspects of the study, including its aim and purpose. Furthermore, participants were advised of their right to withdraw from the study at any time. Subject confidentiality was ensured by not disclosing their names or personal information in the research. Only relevant data pertinent to answering the research questions were included. Conclusion This study employed a qualitative method to answer the three stated research questions. Data collected from focus groups, questionnaires and evaluations of existing augmented reality applications were thematically analyzed to gauge the viability of mobile augmented reality as a medium for visual information communication. CHAPTER 4: RESULTS This study employed a qualitative methodology to (1) discover and identify the initial perceptions of first time augmented reality users and (2) identify the advantages and limitations of mobile augmented reality systems from a design perspective. Focus groups were used to collect data in the areas of usability and desirability, along with the perceived advantages and disadvantages of using mobile augmented reality as a medium for information delivery. Additionally, questionnaires submitted by two professional designers with experience in the field provided supplemental insights into mobile augmented reality from a designer’s perspective. Finally, a heuristic evaluation was conducted to evaluate how existing mobile augmented reality applications conform to established information design principles. A thematic analysis revealed that mobile augmented reality is struggling to find its niche among consumers. Themes derived from participant responses were problematic technical limitations and a lack of compelling content and user experiences. Additionally, a heuristic evaluation of two augmented reality applications discovered limitations in how augmented reality content is displayed on mobile devices. 36 First Impressions Focus group discussions provided insight into how participants perceived the technology before using it for the first time. Prior to discussion, participants watched two short videos (totaling 10 minutes in length) that introduced the concepts of mobile augmented reality and the Layar application. Both videos demonstrated Layar and the social media, gaming and information-­‐delivery capabilities of mobile augmented reality. Following this brief overview, the researcher facilitated discussion by asking participants to share their initial thoughts and opinions based on the introductory videos. Additional questions explored how they might use the application, its desirability and/or potential concerns. Although the discussions were structured, participants were encouraged to speak freely about their thoughts and opinions. In addition to the group’s first impressions, two themes emerged from the patterns of discussion: perceived uses with an emphasis on information-­‐based services and a potential invasion of user privacy. First impressions: Initial judgments about mobile augmented reality were generally positive, with most participants viewing mobile augmented reality as potentially useful. The entire group was in agreement that this was an application they would be willing to try out for themselves. Participants described the application as a potential tool they could use for socializing with friends or as an immersive information guide they could employ when exploring a new city. Some 37 participants viewed mobile augmented reality as the next evolution of virtual spaces: “(Mobile Augmented Reality) Almost seems like a cross between second life and reality. You are meeting in a virtual space, but you are actually meeting in the same space. It's kind of like Second Life 2.0, the evolution of second life. Instead of trying and take everything that is reality and make it virtual, it creates the best of both worlds.” Perceived information-­‐based uses: When participants were asked to describe how they might use an augmented reality application such as Layar, each person described a function or use relating to information based services. The most common responses could be subdivided into information-­‐based searches, navigation, tourism and shopping. Social networking and gaming were also mentioned as potentially valuable reasons for using mobile augmented reality. However, all participants considered these to be secondary. Being able to point your mobile phone at an artifact within one’s own surroundings was of particular interest to these potential mobile augmented reality users. They were intrigued by the possibility of being able to use their mobile devices to discover additional information, such as a restaurant’s hours of operations or the historical significance of a local landmark. A common idea among all participants was to use this technology within a museum as a personal tour guide for learning more about artifacts on display: 38 “I would use AR for information purposes. Like if I was going to a museum, finding out about the artifacts that I'm looking at. If you put the phone over the art, it gives you information about the piece and it's history. I recently went over to the Air Force museum in Dayton over spring break. They have some of the stations have podcasts, but I think an augmented reality would be a better suited since it could feed more content.” Participants recognized tourism-­‐based information as one of the most potentially useful applications of mobile augmented reality. Everyone in the focus group saw this as a tool they would want to take with them on their next vacation or use to explore an unknown destination or city. Paired with the built in navigational features of a smart phone, several participants described mobile augmented reality as a “personal tour guide:” “I like the idea of having a device like this when I was traveling last year. I had never been to the place, and it would have been great to have a guided tour that continually updates live information like traffic flow, highlights in the city, that sort of thing.” This concept of using mobile augmented reality as a personal tour guide was also used as a metaphor for describing how this technology could be employed as a personal shopping guide: “If you tell the app what you want, and the app points out a route for you... so as you are walking through the store, telling you where the 39 items are located. A guided experience, like walking through IKEA... A virtual tour guide as you are looking for your products.” Being able to use a mobile phone as a shopping guide was conceived as a valuable tool for both consumers and marketers. Participants envisioned using mobile augmented reality for comparative shopping, virtually trying out merchandise and researching additional information about a product by simply pointing the cell phone’s camera at the item of interest. Many also recognized the value to marketers and companies who could use the application to attract customers by offering special deals to mobile augmented reality users. Other uses: In addition to information-­‐based services, several focus group members were also attracted to the notion of using mobile augmented reality for social networking and gaming. Students expressed how essential social networking tools such as Twitter and Facebook are to how they communicate with their peers. Participants conceptualized using mobile augmented reality as a way to virtually communicate and/or meet new people in their immediate surroundings. Participants suggested they would want to use this technology as a means for communicating anonymously with others on their daily commute to class or work. They saw this as an innovative way for connecting to new people in their communities. Gaming was also mentioned both as an extension of social networking and as a potentially attractive feature of mobile augmented reality. Younger students, who did not own a car and/or had limited spending money, saw mobile augmented 40 reality games as a cheap and entertaining way to socialize with friends. Other participants described mobile augmented reality gaming as a “virtual theme park” in which children could play within the confinements of their own backyards. Privacy Risks: The perceived risk of privacy when operating a mobile augmented reality application was of significant concern to many of the subjects. Participants feared that their actions and locations could potentially be tracked through location-­‐based technologies: “I have issues with privacy; I don't use any applications that are location-­‐based because I don't want people following me. Not that I'm afraid of a ‘Big Brother’ idea. But at the same time, I like having some anonymity of where I’m at. I don't think it’s really relevant that people know my whereabouts. With augmented reality, it's obvious that people would know where I or other people are located.” The majority of participants in this study made clear that they want direct control over what information may potentially be shared: “I do want control. All of the stories that have come out in the news about these privacy issues are causing me to trust these entities less and less. Privacy controls absolutely need to be involved. You need to be able to have the option to turn off your location, because if you don't you are going to get people who don't understand the risks involved.” 41 Although the majority of students had concerns about potential privacy risks, a few members of this study considered it a non-­‐issue. Those who were concerned however, also recognized that a minimum amount of personal information sharing would be necessary. Otherwise, the user experience could be significantly crippled. User Experiences and Feedback Following the group’s sharing of their first impressions about mobile augmented reality, the research subjects were instructed to try out the Layar application on their own and report back in one week to discuss their experience. Participants were asked to discuss what they liked and disliked about the application, any problems they incurred, suggestions for improvement, and their desire and willingness to use the application in the future. After having a chance to use the application for the first time, a majority of the participants reported that they had “mixed feelings” about their experiences. Although they liked the concept of mobile augmented reality, in practice the application left much to be desired. Participants pointed out design, contextual and technical limitations which hindered their experience. Additionally, several of the features they reported trying did not work as expected (or at all), while those that did were not considered to be intuitive: “I feel it’s not incredibly intuitive. It took a lot of poking around to figure out exactly what I was actually doing. I considered at many 42 points to go online to look up what I was doing and try to find a ‘How To’ guide.” Other students shared this same sentiment and suggested that an in-­‐
application help guide would be both helpful and necessary, since the types of interfaces used for mobile augmented reality applications are likely unfamiliar to most new users. “A tutorial would be good, just like a lot of mobile phone games have a tutorial option. The same thing would work here. This is what you can do.... even if your first screen options are asking what do you want to do, and list specific options ... you tap that.... then it guides you through that process.” Understanding how to use the application was not the only concern. By far, user interface and information design problems were the most significant complaints among all participating users. Analysis of the responses revealed that information overload and a lack of hierarchy between displayed elements caused the greatest hindrance to usability as a result of the application’s design. Other notable factors included technological constraints, a lack of content in rural areas, and a general lack of compelling content. Design concerns: Participants were dismayed by how the application displayed augmented reality content. Although they found the aesthetics of the interface to be visually pleasing, they also found it difficult to differentiate between the multiple data points displayed on screen. The larger the pool of content 43 available within the user’s vicinity, the harder it was to navigate through the information displayed on screen. Content was continually overlapping, making it nearly impossible to see or access the information that was covered up. Most of the subjects described the situation as “overwhelming,” especially within larger metropolitan areas. As one student describes: “I went downtown to Indianapolis circle and the application was throwing hits from every store in the mall. The screen on my phone was overloaded and I wasn’t even in the mall!” Even limiting the distance parameters within the application did not improve the experience. Another subject explained: “I don't know what's the best way to go about solving that issue, I did mess with the parameters a bit, it helped some, but at the same time on the lowest area in a heavy populated area, you are going to get overlap.” Participants also suggested they would like to see options for hiding content, such as a particular user’s tweets or a retail location tag they are not interested in. They said this might serve as a means for combating the application’s display issues. Additionally, many participants considered that it may be valuable to add “like” and “dislike” buttons as a way for the application to learn what types of content a user might be interested in and filter out any irrelevant data. Technological constraints: Most subjects reported that the quality of the application’s performance on their smartphones was less than adequate. Several 44 issues were raised, including slower processing speeds when running the application and detrimental effects on battery life. Several subjects described their experiences as “glitchy” and often reported the incorrect location of data. A subject describes his experience: “My phone wouldn't calibrate proper north and south. I tried it out a couple different times and it would tell me that Thai Smile was ‘that way’, while I'm standing right there staring at it. I don't know, I really hate things that are glitchy, so I'm really hesitant to use it again.” Subjects suggested that a “lite” version of the Layar application should be released. They said they are willing to accept fewer features in return for better hardware performance and battery life. The participants were also in agreement that the screen size of a typical smartphone is inadequate for an application of this nature. One of the subjects commented, “I think the screen size is too small to cram so much information into it.” Others suggested that the larger screens employed by tablet devices such as the iPad would be a more suitable medium for displaying augmented reality content. The design respondents shared this same sentiment. Smaller screens, in combination with limited tracking systems and a lack of computing power, diminish a designer’s ability to create immersive, three-­‐dimensional experiences. Instead, designers are limited to developing two-­‐dimensional overlays in a three-­‐dimensional environment, which designers and end-­‐users find to be limiting. Display technologies must evolve to the point that a 45 seamless presentation of augmented reality content becomes possible within a physical space. As one designer stated, “3D content is really what works best for an augmented reality application.” Furthermore, both designers argued that today’s display technologies offer more of a “forced” experience, rather than an immersive one. The user is limited by what they see and is forced to experience augmented reality through the perspective of a hand-­‐held device, rather than through the user’s own eyes. As one designer explained: “When you are looking around through your device, you are always being mediated through something. You are always looking through the screen. The point of focus is always the device. You're not really experience augmented reality, you are experiencing a trick on your mobile phone.” In one designer’s opinion, head-­‐worn display devices will eventually mature to the point that they will usurp smartphones and the leading display device for mobile augmented reality content. A head-­‐worn display device would be able to provide a seamless, immersive viewing experience and open the door to new forms of interactivity based on physical movement, rather than a touchscreen interface. Lack of content in rural areas: In contrast to the “information overload” concerns many of the participants had, several subjects felt that the amount of content available within smaller communities was significantly lacking. Although 46 most of the subjects had the opportunity to try the application in larger metropolitan cities, nearly all of them reside in a smaller mid-­‐western city with approximately 70,000 citizens.4 Subjects argued they would be less likely to regularly use the application in their own neighborhoods if local content was limited. Lack of compelling content: At the conclusion of the study, participants were asked if they intended to use Layar in the future. Each stated they would either seldom use the application, or not at all. Despite its glitches, the students argued that the application itself offers little compelling content to warrant repeated use. Rather than focusing on static content such as location-­‐based information, the subjects said they would be more interested in seeing non-­‐static content such as local news, events such as concerts and sports, or current specials run by local business. Likewise, the design respondents agreed that existing applications have little to offer. They blame this lack of quality content on an existing top-­‐
down application development cycle where corporate objectives overshadow the need for a compelling user experience. In their opinion, most mobile applications are being employed as marketing tools, which can be seen as overhyped and gimmicky. A culture driven by content developers is necessary to support the maturation of this new form of interaction and 4 Population count as reported by 2010 census. 47 presentation. In the meantime, as one designer laments: “Augmented reality is going to be a really gimmicky field for a very long time.” Heuristic Evaluations In order to validate the design limitations raised by the focus group, this study employed a heuristic evaluation process to appraise mobile augmented reality applications on their ability to effectively present information. The evaluation was conducted by the researcher using information design standards as evaluative criteria. A set of heuristics was devised from Edward Tufte’s principles of information design to appraise the mobile applications Layar and Junaio on the basis of structure, presentation and dynamics in the applications’ information design (Appendix C). For this study, five heuristics were used to examine how Layar and Junaio present augmented reality content within a smartphone camera’s view. The research compared how each application displayed Twitter and Wikipedia content. The purpose of this evaluation was to identify major information design flaws within Layar and Juniao through the application of five heuristics: 1. Presents individual data points within a framework relating to a larger context; 2. Effectively layers and separates information to emphasize hierarchy and structure; 3. Employs small multiples as a means of comparing differences between related data; 48 4. Effectively uses color for information display purposes; 5. Integrates words and images effectively. Although usability is an important component of interactive information design, a usability evaluation is beyond the scope of this study. Usability studies are standard practice for any interaction design project prior to deployment. Instead, this study examines the unique problems mobile augmented reality faces when visually presenting information. After the evaluation was complete, 10 problem areas were identified as violations of established information design principles and rated according to severity: 1 — Design problem that occurs infrequently and/or with minor user hindrance. 2 — Moderate design problem that occurs occasionally and may result in readability issues. 3 — Severe design flaws that occurs frequently and damages the readability of data. Issue 1: Oversized icons cover up the background (camera view) and other data points. # Problem Oversized icons cover up the background (camera 1 view) and other data points. Severity Rating 3 Application: Layar Heuristic Number Broad Heuristic 1, 2 Micro design elements should not overshadow the main view. 49 Problem: Although this problem does not occur within the Twitter and Wikipedia components in the application, it is worth mentioning since it severely affects other layers within the application such as Livespot (See image 4.1). By covering up the entire screen, the application violates the micro/macro design principle. The minor detail (restaurant icon) overshadows the macro view by failing to display its location in context to the user’s visual perception of the environment as seen through the smartphone’s camera. It is also problematic when content completely covers and eliminates access to other points of interest; this is a violation of Tufte’s layering and separation principle. Image 4.1: Livespot on Layar Image 4.2: Layar’s “Bird’s Eye” view Image 4.3: Reality view on Layar Image 4.4: Twitter on Layar 50 Issue 2: Oversized icons cover up the background (camera view) and other data points. Severity Heuristic # Problem Broad Heuristic Rating Number “Bird’s eye” view feature Data should be appropriately 2 results in data point 2 2 layered in proximity to the overlap and diminishes user’s location location-­‐based context. Application: Layar Problem: Layar offers two options for viewing augmented reality data in camera mode: “Reality” view and “Bird’s Eye” view. The reality view (Image 4.3) employs the layering and separation principle by presenting data points in a visual hierarchy according to its geographical proximity to the user. Closer points of interest appear closer in the foreground, while distant location points are placed further back. Additionally, the subtle, yet noticeable changes in dot sizes are an effective execution of the small multiples rule. The size differences between the dots represent their relative distance to the user, thus maintaining an accurate visual perspective. In contrast, the “bird’s eye” view (Image 4.2) fails to layer information points appropriately in addition to stripping much of its location-­‐based context in relation to the user’s position. Furthermore, large data sets, such as the one depicted in image 4.1, can result in unnecessary data point overlapping. This can lead to usability issues since information can be missed or difficult to access. 51 Issue 3: Difficult to recognize which tweet is currently selected. Severity Heuristic Problem Broad Heuristic # Rating Number Difficult to recognize Selected objects should be 3 which tweet is currently 2 2, 3 highlighted. Icons should be selected simple, and consistent. Application: Layar Problem: As seen in image 4.4, Layar uses Twitter avatars to represent geographically tagged tweets within the camera view. While this method offers the benefit of placing tweets in context with its creator, it does create confusion for the user. The only way to determine which tweet is currently being viewed is the small arrow connected to the corresponding dialog box and the identical avatar image located within. This is the result of a lack of separation between other displayed avatar images. A better approach would be to model it after the Wikipedia layer (Image 4.3), which represents Wikipedia entries as consistent visual icons, while separating selected data points by a change in color. Issue 4: A visual disconnect exists between data points and their related text and/or images. Severity Heuristic # Problem Broad Heuristic Rating Number A visual disconnect exists Data points should have 4 between data points and 2 5 strong visual associations with their related text and/or related images and text. images 52 Application: Layar Problem: Layar loosely presents text and images relating to a point of interest by means of a static dialog box located always at the bottom of the screen. Content appearing inside the box is dependent on which data point is currently being viewed. The application employs a small arrow that points at the highlighted dot to create a visual association. (images 4.3, 4.4) Although this presentation form loosely adheres to the integrating words and images principle, it does create an unnecessary burden for the user who is tasked to associate related content by means of a subtle visual clue. This problem is particularly troublesome when it occurs in conjunction with issue 3. Image 4.5: Layar (4.0) dialog box Image 4.6: Layar (3.0) dialog box Issue 5: Large text boxes take up a considerable amount of screen real estate. # Problem Large text boxes take up 5 a considerable amount of screen real estate. Severity Rating 2 Heuristic Number Broad Heuristic 1, 2 The display of secondary content should not overpower the framework of the bigger scene. 53 Application: Layar Problem: Dialog boxes in Layar (version 4.0) take up one-­‐third of the available screen real estate, thus diminishing the environmental view. Tufte’s layering and separation principle suggests this content requires additional visual emphasis since it takes priority over other content on screen. However, this execution unnecessarily obscures the user’s view by covering up other data points and their surroundings. Furthermore, the information presented with the dialog box is poorly laid out, resulting in a waste of precious space on the small screen. Earlier versions of Layar were more conservative in the dialog box design. Version 3.0 dialog box designs (Image 4.6) presented only basic identification information in one-­‐sixth of the available screen space. If a user wanted more information, she could open an expanded dialog box that overlays on top of the camera view. This display option appropriates layered content to establish visual dominance by dimming the background. Issue 6: White text on bright colored backgrounds difficult to read. # Problem White text on bright 6 colored backgrounds difficult to read. Severity Rating Heuristic Number 1 4 Application: Layar Broad Heuristic Poor color selection and lack of contrast. 54 Problem: Brightly colored backgrounds can make white type illegible. This is not a consistent problem within the Layar application, since each layer employs different visual styles. However, when it does occur, the small type size, combined with a lack of contrast with the background, makes the text especially hard to read. Tufte suggests that designers avoid brightly colored backgrounds for this very reason, since it causes too much visual distortion. Instead, designers should use earth tones such as lighter colors of blue, yellow and gray with black type. Issue 7: User-­‐interface controls do not orient correctly when viewing in landscape mode. Severity Heuristic # Problem Broad Heuristic Rating Number User-­‐interface controls User interface should be 7 do not orient correctly 3 3 consistent throughout the when viewing in application. landscape mode. Application: Junaio Problem: When switching from portrait to landscape mode, the navigational interface elements do not reorient to conform to the new view (image 4.7). This is more of a usability issue, but it also presents a flaw in the information design as it hinders the readability of presented information. The small multiples principle suggests that repeated navigational devices such as links, arrows, or icons should maintain a consistent design. Failing to orient navigational elements correctly is both visually unappealing and unnecessarily burdens the user. 55 Image 4.7: Junaio Issue 8: User-­‐interface controls do not orient correctly when viewing in landscape mode. Severity Heuristic # Problem Broad Heuristic Rating Number Expanded dialog boxes Text presented sideways is 8 are presented sideways 3 5 when using application difficult to read. in l
andscape m
ode. Application: Junaio Problem: When operating the application in landscape mode, expanded dialog boxes display text sideways. This problem is the result of the same design flaw as issue 7. In this instance, the poor alignment of text creates significant readability 56 issues, thus breaking the principle of integrating words effectively. Large amounts of copy should never be oriented on its side, as it makes the text difficult to read. Issue 9: Radar size is inadequate for presenting surrounding information in 360 degrees. # Problem Radar size is inadequate for presenting 9 surrounding information in 360 degrees. Severity Rating Heuristic Number Broad Heuristic 2 3 Difficult to separate content effectively on a smaller scale. Application: Junaio Problem: The radar graphic (see image 4.7) used to display nearby points of interest within a 360-­‐degree radius of the user is too small. There is not enough separation between elements to adequately gauge the distance between them. Layar’s execution of the radar navigational element would be more appropriately size (image 4.4). Issue 10: Text displayed within individual data points is difficult to read. # Problem Text displayed within 10 individual data points is difficult to read. Severity Heuristic Rating Number 1 4 Application: Junaio Broad Heuristic Reversing type out of dark backgrounds requires strong color contrast and larger type sizes. 57 Problem: In contrast to Layar, the Junaio application employs a tighter integration of words and images using the dialog boxes themselves as geographically pinned data. This execution offers many advantages over Layar since it does not unnecessarily burden the reader with having to visually associate separate components. However, the execution could use refinement of the color of the text. White text, instead of gray over the dark translucent background would provide greater contrast, thereby making the text easier to read. Conclusion According to the respondents, mobile augmented reality is a compelling medium for displaying visual information. Participants in this study see mobile augmented reality as an immersive tool for engaging everyday information in a more personal context. However in its present form, mobile augmented reality is lacking in functionality and desirability. User studies and design evaluations suggest that mobile augmented reality is limited by poor information display factors and technological limitations of mobile devices. Furthermore, there is a lack of compelling content and an overhyped push by corporations and marketers. Despite these limitations, the participants in this study agreed that mobile augmented reality has potential. However, it needs time to mature. As one student concluded, “I see potential in this, but it's just like anything else, it's just going to take time before it finds its niche.”
CHAPTER 5: DISCUSSION This thesis postulated that a usability and design analysis is a feasible and useful method for comprehending the value of mobile augmented reality as a vehicle for the visual communication of personally relevant information. To ascertain this theory, the following questions were posed and answered by means of user studies, questionnaires and application evaluations. RQ1: What are mobile augmented reality users' impressions of the technology before and after using it for the first time? Users welcome the idea of mobile augmented reality with open arms. They recognize its potential for revolutionizing social media and information services by connecting with content on a more personal level. In their opinions, mobile augmented reality applications could be employed as immersive tour guides for exploring one’s community or to interact with neighbors whom they are unlikely to meet by conventional means in a face-­‐to-­‐face reality. Additionally, these same users perceive mobile augmented as a new tool for marketers and companies to employ as immersive advertising and shopping experiences. Commercial applications of mobile augmented reality could offer 59 guided tours through shops, comparative pricing tools or bargains tailored specifically for mobile users. These same users were less optimistic after using a mobile augmented reality application for the first time. Participants reported that their experience was hindered by design, contextual and technical limitations. Oftentimes the test application, Layar, would perform erratically and failed to operate as expected. Subjects frequently complained about information overload due to the small size of mobile phones. Additionally, users were dismayed by a general lack of compelling content. Participants were more interested in seeing non-­‐static content such as local news and social media, rather than commercially based information. Although mobile augmented reality users are open to using such applications, they are weary of the privacy risks that come with sharing their personal information, particularly their current locations. Prior to using any mobile augmented reality service, users want direct control over what information they are sharing and or receiving from content providers. RQ2: What do users and content developers see as current advantages and limitations of mobile augmented reality? Mobile augmented reality currently offers little value for smartphone users and content designers. Technical limitations and a lack of compelling features are the leading factors for a poor user experience. Consumers frequently complained that the test application Layar was plagued by slow operating speeds, data 60 inaccuracies and detrimental effects on battery life. Additionally, smaller screens were found to be inadequate for displaying augmented reality content. Smaller screens forcibly burden the user by requiring the use of a hand held device as a point of visual perception when displaying augmented reality content. Furthermore, using the application within heavily populated areas may result in information overload for the user since there are limits to how much information can be efficiently displayed on a smaller screen at one time. These same technical limitations have constrained the potential for developing content by designers as well. Mobile phone technologies have not yet advanced to the point in which they can accurately and efficiently display three-­‐
dimensional visualizations. Instead, designers are limited to creating mobile applications that provide geographically tagged, augmented reality content within a two-­‐dimensional overlay on a camera screen. Designers consider these limitations to be frustrating and as a result, are finding that mobile augmented reality has yet to live up to its expectations. Current applications are being viewed as limited and boring, while consumers are left wanting. Instead of geographically tagging information one could easily find in the yellow pages, consumers are more interested in seeing dynamic content such as local news and events or daily specials currently offered by local business. Mobile augmented reality does have a potential advantage over other communication technologies in that it offers an immersive experience by merging the physical and virtual worlds. However, this advantage has yet to be realized. Both 61 designers and consumers foresee mobile augmented reality having a major influence on how we play games or access information in the future. But due its limitations, it remains an underdeveloped technology. RQ3: Do current information design principles need to be revised in order to apply them to augmented reality experiences? A heuristic evaluation of two mobile augmented reality applications revealed the traditional principles of information design remain applicable for augmented reality content displayed on hand held devices. Furthermore, these principles can also serve as an effective benchmark for measuring the viability of mobile augmented reality as a medium for presenting visual information from a user’s geographical context. Using Edward Tufte’s principles of information design for print and multimedia as judging criteria, several presentation issues were noted ranging from poor color usage to a failure to properly separate and layer visual information. Furthermore, the identified limitations were validated by prior focus group responses pertaining to the same issues. Many of these problems are a result of limited screen space offered by the mobile device. To rectify this, a conservative approach is required where only the minimum amount of detail is presented without loosing context. Tufte refers to this concept as the “Data-­‐Ink ratio,” where the largest use of ink (or pixels) should present data-­‐information. Everything else is expendable (Tufte, 1983). Oversized icons and wasted space in dialog boxes take up 62 valuable screen real estate, which often conceals other sets of information. This is especially problematic in larger populated areas where the amount of augmented reality content available outpaces the visual limitations of the small screen. The resulting effect is information overload for the user. It is evident applications designers have attempted to make the best presentation decisions possible within the technical limitations. By following the suggestions mentioned above, designers can improve the presentation of information and create a better experience for the user. However, this will not solve the underlying problem presented by all mobile augmented reality applications: The ineffective use of a two-­‐dimensional content overlay within a three-­‐dimensional space. Augmented reality content is often displayed within a location-­‐based context, and current mobile applications are failing in this regard. Simply overlaying a dot on screen to signify a location provides no context to distance or relative proximity to any nearby points of interest. It merely provides a compass direction. This may work if there are only one or two information points displayed, but when there are several present, the metaphor is less effective. Tufte’s micro/macro design principle states that the smaller details must be visualized in context to the greater whole. When applied to augmented reality, it’s not enough to place individual location markers in context with other data points, since it shares a direct relationship with its environment as well. 63 The user’s environment displayed on screen is visual information that also needs to be taken into account when it’s augmented with supplemental information. The principle of layering and separation states that a visual hierarchy should be established to emphasize which content is most important to the reader, while deemphasizing what is not. For the user, this means distinguishing which content is in close proximity in relation to physical objects in their surroundings. The optimum way to accomplish this is to spatially display data points within three dimensions. However, this option is not yet feasible due to the technical limitations of modern mobile technology. Guidelines for Future Mobile Augmented Reality Applications While this study has highlighted the shortcomings of mobile augmented reality, it also recognized potential areas for improvement. Based on user feedback and heuristic design evaluations of current applications, the following guidelines would serve to improve the design and user experience of future applications: Applications should implement navigational controls that are both intuitive and easy to use. User controls should be easy to navigate with minimum instruction or prompting. Users find multiple viewing options confusing and would prefer a single modular interface. Additionally, users would also benefit from a simple visual cue that highlights the central viewpoint on their mobile devices. By employing a small box or cross hairs in the middle of the screen, users would better gauge the geographical location of mobile augmented reality content. This implementation 64 would effectively work in concert with the existing practice of highlighting selected data points for separation purposes. Augmented content should be spatially oriented in 3D. Mobile display technologies will eventually mature to the point that they can accurately display augmented reality content in 3D. When this occurs, developers should move beyond the magic mirror approach and seek to seamlessly place points of interest within the users environment. Such implementations would offer significant advantages over traditional 2D overlays since it presents an accurate representation of geophysical locations. Augmented reality content should be filtered on the basis of user interests and proximity. Information overload is a significant usability hindrance of existing applications due to the limited screen space afforded by mobile phones. Developers must understand these limitations and design around them. Existing applications are currently addressing this issue by filtering content based on user proximity. Filtering content based on user interests could strengthen this system, thus reducing the risk of information overload. Participants in this study expressed their desire for a filtering features that hiding uninteresting or non-­‐applicable information. A progressive approach would be to devise an educated filter that learns by means of users marking content as “liked” or “disliked.” Applications should emphasize social bonding and sharing of information to create compelling user experiences. Social media has taken a central role in how people communicate today. Applications like Facebook and Twitter have 65 strengthened social bonds while offering new ways of connecting to new people. The participants of this study have noted that social media is an important part of their lives and expect mobile augmented reality applications to take advantage of this. Furthermore, these same users are also interested in seeing local content that is both dynamic and useful such as restaurant specials or community events. If this content is both useful and engaging, users are more likely to share it with friends, thus expanding an application’s active user base. Information should be interoperable between all applications. If augmented reality is to become a successful medium for information delivery, there needs to be open standards for sharing content among all applications. This current lack of interoperability burdens content developers, as they must reformat information for multiple augmented reality applications. It’s especially problematic when formatting dynamic content that must be updated on a regular basis. Consumers would also benefit from standardization. In the future, users will be the driving force in generating new augmented reality content by means of geographically tagging social media and personally relevant content. These users will be able to reach a wider audience if tagged information can be shared between multiple applications. Limitations Although this research helps to explain how users and designers perceive mobile augmented reality, there were several limitations. The first was related to the participant sample for the user focus groups. Participants in this study were pooled from a smaller audience (college students) rather than addressing all 66 demographics representing smart phone users. A second limitation was that while student participants tested the Layar application in both rural and urban settings, the primary testing location was in a smaller community. Since the analysis of this study suggests that population density has a direct effect on the user experience, a second sample derived from residents in a metropolitan area would have been beneficial. Additional limitations were related to the implementation of the heuristic evaluation of the Layar and Junaio applications. Ideally, multiple expert reviewers are employed to access and document the limitations of an application when applying the heuristic evaluation method. Due to time and financial constraints, the evaluation was limited to the findings and opinions of a single researcher. Although the final analysis has merit, multiple evaluators could have further validated the results. Furthermore, Edward Tufte’s information design principles required modification in order to apply them to mobile augmented reality. While Tufte’s principles are applicable to this medium, they were originally intended for print or static mediums. In his original works, Tufte never addressed interactivity or spatial design within a 3D environment. While Tufte’s guideline for presenting information within a 3D projection presented in 2D proved applicable to augmented reality, it’s meaning had to be inferred when applied to spatial environments. 67 Recommendations for Future Research As mobile augmented reality is relatively young, there are many avenues for future research in this area. New applications are introduced on a weekly basis and mobile technologies continue to improve every year. Future studies would benefit by examining how users perceive and interact with other forms of mobile augmented reality applications besides information browsers. Applications in social media, gaming and navigation are worthy of a closer look. Furthermore, as augmented reality applications are refitted for tablet computers, it would be valuable to reassess the limitations found in this study’s analysis in context to mobile phones to see if they still apply. Finally, it would be advantageous to expand the designer’s perspective component into a full research study. The initial research in this area suggests that designers have a central role in defining the direction mobile augmented reality takes. Their thoughts and opinions would be valuable to future studies exploring mobile augmented reality content. Conclusion Mobile augmented reality is on the cusp of revolutionizing how people interact with digital information in their daily lives. Content will no longer be tied to a specific medium such as print or the web; rather it will seamlessly blend with one’s surroundings. This disruptive technology will soon redefine how consumers engage with news, entertainment, and other forms of data on a daily basis. In all 68 respects, mobile augmented reality may one day become the ultimate immersive user experience. Nevertheless, mobile augmented reality requires time to mature. Just as every disruptive technology that has come before, mobile augmented reality must address its initial flaws and limitations if it’s to become a commercial success. While augmented reality has demonstrated that it can be an innovative medium for displaying information, it’s mobile counterpart has yet to create a sense of “need” for consumers. Further advancements in content development are necessary in order to generate universal acceptance and demand and to shed the “gimmicky” perceptions that users and content designers have about this technology. Additionally, mobile technologies and applications have not advanced to the point in which they can deliver on augmented reality’s promise of immersive information delivery. Despite the hype generated by marketers and corporations, current hand-­‐held devices are incapable of displaying pure augmented reality experiences. By definition, augmented reality systems must be able to register and align virtual and physical objects in relation to each other within a 3D space. Current mobile offerings such as Layar or Junaio fail to meet this criterion, as they instead rely on visual trickery by employing a two-­‐dimensional overlay on top of a smartphone camera-­‐viewfinder. To that end, consumers are left with a “magic mirror,” which plasters virtual content over their small screens rather than fully integrating within the user’s surroundings. 69 Although mobile augmented reality has been limited by technical constraints, the recent arrival of faster processers and larger displays will have a profound impact on the depth and viability of future applications. Tablet computers such as the iPad are beginning to redefine how developers approach augmented reality content. These devices are equipped with powerful 3D graphics hardware and larger screens, which move beyond the limitations of mobile phone displays. However, tablet and mobile devices are only the next evolution, not the final step in mobile augmented reality’s development. Most designers and users find using a hand-­‐held device to be cumbersome since it restrains their visual perception to the device’s limited perspective. Eventually head-­‐worn displays such as augmented reality glasses will become the standard. Users will gain unparalleled access to content from a first-­‐person viewpoint, thus eliminating the need for employing hand-­‐held displays as windows for viewing augmented reality content. Furthermore, augmented reality systems will expand to allow users to interact with virtual content in the same way they would use physical objects by means of gestural interfaces. Once mobile augmented reality moves beyond its limitations, the potential benefits for information delivery and entertainment are enormous. Mobile augmented reality could be used to hyper-­‐localize content such as dining reviews or present statistical overlays during live sporting events. Additionally, the immersive nature of augmented reality could also be used to pull gamers and readers into stories by interacting with virtual characters in their own space in real time. The 70 opportunities for content designers and users alike are potentially limitless. Unlike overhyped technologies of the past, mobile augmented reality has the greatest potential for redefining how consumers view the world by merging the physical and virtual into a single reality. REFERENCES 71 AIGA (2004). 2004 AIGA Medaliists -­‐ Edward Tufte. Retrieved January 11, 2011, from AIGA web site: http://www.aiga.org/content.cfm/medalist-­‐
edwardtufte Aukstakalnis, S. & Blatner, D. (1992). Silicon Mirage -­‐ The Art and Science of Virtual Reality. Berkeley, CA: Peachpit Press. Azuma, R. (1997, August). A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 355-­‐385. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B. (2001). Recent Advances in Augmented Reality. Computers & Graphics, November 2001. Baer, K. (2009). Information Design Workbook: Graphic Approaches, Solutions, and Inspirations + 30 Case Studies. Beverly, MA: Rockport Publishers. Boyatzis, R. (1998). Transforming Qualitative Information. Thousand Oaks, CA: SAGE Publications. Bryne, C. (2010, December 22). Most augmented reality companies not doing augmented reality? Retrieved January 13, 2011, from VentureBeat: http://venturebeat.com/2010/12/22/forrester-­‐most-­‐augmented-­‐reality-­‐
companies-­‐not-­‐doing-­‐augmented-­‐reality/ Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 77-­‐101. Chen, B. (2009, August 25). If You’re Not Seeing Data, You’re Not Seeing. Retrieved April 17, 2010, from Wired: http://www.wired.com/gadgetlab/2009/08/augmented-­‐reality/ Dervin, B. (1999). Chaos, Order and Sense-­‐Making: A Proposed Theory for Information Design. In R. Jacobson, Information Design (pp. 35-­‐57). Cambridge: MIT Press. Dünser, A., Grasset, R. & Billinghurst M. (2008). A Survey of Evaluation Techniques Used in Augmented Reality Studies. HIT Lab NZ Technical Report. 72 Entner, R. (2010, June 30). Quantifying the Mobile Data Tsunami and its Implications. Retrieved July 19, 2010, from Nielsen: http://blog.nielsen.com/nielsenwire/online_mobile/quantifying-­‐the-­‐mobile-­‐
data-­‐tsunami-­‐and-­‐its-­‐implications/ (2009, December). Esquire Magazine . Gotow, J., Zienkiewicz, K., White, J., Schmidt, D. (2010). Addressing Challenges in Delivering Augmented Reality Applications to Smartphones. Third International ICST Conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications. Chicago: Mobileware. Haller, M., Billinghurst, M., & Thomas, B. (2006). Emerging Technologies of Augmented Reality: Interfaces and Design. Hershey, PA: Idea Group Publishing. Horn, R. (1999). Information Design: Emergence of a New Profession. In R. Jacobson, Information Design (pp. 15-­‐33). Cambridge: MIT Press. Hurley, W. (2009, November 3). Augmented Reality: Getting Beyond the Hype. (Bloomberg) Retrieved January 30, 2011, from Businessweek: http://www.businessweek.com/technology/content/nov2009/tc2009112_3
53477.htm Husson, T. (2010, December 20). Mobile Augmented Reality: Beyond The Hype, A Glimpse Into The Mobile Future. Retrieved January 9, 2011, from Forrester Blogs: http://blogs.forrester.com/thomas_husson/10-­‐12-­‐20-­‐
mobile_augmented_reality_beyond_the_hype_a_glimpse_into_the_mobile_futu
re Junaio. (2011). (Metaio, Producer) Retrieved from http://www.junaio.com/ Kato, H. & Billinghurst, M. (1999). Marker tracking and hmd calibration for a video based augmented reality conferencing system. 2nd IEEE and ACM International Workshop on Augmented Reality, (pp. 85-­‐94). Kirkpatrick, M. (2009, August 24). Augmented Reality: 5 Barriers to a Web That's Everywhere. Retrieved April 2, 2010, from Readwriteweb: http://www.readwriteweb.com/archives/augmented_reality_five_barriers_t
o_a_web_thats_eve.php 73 Maes, P. & Mistry, P. (2009, November). Pranav Mistry: The thrilling potential of SixthSense technology. Retrieved March 9, 2010, from TED: http://www.ted.com/talks/pranav_mistry_the_thrilling_potential_of_sixthse
nse_technology.html Metz, R. (2009, October 4). Another use for your phone: augmented reality. Retrieved from Associated Press. Milgram, P. and Kishino, F. (1994, September). A Taxonomy of Mixed Reality Virtual Displays. IEICE Transactions on Information and Systems. Nielson, J. & Molich, R. (1990). Heuristic evaluation of user interfaces. CHI '90 Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, (pp. 249-­‐256). Rogers, E. (2003). Diffusion of Innovations (5th ed.) New York: Free Press. Rutledge, P. (2010, August 26). Augmented Reality: Who wants more reality? Retrieved January 22, 2011, from Media Psychology Research Center: http://mprcenter.org/blog/2010/08/26/who-­‐wants-­‐more-­‐reality/ Sharp, H., Rogers, Y. & Preece, J. (2007). Interaction Design: Beyond Human-­‐Computer Interaction. West Sussex, England: John Wiley & Sons. Schmalstieg, D., Langlotz, T., and Billinghurst, B. (2011). Augmented Reality 2.0. In Virtual Realities. New York: Springer Vienna. Schroeder, S. (2010, September 29). How Popular Is the iPhone, Really? [INFOGRAPHIC]. Retrieved February 4, 2011, from Mashable: http://mashable.com/2010/09/29/popular-­‐iphone-­‐infographic/ Shedroff, N. (1999). Information Interaction Design: A Unified Field of Theory of Design. In R. Jacobson, Information Design (pp. 267-­‐292). Cambridge: MIT Press. Sterling, B. (2009, August 20). At the Dawn of the Augmented Reality Industry. Retrieved March 29, 2010, from Layar conference keynote (video): http://site.layar.com/company/blog/video-­‐bruce-­‐sterlings-­‐keynote-­‐at-­‐the-­‐
dawn-­‐of-­‐the-­‐augmented-­‐reality-­‐industry/ Swan J., & Gabbard J, (2005). Survey of User-Based Experimentation in Augmented
Reality, 1st International Conference on Virtual Reality, Las Vegas.
74 Toto, S. (2010, March 15). Crimsonfox: Augmented reality-­‐powered scavenger hunt n Tokyo. Retrieved January 29, 2011, from CrunchGear: http://www.crunchgear.com/2010/03/15/crimsonfox-­‐augmented-­‐reality-­‐
powered-­‐scavenger-­‐hunt-­‐in-­‐tokyo-­‐video/ Tufte, E. (1997). Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire: Graphics Press. Tufte, E. (1990). Envisioning Information. Cheshire: Graphics Press. Tufte, E. (2008). iPhone video. Retrieved March 1, 2011, from The Work of Edward Tufte and Graphics Press: http://www.edwardtufte.com/bboard/iphone-­‐
video.adp Tufte, E. (1983). The Visual Display of Quantitative Information. Cheshire: Graphics Press. Uger, R. & Chandler, C. (2009). A Project Guide to UX Design: For User Experience Designers in the Field or in the Making. Berkeley, CA: New Riders. Vaughan-­‐Nichols, S. (2009). Augmented Reality: No Longer a Novelty?. IEEE Computer, 4. Wagner D., L. T. (2008). Robust und Unobtrusive Marker Tracking on Mobile Phones. ISMAR’08, (pp. 121–124) Wen, H. (2010, January 26). Augmented reality: Pure hype or Next Big Thing in mobile? Retrieved December 19, 2010, from Computerworld: http://www.computerworld.com/s/article/9145418/Augmented_reality_Pu
re_hype_or_Next_Big_Thing_in_mobile_ Zhou, F., Duh, H. & Billinghurst, M. (2008). Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR. ISMAR 2008. Cambridge. Zimmermann, B. (1997). Applying Tufte's principles of information design to creating effective Web sites. 15th annual international conference on Computer documentation (pp. 309-­‐317). New York: ACM. 75 APPENDIX A: FOCUS GROUP DISCUSSION GUIDE PROTOCOL Session 1 Introduction (10 minutes) Explanation of Focus Group: Today I am going to be introducing to you a mobile augmented reality application called Layar. Over the next 1 ½ hours we are going to be discussing, sometimes individually, and other times as a whole group your thoughts about augmented reality and the Layar application. To begin, we’re going to introduce ourselves. If you would, please state your name, major, year in school and a little bit about the types of mobile apps you typically use. Activity: Presentation of Augmented Reality and Layar concept videos (15 minutes) Participants will be presented with two videos about augmented reality and the Layar mobile application, produced by Layar. Video 1: http://www.youtube.com/watch?v=b64_16K2e08 Video 2: http://www.youtube.com/watch?v=EtpNx7Y14d0&feature=related After watching the video clips, the moderator will open the floor to discussion asking participants what do they think of augmented reality and the Layar app based on what they just watched. Discussion: First impressions of mobile augmented reality (40 minutes) The moderator will facilitate discussion about the participants first impressions about mobile augmented reality and the application, Layar by utilizing the following probing questions: • Does Layar seem like an application you would be interested in trying out for yourself? Why or Why not? •
How do you think you might use this application? •
Are you more likely to use Layar for tourism, social networking, shopping, or games? Why did you choose that category? •
Can you think of any other potential augmented reality applications you would like to see? How would they work, and why would you want to use it? •
76 Can you think of reasons why you would not use an application like Layar? Assignment: Testing out the Layar application (10 minutes) Participants will be provided directions on how to download the Layar application to their smart phone, and asked to try out the application for a period of at least two hours over the next week. The focus group will reconvene a week from today. Session 2 Introduction (5 minutes) Explanation of Focus Group: Today we are going to share and discuss your experiences using the Layar mobile reality application. What I’m interested in hearing from you are you’re your likes and dislikes about the app, how you used the app, and whether or not you intend to continue using this app in the future. Discussion: First impressions of mobile augmented reality (70 minutes) The moderator will facilitate discussion about the participants’ first experiences using a mobile augmented reality application, by utilizing the following probing questions: • So, what did you think of the Layar application now that you have had a chance to try it out? •
Did the application work as you expected it to? If not, what surprised you? •
Did you have any problems using the application? If so, what were they? •
If you could change one aspect of the application or add one feature, what would it be? •
Please fill in the blank. It would be really cool if an augmented reality application could do __________. •
If you were to recommend Layar to a friend? Who would it be, and why do you think they would like using it? •
Do you plan to continue using this application in the future? If not, why? 77 APPENDIX B: DESIGNER PERCEPTIONS QUESTIONARE Name: Occupation: 1. What types of content do you think works best in an augmented reality application? 2. How do you utilize mobile augmented reality as a medium for content creation? What are your current plans on using augmented reality in the future? 3. Please fill in the blank. It would be really cool if an augmented reality application could do __________. 4. How is the design process for creating augmented reality applications different from other forms of interactive media? 5. What do you see as the advantages and limitations to developing augmented reality information and storytelling based applications? 6. What are the must have elements to every augmented reality design? 7. What do you think are mobile augmented reality’s greatest design challenges? 8. In what direction do you see augmented reality taking information design and digital storytelling in the future? 78 APPENDIX C: HEURISTIC EVALUATION PROTOCOL This document provides a detailed analysis of the heuristic evaluation of two mobile augmented reality applications: Layar and Junaio. Evaluation criteria were derived from established principles of information design written by Edward Tufte in his books: The Visual Display of Quantitative Information (1983), Envisioning Information (1990) and Visual Explanations (1997). Problems and instances are noted for each applied principle on the basis of structure, presentation and dynamics in the application design. Evaluative heuristics 1. Individual elements are presented within a framework relating to a larger context 2. Effectively layers and separates information to emphasize hierarchy and structure 3. Employs small multiples as a means of comparing differences between related data 4. Effectively uses color for information display purposes 5. Integrates words and images effectively Severity rankings 1 — Minimum design problem that occurs infrequently and/or with minor user hindrance. 2 — Moderate design problem that occurs occasionally and may result in readability issues. 3 — Severe design flaws that occurs frequently and damages the readability of data. 79 Evaluation 1: Layar # Problem Oversized icons cover up 1 the background (camera view) and other data points. “Bird’s eye” view feature results in data point overlap 2 and diminishes location-­‐
based context. Severity Heuristic Rating Number 3 1, 2 2 2 2 2, 3 A visual disconnect exists between data points and 4 their related text and/or images 2 5 Large text boxes take up a 5 considerable amount of screen real estate. 2 1 White text on bright colored 6 backgrounds difficult to read. 1 4 3 Difficult to recognize which tweet is currently selected Broad Heuristic Micro design elements should not overshadow the main view. Data should be appropriately layered in proximity to the user’s location Selected objects should be highlighted. Icons should be simple and consistent. Data points should have strong visual associations with related images and text. The display of secondary content should not overpower the framework of the bigger scene. Poor color selection and lack of contrast. 80 Evaluation 2: Junaio # Problem User interface controls do not orient correctly when 7 viewing in landscape mode Expanded dialog boxes are presented sideways when 8 using application in landscape mode Radar size is inadequate for presenting surrounding 9 information in 360 degrees. Text displayed within 10 individual data points is difficult to read Severity Heuristic Rating Number Broad Heuristic 3 3 User interface should be consistent throughout the application. 3 5 Text presented sideways is difficult to read. 2 3 Difficult to separate content effectively on a smaller scale. 5 Reversing type out of dark backgrounds requires strong color contrast and larger type sizes. 1