School of Information Technology and Mathematical Sciences
Division of Information Technology, Engineering and the Environment
Minor Thesis for the Bachelor of Computer Science (Honours)
Informing the Presentation of Spatial Augmented
Reality Annotations with Subliminal Cueing
James Baumeister
Supervisor: Prof. Bruce H. Thomas
ii
Abstract
Augmented reality systems commonly employ overt cueing to direct a user’s
attention and guide their actions. This human-computer interaction delves
into the psychology of how people engage with such an immersive technology.
Spatial augmented reality is a projector-based version of augmented reality
that enables direct appearance manipulation of objects in the physical world.
This allows for novel interactions with the objects that are not possible when
constrained to imagery seen through a typical head-mounted display or
screen. While current research is expanding the applications of spatial
augmented reality, little work has examined how people interact with the
technology at a psychological level.
This thesis first explores adopting some of the concepts of cognitive
psychology into future research into spatial augmented reality. One of these
concepts, subliminal cueing, was taken and applied to information
presentation in spatial augmented reality to analyse its impact on task
performance. A series of user studies were conducted to explore subliminal
cues; annotations that exist below the threshold of consciousness. It was found
that subliminal cueing is technically possible with standard data projectors,
and that subliminal cues can further improve users’ reaction time in
procedural tasks. This research introduces a novel technique for temporal
subliminal cues in spatial augmented reality.
iii
iv
Declaration
I certify that this thesis does not contain material that has been accepted for
the award of any other degree or diploma in any other institution, and that to
the best of my knowledge, it contains no material previously published or
written by another person except where due reference is made.
James Baumeister
Adelaide, November 2014
v
vi
Acknowledgements
The biggest thank you is to my supervisor, Professor Bruce Thomas. At the
beginning of the degree I would never have thought that my previous studies
would ever become relevant to computer science. Not only did Bruce show an
interest, but he also gave me a research project that played to my strengths
and allowed me to think in multiple directions. His wisdom and guidance kept
me on track and the project rolling forward, even when met with hurdles.
Thank you very much, Bruce.
There have been a number of collaborators on this project who also deserve
thanks. Much of my work was built upon existing frameworks. Thank you very
much to Doctor Michael Marner for his open door policy and help with more
questions than I can remember. Also, thank you to Doctor Ross Smith for his
advice and encouragement throughout the year. Finally, thank you to Doctor
Mark Kohler for his insights into cognitive psychology.
The Wearable Computer Lab is a very nice environment to work in every day. I
have enjoyed the laughs and welcome distractions, but also the availability of
everybody to lend a hand when needed. Thank you to Doctor James Walsh,
Andrew Irlitti, Tim Simon, and Neven Elsayed for making the lab such a great
place.
Finally, a big thank you to my family. Kat, I did it to you again. Her time and
efforts reading countless drafts will always be appreciated. To my parents,
Angela and Harry, thank you for supporting me. I’ve chosen well this time, I
promise.
vii
viii
Contents
Introduction ........................................................................................................................1
Research Questions ................................................................................................................... 3
Thesis Structure.......................................................................................................................... 4
Background .........................................................................................................................5
Virtual and Augmented Reality ............................................................................................. 5
Virtual Reality........................................................................................................................................ 6
Augmented Reality .............................................................................................................................. 8
Limitations............................................................................................................................................ 12
Spatial Augmented Reality ................................................................................................... 13
Applications of Spatial Augmented Reality .................................................................... 14
Industry and Design.......................................................................................................................... 14
Entertainment ..................................................................................................................................... 16
Instruction and Information Presentation .............................................................................. 17
Human-Computer Interaction and Cognitive Psychology ............................... 21
Attention..................................................................................................................................... 21
Cognitive Load Theory........................................................................................................... 23
Sleep Deprivation .................................................................................................................... 25
Visual Subliminal Cues in Spatial Augmented Reality ...................................... 27
Subliminal Cueing ................................................................................................................... 27
Experiment 1 – Effects of Subliminal Cues in a Procedural Task ........................... 30
Method ................................................................................................................................................... 31
Results .................................................................................................................................................... 35
Discussion ............................................................................................................................................. 37
Limitations............................................................................................................................................ 39
Experiment 2 – Perception of Two New Subliminal Cues ......................................... 40
Method ................................................................................................................................................... 42
Results .................................................................................................................................................... 45
Discussion ............................................................................................................................................. 46
Experiment 3 - Effectiveness of Two New Subliminal Cues...................................... 47
Method ................................................................................................................................................... 48
Results .................................................................................................................................................... 51
Discussion ............................................................................................................................................. 57
General Discussion ................................................................................................................. 59
Conclusion ........................................................................................................................ 61
ix
x
List of Figures
Figure 1: SAR used for different purposes and scales ...................................................... 2
Figure 2: AR annotation providing guidance on an assembly task ............................. 9
Figure 3: A MAR application .................................................................................................... 11
Figure 4: Blank control panel with SAR added detail .................................................... 13
Figure 5: Foam sculpting with SAR guides ........................................................................ 16
Figure 6: The button box used for the trials...................................................................... 31
Figure 7: A left-handed participant performing the task on the button box. ....... 32
Figure 8: Timing for a helpful cue in the right-handed configuration .................... 33
Figure 9: Timing for an unhelpful cue in the left-handed configuration................ 34
Figure 10: Timing for the perception analysis test ........................................................ 35
Figure 11: Mean reaction times for each block of trials ............................................... 36
Figure 12: The dome apparatus ............................................................................................. 40
Figure 13: The hash cue style ................................................................................................. 41
Figure 14: Masking techniques for the green cue style ................................................ 43
Figure 15: Masking technique for the hash cue style .................................................... 44
Figure 16: Green cue styles in Experiment 3 .................................................................... 49
Figure 17: Hash cue styles in Experiment 3 ...................................................................... 50
Figure 18: Mean reaction times for the green cue on the box .................................... 53
Figure 19: Mean reaction times for the hash cue on the box ...................................... 53
Figure 20: Mean reaction times for the green cue on the dome................................ 55
Figure 21: Mean reaction times for the hash cue on the dome .................................. 55
xi
xii
1
Introduction
Augmented Reality (AR) [3, 34] supplements the physical world with
registered computer-generated imagery and information to enhance our
experience. This imagery can serve a multitude of purposes, from
entertainment [90], to providing valuable instructional guides for complex
tasks [55]. A common method of providing AR visual information to a user is
with a Head-Mounted Display (HMD) [4, 13], or a handheld display device
through which the user can see the physical world [5, 48]. Although similar,
AR is not to be confused with Virtual Reality (VR). VR also uses HMDs, but
instead of the physical world being augmented with the computer imagery, the
user is immersed with only the virtual world information, solely experiencing
the world as presented by the VR system [86]. As an adaptation of AR, Spatial
Augmented Reality (SAR) [8, 72] uses projectors to illuminate and modify the
appearance of physical objects. SAR can be used to project perspectively
correct graphics onto surfaces and objects in the physical world, with uses
ranging from interfaces on a table top scale, to design mock ups on complex
objects such as cars [82, 100] and even buildings (see Figure 1 for examples of
this range).
Information presented with SAR has a number of benefits to that with AR,
which make it uniquely suited to guidance and instructional tasks [36, 37].
Unlike AR, SAR actively modifies the appearance of physical objects with
1
controlled illumination from projectors. In this way, users of a SAR system can
view an object in their environment without the need to wear a HMD or hold a
handheld device. Instead of having their views of the world augmented, the
objects they are seeing are augmented. Moreover, as the projections do not
require any additional viewing technology, multiple users can view the same
information [56, 71].
Figure 1: SAR used for different purposes and scales. SAR used to designate weld points on a car
chassis a). SAR used to modify the appearance of a building as an art installation b). (Image
sources: [84], [56])
The SAR style of presenting information and imagery is unique and its
development is quite recent. As such, the exploration of how best to present
information, and how its style alters the perception and behaviour of its
viewers, has not been explored in depth. The field of Human-Computer
Interaction (HCI), however, has been well researched and has a long history of
taking influence from cognitive psychology [15, 61] on how best to present
information and how people naturally interact with computers.
With SAR’s use as an instructional display medium being actively explored
[55, 83, 88], this thesis presents how the influences of cognitive psychology can
inform improvement of the presentation of SAR information.
2
Research Questions
The underlying motivation for this thesis is to explore how the influences of
cognitive psychology can help shape the presentation of information in SAR.
Of particular interest is how subliminal cues can improve a user’s performance
when presented with SAR techniques. Thus, the following research questions
were formulated:

How can concepts of cognitive psychology shape the presentation of SAR
annotations? SAR information can be presented in a number of ways.
Cognitive psychology, and in particular the theories of attention and
cognitive load, can provide clues as to the best practices of information
presentation.

Can information be provided subliminally to users of a SAR system?
Subliminally presented information has a number of benefits, including
lowering cognitive load and reducing interface clutter. Information
presented in this way with SAR would open a new possibility for SAR
interfaces.

Does subliminal priming with SAR affect the reaction time of a user
performing a procedural task? If it is indeed possible to present
subliminal information, the next step is to examine how this
information impacts the experience of the SAR system user.
The results of the investigation of these questions provide a number of
contributions to the fields of SAR and cognitive psychology. While subliminal
cueing has been researched for decades [33, 84, 97], the viability for use in SAR
has not been demonstrated to date. This thesis presents a number of SAR
cueing techniques that have been implemented into an operational SAR
system. This thesis also details a number of experiments that test the viability
of this pairing and identifies how subliminal information could be
incorporated into SAR interfaces. Furthermore, as SAR is a relatively new field
of study, subliminal cueing within the paradigm has not been explored in the
cognitive psychology literature. These findings also contribute to that field of
knowledge.
3
Thesis Structure
This thesis is structured as follows: The following chapter provides a
background to related work in SAR, using its parent fields of VR and AR to
examine how SAR solves some problems in AR as a whole. The research
detailed later explores some uses of SAR, so this chapter provides the
necessary background for these descriptions. Chapter 3 examines some of the
influences of cognitive psychology on HCI and outlines potential areas where
SAR could also benefit. A number of concepts from cognitive psychology are
introduced and their influences are discussed. The fourth chapter focuses on
the main research area of this thesis: visual subliminal cueing in SAR. The
chapter is divided to first give an overview of subliminal cueing, including a
brief history and its use within computing fields, then three separate
experiments are presented that test the impact of subliminal cueing in SAR
procedural tasks. The chapter concludes with a general discussion of these
experiments and their contribution to the field of SAR research. Finally, an
overall conclusion will be presented, including an overview of future research
directions.
4
2
Background
HCI centres upon the facilitation of effective and enjoyable experiences when
using computers [25]. Of particular importance when designing SAR interfaces
with HCI optimisation is that the information is presented in a way that is
easily comprehended, and preferably, more effective than that same
information being presented by a more traditional means (such as a monitor).
Computers are ubiquitously used for instruction and guidance in a myriad
of workplace and everyday tasks [41]. In order to do this effectively, the
hardware and software must work with the human user in a way that
facilitates a natural translation from the computer instruction to the task at
hand. SAR provides the means to perform this translation with fluidity, due to
its in situ and intuitive presentation style. The following sections will provide
an overview of the parent fields of SAR, VR and AR, to highlight their uses,
strengths, and weaknesses. Finally, the SAR literature will be reviewed to
identify its unique advantages over VR and AR. A set of applications using SAR
will be examined to gain a better understanding of these advantages.
Virtual and Augmented Reality
Virtual and augmented realities are a means of presenting digital information
and imagery to users through a form of computerised immersion. In the case
of virtual reality, the user is “placed” into an entirely computer-generated
virtual world. VR environments may be manipulated by researchers to elicit
5
specific responses from the user, such as joy or fear. These virtual worlds
provide an experience that resembles reality or a particular aspect of reality.
Conversely, augmented reality presents the computer-generated imagery
superimposed on the physical world. In this way, a viewer’s perception of
reality is altered, but the experience of the physical world and its objects is
maintained.
As an example of the application of these concepts, virtual and augmented
reality systems facilitate the avoidance of a disconnection between the user’s
environment and the computer-generated imagery. In a VR setting, the
disconnection is avoided due to the world of the user being entirely virtual.
Similarly, AR brings the computer-generated imagery into the physical world,
also avoiding a disconnection. Traditional paper or separate screen
instructions require a momentary abandonment of the task to attend. AR can
provide virtual information without a user needing to direct attention
elsewhere. Despite many similarities between the two modes of digital display,
VR and AR have distinct differences, each with advantages in their appropriate
setting.
Virtual Reality
The most prominent, non-research, use of VR is for entertainment, with many
companies developing VR technologies for gaming purposes. For example, the
Oculus Rift1 is a HMD resembling a ski mask, which presents a scene to each
of the eyes when worn. The first generation of the Rift included orientation
only tracking with three Degrees-Of-Freedom (3DOF), but the second
generation included a limited volume 6DOF tracking system. The display,
combined with the included gyroscope, accelerometer, and magnetometer,
allows for a realistic and immersive 3D virtual world to be experienced by the
user, with their motions in this world influencing their viewpoint. The latest
generation of the Rift allows for natural, leaning motions to be translatable to
the game, meaning that a player can move in for a closer look at game objects.
Sony’s upcoming contribution to the VR market, dubbed Project Morpheus
1
oculus.com
6
[31], will work exclusively with the PlayStation 4 gaming console and its
included controllers. The potential interactions with the 3D imagery through
the proprietary controllers, as well as the inclusion of 3D, position-determined
audio, may create a new level of gaming interactivity. These uses of virtual
reality can provide users with deeply immersive experiences, bridging the
disconnection between the game screen and the player.
Many researchers have explored the use of this deep immersion in
providing a controlled environment through which to treat phobias [14, 63]. In
traditional phobia therapy, physical exposure to the phobic stimuli was used.
This approach is susceptible to great variability in intensity, and in the
example of spider phobia treatment, the spiders themselves may not react as
the therapist intends [14]. This can be traumatic to the patient and possibly
counter-productive. Virtual reality is aptly suited for use in phobia therapy as
it can generalise the phobic stimuli and elicit phobic responses, but the
therapist has a high degree of control over the intensity at which they are
presented.
Similarly, virtual reality technologies have been utilised for physical and
motor skill rehabilitation [38, 49]. In situations where the brain has been
damaged and the ability to perform motor movement hindered, virtual reality
can provide favourable conditions in which a patient can practice such
movements [95]. Moreover, virtual reality environments can offer forms of
sensory feedback to the patient where physical world practice cannot. Some of
these include the ability to play back interactions and gain feedback,
manipulate sensory information such as lighting and auditory input, and view
an interaction from multiple vantage points.
As an alternative to HMDs, a CAVE Automatic Virtual Environment
(CAVE) is a large, walled space onto which computer generated imagery is
projected [20]. A user of a CAVE stands inside the cube, with projectors
presenting imagery onto the four walls and sometimes the ceiling and floor.
Each wall of the cube acts as a projection plane, meaning that when standing
inside the cube, a user is presented with perspective-correct surround vision
7
[21]. Freed from a wearable display device, the user is able to move within the
space, and their movements are tracked and the visualisation updated in real
time to maintain the correct perspective [19]. This view-dependent nature of
the projections makes the experience very immersive and realistic for single
users, but does limit the number of users able to simultaneously use the space.
Despite the freedom of motion given to users of CAVE systems and removal of
a worn device, a user is limited to viewing the virtual world through windows,
with each of these windows being a planar surface. Any interactions with the
world and its objects must be done from a distance, leading to a disjunction of
experience. Stereoscopic displays can provide the illusion of a virtual object
being projected out of the planar surface into the cube. While these forms of
CAVEs provide a greater sense of immersion, a CAVE experience is quite
different from one gained from an HMD [12].
Augmented Reality
AR is similar to VR, but instead of the experience being solely virtual, the
reality from the user’s perspective is a combination of their experience of the
physical world, and registered computer generated imagery and information.
The virtual aspects are superimposed onto the physical world [4], preserving
the user’s natural experience. In addition, AR opens up new interactions that
are unavailable to virtual reality displays; users can directly interact with
objects in the physical world, with their appearance or function altered by the
AR software.
HMDs are commonly used in AR applications to provide the digital
imagery directly to the eye. While HMDs are also used in VR, those in AR are
see-through (optical and video), which enables the digital imagery to be
presented in conjunction with the physical world. Optical see-through HMDs
work by half-transparent mirrors allowing the wearer to view the physical
world and reflect the computer-generated imagery into the eyes [13, 77]. As an
alternative, video see-through HMDs operate by using cameras that capture
the physical world from the wearer’s perspective. This video is combined with
registered computer generated graphics and presented through the HMD [78].
8
Like in VR, applications of AR suitably lend to the treatment of various
disorders and disability, but with the distinct advantage of them being more
translatable to the natural environment. More importantly to the current
research is that the culmination of physical and computer generated imagery
lends well to instructional and training applications, in the form of virtual
annotations presented directly onto objects in the physical world.
Henderson and Feiner [37] explored the use of AR presented annotations
through a HMD in the context of procedural task instruction. In their study,
participants were required to complete a procedure comprising the assembly
of a combustion chamber (see Figure 2). In the AR condition, the participants
wore a HMD and annotations detailing how and where to perform the next
task were presented directly on the relevant objects. Performance was analysed
when aided by AR versus documentation shown on an LCD screen. Overall,
the authors found that the AR was more helpful in the psychomotor task than
the instructions on a separate LCD screen, and that the users preferred the AR
condition.
Figure 2: AR annotation providing guidance on an assembly task. (Image source: [72])
Building on using AR annotations as instructional or guidance cues, Lu et
al. [52] introduced the concept of subtle cueing with HMDs. Designed to
9
reduce the visual occlusion and interface clutter presented by the use of overt
cues, subtle cues still offer a means by which to redirect user attention, but
avoid the aforementioned drawbacks. In order to test the effectiveness of the
subtle cue, the researchers had participants complete a search task while
wearing a HMD. The participants looked for a small cross on a large
panoramic image guided by: an overt cue, a distinct, white box surrounding
the cross; a subtle cue, which was a much less overt, surrounding white box;
and no cue. Their results showed that the subtle cue significantly reduced the
search time, compared to the no cue condition.
Bruder et al. [10] used a HMD to generate computer-mediated reality, and
manipulate the wearer’s perception of self-motion and direction. Employing
the phenomenon of change blindness, which describes how viewers can be
oblivious to changes in their environment if there is a visual disruption, the
authors altered the participants’ perception of their position in the room, and
also how far they had moved, versus how far they thought they had moved.
The authors showed that temporal manipulation of imagery to the participants
meant that their movements could also be manipulated. This could be
particularly useful for simulating large-scale environments in much smaller
spaces. To give a gaming example, a player could travel through a map much
larger than the physical room they are in, with their movements imperceptibly
altered to remain within the present confines.
Presenting information with AR need not always involve a HMD. Static
displays, such as computer monitors and television screens, coupled with
cameras, can be used to present a version of reality that has been altered with
computer-generated imagery. Bai, Blackwell, and Coulouris [6] developed a
system to foster imagination and pretend play behaviour in children with
autism. Their system used a camera and tracking to record the children and
also some blocks the children were to play with. AR was used to change the
appearance of the blocks on a television screen in front of the children to make
them look like other objects, such as cars and trains. In this way, the children
were able to see themselves on the screen playing with something other than
10
blocks, helping form imaginary connections between a plain object, and
something more interesting.
Another field of AR that moves away from the use of a HMD, is mobile AR
(MAR) [27, 39]. MAR facilitates the supplementation of reality with computergenerated imagery through the viewport of the handheld device, such as a
mobile phone screen. Commercial packages, such as Layer2 and Wikitude3
provide the software platforms on which many MAR applications can be built.
This expands the possibilities of the paradigm and brings the concept of truly
portable AR to millions of potential users. Figure 3 shows how augmented
reality can be implemented on an everyday handheld device. MAR is also
being explored in the research community [92].
Figure 3: A MAR application. A user selects objects aided by real-time visualisations. (Image
courtesy of Neven Elsayed)
Langlotz et al. [48] developed a MAR real-world sketching application
where users sketched digital 2D or 3D imagery onto the physical world as seen
through their mobile device. They then attached this imagery to the scene in
real time and saved the sketch as a real-world object; the added sketch
appeared in coherence with the physical scene, as viewed on the device. Other
users may view the sketch with their own devices when looking at that same
2
3
layar.com
wikitude.com
11
scene. Applications like this could provide useful information to designers or
tourists, to name just two. Olsson et al. [62] presented participants with a
number of everyday scenarios and described how MAR would be used in those
situations. Participants generally reported support of MAR services when the
use served a pragmatic purpose; when they made performing tasks easier and
more efficient. MAR presents an alternative AR medium to HMDs.
Limitations
Despite the powerful platforms on which VR and AR systems can present
information, they both suffer from a number of weaknesses. HMDs are a
particularly limiting factor, with their cumbersome, uncomfortable, and
sometimes sickness-inducing [51] nature making them inappropriate for
everyday use. In addition, HMDs are often tethered to a computer, limiting
portability, and where the computer is designed into a wearable form this can
add to the discomfort [85]. New technologies, such as the Google Glass4 and
the Epson Moverio5 are addressing a number of these issues, but to date the
use of HMDs in everyday life is not commonplace.
Beyond the debate of comfort and effectiveness, personal devices like
HMDs are constrained to the individual, and are not always suitable for group
interaction. Although multiple users could experience the same environment
through their respective displays, the worn device acts as a barrier between
users, making social interaction awkward and impersonal [91]. Immersive
display systems, like CAVEs as discussed above, address this particular
limitation, but other limitations are introduced. Users in such systems are free
to interact and experience the visualisation as a group, but are limited by the
shape and configuration of the displays. In the case of fixed displays, the users
also cannot freely move around objects of interest, but rather view them
through the “window”. MAR does address this weakness by allowing each user
to view information through their own mobile device display. The use of a
mobile device, however, leads to an impersonal and disconnected form of
4
5
http://www.google.com/glass/start/
http://www.epson.com/cgi-bin/Store/jsp/Landing/moverio-bt-200-smart-glasses.do
12
collaboration. Weaknesses such as these reveal an opportunity to an
alternative form of AR: SAR.
Spatial Augmented Reality
SAR enables the same interaction with the real world as AR, but frees the user
from HMDs or fixed displays by using projectors to illuminate and alter the
visual presentation of physical objects [72]. SAR facilitates the augmentation of
the physical environment itself, enabling the users to view the augmented
physical object from multiple perspectives. Unlike VR and AR, SAR projections
can alter the appearance of the physical surface for arbitrary physical objects in
the environment. To increase the photo-realism of the final visual effect,
6DOF tracking technologies may be used to determine a viewer’s perspective
of the augmentation and change with their movement. This information may
be used to improve the 3D graphical rendering to enhance the appearance of
the augmentation [9, 70]. The use of 6DOF tracking is not a requirement for
all SAR systems, but this sensing technology may be used to improve the
usefulness of a SAR system.
Figure 4: Blank control panel with SAR added detail. (Image courtesy of the Wearable Computer
Lab)
In their work defining the “office of the future,” Raskar et al. [71]
pioneered the idea of SAR by presenting a room, resembling an office, where
all the surfaces were potential displays, illuminated by projected imagery. In
this office, the workers are tracked in real-time, with their view of the imagery
13
updated to maintain perspective correctness. Later work incorporated the idea
of Shader Lamps [73] to project onto distinct objects, rather than surfaces, and
alter the visual appearance of that object. Using this method, a white model is
constructed, including the basic shape and contour of the desired object.
Then, a 3D computer model is projected onto the object, including all colour
and texture information, to transform the object into a realistic and detailed
representation. The development of this concept and technology revealed SAR
as a powerful means of altering the appearance of surfaces and objects, thus
identifying its suitability for a wide range of applications. The following subsections will discuss some of these applications.
Applications of Spatial Augmented Reality
As previously mentioned, SAR is preferable to HMD and handheld AR in a
number of contexts. SAR environments do not require HMDs or any other
viewing aid, removing technology from the user’s head and hands, facilitating
SAR as a collaboration tool. Viewers can physically move around the
environment; they are not limited to a presenter’s perspective. Annotations
and images being directly projected onto physical surfaces also means that
SAR could be utilised in industries such as manufacturing [100], medical
surgery [83], and architecture [46]. Guides can be projected onto objects to
provide information to the user without inhibiting them with a wearable
device or shifting their focus to a secondary display. Users can also manipulate
the projections to alter the appearance of an object at will [96]. SAR’s ability to
alter the appearance of an object could be beneficial for designers and artists
[54].
Industry and Design
Schwerdtfeger, Pustka, Hofhauser, and Klinker [82] investigated the use of
laser projections for identifying weld points on a car chassis. The authors
considered two styles of projecting the weld information. One, a head-worn
projector, allowed the wearer a large field in which they could receive the SARprovided weld annotations. It was, however, too cumbersome to be practical.
The other style, a tripod-mounted projector, was preferred. The authors
14
identified the freedom from uncomfortable worn devices, and the ability to
present the information to multiple users of the system as advantages of using
projectors over HMD-based AR. Similarly, Zhou et al. [100] developed a system
for providing SAR-based projections directly onto a car chassis to facilitate a
more efficient manufacturing process. The projected annotations removed the
need for workers to consult diagrams and manually locate the respective weld
point on the chassis.
Zaeh and Vogl [99] employed laser projectors for the programming of
industrial robots, designing a user interface whereby a user could control a
tracked input device to outline a robot path. The projector then displayed the
path with the robot performing its task by following the tracked trajectory.
The interactivity gains between the programmer and robot facilitated by SAR
helped produce a reduction of up to 80% in robot programming time, when
compared to ordinary methods.
To create an interactive architecture design environment, Kurz et al. [46]
designed a system coupling SAR with laser pointer tracking, and cameras. The
system used an automated calibration routine whereby the geometry and
reflectivity of the space was ascertained by a camera and laser pointer
combination. This calibration was then used by the computer system to track
a handheld laser pointer throughout the space, enabling interactivity with the
projections on all surfaces. This system allowed architects and designers a
flexible way in which to interact with the environment and simulate a final
structure configuration or design.
Marner and Thomas [57] adapted existing foam sculpting techniques for
the industrial design domain to the SAR paradigm. The authors projected the
desired geometry onto a block of foam. Using a tracked, hot-wire, cutting tool
the users’ cuts into the foam were measured, and the SAR-based projections
updated to indicate how much foam had been removed. SAR imagery was
projected to indicate where the foam block still needed to be altered to achieve
the desired shape (see Figure 5). Expert users, comprising final year design
students, received the system positively, identifying its potential to eliminate
15
mistakes when modelling without guides. The extra information provided by
SAR gave the users a means by which to improve their modelling and overall
visualisation of the end product.
Figure 5: Foam sculpting with SAR guides. The user cuts the foam with the SAR wireframe
updating to reflect remaining cuts. (Image courtesy of the Wearable Computer Lab)
Entertainment
SAR’s ability to alter the appearance of surfaces and objects, and its
applicability to collaboration environments, make it an intriguing medium for
entertainment. Marner et al. [54] created a SAR system for use in a live theatre
production. The theatre set was represented using projected imagery and
textures, and the movement of the actors and various objects across the stage
was tracked with a non-invasive Kinect. The imagery reacted with this
movement to give a sense of dynamism to the production. In addition,
annotations were projected onto the set “attached” to the actors, and followed
them as they moved, giving extra information to the audience.
As previously mentioned, SAR can be used in the design phase in the field
of architecture. This same concept has also been exploited for entertainment.
Large-scale art installations have been taken to the street, with projectors
altering the appearance of entire buildings; lighting up city streets with
16
colourful, augmented buildings. A Ukrainian specialist 6 in projected art
installations augmented the appearance of an Islamic mosque in Grozny,
Russia. The installation used a series of projectors to display animated art on
the building for the enjoyment of onlookers. Similarly, the Northern Lights
installation illuminated significant buildings in Adelaide, Australia with
colourful graphics as part of the Adelaide Arts Festival [64]. Displays such as
these highlight the large-scale, cultural impact that SAR can have when used
as an entertainment vessel.
Gaming is also an area that has been explored using SAR techniques. Jones
et al. [42] developed a table top game where players first constructed a gaming
level, then placed virtual imagery throughout the level, and then were able to
interact with the imagery. Players constructed the level using foam blocks in
any configuration they desired. The resulting configuration was captured by a
structured light scan, utilising both the projector and camera. The players then
mapped virtual content onto the playing field with a tracked stylus. Once
done, the game could be played with the virtual content reacting to gestures
made by the players with the stylus.
Instruction and Information Presentation
Being that SAR annotations are projected directly onto objects and surfaces of
interest, it is an ideal technology for task instruction. In this vein, Suganuma et
al. [88] developed a SAR-aided billiards instruction system. By using a camera
and projector mounted directly above the billiard table, the authors tracked
the balls and the player’s cue, and projected onto the table the optimal angle
at which to strike the white ball in order to pocket the target ball. In a similar
style of instruction, Seo et al. [83] used SAR projections to provide guides for
performing medical surgery. In order to address the limitations of using a
regular ink pen to mark the cut locations on a patient, the authors used a
tracked stylus to mark the incision path, with projectors replicating this path
directly onto the patient model.
6
promo-promin.com/en
17
Piper and Ishii [68] and Rosenthal et al. [79] have utilised SAR projection
techniques to develop systems that allow novice user to perform complex
tasks. The Rosenthal et al. research used projected guides to aid the
completion of tasks involving cutting and folding, amongst others. Their
investigation showed that the SAR instructions, projected directly onto the
task surface resulting in faster completion times and fewer errors than when
following on-screen instructions. Similarly, Piper and Ishii used a projector to
instruct the building of a complex structure from a computer-aided design
(CAD) drawing. Users of the system were instructed by projected imagery
where a series of blocks needed to be placed in order to recreate the final CAD
model. The benefit found in both studies was that the colocation of
instructions and task materials led to a more cohesive understanding of the
task requirements.
Porter et al. [69] investigated the rapid prototyping potential of SAR by
projecting virtual buttons onto a physical substrate. Varying visual
appearances could be projected onto the blank substrates, and the interaction
with the virtual buttons was calculated in real-time with finger tracking. The
benefit of such a prototyping system is that designers could rapidly generate
many design variations for a given object and test each of these without having
to construct a complete model. The authors tested the usability of the virtual
buttons with a user study, but found that interactions with the virtual buttons
were slower than with prototypes with physical buttons.
Moving away from virtual buttons, Marner, Irlitti, and Thomas [55]
conducted a study into the use of SAR annotations for procedural tasks, using
physical buttons. Participants were required to press a series of buttons in
accordance with instructions presented on either a separate computer monitor
or projected directly onto the button interface. Two interface types were used.
First was a non-planar, dome-shaped device, on which participants were
required to press the buttons with their preferred hand. Next, they performed
the same task on another device, fashioned like a car dashboard. On this
device, the buttons were divided into two distinct groups; one for the left
18
hand, another for the right. Green annotations were shown around the button,
with a numerical label identifying the order in which they were to be pressed.
The researchers found a significant positive effect of the SAR condition on
performance, both in reducing time and errors. The study showed that when
the previous limitations of AR (such as lag, mobility inhibition due to HMDs,
and poor tracking sensors) are removed, SAR could produce a strong and
positive effect.
19
20
3
Human-Computer Interaction and
Cognitive Psychology
Theories of behaviour, attention, brain functioning, and emotions have been
borrowed from cognitive psychology and applied to a number of studies in the
field of HCI. Researchers have learned from past findings and improved their
understanding of human interaction, and learned how to apply cognitive
psychology principles to computing fields such as user interface design, HCI,
and computer vision. This section provides a brief overview of some of the
areas in which cognitive psychology has informed the field of HCI and how
this knowledge may be used for the improvement of user interaction within
SAR.
Attention
One area of human cognition that plays a central role in governing daily
function is attention. Attention determines how we perceive or do not
perceive the environment around us [76]. How we attend to our environment
and situation also shapes our use of language and our ability to problem solve.
A person’s attention to specific features of an environment can lead to the
inattentive exclusion of other features. Theories of divided attention have
shown that splitting attention amongst multiple secondary tasks is
detrimental to the primary task [80]. When learning to use a new system,
users who are required to divide their attention between a manual, an
21
interface, and unfamiliar surroundings will experience a high level of mental
load, slowing their performance and skill learning. With practice these effects
can be mitigated, but until that practice occurs, the user will be inhibited by
the divided nature of the task. SAR’s in situ nature makes it ideal for limiting
the need for divided attention in the system’s design. If a user is required to
learn how to use a control panel, for example on a train, the process may be
directly projected onto the control panel itself instead of using a paper manual
or instructions on an external screen. Using this approach, the user is only
interacting with the augmented control panel, ensuring their attention is not
divided between multiple mediums.
SAR can also provide a controlled and isolated practice environment.
Raskar and Low [73] designed a Styrofoam model of a blank room that they
prototyped into a detailed space with SAR projections. This enabled easy
manipulation of the appearance of the room, and gave the user freedom to
move around and experience the room’s interior. This concept could be
applied to interface design of a complex system, or any scenario in which
attention training needs to occur. If training in the actual scenario was not
possible or feasible, SAR could be employed to replicate the environment and
interface and allow trainers to dynamically control it at will.
The concept of attention orienting is opposite to distraction and divided
attention. In many situations, it is important to orient the attention of a user;
for example, personnel in airplane cockpits, traffic monitoring stations, and
emergency service control stations need to be immediately alerted to high
priority tasks. It may not be feasible for these personnel in such environments
to wear HMDs, as they may impact upon their field of view or be cumbersome
after long periods of wear. Bright LED small form factor (105mm x 106mm x
39mm, 380gms) projectors are currently available. The LED technology means
that projectors required little power and have very low heat emission, making
the projectors suitable for enclosed areas, such as a cockpit. As the projectors
can be roof mounted, they are also out of the way of the personnel. SAR is
uniquely suited in such circumstances to visually draw attention to specific
22
focal points. Implementation of a system of projectors would be non-invasive
in such environments.
Cognitive Load Theory
The theory of attention naturally leads to cognitive load theory (CLT) [65, 66].
Humans have only a finite level of cognitive ability with which to perform
tasks and calculations. As task difficulty and complexity rises, the task requires
more concentration and cognitive resources to learn and complete [65].
Cognitive load theory breaks the load rating of tasks into three categories:
intrinsic, extraneous, and germane [89]:

Intrinsic load is caused by the difficulty of the task itself, and the
demands that task places on working memory. As the strain on
working memory is part of the task itself, and not the instruction
method, a better process cannot reduce intrinsic load. Intrinsic
load is only reduced when the task is simplified.

Extraneous load is mental strain that is imposed upon the task
learner by ineffectively presented information or task completion
processes. The load is unnecessary and it interferes with the user’s
ability to generate schemas and automation. Increasing the
extraneous load reduces user performance.

Germane load, like extraneous load, is a product of the
instructional or process design. It differs, however, in that it
enhances the learning of the skill by encouraging the formulation
of schemas and automation [65, 81]. Increasing the germane load
improves user performance.
Together these three types of cognitive load form the overall load rating. If
somebody is learning a new skill, the overall load cannot exceed their working
memory if learning is to occur.
Many investigations have explored the effects of instructional task load.
Kester, Kirschner, and van Merrienboer [45] conducted a study where
participants were required to complete a series of electronic circuit problems.
23
The participants were randomly assigned to two conditions that impacted how
the information was displayed to them on a computer screen. In one
condition, the two parts of the problem were integrated, and in the other they
were divided. They found that learning material presented in an integrated
manner leads to better test results than when presented in a divided manner.
In this study, the divided condition led to an increased level of extraneous
load, slowing down performance and schema generation. This finding relates
to SAR, which integrates virtual information with the physical world, rather
than requiring divided attention.
Cheon and Grant [17], in their study into Web-based instruction,
employed a number of cognitive load measures. The authors were interested
in whether increased germane cognitive load would help increase performance
in the learning task. They designed a metaphoric interface to help enhance
schema acquisition. The authors found that the increased germane load
positively affected performance, and that the mental models offered by the
metaphorical interface aided the participants in forming their own schemas.
Schultheis and Jameson [81] conducted an experiment that used an
electroencephalogram (EEG), event-related potential (ERP), and pupil dilation
to assess participant cognitive load. The authors used a specialised ERP device,
which has been shown to have a correlation between a high amplitude and
high mental load. The participants completed a series of reading tasks,
receiving both easy and difficult passages. As they recited the passages to the
computer, the various measures were taken. The results of the experiment
found that text difficulty significantly affected reading speed, subjective load,
and the ERP amplitude. The advent of sophisticated measuring devices has
identified how complex neurocognitive processes respond to varying task
difficulty.
Measures for testing the cognitive load inherent in particular tasks usually
include qualitative surveys [65]; performance scores; and brain analyses, such
as the previously mentioned EEG, fMRI [11], and ERP [81]. Despite the high
reliability of self-reports of load given by participants [67], empirical measures
24
can give a unique picture of not only the level of brain activity, but also the
regions of the brain where the activity occurs. These cognitive measures,
therefore, have the potential to show not only how SAR instructions differ
from those on a computer display or paper instructions, but also to provide
insight into how SAR instructions are processed differently by the user’s brain.
These investigations could potentially lead to new models of human learning.
Sleep Deprivation
Applying SAR to industries such as manufacturing would result in SAR
systems being used by workers performing simple, repeated tasks that require
constant attention. Additionally, shift workers who work long irregular hours
may conduct these tasks. This style of employment makes the effects of sleep
deprivation a serious consideration. These effects have long been established
to degrade neurocognitive and psychomotor performance [26, 35]. In addition,
sleep deprivation increases the risk of harmful accidents, such as motor vehicle
crashes. In fact, Williamson and Feyer [98] found that being awake for any
time exceeding 16 hours during the day produced the same decrease in
psychomotor performance as a blood alcohol content between 0.05 and 0.1%.
In their meta-analysis of a multitude of sleep studies, Lim and Dinges [50]
noted that the effect of sleep deprivation on different cognitive tasks varies
greatly in their performance reduction. Despite the propensity of sleep
deprivation to cause serious deleterious effects across many domains of
cognition, the authors note that task performance on simple attention tasks
suffers the most. Harrison and Horn found tasks that are sustained and
monotonous are most susceptible to sleep deprivation based on their inability
to encourage compensatory effort to overcome sleepiness [35]. In a sleep
deprivation state, a sleepy person also suffers from wake-state instability,
meaning that micro-sleeps could occur without the person’s awareness or
control.
As previously mentioned, some of the potential applications of SAR
include the manufacturing industry, control stations, medical surgery, and
industrial design. Some of these occupations include an element of time
25
pressure,
while
still
requiring
accurate
and
efficient
performance.
Furthermore, some of these occupations require sustained and monotonous
tasks. It is likely that sleep deprivation could negatively impact workers in
such situations, with potentially catastrophic outcomes. An interesting finding
for SAR research would be to compare task performance in sleep deprived
circumstances. If there were a positive difference between a SAR and non-SAR
condition, SAR could be offered a means for a safer and more effective work
environment.
26
4
Visual Subliminal Cues in Spatial
Augmented Reality
This chapter details the main experimental focus of this thesis, subliminal
cueing in SAR. The following subsections define each of the three experiments
that were conducted, with their respective methodologies, results, and a
discussion outlined in order. The chapter will conclude with an overall
discussion outlining the findings of these three experiments and propositions
for future research in the area.
Subliminal Cueing
A subliminal stimulus is a visual, haptic, or auditory stimulus that occurs
below a person’s conscious awareness [16]. Despite this lack of awareness, such
stimuli are still processed by the brain at an unconscious level. Subliminal
processing is thought to rely on distinct, reduced, neural processes, compared
to conscious processing [22]. Achieving subliminal visual presentation of a
stimulus can be done in a number of ways. The first is by a rapid temporal
display; a stimulus is presented for a very brief period of time, and then
removed. The rapid nature means that the visual impulses cannot be
consciously perceived before the viewer must attend to other, active visual
stimuli. Another method is to employ masking to conceal the presentation [28,
30, 53]. A subliminal prime can be masked using two techniques: forward and
backward masking. A forward mask is a stimulus presented for a short period
before the subliminal cue, and a backward mask is a stimulus presented for a
short period after the subliminal cue. The mask acts as visual “noise,”
27
disrupting the conscious (but not unconscious) processing of the subliminal
cue. The final consideration that can impact the effectiveness of a subliminal
prime is the stimulus onset asynchrony (SOA) [7, 18, 32, 60], that is, the
amount of time that elapses from the point of presentation of the prime, to the
time when the viewer is expected to respond to the prime. If this delay is too
great, the viewer will experience an inhibition of return and the effect of the
prime will diminish.
Although it has long been experimented with, subliminal suggestion
created controversial interest in the 1950s with the Vicary [44] experiment
claiming to use embedded messages to prompt patrons to buy Coke and
popcorn during a film. Now proven to be a hoax, with many similar
experiments from that period suffering serious methodological flaws [75], the
effectiveness of subliminal messages were questioned by researchers in the
field for decades [59]. More recent studies [23, 87, 94], however, showed that
not only can subliminal messages be effective in influencing reaction time to a
stimulus, but also that there is a much higher level of cognitive processing
than many researchers thought possible.
A study by Strahan, Spencer, and Zanna [87] revisited the usage of
subliminal suggestion for influencing particular brands over others.
Participants in their study were required to attend the experiment without
having eaten or drunk for three hours prior. The participants partook in a
lexical decision task on a computer, with half of them receiving subliminal
cues with thirst-related words. The authors found that the participants who
were thirsty did not report being significantly more thirsty than the control
group, but drank significantly more. In another experiment, they found that
participants who had been cued with thirst-related words were more likely to
want a drink that the authors had marketed as “thirst-quenching”, rather than
another that was “electrolyte-restoring”. This study showed that not only
could subliminal cues influence participant decision-making, but also modify
their behaviour.
28
Chalfoun and Frasson [16] found that by using subliminal priming in a
magic square solving task participants were able to complete the task
significantly faster. Dijksterhuis et al. [24] found that they were able to
influence attribution of self-authorship in a task where the participants
competed against a computer to remove words from the screen, but were not
told who had completed the task first, the computer or themselves. The
authors subliminally cued the participants with words relating to self (such as
‘I’ and ‘me’) or the computer and found that if cued with words relating to the
self, participants were more likely to believe they had beaten the computer in
that task. A study by Wallace et al. [93] found subliminal cueing could improve
participant learning of a text editor program, and Jraidi and Frasson [43] found
that subliminal cueing could increase participant self-esteem.
These examples highlight the level of cognition that is occurring, even
when the stimulus is unperceived and unattended. These effects of improved
task completion and unconscious processing could possibly be utilized in SAR
as means of overcoming interface clutter [74] (interfaces with large amount of
complex information and control sets) where SAR overt (supraliminal) cueing
would compound the clutter. If the clutter is difficult or impossible to remove,
then SAR based subliminal cueing may offer an alternative. This form of
cueing may assist expert users without detrimentally impacting their focus by
shifting attention to overt cues and information.
Subliminally presented cues have been shown to enable limited high-level
processing [16, 23, 24, 32, 43, 87, 93] while maintaining a lack of viewer
awareness. The use of SAR could extend the cognitive psychology knowledge
of subliminal cues when presented in the following ways:

in situ with the physical world,

on non-planar surfaces, and

in context with physical tasks.
Applying cognitive psychology’s knowledge of subliminal cues to SAR is a
new research direction in the virtual environments domain. Divided attention
29
has been shown to decrease the performance of a primary task [40]. If an
interface for controlling important systems (such as a nuclear reactor
workstation) is cluttered and forces divided attention causing an increased
mental load to operate, there could be severe consequences. Subliminal cues
have been shown to improve reaction times and elicit semantic processing
without negatively impacting cognitive load. No study, however, has been
conducted to present subliminal information using a SAR-based methodology.
Thus, this thesis presents the first study conducted to examine the
effectiveness of subliminal cueing with projector-based augmented reality
technology. The methodology involved a similar procedure to that used by
Marner et al. [55]. Participants were required to interact with a series of
physical pushbuttons, with the targets annotated using SAR-based projections.
Subliminal cues were presented temporally prior to the target annotation in
order to prime the participant to the target’s upcoming location. The
experiment was designed to test if subliminal priming enhanced participant
performances.
Experiment 1 – Effects of Subliminal Cues in a Procedural Task
Two tasks were conducted to evaluate the efficacy of subliminal cueing in SAR.
The experimental procedure was based on the work conducted by Mulckhuyse
et al. [60]. The purpose was to enquire as to whether subliminal cueing could
actually increase performance in a procedural task using SAR projections.
Unlike the Mulckhuyse et al. study that only measured perception of a target
by having them press the “spacebar,” this study required that participants push
a target button. Furthermore, in order to show practicability and impact on
performance when explored in a SAR paradigm, a comparison between a cued
and non-cued condition was required.
The first task in Experiment 1 evaluated user performance in a buttonpressing task. Three types of subliminal cue were used in order to compare
their effects: helpful cues, cues that correctly direct the user to the target;
unhelpful cues, cues that direct the user to a button that is not the target; and
no cue. The second task evaluated whether the masked cue technique did
30
produce subliminal stimuli that are imperceptible to users. The hypotheses
tested in the experiments were as follows:
[Hypothesis H1] Helpful cues lead to faster reaction time, compared to
no cue.
[Hypothesis H2] Helpful cues lead to faster reaction time, compared to
unhelpful cues.
[Hypothesis H3] No cues lead to faster reaction time, compared to
unhelpful cues.
[Hypothesis H4] Participants would be able to detect the cues less than
50% of the time, indicating the cues are subliminal.
Hypotheses H1, H2, and H3 were addressed by Task 1. H4 was addressed with
Task 2.
Figure 6: The button box used for the trials. The physical buttons are annotated with projected
labels. The centre-left button is indicated as the target.
Method
A typical office desk and chair were used for participants to be seated during
the study. A wooden box, measuring 480mm x 330mm x 110mm, with four
buttons in a straight line on top was positioned in front of the participant, see
31
Figure 6. Participants were seated approximately 30 cm from the button box,
with their body central to the middle two buttons. A left or right-handed
configuration was provided, based on participant preference. If left-handed,
the origin button was the leftmost of the four and the targets the remaining
three, and the reverse if right-handed. An NEC NP510W projector with
1280x800 resolution and 50hz refresh rate was mounted overhead, angled
towards the box to display the cues. Figure 7 depicts a left-handed participant
operating the button box.
Figure 7: A left-handed participant performing the task on the button box.
The first task evaluated the effect that the cueing had on task completion
time in a button-pressing task. The task consisted of three conditions: helpful
subliminal cueing, unhelpful subliminal cueing, and no subliminal cueing.
Helpful cues primed the participant to their next target, whilst an unhelpful
cue primed a button other than the target. The target was identified with a
circle symbol and the participants were told to press the button as quickly as
possible when it appeared.
32
The task was conducted as a within subject, repeated measures design.
There were a total of 315 button press trials, divided into 5 blocks of 63 trials,
with a minimum 30-second break between blocks. Each condition was tested
21 times per trial block, with the order randomised. The target button was
randomised for each trial, with each button receiving the same number of
targets overall.
A practice version of the task, consisting of four sets of three targets with
no subliminal cues, was presented to allow familiarisation with the process
and reduce any learning effects. The researcher instructed the participant to
use only their dominant hand. The participant was to keep their hand rested
over the origin button until the target was identified, at which time they were
to press the target button as quickly as possible. Their hand was also to return
to the origin position between each press. This, along with even target
dispersal between buttons, ensured consistent hand travel distance between
participants.
Figure 8: Timing for a helpful cue in the right-handed configuration. A helpful cue is displayed
for 20ms after the trial is started by pressing the red origin button.
When ready, the participant began the main task by pressing the origin
button, identified by a red annotation. This was followed by a delay of 1000ms.
33
If in the helpful (see Figure 8) or unhelpful (see Figure 9) condition, a cue
would display for 20ms prior to the display of the annotations for the other
two buttons. After the 20ms all three buttons were annotated and the target
was indicated by a circle symbol.
Figure 9: Timing for an unhelpful cue in the left-handed configuration. An unhelpful cue is
displayed for 20ms after the trial is started by pressing the red origin button.
In the non-subliminal cue condition, all three annotations appeared
concurrently after the 1000ms delay. When the target was pressed, the green
annotations were removed and the red origin annotation appeared. The cycle
would then repeat. Data were recorded capturing the time taken to press the
target button, in milliseconds, from the time the circle symbol appeared.
A second experimental task was designed to measure how effectively
participants could perceive the earlier onset of a cue. In this task, a cue was
pseudo-randomly selected to appear on any of the three target buttons. The
participant was asked to press the button they believed was cued first. This
experiment consisted of four sets of 21 button presses. Upon pressing the
origin button, and after a delay of 1000ms, a cue was displayed for 20ms.
Following that, the remaining two annotations were displayed, without a
target
indicator.
Participants
were
34
instructed
to
press
the
button
corresponding to the annotation they believed appeared before the others. The
timing of annotations for this experiment is shown in Figure 10. Only
information regarding a correct or incorrect press was recorded.
Figure 10: Timing for the perception analysis test.
Results
Twenty-one participants took part in the experiment. Seventeen participants
were male, and four were female. Three participants were left handed, with the
remaining eighteen right handed. The participants were aged between 22 and
54 years of age, with a mean age of 29.5 years (SD = 7.9), and the participants
all had normal or corrected to normal vision.
The mean reaction time across all fifteen conditions was 623.52ms (SD =
88.42). The overall means for each of the cueing conditions are as follows:
helpful = 604.64ms (SD = 81.30), unhelpful = 632.43ms (SD = 76.36), and nocue = 633.49ms (SD = 103.12).
35
Figure 11: Mean reaction times for each block of trials. Reaction times were faster for helpful cue
across all trials when compared to unhelpful cue, and faster when compared to no-cue during
first and last trial blocks only.
A five (trial block) by three (cueing condition) repeated measures ANOVA
was performed to determine differences in reaction time across trials and
between conditions. Sphericity was violated, χ2(2) = 11.81, p = .003, therefore a
Greenhouse-Geisser estimates of sphericity (ε = 0.68 for the cueing effect) was
used. There was a significant main effect for cueing condition, F(1.37,27.34) =
16.08, p < 0.001. Post-hoc analysis revealed faster reaction times in the helpful
cue condition compared to both unhelpful cue and no-cue (both p < 0.001)
Figure 11. No overall differences were observed between unhelpful and no-cue
condition. There was also a significant main effect for trial, F (2.31, 46.29) =
5.03, p < 0.01, however, no overall reaction time differences between pairs of
trials were found in post hoc analysis. Finally, a significant condition by trial
interaction was found, F (2.72, 54.44) = 3.13, p < 0.05. Post hoc comparisons
suggest this effect was due to the slower reaction time observed for the no-cue
36
condition at trial one compared to other trials, however, this difference was
only significant when comparing trial one to trial four, p < 0.05.
Further pair-wise post hoc analyses shows that reaction times were
consistently faster in the helpful cue condition compared to the unhelpful cue
condition at every trial (all p < 0.01). However, reactions times were
significantly faster for the helpful cue compared to no-cue condition in the
first and last trials only (all p < 0.05). Examination of response errors, to
determine any possible trade-off between response time and response
accuracy, showed a very small error rate ranging from 0.3% to 0.8% across
trials (average = 0.5%). Chi-squared analysis indicated there was an even
distribution of errors across participants (p > 0.05).
For the second task the mean number of correct responses was 9.91 (SD =
4.32) or 47.2% and the mean number of incorrect correct responses was 11.10
(SD = 4.32) or 52.8%. A one-way ANOVA was conducted on the results. This
analysis shows there was no change in number of correct (or incorrect)
responses across trials (p > 0.05).
Discussion
These results have successfully, and for the first time, demonstrated the
application of subliminal cueing to facilitate improved procedural task
performance in a SAR environment. The findings are as follows:
1.
Standard, consumer-grade, projection devices are sufficient to
support subliminal cueing.
2.
Subliminal cues presented as a primer, improve a user’s reaction
time for a procedural task.
3.
Subliminal cues are effective for spatial augmented reality
annotations.
To demonstrate that standard consumer-grade projection devices are
sufficient to support subliminal cues, a threshold test was conducted to
determine if the presented cues were subliminal or supraliminal. A subliminal
threshold is typically determined by the ability to detect a stimulus on average
37
50% of the time. Consistent with this definition, the results for the assessment
of participant perception of the subliminal cue were reported at just under
50%. High-speed videography was employed to confirm that the subliminal
cue correctly appeared first, and the annotations displayed simultaneously.
In addition, the results clearly indicate that the subliminal cues improved
the reaction time of a user selecting a particular button highlighted with a
spatial augmented reality annotation. Specifically, reaction times were faster in
the cued condition than either a cue in an incorrect location (unhelpful cue)
or no cue at all. All effects were adequately powered (all above 0.9) and effect
sizes were large. These results are consistent with the psychological literature
demonstrating that a valid subliminal cue facilitates target identification in
non-SAR spatial tasks [6, 26].
Figure 11 depicts the relative mean reaction times over the five blocks.
While there is an indication of a subtle training effect beyond the practice task
given to participants, the change in reaction time across trials within each
condition was relatively small, and statistically non-significant. Closer analysis
revealed that response times were consistently faster in the helpful cue
condition compared to unhelpful cue across the trial blocks, however
differences between helpful cue and no-cue conditions were only robustly
established in first and last trial blocks. This result may be explained by carryover learning effects in the first trial, whereby helpful priming facilitated the
learning process as well as leading to maintenance of boosted performance
across trials. Conversely, task fatigue would be expected to be maximal by the
final trial block, where again helpful cueing could be expected to provide
maximal benefits. The implications of this latter finding are that there are
applications to cueing in real-world scenarios that are fatiguing (e.g. shiftwork) and require fast and accurate performance.
While projectors have been previously employed to depict subliminal
cues, these were typically in cinema, or cinema-like settings [30]. The key
difference between this format and SAR annotations is that in the cinema the
user only focuses on the virtual information (i.e. the film), and in an AR setting
38
the user must focus on both virtual information and the physical world. This
investigation demonstrated that projected information onto physical objects
could contain subliminal information. The second major presentation of
subliminal information in the literature is that with a workstation monitor. In
the literature the most common experiments were in the form of the user
viewing information on a workstation monitor and being asked to respond via
a keyboard or mouse button press. The user in these previous experiments had
to view virtual information and then they physically responded to this
information with an input device not in their field of vision. This investigation
required to the user to view virtual information in-situ with the physical world
(i.e. control panel with the physical button to be pressed).
Limitations
A number of methodological limitations present in the experiment must be
addressed. Despite the perception test finding that the cues were not visibly
perceived in over 50% of the presentations, the claim of this phenomenon
being subliminal, as opposed to pre-attentive or unattended, cannot be
definitively stated with the discussed methodology. It is possible that the cues
presented 20ms prior to the targets were not subliminal, but rather that their
existence remained largely unattended by the participants who were focussed
upon finding and reacting to the target stimuli, namely the ‘0’ symbol. This
limitation could be addressed by introducing a qualitative measure; directly
asking participants to look for, and identify the cue.
Another limitation is the suitability of this style of cue in a practical or
applied scenario. While this initial experiment was only intended as a proofof-concept, a practical use for the finding would be ideal. The lack of any
stimulus onset asynchrony means that the cue used is display immediately
before the presentation of the target. At best, an improvement in reaction time
could only be the duration of the cue, 20ms in this case. An improved version
would introduce a SOA, allowing for a delay between the presentation of the
cue and the target stimulus.
39
The final limitation is the nature of the button apparatus itself. Being a
stark white, planar box, it bears little resemblance to control panels that might
be found in an airplane cockpit or control station. Such panels may include an
abundance of extra information and many more buttons and lights. Future
work could replicate these more complex interface designs and examine
whether the positive effects of subliminal cueing still exist.
Experiment 2 – Perception of Two New Subliminal Cues
Figure 12: The dome apparatus. The target is identifiable by the '0' symbol.
In order to address the limitation of the questionable subliminal nature of the
cues presented in Experiment 1, a second experiment was designed and
conducted. The major concern stemming from this limitation was that the
participants might just be not attending the cue, making it pre-attentive rather
than subliminal. The quantitative data that were collected in the perception
analysis test were not sufficient to completely eliminate this concern, so the
development of a more comprehensive test was required. This experiment
incorporated a version of the perception analysis test used in Experiment 1, but
with two adaptations of the cueing technique, and qualitative responses taken
from the participant on the visibility of the cue presentation. In this version of
the perception analysis new conditions were also introduced. As well as the
button box, a button dome was introduced (see Figure 12). The button box
40
included four buttons, of which three could be potential targets, while the
dome included sixteen buttons, of which fifteen could be potential targets. In
addition, two cue styles were tested, which allowed for the use of masking
techniques to lengthen the SOA (another limitation of Experiment 1), and
provide a more complex and practical subliminal cue.
The two cue styles, referred to as green and hash, were introduced to test
the effectiveness of a slightly more complex stimulus than that presented in
Experiment 1. The green cue used an annotation similar to the previous study.
The difference, however, is that after the 20ms presentation of the green cue,
identical green annotations were presented on the remaining buttons for
200ms, representing a backward masking and a modest SOA.
Figure 13: The hash cue style.
The hash cue was a modified green cue with a hash pattern, see Figure 13.
This was introduced in order to test the viability of backward and forward
masking to conceal the subliminal cue. Moreover, the hash cue more closely
resembles the environment of a worker at a control station. The control
station may feature many lights and flashing stimuli, not just one style of cue,
such is the case with the green cue. The hash pattern annotation was displayed
on all buttons for 60ms, acting as a forward mask. Then, for 20ms, the ‘0’
symbol would appear on one of the hash labels, cueing to that button. During
this time, the remaining buttons would show the plain hash label. Following
41
this, a backward mask was shown for another 60ms, represented by all buttons
showing the plain hash label. Then, all buttons showed the blank green label
for a 200ms SOA before the ‘0’ identified one of the buttons as a target.
The hypotheses for Experiment 2 were as follows:
[Hypothesis H1] Participants would be able to detect the cues less than
50% of the time, as reported by their verbal response.
[Hypothesis H2] Participants would be able to detect the cues less than
50% of the time, as reported by their guess.
[Hypothesis H3] Participants would be able to detect the cues less than
50% of the time, irrespective of the cue label.
[Hypothesis H4] Participants would be able to detect the cues less than
50% of the time, irrespective of the button apparatus.
Method
One NEC NP510W projector with 1280x800 resolution and 50hz refresh rate
was used to display cue on the box. Two of the same model projectors were
used to display cue to their respective sides of the box. All projectors were
mounted overhead, angled towards the apparatus. The participant was greeted
by the researcher and given an overview of the task, which was to test the
visibility of two cueing styles. Participants were shown images of the
annotations they would see projected onto the button apparatuses. The
experimenter explained that they would see a series of flashes and that there
was a subliminal cue that would be displayed for a brief time. The exact
sequence of the flashes was explained to the participants. The experimenter
also explained that they were to specifically look for the subliminal cue and
respond “yes” or “no” as to whether they had seen the cue. Figure 14 and Figure
15 show the masking techniques that was used to conceal the subliminal cue.
The perception analysis test consisted of 8 blocks of 10 button presses on
each of the box and dome apparatuses. The first four blocks used the green
cueing style, and the second four used the hash cueing style. On the box, only
the bottommost row of buttons was used, with the rightmost button
42
representing the trial start button, and the other three as potential targets for
the subliminal cue. On the dome, there were 16 buttons scattered across the
surface. The centremost on the bottom row was the trial start button. The
remaining 15 were potential targets for the subliminal cue
Figure 14: Masking techniques for the green cue style. The scale shows the display for each stage
of the mask and cue. The green cue features a backward mask.
Pressing the trial start button, as identified by a red annotation, started
each trial. After a delay of 1000ms on the box and 1500ms on the dome, the cue
would be displayed. The delay was longer on the dome to allow time for the
participant’s hand to be moved outside of the projection area before the
presentation of the next cue. After the cue was displayed, all the target buttons
were annotated with a blank, green label. The participant was then required to
verbally acknowledge whether or not they saw the subliminal cue by stating
“yes” or “no.” The researcher then recorded this response. Then, irrespective of
the response, the participant was asked to guess where the cue appeared by
pressing the button on which they believed the subliminal cue to have been
displayed. The annotations were then all removed and the next trial began.
This was repeated for 10 trials. Between each block of 10 presses, the
43
participant was required to press the start trial button, identified by a red
annotation.
Figure 15: Masking technique for the hash cue style. The scale shows the display duration for
each stage of the mask and cue. The hash cue features a forward and backward mask.
The subliminal cues were divided randomly amongst the potential target
buttons. In 80% of the trials there was a subliminal cue. The remaining 20%
acted as catch trials. Thus, the response and guess combinations gave the
following possibilities:

“yes” response with a correct guess,

“yes” response with an incorrect guess,

“no” response with a correct guess,

“no” response with an incorrect guess, and

the catch trial with a “yes” or “no” response.
44
In order to claim that the presented cues were truly subliminal, less than
40% of the responses would need to be “yes” with a correct guess. The 40% is
due to 20% of the trials being catches, and the definition of subliminal being
accepted as perceived less than 50% of the time.
Results
Ten participants were recruited from the general public and students of the
University of South Australia, and they all completed the experiment. Ages
ranged from 18 to 30 years (M = 26.40 years, SD = 3.75), and 5 were female. All
participants had normal or corrected to normal vision.
Table 1: Mean number of responses per cue type.
“Yes”
Correct
“Yes”
Incorrect
“No”
Correct
“No”
Incorrect
“Yes”
Catch
“No”
Catch
Box –
Green
M = 8.50,
SD = 4.06
M = 7.20,
SD = 5.43
M = 6.10,
SD = 4.12
M = 10.20,
SD = 2.78
M = 1.90,
SD = 1.60
M = 6.10,
SD = 1.60
Box –
Hash
M = 12.50,
SD = 8.07
M = 2.30,
SD = 1.91
M = 7.80,
SD = 5.41
M = 9.40,
SD = 6.50
M = 1.40,
SD = 1.71
M = 6.60,
SD = 1.71
Dome –
Green
M = 1.70,
SD = 1.77
M = 2.80,
SD = 2.97
M = 3.60,
SD = 3.44
M = 23.90,
SD = 5.09
M = 2.80,
SD = 2.04
M = 5.20,
SD = 2.04
Dome –
Hash
M = 1.10,
SD = 0.88
M = 6.30,
SD = 8.17
M = 1.20,
SD = 0.92
M = 23.40,
SD = 8.48
M = 1.70,
SD = 2.21
M = 6.30,
SD = 2.21
Overall, on the box apparatus and for both cue styles, “yes” responses were
given in less than 50% of the 80 presentations (M = 33.80, SD = 12.03).
Analysing this further, both the green cue (M = 16.60, SD = 5.76) and the hash
cue (M = 17.20, SD = 6.75) received “yes” responses in less than 50% of the
trials.
Table 1 shows the mean scores and standard deviations for the six possible
responses listed above, for each of the apparatus and label conditions. Each of
the cue styles was presented in 40 trials. Of those, eight were catch trials. For
each for the cueing styles, and on both apparatuses, the combination of a “yes”
45
response and a correct guess occurred in less than 50% of the trials with a
subliminal cue.
Discussion
Experiment 1 was conducted primarily to provide a more definitive
groundwork for referring to the cueing style as subliminal. The methodology
was designed to give not only a quantitative measure of a correct or incorrect
button press guess, but also to gain a qualitative response to each trial; a
simple “yes, I saw it”, or “no, I did not see it.” These two forms of response
coupled with the explicit instructions given to participants to only search for
and attend to the subliminal cue serves as a confident basis for claiming the
cues are imperceptible rather than unattended.
The response with the guess combination was intended to eliminate the
element of chance that could have influenced the analysis of the perception
test in Experiment 1. The null hypothesis, that the cues would be detected in
greater than 50% of the trials, would only be true if the “yes” with a correct
guess case occurred for over 50% of the trials. Each cue type, on each
apparatus, was presented 32 trials, once subtracted the catch trials. The mean
scores in
Table 1 shows that the “yes” and correct response occurred in fewer than 50%
of the trials, for both cue types and apparatuses. By definition, the green and
hash cue styles are subliminal.
Table 1 also reveals an interesting observation from the high occurrence of
the “no” and incorrect combination, particularly on the dome apparatus. The
argument could be made that this shows that rather than being subliminal, the
cues were altogether unseen and unprocessed by the participant. On the box,
the levels of the “no” and correct combination does counter this somewhat,
but the occurrence was less than “no” and “incorrect”. This does not apply to
the dome apparatus where the “no” and “correct” combination had far fewer
occurrences. A better explanation is that the participants were given explicit
instructions to look for the cue, and the press was a secondary task. The
motivation of the participant, therefore, was to perceive the cue, not to press
46
the correct button. Research in the psychological literature [16, 59, 84, 87] has
shown that subliminal suggestion does not have the power to change a
person’s motivation and influence them to perform an action they would not
otherwise. Rather, subliminal suggestion can increase the likelihood of
performing a task in congruence with their motivations. This relates to
Experiment 2 where if the participant did not see the cue, they responded “no,”
placed a guess, and was likely motivated to see the next presentation. The
experimenter had stressed that the task was to see the cue, and that the button
press was a guess.
Beyond testing to meet the definition of subliminal, Experiment 2 offered
an opportunity to trial two variations of cues, and trial a secondary apparatus;
the non-planer dome with 15 target buttons. This was designed as a precursor
to an extended version of Experiment 3, which was to feature a more complex
design. The results outlined in Table 1 showed that both cues met the
definition of being subliminal, on both apparatuses. Hypotheses H3 and H4
were shown to be true, and the cues were validated for use in future
experiments.
Experiment 3 - Effectiveness of Two New Subliminal Cues
An extension of Experiment 1’s main task was designed using the validated
subliminal cueing techniques of Experiment 2. The main objectives of this
experiment were to replicate the findings of the first, and also test whether the
effect continues with more complex cueing styles and on a more complex
button apparatus. This increase in task complexity is a stepping-stone from
the inherent simplicity of methodology presented in Experiment 1 to a cueing
technique and procedure that is more closely related to what could be seen in
a practical, non-laboratory environment. The hypotheses for this experiment
were as follows:
[Hypothesis H1] Helpful cues lead to faster reaction time, compared to
no cue.
47
[Hypothesis H2] Helpful cues lead to faster reaction time, compared to
unhelpful cues.
[Hypothesis H3] No cues lead to faster reaction time, compared to
unhelpful cues.
[Hypothesis H4] Helpful cues lead to fewer errors, compared to no cue.
[Hypothesis H5] Helpful cues lead to fewer errors, compared to
unhelpful cues.
[Hypothesis H6] No cues lead to fewer errors than unhelpful cues.
In order to encompass the two cueing types (hash – h and green - g) and the
two apparatuses (box - b and dome - d), each hypothesis will be encoded with
a suffix for the condition to which it is referring. For example, H1 for the box
with the green cue type would be H1_b_g.
Method
One NEC NP510W projector with 1280x800 resolution and 50hz refresh rate
was used to display cue on the box. Two of the same model projectors were
used to display cue to their respective sides of the box. All projectors were
mounted overhead, angled towards the apparatus. Participants completed 24
blocks of 12 button presses on the box button apparatus and then the dome.
The button task was performed with only one hand, with the participant
choosing whichever they preferred. The 24 blocks were equally divided into
the helpful, unhelpful, and no cue conditions, as described in Experiment 1.
These were further divided into the two cueing techniques described in
Experiment 2; green and hash. On each apparatus there were 4 blocks testing
the helpful, unhelpful, and no cue conditions with each of the cue styles. The
24 blocks were arranged pseudo-randomly.
On the button box only the four bottommost were used. Three were used
for target buttons, the other for the origin button that began each trial. Two
configurations were provided, depending on the hand preference of the
participant. The left-handed configuration used the leftmost button as the
origin, and the right-handed used the rightmost. Each button was assigned an
48
equal number of targets, four each, pseudo-randomly dispersed in each 12-trial
block.
Figure 16: Green cue styles in Experiment 3. Helpful cue with forward masking and 200ms SOA
a). Unhelpful cue with forward masking and 200ms SOA b).
On the dome 16 buttons were used. 15 were used for target buttons, and
the origin button was the central button on the bottommost row (see Figure
12). The configuration was the same, irrespective of the hand used to press the
buttons. The targets were pseudo-randomly allocated, but the overall hand
travel length was calculated to ensure equivalent task length for each
participant. To do this 12 of the 15 buttons were pseudo-randomly chosen
100,000 times. The total travel distance from the origin button to the targets
was summed, and an average calculated. Blocks of 12 trials generated for the
task were then required to be in the range of the average to the average + 10%.
The choice of 12 buttons per blocks was chosen to maintain consistency with
the box, on which the 12 could equally be divided between the three buttons,
and also to reduce the time taken to complete the task. Furthermore,
49
generating approximately equal pseudo-random paths removed any ability of
the participant to deduce the upcoming target.
Figure 17: Hash cue styles in Experiment 3. Helpful cue with forward and backward masking, and
200ms SOA a). Unhelpful cue with forward and backward masking, and 200ms SOA b).
The timing of the subliminal cue and target presentation differed slightly
for the green and hash cueing styles. For the green cue, after pressing the
origin button, a delay of 1000ms occurred, with no annotations presented.
Then, if in the helpful or unhelpful cueing condition, a blank green annotation
appeared for 20ms on one button, representing the subliminal cue. In the nocue condition, no annotations were displayed. Following this, a backward
mask consisting of blank green annotations was presented on all potential
target buttons (see Figure 14). This continued for 200ms, representing a period
50
of SOA. Then, the target ‘0’ symbol was displayed on the target button (see
Figure 16).
For the hash cue, after pressing the origin button, a delay of 1000ms
occurred, during which blank green annotations were display on all potential
target buttons. Then, a forward mask consisting of hash labels on all potential
target buttons was displayed for 60ms. After this, if in the helpful or unhelpful
cueing condition, a subliminal cue consisting of a ‘0’ symbol was presented for
20ms on one button, representing the subliminal cue. In the no-cue condition,
all potential targets buttons continued to show plain hash labels. Following
this, a backward mask consisting of blank hash labels was presented for 60ms
(see Figure 15). Finally, green labels were shown on all potential targets
buttons for a 200ms SOA before the target ‘0’ symbol appeared on one of the
buttons (see Figure 17).
To begin the trial, a participant was seated at the button box
approximately 30cm away, with their body centred to the middle buttons. A
practice version of the test was run, involving seven trials of each the green
and hash labels with no subliminal cue. Participants were told to react to the
target stimulus as quickly as possible by pressing the corresponding button.
When ready, the participant began the task. After every three blocks, a 30
second rest was enforced by the system. After completion of the 24 trials, the
participant took a longer, five minute break before repeating the task on the
dome.
Results
Sixteen participants were recruited from the general public and students of the
University of South Australia and completed the experiment. Ages ranged
from 18 to 58 years (M = 29.81 years, SD = 9.23), and seven were female. All
participants had normal or corrected to normal vision. The presentation of the
results will be divided into two parts: box and dome. There are no comparisons
made between the two button apparatuses. The investigation is interested in
determining if the subliminal cues work under different physical conditions
and not to determine the optimal physical condition to apply them.
51
Box
The overall mean reaction time on the box for all cues conditions was
574.73ms (SD = 93.85). Overall means for the green and hash reaction times
were 582.16ms (SD = 96.79) and 567.30 (SD = 96.18), respectively. Overall
mean reaction times for the green cue for the three cue conditions are as
follows:

helpful was 579.92ms (SD = 100.22),

unhelpful was 581.66ms (SD = 114.15), and

no-cue was 584.90 (SD = 113.09).
Overall mean reaction times for the hash label in the three cue conditions are
as follows:

helpful was 550.49ms (SD = 94.34),

unhelpful was 583.02ms (SD = 111.78), and

no cue was 568.39 (SD = 134.88).
A four (block) by three (cue type) repeated measures ANOVA was
conducted for the green cue to compare reactions times across the three
cueing conditions: helpful, unhelpful and no-cue. Mauchly’s test of sphericity
indicated that sphericity was not violated. There was no significant main effect
found for the cueing condition, F(2, 30) = 0.303, p > 0.05. Figure 18 shows
reactions times per cue type over each block. An improvement in reaction
time across the blocks is observable, but this effect was not significant.
A second four (block) by three (cue type) repeated measures ANOVA was
conducted for the hash label, measuring the same conditions. Mauchly’s test
of sphericity indicated that sphericity was violated χ2(5) = 106.59, p = > 0.01 so
Greenhouse-Geisser estimates of sphericity were used (ε = 0.745 for the cueing
effect). The results show that there was a significant effect of the cue condition
on the reaction time F(2.24, 426.81) = 4.77, p = < 0.01. The helpful cue was
found to be significantly faster across all trials than the unhelpful cue (p <
0.01). It was also significantly faster than no cue, but only in the final three
blocks (p = 0.05). Overall, the unhelpful cue was not found to lead to
52
significantly slower reaction times than the no cue condition. It was, however,
found to lead to significantly slower reaction times in the final three blocks (p
< 0.05). Figure 19 gives a graphical representation of the reaction times for
each cue type.
Figure 18: Mean reaction times for the green cue on the box. No significant effect of cue type on
reaction time was found. There was no significant change of reaction time over the block,
despite an observable improvement.
Figure 19: Mean reaction times for the hash cue on the box. Reaction times were faster for
helpful cue across all trials when compared to unhelpful cue. This effect was consistent across
blocks.
53
Dome
The overall mean reaction time on the dome was 847.70ms (SD = 119.31).
Overall means for the green and hash reaction times were 835.81ms (SD =
122.24) and 859.58ms (SD = 136.25), respectively. Overall mean reaction times
for the green cue in the three cue conditions are as follows:

helpful was 835.12ms (SD = 168.94),

unhelpful was 835.91ms (SD = 172.79), and

no-cue was 836.37ms (SD = 131.05).
Overall mean reaction times for the hash label in the three cue conditions are
as follows:

helpful was 863.45ms (SD = 168.98),

unhelpful was 857.65ms (SD = 171.49), and

no-cue was 857.64ms (SD = 190.83).
A four (block) by three (cue type) repeated measures ANOVA was
conducted for the green cue to compare reactions times across the three
cueing conditions: helpful, unhelpful, and no-cue. Mauchly’s test of sphericity
indicated that sphericity was not violated. There was no significant main effect
found for the cueing condition F(2, 382) = 0.005, p > 0.05 (see Figure 20).
A second four (block) by three (cue type) repeated measures ANOVA was
conducted for the hash label, measuring the same conditions. Mauchly’s test
of sphericity indicated that sphericity was violated χ2(5) = 14.05, p = 0.15 so
Greenhouse-Geisser estimates of sphericity were used (ε = 0.958 for the cueing
effect). There was no significant main effect found for the cueing condition
F(1.98, 378.29) = .111, p > 0.05 (see Figure 21).
54
Figure 20: Mean reaction times for the green cue on the dome. No significant effect of cue type
on reaction time was found. There was no significant change of reaction time over the block,
despite an observable improvement.
Figure 21: Mean reaction times for the hash cue on the dome. No significant effect of cue type on
reaction time was found. There was no significant change of reaction time over the block,
despite an observable improvement.
Errors
Examination of response errors, to determine any possible trade-off between
response time and response accuracy, showed a very small error rate across the
55
trials of on the box 0.85%, and 0.82% on the dome. In order to identify if there
was an interaction between errors and the cue style further analyses were
performed. A series of ANOVA tests were conducted for the two label types on
both apparatuses. These compared the error rates across the three cueing
conditions: helpful, unhelpful, and no-cue.
Box
A four (block) by three (cue type) repeated measures ANOVA was
conducted for the green cue on the box. Mauchly’s test of sphericity indicated
that sphericity was violated χ2(5) = 6.48, p < 0.001 so Greenhouse-Geisser
estimates of sphericity were used (ε = 0.686 for the cueing effect). The results
showed no significant effect of errors on the cueing condition F(1.92, 382) =
0.362, p = 0.689.
A second four (block) by three (cue type) repeated measures ANOVA was
conducted for the hash label on the box. Mauchly’s test of sphericity indicated
that sphericity was violated χ2(5) = 126.24, p < 0.001 so Greenhouse-Geisser
estimates of sphericity were used (ε = 0.689 for the cueing effect). The results
showed no significant effect of errors on the cueing condition F(1.53, 292.01) =
1.30, p = 0.269.
Dome
A third four (block) by three (cue type) repeated measures ANOVA was
conducted for the green cue on the dome. Mauchly’s test of sphericity
indicated that sphericity was violated χ2(5) = 60.73, p < 0.001 so GreenhouseGeisser estimates of sphericity were used (ε = 0.817 for the cueing effect). The
results showed no significant effect of errors on the cueing condition F(1.83,
349.14) = 0.92, p = 0.39.
A final four (block) by three (cue type) repeated measures ANOVA was
conducted for the hash label on the dome. Mauchly’s test of sphericity
indicated that sphericity was violated χ2(5) = 220.14, p < 0.001 so GreenhouseGeisser estimates of sphericity were used (ε = 0.616 for the cueing effect). The
56
results showed no significant effect of errors on the cueing condition F(1.44,
274.50) = 0.1.83, p = 0.17.
Discussion
Experiment 3 used the validated subliminal cue techniques from Experiment 2
to determine the effectiveness of introducing a delay between the presentation
of the cue and the target stimulus, and also the effect of scale the methodology
from Experiment 1 to a non-planar apparatus with more buttons. The results
showed that:

the methodology of Experiment 1 can be replicated on the box, with the
hash cue style impacting reaction time on a procedural task,

the green cue style did not have any effect on reaction time when a
delay was introduced,

error rates were minimal and not impacted by the subliminal cues, and

the methodology of Experiment 1 cannot be naïvely scaled to a larger,
more complex apparatus without a redesign.
The results showed H1_b_h to be true, with the hash cue significantly
improving participant reaction time, compared to the unhelpful cue. This was
also consistent across the four blocks, as shown by Figure 19. Figure 19 also
shows a general trend in support of H2_b_h, but this effect was only
significant in the final three blocks. This is likely due to learning effects in the
first block. H1_b_g, however, was shown to be false, with erratic reaction times
across all blocks, perhaps indicating that the cue was not being registered by
the participants. It is possible that the backward mask and SOA led to the
failure to replicate the reaction time improvement of Experiment 1 with the
green cue. In Experiment 1, the cue was followed by the target, representing an
immediate shift of stimulus. Conversely, the green cue in this experiment was
followed with 200ms of the presentation of identical green cues. The lack of a
visual difference may have overridden any effect of the temporal prime.
The lack of any effect on the dome is most likely due to the naïve
approach used to scale the established methodology on the box to a larger,
57
more complex design. When seated in front of the dome apparatus, only a
small portion of the potential buttons is within the foveal vision, with many
buttons on the left and right sides falling outside the field of view. Many
authors in the psychological literature have shown positive effects of
peripheral subliminal cueing [1, 2, 30, 47, 60], so it is likely that the subliminal
cue were not within the participants’ field of view. This assumption is also
supported by the findings of Experiment 2, which showed a high number of
“no” and incorrect response combinations.
In order to scale subliminal to larger, more complex devices than the box
apparatus, more research must be done. Instead of using the same cues as on
the box that appeared directly on the target button, the subliminal cue could
direct to the region of the upcoming target. Watanabe, Nanez, and Sasaki [94]
used visual noise dots to invisibly indicate motion. While the motion was
consciously imperceptible, the authors found that the directional suggestion
led to increased performance. This form of subliminal cueing does not directly
identify an upcoming target, but the gaze orientation should improve reaction
time on a larger device such as the dome. Alternatively, a more overt cue, such
as an arrow, could be presented subliminally, masked in a similar way to the
hash cue. This strategy has also been found to be effective in subliminally
directing attention [58].
An examination of errors on both apparatuses revealed that the error rate
was extremely low, less than 1% for each. This is likely due to the very simple
nature of the task. Consequently, hypotheses 4 through 6 were shown to be
false. Although unhelpful cues may have introduced an element of
misdirection, the task only required that the participants push the marked
button. Another version of the task could involve all buttons being marked
along with the target symbol, albeit with a different stimulus. If the target
were more difficult to identify, the misdirection may elicit higher error rates.
Overall, Experiment 3 showed that more work is required in order to scale
subliminal cueing in SAR from a simple, planar surface to a larger, more
complex surface. It was shown that masking does allow for an alternative cue
58
to that used in Experiment 1 to be effective. The hash cue was effective with
the introduction of a SOA.
General Discussion
The results of the three experiments showed that subliminal cueing is
achievable with SAR display techniques. Experiment 1, with the basic green
cue and target mask showed a significant reaction time improvement over
both the unhelpful and no cue conditions. The hash cue then invoked a faster
reaction time versus the unhelpful cue, and a general trend was seen against
the no cue condition. It is clear that by definition, the cues used meet the
definitions required to be called subliminal, and further, that they are having
an impact on reaction times.
That a positive effect of the hash cue on the box was found has
encouraging implications for continued subliminal cue research in SAR.
Experiment 3 introduced a period of SOA. This was done to improve
practicability of the cue for use in SAR applications. The 200ms SOA used
shows that the cue can be shifted away from the targets, allowing for more
time between the two presentations. In an applied context, this would
facilitate priming to a particular action or area prior to the action needing to
occur. This differs from the findings presented by Experiment 1 where the
potential benefit of the prime was only the duration of the cue itself (20ms).
Some psychological literature suggests the efficacy of such a delay has an
upper limit, known as inhibition of return [32, 60], but effects of subliminal
priming with an SOA as large as 1000ms have been found [18, 29]. The ability
to prime a user with a modest delay gives subliminal cueing with SAR more
real-world applicability.
The next stage for subliminal cueing in SAR is to develop a technique that
enables cueing to be taken from a simple surface where all interactions are
done within a user’s field of view and scale it to a more complex and nonplanar surface. The experimentation with the dome apparatus showed that the
method is not simply to use the same methodology, but rather directional cues
may be more effective. Also, the cue styles, hash and green, and also the
59
masks, were formed from the same style of green label (see Figure 9). A future
experiment could investigate the effects of a noisy background. This noise may
be achieved with a procedural texture, or may be an array of flashing physical
and projected lights. These experiments served as a proof of concept and have,
for the first time, shown that a basic form of subliminal cueing can be
implemented
in
SAR
and
improve
60
the
reaction
time
of
a
user.
5
Conclusion
This thesis has explored the presentation of SAR information from a cognitive
psychological viewpoint and used the phenomenon of subliminal suggestion
to develop an alternative cueing technique. Throughout the investigation, a
number of contributions to the body of research in SAR have been made and
are as follows:

Subliminal cueing is possible with consumer grade, non-high-speed
projectors.

Backward and forward masking techniques are able to conceal a cue in
at least 50% of the presentation cases, indicating that it is subliminal.

Subliminal cueing with SAR can increase the reaction time of a user
performing a procedural task.
This thesis represents the first exploration into the implementation of
subliminal priming as an interface aid in SAR. The development process went
through a number of iterations, beginning with an overview of cognitive
psychology theory and how subliminal cueing might be of benefit to interface
design in SAR. Taking a multidisciplinary approach to thinking about the
improvement of SAR information presentation meant that alternative methods
were considered that might not have been otherwise.
Experiment 1 focused the research questions centred on determining
whether it was possible to achieve a subliminal effect with ordinary projectors,
with only a 50hz refresh rate. Finding a positive effect of helpful subliminal
61
cues on reaction time provided the first clue that subliminal cues could be
used in time-critical and stressful environments, but led to more questions
regarding whether the cues were actually subliminal. Experiment 2 provided
the necessary evidence that, by definition, the cues were subliminal, and that
effect on reaction time deserved further investigation. Experiment 3 trialled
two cue styles, scaling the potential targets five-fold onto a more complex
apparatus. The methodology presented in this thesis reached its scaling limit
and requires further study.
Future work into subliminal cueing should address some of the hardware
and methodological limitations highlighted by this thesis. A 50hz refresh rate
is not fast enough for individualised perception thresholds to be implemented,
meaning that a 20ms cue is a “one size fits all” approach. Although the aim of
this thesis was to explore the capabilities of consumer-grade technology,
further research should first analyse the effectiveness of shorter cue lengths
with faster projectors. This could allow for a number of cue styles that do not
require any masking, courtesy of the speed at which they are presented. In
addition to trialling alternative cue timing, it would be beneficial for the
application of subliminal cueing to study the limits of SOA in SAR. If
subliminal cues are to serve a practical purpose, a longer delay between the
presentation of the cue and the primed task is beneficial.
In order to move subliminal cueing from controlled, laboratory settings
into a practical and complex real-world environment, directional and attention
orienting cues could be investigated. This thesis has identified that subliminal
cueing is possible with SAR projector technology, and that it can have an
impact on user performance when implemented as part of a system’s interface.
The research described in this thesis has provided the groundwork for the
development of a more robust subliminal cueing knowledgebase in SAR.
62
References
1.
Ansorge, U. and Heumann, M. Shifts of visuospatial attention to
invisible (metacontrast-masked) singletons: Clues from reaction times
and event-related potentials. Advances in Cognitive Psychology, 2 (1). 6176.
2.
Ansorge, U., Horstmann, G. and Worschech, F. Attentional capture by
masked colour singletons. Vision Research, 50 (19). 2015-2027.
3.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. and MacIntyre,
B. Recent advances in augmented reality. Computer Graphics and
Applications, IEEE, 21 (6). 34-47.
4.
Azuma, R. and Bishop, G., Improving static and dynamic registration in
an optical see-through HMD. in Proceedings of the 21st annual
conference on Computer graphics and interactive techniques, (1994),
ACM, 197-204.
5.
Azuma, R.T. A survey of augmented reality. Presence, 6 (4). 355-385.
6.
Bai, Z., Blackwell, A.F. and Coulouris, G., Through the looking glass:
Pretend play for children with autism. in Mixed and Augmented Reality
(ISMAR), 2013 IEEE International Symposium on, (2013), IEEE, 49-58.
7.
Bayer, U.C., Achtziger, A., Gollwitzer, P.M. and Moskowitz, G.B.
Responding to subliminal cues: Do if-then plans facilitate action
preparation and initiation without conscious intent? Social Cognition,
27 (2). 183-201.
8.
Bimber, O. and Raskar, R. Spatial augmented reality. Peters, 2005.
63
9.
Broecker, M., Thomas, B.H. and Smith, R.T., Adapting ray tracing to
Spatial Augmented Reality. in Mixed and Augmented Reality (ISMAR),
2013 IEEE International Symposium on, (2013), IEEE, 1-6.
10.
Bruder, G., Wieland, P., Bolte, B., Lappe, M. and Steinicke, F., Going
with the flow: Modifying self-motion perception with computermediated optic flow. in Mixed and Augmented Reality (ISMAR), 2013
IEEE International Symposium on, (2013), IEEE, 67-74.
11.
Brunken, R., Plass, J.L. and Leutner, D. Direct Measurement of
Cognitive Load in Multimedia Learning. Educational Psychologist, 38
(1). 53-61.
12.
Buxton, B. and Fitzmaurice, G.W. HMDs, caves & chameleon: a humancentric analysis of interaction in virtual space. ACM SIGGRAPH
Computer Graphics, 32 (4). 69-74.
13.
Cakmakci, O. and Rolland, J. Head-worn displays: a review. Display
Technology, Journal of, 2 (3). 199-216.
14.
Carlin, A.S., Hoffman, H.G. and Weghorst, S. Virtual reality and tactile
augmentation in the treatment of spider phobia: a case report.
Behaviour research and therapy, 35 (2). 153-158.
15.
Carroll, J.M. Designing interaction: Psychology at the human-computer
interface. CUP Archive, 1991.
16.
Chalfoun, P. and Frasson, C. Subliminal Cues While Teaching: HCI
Technique for Enhanced Learning. Advances in Human - Computer
Interaction.
17.
Cheon, J.M. The effects of metaphorical interface on germane cognitive
load in Web-based instruction. Educational Technology Research &
Development, 60 (3). 399-420.
18.
Chou, W.-L. and Yeh, S.-L. Subliminal spatial cues capture attention
and strengthen between-object link. Consciousness and Cognition, 20
(4). 1265-1271.
19.
Creagh, H., CAVE automatic virtual environment. in Electrical
Insulation Conference and Electrical Manufacturing &amp; Coil Winding
Technology Conference, 2003. Proceedings, (2003), IEEE, 499-504.
64
20.
Cruz-Neira, C., Sandin, D.J. and DeFanti, T.A., Surround-screen
projection-based virtual reality: the design and implementation of the
CAVE. in Proceedings of the 20th annual conference on Computer
graphics and interactive techniques, (1993), ACM, 135-142.
21.
Cruz-Neira, C., Sandin, D.J., DeFanti, T.A., Kenyon, R.V. and Hart, J.C.
The CAVE: audio visual experience automatic virtual environment.
Communications of the ACM, 35 (6). 64-72.
22.
Dehaene, S., Changeux, J.-P., Naccache, L., Sackur, J. and Sergent, C.
Conscious, preconscious, and subliminal processing: a testable
taxonomy. Trends in Cognitive Sciences, 10 (5). 204-211.
23.
Dehaene, S., Naccache, L., Le Clec'H, G., Koechlin, E., Mueller, M.,
Dehaene-Lambertz, G., van de Moortele, P.-F. and Le Bihan, D. Imaging
unconscious semantic priming. Nature, 395 (6702). 597-600.
24.
Dijksterhuis, A., Preston, J., Wegner, D.M. and Aarts, H. Effects of
subliminal priming of self and God on self-attribution of authorship for
events. Journal of experimental social psychology, 44 (1). 2-9.
25.
Dix, A. Human-computer interaction. Springer, 2009.
26.
Durmer, J.S. and Dinges, D.F., Neurocognitive consequences of sleep
deprivation. in Seminars in neurology, (2005), 117-129.
27.
Feiner, S., MacIntyre, B., Höllerer, T. and Webster, A. A touring
machine: Prototyping 3D mobile augmented reality systems for
exploring the urban environment. Personal Technologies, 1 (4). 208-217.
28.
Forster, K.I. and Davis, C. Repetition priming and frequency
attenuation in lexical access. Journal of experimental psychology:
Learning, Memory, and Cognition, 10 (4). 680.
29.
Fuchs, I., Theeuwes, J. and Ansorge, U. Exogenous attentional capture
by subliminal abrupt-onset cues: Evidence from contrast-polarity
independent cueing effects. Journal of Experimental Psychology: Human
Perception and Performance, 39 (4). 974-988.
30.
Gayet, S., Stigchel, S. and Paffen, C.L.E. Seeing is believing: Utilization
of subliminal symbols requires a visible relevant context. Attention,
Perception, & Psychophysics.
65
31.
Goradia, I., Doshi, J. and Kurup, L. A Review Paper on Oculus Rift &
Project Morpheus.
32.
Greenwald, A.G., Draine, S.C. and Abrams, R.L. Three cognitive markers
of unconscious semantic activation. Science, 273 (5282). 1699-1702.
33.
Guthrie, G. and Wiener, M. Subliminal perception or perception of
partial cue with pictorial stimuli. Journal of Personality and Social
Psychology, 3 (6). 619-628.
34.
Haniff, D.J. and Baber, C., User evaluation of augmented reality
systems. in Information Visualization, 2003. IV 2003. Proceedings.
Seventh International Conference on, (2003), IEEE, 505-511.
35.
Harrison, Y. and Horne, J.A. The impact of sleep deprivation on
decision making: a review. Journal of Experimental Psychology: Applied,
6 (3). 236.
36.
Henderson, S.J. and Feiner, S., Evaluating the benefits of augmented
reality for task localization in maintenance of an armored personnel
carrier turret. in Mixed and Augmented Reality, 2009. ISMAR 2009. 8th
IEEE International Symposium on, (2009), IEEE, 135-144.
37.
Henderson, S.J. and Feiner, S.K., Augmented reality in the psychomotor
phase of a procedural task. in Mixed and Augmented Reality (ISMAR),
2011 10th IEEE International Symposium on, (2011), IEEE, 191-200.
38.
Hoffman, H.G., Patterson, D.R., Carrougher, G.J. and Sharar, S.R.
Effectiveness of virtual reality-based pain control with multiple
treatments. The Clinical journal of pain, 17 (3). 229-235.
39.
Höllerer, T., Feiner, S., Terauchi, T., Rashid, G. and Hallaway, D.
Exploring MARS: developing indoor and outdoor user interfaces to a
mobile augmented reality system. Computers & Graphics, 23 (6). 779785.
40.
Howard, M.E., Jackson, M.L., Berlowitz, D., O’Donoghue, F., Swann, P.,
Westlake, J., Wilkinson, V. and Pierce, R.J. Specific sleepiness
symptoms are indicators of performance impairment during sleep
deprivation. Accident Analysis & Prevention, 62. 1-8.
66
41.
Johnson, D.A. and Rubin, S. Effectiveness of interactive computer-based
instruction: A review of studies published between 1995 and 2007.
Journal of Organizational Behavior Management, 31 (1). 55-94.
42.
Jones, B.R., Sodhi, R., Campbell, R.H., Garnett, G. and Bailey, B.P., Build
your world and play in it: Interacting with surface particles on complex
objects. in Mixed and Augmented Reality (ISMAR), 2010 9th IEEE
International Symposium on, (2010), IEEE, 165-174.
43.
Jraidi, I. and Frasson, C., Subliminally enhancing self-esteem: impact on
learner performance and affective state. in Intelligent Tutoring Systems,
(2010), Springer, 11-20.
44.
Karremans, J.C., Stroebe, W. and Claus, J. Beyond Vicary’s fantasies: The
impact of subliminal priming and brand choice. Journal of Experimental
Social Psychology, 42 (6). 792-798.
45.
Kester, L.P.A.v.M. and eumlnboer, J.J.G. The management of cognitive
load during complex cognitive skill acquisition by means of computersimulated problem solving. British Journal of Educational Psychology, 75
(1). 71-85.
46.
Kurz, D., Hantsch, F., Grobe, M., Schiewe, A. and Bimber, O., Laser
pointer tracking in projector-augmented architectural environments. in
Proceedings of the 2007 6th IEEE and ACM International Symposium on
Mixed and Augmented Reality, (2007), IEEE Computer Society, 1-8.
47.
Lambert, A., Naikar, N., McLachlan, K. and Aitken, V. A new
component
of
visual
orienting:
Implicit
effects
of
peripheral
information and subthreshold cues on covert attention. Journal of
Experimental Psychology: Human Perception and Performance, 25 (2).
321.
48.
Langlotz, T., Mooslechner, S., Zollmann, S., Degendorfer, C., Reitmayr,
G. and Schmalstieg, D. Sketching up the world: in situ authoring for
mobile augmented reality. Personal and ubiquitous computing, 16 (6).
623-630.
49.
Laver, K., George, S., Thomas, S., Deutsch, J.E. and Crotty, M. Virtual
reality for stroke rehabilitation. Stroke, 43 (2). e20-e21.
67
50.
Lim, J. and Dinges, D.F. A meta-analysis of the impact of short-term
sleep deprivation on cognitive variables. Psychological bulletin, 136 (3).
375.
51.
Lin, J.-W., Duh, H.B.-L., Parker, D.E., Abi-Rached, H. and Furness, T.A.,
Effects of field of view on presence, enjoyment, memory, and simulator
sickness in a virtual environment. in Virtual Reality, 2002. Proceedings.
IEEE, (2002), IEEE, 164-171.
52.
Lu, W., Feng, D., Feiner, S., Zhao, Q. and Duh, H.B.-L., Subtle cueing
for visual search in head-tracked head worn displays. in Mixed and
Augmented Reality (ISMAR), 2013 IEEE International Symposium on,
(2013), IEEE, 271-272.
53.
Marcel, A.J. Conscious and unconscious perception: Experiments on
visual masking and word recognition. Cognitive Psychology, 15 (2). 197237.
54.
Marner, M.R., Haren, S., Gardiner, M. and Thomas, B.H., Exploring
interactivity and augmented reality in theater: A case study of Half Real.
in Mixed and Augmented Reality (ISMAR-AMH), 2012 IEEE International
Symposium on, (2012), IEEE, 81-86.
55.
Marner, M.R., Irlitti, A. and Thomas, B.H., Improving procedural task
performance with Augmented Reality annotations. in Mixed and
Augmented Reality (ISMAR), 2013 IEEE International Symposium on,
(2013), IEEE, 39-48.
56.
Marner, M.R., Smith, R.T., Porter, S.R., Broecker, M.M., Close, B. and
Thomas, B.H. Large Scale Spatial Augmented Reality for Design and
Prototyping. Springer, 2011.
57.
Marner, M.R. and Thomas, B.H., Augmented foam sculpting for
capturing 3D models. in 3D User Interfaces (3DUI), 2010 IEEE
Symposium on, (2010), IEEE, 63-70.
58.
McCormick, P.A. Orienting attention without awareness. Journal of
Experimental Psychology: Human Perception and Performance, 23 (1).
168.
68
59.
Moore, T.E. Subliminal advertising: What you see is what you get. The
Journal of Marketing. 38-47.
60.
Mulckhuyse, M.D.J. Grabbing attention without knowing: Automatic
capture of attention by subliminal spatial cues. Visual Cognition, 15 (7).
779-788.
61.
Norman, D.A. Cognitive engineering. User centered system design. 31-61.
62.
Olsson, T., Kärkkäinen, T., Lagerstam, E. and Ventä-Olkkonen, L. User
evaluation of mobile augmented reality scenarios. Journal of Ambient
Intelligence and Smart Environments, 4 (1). 29-47.
63.
Opdyke, D., Williford, J.S. and North, M. Effectiveness of computergenerated (virtual reality) graded exposure in the treatment of
acrophobia. Am J Psychiatry, 1. 52.
64.
Osborne, M. Handle with Care: 2008 Adelaide Biennial and Adelaide
Festival Visual Arts Program. Art Monthly Australia (209). 5.
65.
Paas, F., Renkl, A. and Sweller, J. Cognitive Load Theory and
Instructional Design: Recent Developments. Educational Psychologist,
38 (1). 1-4.
66.
Paas, F., Tuovinen, J.E., Tabbers, H. and Van Gerven, P.W.M. Cognitive
Load Measurement as a Means to Advance Cognitive Load Theory.
Educational Psychologist, 38 (1). 63-71.
67.
Paas, F.G.W.C. and Van Merriënboer, J.J.G. The efficiency of
instructional conditions: An approach to combine mental effort and
performance measures. Human Factors: The Journal of the Human
Factors and Ergonomics Society, 35 (4). 737-743.
68.
Piper, B. and Ishii, H. CADcast: a method for projecting spatially
referenced procedural instructions, Tech. rep., MIT Media Lab, 2001.
69.
Porter, S.R., Marner, M.R., Smith, R.T., Zucco, J.E. and Thomas, B.H.,
Validating spatial augmented reality for interactive rapid prototyping.
in Mixed and Augmented Reality (ISMAR), 2010 9th IEEE International
Symposium on, (2010), IEEE, 265-266.
69
70.
Raskar, R. and Low, K.-L., Interacting with spatially augmented reality.
in Proceedings of the 1st international conference on Computer graphics,
virtual reality and visualisation, (2001), ACM, 101-108.
71.
Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L. and Fuchs, H., The
office of the future: A unified approach to image-based modeling and
spatially immersive displays. in Proceedings of the 25th annual
conference on Computer graphics and interactive techniques, (1998),
ACM, 179-188.
72.
Raskar, R., Welch, G. and Fuchs, H., Spatially augmented reality. in
First IEEE Workshop on Augmented Reality (IWAR’98), (1998), 11-20.
73.
Raskar, R., Welch, G., Low, K.-L. and Bandyopadhyay, D. Shader lamps:
Animating real objects with image-based illumination. in Rendering
Techniques 2001, Springer, 2001, 89-102.
74.
Riener, A., Reiner, M., Jeon, M. and Chalfoun, P., Methodical
approaches to prove the effects of subliminal perception in Ubiquitous
Computing Environments. in Proceedings of the 2012 ACM Conference
on Ubiquitous Computing, (2012), ACM, 1120-1121.
75.
Ritter, W. Benefits of subliminal feedback loops in human-computer
interaction. Advances in Human-Computer Interaction, 2011. 1.
76.
Robinson-Riegler, B. and Robinson-Riegler, G.L. Cognitive Psychology:
Applying The Science of the Mind. Pearson, USA, 2011.
77.
Rolland, J.P. and Fuchs, H. Optical versus video see-through headmounted displays in medical visualization. Presence: Teleoperators and
Virtual Environments, 9 (3). 287-309.
78.
Rolland, J.P., Holloway, R.L. and Fuchs, H., Comparison of optical and
video see-through, head-mounted displays. in Photonics for Industrial
Applications, (1995), International Society for Optics and Photonics,
293-307.
79.
Rosenthal, S., Kane, S.K., Wobbrock, J.O. and Avrahami, D.,
Augmenting on-screen instructions with micro-projected guides: when
it works, and when it fails. in Proceedings of the 12th ACM international
conference on Ubiquitous computing, (2010), ACM, 203-212.
70
80.
Schneider, W. and Shiffrin, R.M. Controlled and automatic human
information
processing:
I.
Detection,
search,
and
attention.
Psychological review, 84 (1). 1.
81.
Schultheis, H. and Jameson, A., Assessing cognitive load in adaptive
hypermedia systems: Physiological and behavioral methods. in Adaptive
hypermedia and adaptive web-based systems, (2004), Springer, 225-234.
82.
Schwerdtfeger, B., Pustka, D., Hofhauser, A. and Klinker, G., Using laser
projectors for augmented reality. in Proceedings of the 2008 ACM
symposium on Virtual reality software and technology, (2008), ACM,
134-137.
83.
Seo, B.-K., Lee, M.-H., Park, H., Park, J.-I. and Kim, Y.S., DirectProjected AR based interactive user interface for medical surgery. in
Artificial Reality and Telexistence, 17th International Conference on,
(2007), IEEE, 105-112.
84.
Sharp, H.C. Effect of subliminal cues on test results. Journal of Applied
Psychology, 43 (6). 369-371.
85.
Sharples, S., Cobb, S., Moody, A. and Wilson, J.R. Virtual reality
induced symptoms and effects (VRISE): Comparison of head mounted
display (HMD), desktop and projection display systems. Displays, 29
(2). 58-69.
86.
Sherman, W.R. and Craig, A.B. Understanding virtual reality: Interface,
application, and design. Elsevier, 2002.
87.
Strahan, E.J., Spencer, S.J. and Zanna, M.P. Subliminal priming and
persuasion: Striking while the iron is hot. Journal of Experimental Social
Psychology, 38 (6). 556.
88.
Suganuma, A., Ogata, Y., Shimada, A., Arita, D. and Taniguchi, R.-i.,
Billiard instruction system for beginners with a projector-camera
system. in Proceedings of the 2008 International Conference on Advances
in Computer Entertainment Technology, (2008), ACM, 3-8.
89.
Sweller, J., Van Merrienboer, J.J.G. and Paas, F.G.W.C. Cognitive
architecture and instructional design. Educational psychology review, 10
(3). 251-296.
71
90.
Thomas, B.H. A survey of visual, mixed, and augmented reality gaming.
Computers in Entertainment (CIE), 9 (3). 3.
91.
Toney, A., Mulley, B., Thomas, B.H. and Piekarski, W., Minimal social
weight user interactions for wearable computers in business suits. in
Wearable
Computers,
2002.(ISWC
2002).
Proceedings.
Sixth
International Symposium on, (2002), IEEE, 57-64.
92.
Wagner, D., Reitmayr, G., Mulloni, A., Drummond, T. and Schmalstieg,
D. Real-time detection and tracking for augmented reality on mobile
phones. Visualization and Computer Graphics, IEEE Transactions on, 16
(3). 355-368.
93.
Wallace, F.L., Flanery, J.M. and Knezek, G.A. The effect of subliminal
help presentations on learning a text editor. Information processing &
management, 27 (2). 211-218.
94.
Watanabe, T., Nanez, J.E. and Sasaki, Y. Perceptual learning without
perception. Nature, 413 (6858). 844.
95.
Weiss, P.L.T., Keshner, E.A. and Levin, M.F. Virtual Reality for Physical
and Motor Rehabilitation. Virtual Reality.
96.
Wetzstein, G. and Bimber, O., Radiometric Compensation through
Inverse Light Transport. in Pacific conference on computer graphics and
applications, (2007), 391-399.
97.
Wiener, M. and Schiller, P.H. Subliminal perception or perception of
partial cues. The Journal of Abnormal and Social Psychology, 61 (1). 124137.
98.
Williamson, A.M. and Feyer, A.-M. Moderate sleep deprivation
produces impairments in cognitive and motor performance equivalent
to legally prescribed levels of alcohol intoxication. Occupational and
environmental medicine, 57 (10). 649-655.
99.
Zaeh, M. and Vogl, W., Interactive laser-projection for programming
industrial robots. in Proceedings of the 5th IEEE and ACM International
Symposium on Mixed and Augmented Reality, (2006), IEEE Computer
Society, 125-128.
72
100.
Zhou, J., Lee, I., Thomas, B., Menassa, R., Farrant, A. and Sansome, A.
In-Situ
Support
for
Automotive
Manufacturing
Using
Augmented Reality. International Journal of Virtual Reality, 11 (1).
73
Spatial