PsychNology Journal, 2003
Volume 1, Number 3, 310 - 325
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions: the V-STORE.
G. Castelnuovo♣ 1-2, C. Lo Priore 3-4, D. Liccione 3, G. Cioffi 5
1
ATN-P LAB, Applied Technology for Neuro-Psychology Lab, Istituto Auxologico
Italiano,
Milan, ITALY
2
Facoltà di Psicologia, Università Cattolica del Sacro Cuore, Milan, ITALY
3
4
5
Presidio di Riabilitazione Extra-Ospedaliera “Paolo VI”
Casalnoceto AL, ITALY
Dipartimento di Psicologia Generale, Università di Padova
Padova, ITALY
Dipartimento di Psicologia Generale, Università degli Studi di Milano Bicocca
Milan, ITALY
ABSTRACT
Cognitive rehabilitation is represented by the application of a lot of procedures in order to
enhance development of skills and strategies necessary to overcome cognitive deficits.
Computer-based tools can be fruitfully used in the assessment and rehabilitation of
cognitive and executive functions. In particular Virtual Reality could play a key role in the
rehabilitation of psychological functions due to a creation of synthetic environments where it
is possible to carry on tasks very similar to the ones experienced in real contexts. After an
analysis of the main characteristics and open issues of the PC and VR based cognitive
rehabilitation, the major aim of this paper is to describe: a) the rationale for the use of Virtual
Reality in this field and b) some VR tools (V-Store, V-Tol, V-Wcst) used with a particular
category of dysfunction, the Dysexecutive Syndrome, typical of the patients with frontal lobe
injuries and other neurological diseases. In particular the description of V-Store is provided.
Keywords: Virtual Reality, Cognitive Rehabilitation, Dysexecutive Syndrome, Frontal Lobe
Syndrome, computer-based tools.
Received 1 January 2003; received in revised form 16 June 2003; accepted 20 June 2003.
1. Open issues about computer-based cognitive assessment and rehabilitation.
Cognitive rehabilitation is characterized by the application of a lot of procedures by a
different group of professionals such as neuropsychologists, occupational therapists,
♣
Corresponding Author:
Gianluca Castelnuovo;
E-mail: gianluca.castelnuovo@auxologico.it
310
PsychNology Journal, 2003
Volume 1, Number 3, 311 - 326
speech pathologists. These procedures aim at enhancing the development of skills and
strategies necessary to overcome cognitive deficits (Diller & Gordon, 1981; Ginarte
Arias, 2002; Gontkovsky, McDonald, Clark, & Ruwe, 2002; Mickey et al., 1998; Riva,
1998a, 1998b), overall with persons characterized by traumatic brain injuries.
Cognitive rehabilitation techniques have a long history since the ancient Greeks, but
only after the World War II they achieved a high level of spread and use in clinical
units. According to Gordon, one of the main reasons that could explain the great
expansion of cognitive rehabilitation in the latest years is represented by the fruitful
technological applications and tools: the potential of computer-based tools could
enhance the administration of cognitive rehabilitation (Gordon, Hibbard, & Kreutzer,
1989); (Mickey et al., 1998). Although the potential of computer-based techniques has
been underlined (Bracy, 1983); (Gontkovsky et al., 2002), controlled research
investigations of computerized interventions have not yet been conducted and
concluded (Gordon et al., 1989; Mickey et al., 1998).
Computer-based tools could be fruitfully used in the assessment of cognitive
functions too. In particular the Tower of London (TOL) Test (Shallice, 1982) has been
studied by Morris et al. (Ager, 1993) in order to obtain a computerized version of TOL,
using a touch-screen. The main advantage of this PC version was the possibility to
carry on a more detailed analysis of the performance due to the registration of all the
movements (and relative times) produced by patients. Also the Wisconsin Carding
Sorting Test (WCST) (Berg & Simple, 1948) has been studied in order to obtain
different computer-based versions (Grant & Berg, 1984; Nelson, 1976) such as the
Bexley-Maudsley Category Sorting Test (Acker, 1982). The advantages of these
computerized tests are many: to save time in the scoring step, to simplify all the
procedures, to create and show new integrates multi-media scenarios more similar to
reality than the traditional pen and pencil approach (Gourlay, Lun, & Liya, 2000).
About rehabilitation, the use of computer-based techniques is now one of the most
promising and discussed field of research. There are a lot of open issues related to
efficacy, ethics and ecological validity.
First of all the evaluation of the results of computerized rehabilitative techniques is
not always univocal due to methodological problems, such as the difficulty to plan
comparison between more groups of patients that can differ for a lot of variables very
hard to control (such as social, cultural and environmental stimuli, spontaneous
retrieval, efficacy of other contemporary treatments, different neurological, cognitive
311
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
and behavioural situation before the rehabilitation, etc.). Moreover many studies in
rehabilitation fields do not include control groups (Mazzucchi & Maravita, 1993).
Another open issue is represented by ethical problems related to the possibility to
experiment not yet validated rehabilitative protocols with patients that could receive
more benefits using traditional approaches.
Moreover another point of discussion is about the ecological validity of computerbased rehabilitation procedures (Gourlay, Lun, & Liya, 2000; Riva & Gamberini, 2000).
Are these techniques useful for patients’ everyday life? Is there a realistic and efficacy
transfer of knowledge between the tasks carried out in labs and the life outside the
hospital?
There also recognized advantages in the use of computerized techniques in the
rehabilitation. First of all many exercises made with a PC could be repeated as
homework without the presence of therapists. Secondary, modern software allow
“customized” programs that can be built according to single patient’s characteristics:
the computerized rehabilitative protocols and exercises are flexible and could be
adapted on user’s needs and features (Skilbeck, 1993).
According to Kurlycheck and Levin (Kurlycheck & Levin, 1987), also in our opinion
PC based rehabilitative programs have to follow the indications reported below:
1. Providing tasks of growing difficulty.
2. Selecting a level of difficulty that allows patients to obtain a high percentage of
success.
3. Controlling the accuracy and speed of responses.
4. Providing frequent feedback information in order to correct the wrong responses
and to confirm the right ones.
5. Reducing gradually the suggestions.
6. Stimulating motivation, continuity and enterprise in patients.
2.
The role of Virtual Reality in the cognitive rehabilitation
Among the different computer-based technologies, Virtual Reality plays a key role in
the assessment and rehabilitation of psychological functions (Cunningham & Krishack,
1999), (Broeren, Bjorkdahl, Pascher, & Rydmark, 2002; Campbell, 2002; Gourlay, Lun,
& Liya, 2000; Grealy & Heffernan, 2000; Jack et al., 2001; Merians et al., 2002; Piron,
312
G. Castelnuovo et al.
Cenni, Tonin, & Dam, 2001; Silva, 2002; Tarr & Warren, 2002; Zhang et al., 2001).
According to Tarr, Virtual reality (VR) has finally come of age for fruitful applications in
neuroscience, cognitive science and psychology. The VR attractive is due to
improvements in computer speed and quality of different tools (head-mounted displays,
wide-area tracking systems, etc.)(Tarr & Warren, 2002).
Moreover VR is considered as the most advanced evolution of the relationship
between man and computers (Vincelli, 2001). VR is different from other technologies
because it offers to users the chance to experience psychological states of immersion
and involvement (A. A. Rizzo, Wiederhold, & Buckwalter, 1998) due to the combination
of hardware and software tools with particular interactive interfaces (such as headmounted display and tracker). So one of the possible added values of Virtual Reality
(with respect to traditional electronic systems of representation and interaction) seems
to be the particular level of presence that subjects can experience in virtual
environments (VEs). With the present and future development of technologies,
synthetic environments will be able to re-create situations more and more similar to the
ones we experiment in everyday life, therefore we can easily imagine that the
possibility to feel presence in virtual environments will increase.
Virtual Reality is a technology used in many applications, from health care to arts,
from e-learning to entertainment. Focusing on the health care field, VR finds a role in
many areas such as psychotherapy (assessment and treatment of pathologies such as
social phobias, eating disorders, sexual disorders, depression, autism, etc.), cognitive
rehabilitation (about memory, attention, spatial abilities and superior cognitive
functions), motor rehabilitation (about paraplegia, parkinsonism and other disabilities).
In particular, in the field of rehabilitation, the possibility to use new technologies has
been studied (Gordon et al., 1989) and the potential of virtual reality based applications
has been recognized (Pugnetti, 1998; A. A. Rizzo & J. G. Buckwalter, 1997).
According to Rizzo et al. (A. A. Rizzo et al., 1998), it is necessary to carry on a
realistic cost/benefit analysis to evaluate which is the added value of VR in different
applications comparing with traditional approaches. A key question is does the
objective that is being pursued require the expense and complexity of a VE approach,
or can it be done as effectively and efficiently using simpler, less expensive, and
currently more available means? (A. A. Rizzo et al., 1998).
There are different issues to consider in order to evaluate the real costs and benefits
of Virtual Reality in mental health applications. One of these important issues, in order
to ensure high benefits in the use of VR, is represented by production of functional and
313
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
useful environments. Traditionally, as indicated by Waterworth, VR designers typically
aim at creating an engaging and convincing environment in which users feel present
(Waterworth & Waterworth, 2001). The focus for VR developers seems to be
“presence” and all the systems to improve it.
But the concept of presence is very complex because this psychological state is
characterized by many factors and so a key issue becomes the choice of the presence
factors that have to be considered in the use of VEs in mental health applications. Riva
(2000) notes that the substantial challenge for the designers and users of VR is how to
use immersive environments to support clinical practice (Riva, 2000). It becomes clear
that a VE built for entertainment has to be different from a one use in psychotherapy or
cognitive rehabilitation. So which are the core characteristics of a virtual environment in
order to ensure a kind of presence that is functional and useful for mental health
applications?
In Riva’s opinion two key characteristics of VR as a clinical oriented tool have to be
the perceptual illusion of non-mediation and the sense of community: in mental health
applications, reproducing physical aspects of virtual environments may be less
important than the possibility of interaction that a VE could allow. According to Riva, in
clinical oriented environments “the criterion of the validity of presence does not consist
of simply reproducing the conditions of physical presence but in constructing
environments in which actors may function in an ecologically valid way” (p. 356, Riva,
2000). Thus the level of presence, connected with the functional use of VEs in clinical
applications, depends also on the level of interaction and possible interactivity (Riva,
2000).
Table 1. Possible issues to consider in designing Virtual Environments
in clinical applications.
Key questions
Possible answers
Applications and
indications for VR designers
1. Are VEs useful,
effective and efficient
in clinical
Evaluation of possible
advantages and limits
Cost/benefit analysis
applications?
2. Do VEs
reproduce the
Development of VEs that
have to ensure only the level
(and quality) of presence
requested by each application.
Attention on graphics and
technical characteristics.
Development of VEs that
have to ensure realism and a
314
G. Castelnuovo et al.
physical and
perceptual
Focus on realism and
technical issues
level of presence as nonmediation and immersion.
characteristics of real
environments?
3. Do VEs allow
users to function in an
ecologically valid
way?
Attention on cultural and
social aspects.
Focus on interaction,
interactivity
Development of VEs that
have to ensure ecological
situations of interaction,
interactivity
Importance of
relationships and context
3. The case of executive functions
In the field of disabilities, the category of cognitive dysfunctions can be classified
making a distinction between the loss (partial or complete) of the basic instrumental
cognitive functions (such as attention, memory, language, visual-spatial abilities, etc.)
and the loss of executive functions (also called central or control functions). These are
generally referred to a group of behavioural skills that includes: the ability of planning a
sequence of actions, the ability of maintaining attention in time, the ability of avoiding
the interfering stimuli and using the feedback provided by others, the capability of
coordinating more activities together at the same time, the cognitive and behavioural
flexibility and the other abilities used to cope new situations and stimuli (Crawford,
1998).
Many terms have been used to define this syndrome: Disorders of Executive
Functions; Dysexecutive Syndrome; Strategy Application Disorder (SAD); (Pre)Frontal
Syndrome. Loss of executive functions is primarily a consequence of brain injury
located in the prefrontal cortex area, but many different categories of subjects can be
characterized by the same syndrome and by similar symptoms, with different levels of
severity and various forms of resulting behaviour: patients suffering of different forms of
dementia (Alzheimer Disease, Frontal or Frontal-Temporal Dementia, etc.), patients
with attention disorders and hyperactivity (i.e. ADD-H), subjects suffering of
Schizophrenia.
Usually the problems related with cognitive disabilities are not evident: Thimble
(1990) noted that the pathological conditions due to a frontal dysfunction, very often are
315
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
not recognized in the clinical practice. This situation could happen, overall in the case
of frontal lobe disease, because cognitive performance of subjects may not be
significantly reduced and many traditional neuropsychological tests may fail to show
significant dysfunctions in the patients’ responses (Damasio, 1995; Gourlay, Lun, Lee,
& Tay, 2000). For example the traditional version of Wisconsin Card Sorting Test
(WCST)(Heaton & Pendleton, 1981) has been usually considered a key measure in the
diagnosis of frontal lobe dysfunction (Bornstein, 1986; Braff et al., 1991; Drewe, 1974;
Janowsky, Shimamura, Kritchevsky, & Squire, 1989; Milner, 1963, 1964), but recently
Stuss et. al noted that “this view of the WCST as a specific measure of impairment in
the frontal lobes has also been seriously questioned” (p. 388, (Stuss et al., 2000).
As Damasio brilliantly pointed out, in the complex tasks of everyday life, the same
patients may show great limits, decisional problems and inabilities connected with high
levels of psychosocial suffering. It has been demonstrated that traditional tests of
frontal lobes function may fail to document abnormality: this “diagnostic insensitivity”
may cause problems between patients and health care services and can determine
incapacity to predict the outcome of treatment.
Damasio’s famous patient Elliot not only had normal performances in the standard
“frontal” cognitive tests, but in the lab assessment he showed normal responses to the
proposed social situations, he planned strategies and he demonstrated to be able to
evaluate correctly the consequences of actions in social situations. The same patient
showed a severe decisional impairment and emotional deregulation in his real life
environment, especially related to social situations. Elliot is described as the prototype
of the hi-functioning frontal patient who experiments severe problems in his daily life.
From the point of view of cognitive rehabilitation, if we consider that the traditional
protocols used in treatment are centered mainly to protect or recover the basic
instrumental cognitive functions, we can understand why, in a clinical or lab setting,
superior executive cognitive disabilities are today particularly hard to treat and thus
receive a very reduced attention in relation to their dramatic real-life consequences.
From a theoretical point of view, we can try to explain the striking differences in
performance and behaviour between lab and life as result of four main failures of the
artificial situations to mimic reality: a) choices and decisions are only to be evoked, not
to be really performed; b) lack of the normal cascade of events as actions and
reactions; c) compressed temporal frame; d) situations are not really presented, but
only described through language (Damasio, 1995). Our diagnostic and rehabilitative
tools, built to be used in clinical or laboratory settings, fail their goal with executive
316
G. Castelnuovo et al.
functions because they cannot adequately reproduce real situations and the perception
of the subject to be really present in them. Cognitive assessment and rehabilitation of
executive functions faces us with the necessity to transfer our work in real life situations
or, as a valid alternative, to be able to build artificial environments who can offer to the
subject similar conditions and the same sense of presence.
4. The added value of VR in the treatment of dysexecutive problems
Cognitive rehabilitation has to allow patients to recover their planning, executing and
controlling skills by implementing sequences of actions and complex behavioural
patterns that are requested in everyday life (A. A. Rizzo & J. G. Buckwalter, 1997; A.A.
Rizzo & J.G. Buckwalter, 1997): with these conditions, VR can be specifically indicated
to reach this goal (Grealy, Johnson, & Rushton, 1999; A.A. Rizzo, 2000). Moreover VR
allows to build realistic spatial and temporal scenarios that can be easily used to
increase diagnostic sensitivity of standard paper&pencil tests (Pugnetti, 1995, 1998).
For example, in the case of early recognition of Alzheimer’s disease, VR may be the
most efficient method for a systematic screening of subjects (A. A. Rizzo et al., 1998).
Due to the great flexibility of situations and tasks provided during the virtual sessions,
considering both the time, difficulty, interest, and emotional engagement, this new
technology allows, besides the diagnostic applications (Zhang et al., 2001), to enhance
the restorative and compensatory approaches in the rehabilitation process of cognitive
functions inside the traditional clinical protocol.
In our opinion, in general cognitive rehabilitation, the added value of VR compared to
the traditional treatment can be summarized according to the following points:
customization on user’s needs: each virtual environment can be produced in a
specific
way
focusing
on
the
patient’s
characteristics
and
demands;
possibility to graduate: each virtual environment can be modified and enriched
with more and more difficult and motivating stimuli and tasks. This last issue shall not
be underestimated, because many subjects with cognitive dysfunctions show a low
level of motivation and compliance about the traditional rehabilitation iter that is usually
repetitive and not stimulating;
317
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
high level of control: the possibility of controlling the rehabilitation process in its
development is very high. This issue allows to professionals to monitor the level of
affordability and complexity of the tasks that can be provided to patients;
ecological validity: a virtual environment allows to stimulate in the subjects
emotional and cognitive experiences like in the real life. So the tasks executed during
the immersion in the VR can induce the patient to reproduce complex cognitive
procedures (such as planning and organizing practical patterns of actions, attention
shift, etc…) that are similar in all the aspects to the ones used in the situations of
everyday life;
costs reduction: rehabilitation with VR can be cheaper than the traditional one,
mostly when it comes to the reconstruction of complex scenarios, also of very complex
ones (such as presence of more persons in the same time, particular environmental
conditions), which avoids the need to leave the rehabilitation office or building.
Nevertheless, when it comes to the specific needs of rehabilitation of dysexecutive
symptoms and frontal lobe syndrome, the employment of virtual environments seems
to have another fundamental advantage with respect to traditional non-immersive
means.
According to Damasio (Damasio, 1995, see 1st par.) diagnostic and rehabilitative
tools used in labs and clinics often fail to assess and treat the frontal patients because
they operate within artificial situations, which are far from reproducing the real
situations. In this view, immersive virtual environments appear to be the best solution to
make lab situations become closer to the natural setting in the subject’s perception.
For these reasons, in our work we decided not only to use and implement immersive
virtual environments, but also to assess very carefully the key variables that support
such choice: subjective sense of presence; qualitative evaluation of virtual experiences;
usability level.
318
G. Castelnuovo et al.
5. A possible VR-based tool for the rehabilitation of executive functions: the VStore.
In the “Paolo VI Centre of Cognitive Rehabilitation” at Casalnoceto (AL, Italy), the
traditional protocol followed in the cognitive rehabilitation has been experimentally
integrated with sessions of Virtual Reality.
V-STORE is a virtual environment that consists of a fruit store internal room: the
subject (or more correctly his “avatar”, his representation within the virtual world) is set
in front of a conveyor belt on which some baskets (from one to three) cross the room.
The subjects’ task is to fill up the baskets with pieces of fruit that can be found in four
shelves located along the other walls of the room.
At the beginning of each trial, a verbal command is communicated through a
loudspeaker, located on the front wall, by which the subject is instructed about what to
do: how to fill the baskets and with what kind of fruit. The task has to be completed
accurately before the baskets run out of the room on the belt, or else the trial will be
repeated from the beginning. It is possible to fix a maximum limit in how many “moves”
the subject can execute to solve the trial, forcing him to follow the most efficient and
quick strategy.
Figure 1. A screenshot of the first level of V-Store
319
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
The tasks are ordered according to their complexity, starting from very quick trials
that need few fruit moves, up to trials that start with a long and verbally complex
command and request special strategies in moving the available fruits from one basket
to another. The trials are currently divided in six levels of ten tasks each.
Other elements which are present in the environment are a waste basket, the light
switch and a wall telephone, located on the rear wall, through which the subject can
receive supplemental orders that integrate the initial verbal command in the most
difficult level. The subject can intervene on some additional commands, by which he
can stop the belt, end the trial, freeze time.
The supervising examiner can introduce a series of distracting events which increase
difficulty and are meant to generate time pressure: room light fainting or progressive
dimming, telephone ring, belt speed modification. In these situations, the most
interesting focus of performance and rehabilitation consists in the managing steps that
the subject will operate and his strategic approach.
For each trial, the system records the following data about the subject’s performance
for further analysis or research: accuracy, execution time, moves and strategical
planning, and furthermore the managing steps taken to face distractors or difficulties,
which often constitute the greatest limit for frontal patients.
Figure 2. A screenshot of the last level of V-Store (room lights dimmed)
Ongoing studies in “Paolo VI Centre of Cognitive Rehabilitation” aim at planning,
developing and testing a rehabilitation protocol with virtual different virtual tools (see
320
G. Castelnuovo et al.
Table 2) for high-level cognitive dysfunctions that affect patients with brain injures,
cerebral diseases, degenerative pathologies and in particular subjects who can be
evaluated as “frontal patients” in scientific literature: we also suppose that the
developed applications could be useful in a broad spectrum of rehabilitative
opportunities.
Table 2. Synthetic description of the V-Store and other tools that ongoing studies are
developing in “Paolo VI Centre of Cognitive Rehabilitation” at Casalnoceto (AL, Italy).
Name
Goal
V-Store
Virtual Store
rehabilitation
V-ToL
Virtual Tower of
London
rehabilitation
diagnosis
V-Wcst
Virtual Wisconsin
Card Sorting Test
rehabilitation
diagnosis
321
Brief description
Virtual environment (internal store) in which
the subject (clinical or experimental) has to solve
a series of tasks ordered in six levels of
increasing complexity. Tasks are designed to
stress executive functions, behavioural control
and programming, categorial abstraction, shortterm memory and attention. A series of distracting
elements are present, aimed to generate timepressure and to elicit managing strategies. The
examiner has full control on all variables, to
enhance usability and to increase the number of
sessions that a single patient will be administered
A virtual version of the well-known Tower of
London Test by Shallice (Shallice, 1982), using
the same environment as V-Store. The test is
meant to evaluate the executive ability to program
behaviors in time. The original paradigm and trial
sequence is carefully respected, to grant the
clinical and scientific equivalence of results. The
task can be used as a one-time assessing test or
repeated as a rehabilitative tool (the examiner
can intervene on all variables implied).
A virtual version of the well-known Wisconsin
Card Sorting Test (Heaton & Pendleton, 1981),
using the same environment as V-Store. The test
is meant to evaluate the executive abilities of
categorization, abstraction and behavioural
flexibility. The original paradigm and trial
sequence is carefully respected, to grant the
clinical and scientific equivalence of results. The
task can be used as a one-time assessing test or
repeated as a rehabilitative tool (the examiner
can intervene on all variables implied).
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
6. References
Acker, W. (1982). A computerized approach to psychological screening: the BexleyMaudsley Automated Psychological Screening and the Bexley-Maudsley Category
Sorting Test. International Journal of Man-Machine Studies, 17(3), 361-369.
Ager, A. (1993). Il personal computer in psicologia clinica. Milano: Franco Angeli.
Berg, E. A., & Simple, A. (1948). A simple, objective technique for measuring flexibility
in thinking. Journal of General Psychology, 39, 15-22.
Bornstein, R. (1986). Contribution of various neuropsychological measures to detection
of frontal lobe impairment. The International Journal of Clinical Neuropsychology, 8,
18-22.
Bracy, O. L. (1983). Computer based cognitive rehabilitation. Cognitive Rehabilitation,
1, 7-8.
Braff, D., Heaton, R. K., Kuck, J., Cullum, M., Moranville, J., Grant, I., et al. (1991). The
generalized pattern of neuropsychological deficits in outpatients with chronic
schizophrenia with heterogeneous Wisconsin Card Sorting Test results. Archives of
General Psychology, 48, 891-898.
Broeren, J., Bjorkdahl, A., Pascher, R., & Rydmark, M. (2002). Virtual reality and
haptics as an assessment device in the postacute phase after stroke. Cyberpsychol
Behav, 5(3), 207-211.
Campbell, M. (2002). The rehabilitation of brain injured children: the case for including
physical exercise and virtual reality: a clinical perspective. Pediatr Rehabil, 5(1), 4345.
Crawford,
J.
(1998).
Assessment
of
attention
and
executive
functions.
Neuropsychological rehabilitation, 8.
Cunningham, D., & Krishack, M. (1999). Virtual reality: a wholistic approach to
rehabilitation. Stud Health Technol Inform, 62, 90-93.
Damasio, A. R. (1995). L'errore di Cartesio: Adelphi.
Diller, L., & Gordon, W. A. (1981). Interventions for cognitive deficits in brain-injured
adults. Journal of Consuliting and Clinical Psychology, 49, 822-834.
Drewe, E. (1974). The effect of type and area of brain lesion on Wisconsin Card
Sorting Test performance. Cortex, 10, 159-170.
Ginarte Arias, Y. (2002). [Cognitive rehabilitation. Theoretical and methodological
aspects]. Rev Neurol, 35(9), 870-876.
322
G. Castelnuovo et al.
Gontkovsky, S. T., McDonald, N. B., Clark, P. G., & Ruwe, W. D. (2002). Current
directions in computer-assisted cognitive rehabilitation. NeuroRehabilitation, 17(3),
195-199.
Gordon, W. A., Hibbard, M. R., & Kreutzer, J. S. (1989). Cognitive remediation: Issues
in research and practice. Journal of Head Trauma Rehabilitation, 4(3), 76-84.
Gourlay, D., Lun, K. C., Lee, Y. N., & Tay, J. (2000). Virtual reality for relearning daily
living skills. Int J Med Inf, 60(3), 255-261.
Gourlay, D., Lun, K. C., & Liya, G. (2000). Telemedicinal virtual reality for cognitive
rehabilitation. Stud Health Technol Inform, 77, 1181-1186.
Grant, D. W., & Berg, E. A. (1984). A behavioural analysis of the degree of
reinforcement and ease of shifting to new responses in a Wiegl-type card sorting
problem. Journal of Experimental Psychology, 38, 401-411.
Grealy, M. A., & Heffernan, D. (2000). The rehabilitation of brain injured children: the
case for including physical exercise and virtual reality. Pediatr Rehabil, 4(2), 41-49.
Grealy, M. A., Johnson, D. A., & Rushton, S. K. (1999). Improving cognitive function
after brain injury: the use of exercise and virtual reality. Arch Phys Med Rehabil,
80(6), 661-667.
Heaton, R. K., & Pendleton, M. G. (1981). Use of Neuropsychological tests to predict
adult patients' everyday functioning. J Consult Clin Psychol, 49(6), 807-821.
Jack, D., Boian, R., Merians, A. S., Tremaine, M., Burdea, G. C., Adamovich, S. V., et
al. (2001). Virtual reality-enhanced stroke rehabilitation. IEEE Trans Neural Syst
Rehabil Eng, 9(3), 308-318.
Janowsky, J. S., Shimamura, A. P., Kritchevsky, M., & Squire, L. R. (1989). Cognitive
impairment following frontal lobe damage and its relevance to human amnesia.
Behavioural Neuroscience, 103, 548-560.
Kurlycheck, R. T., & Levin, W. (1987). Computers in the cognitive rehabilitation of
brain-injured persons. Crit Rev Med Inform, 1, 241-257.
Mazzucchi, A., & Maravita, A. (1993). Quali le indicazioni e i limiti della riabilitazione
computerizzata? In A. Ager (Ed.), Il personal computer in psicologia clinica. Milan:
Franco Angeli.
Merians, A. S., Jack, D., Boian, R., Tremaine, M., Burdea, G. C., Adamovich, S. V., et
al. (2002). Virtual reality-augmented rehabilitation for patients following stroke. Phys
Ther, 82(9), 898-915.
Mickey, D. L., Ross, R. A., Stoll, J. L., Chiang, C. C., Sondberg, H. A., & Dunlop, D. A.
(1998). Brain Injury and Cognitive Retraining: The Role of Computer Assisted
323
Virtual Reality based tools for the rehabilitation of cognitive
and executive functions
Learning and Virtual Reality. Paper presented at the Conference of the International
Cognitive Science Society.
Milner, B. (1963). Effects of different brain lesions on card sorting: The role of the
frontal lobes. Archives of Neurology, 9, 100-110.
Milner, B. (1964). Some effects of frontal lobectomy in man. In J. M. Warren & K. Akert
(Eds.), The frontal granular cortex and behaviour (pp. 313-334). New Jork: McGrawHill.
Nelson, H. E. (1976). A modified card sorting test sensitive to frontal lobe deficits.
Cortex, 12, 313-324.
Piron, L., Cenni, F., Tonin, P., & Dam, M. (2001). Virtual Reality as an assessment tool
for arm motor deficits after brain lesions. Stud Health Technol Inform, 81, 386-392.
Pugnetti, L. e. a. (1995). Evaluation and retraining of adults' cognitive impairment:
which role for virtual reality technology? Comput Biol Med, 25(2), 213-227.
Pugnetti, L. e. a. (1998). VR experience with neurological patients: basic cost/benefit
issues. Stud Health Technol Inform, 58.
Riva, G. (1998a). Virtual environments in neuroscience. IEEE Trans Inf Technol
Biomed, 2(4), 275-281.
Riva, G. (1998b). Virtual reality in neuroscience: a survey. Stud Health Technol Inform,
58, 191-199.
Riva, G. (2000). Design of clinically oriented virtual environments: a communicational
approach. Cyberpsychol Behav, 3(3), 351-357.
Riva, G., & Gamberini, L. (2000). Virtual reality as telemedicine tool: technology,
ergonomics and actual applications. Technol Health Care, 8(2), 113-127.
Rizzo, A. A. (2000). Virtual Environment Applications in Clinical Neuropsychology.
Paper presented at the Medicine meets Virtual Reality 2000.
Rizzo, A. A., & Buckwalter, J. G. (1997). The status of virtual reality for the cognitive
rehabilitation of persons with neurological disorders and acquired brain injury. Stud
Health Technol Inform, 39, 22-33.
Rizzo, A. A., & Buckwalter, J. G. (1997). Virtual reality and cognitive assessment and
rehabilitation: the state of the art. Stud Health Technol Inform, 44, 123-145.
Rizzo, A. A., Wiederhold, M., & Buckwalter, J. G. (1998). Basic issues in the use of
virtual environments for mental health applications. Stud Health Technol Inform, 58,
21-42.
Shallice, T. (1982). Specific impairments in planning. In Philosophical Transactions of
the Royal Society (Vol. 298, pp. 199-209).
324
G. Castelnuovo et al.
Silva, M. R. (2002). Virtual reality in Latin American clinical psychology and the
VREPAR project. Virtual Reality Environments for Psycho-Neuro-physiological
Assessment and Rehabilitation. Cyberpsychol Behav, 5(5), 433-441.
Skilbeck, C. (1993). Riabilitazione cognitiva con personal computer. In A. Ager (Ed.), Il
personal computer in psicologia clinica. Milan: Franco Angeli.
Stuss, D. T., Levine, B., Alexander, M. P., Hong, J., Palumbo, C., Hamer, L., et al.
(2000). Wisconsin Card Sorting Test performance in patients with focal frontal and
posterior brain damage: effects of lesion location and test structure on separable
cognitive processes. Neuropsychologia, 38, 388-402.
Tarr, M. J., & Warren, W. H. (2002). Virtual reality in behavioral neuroscience and
beyond. Nat Neurosci, 5 Suppl, 1089-1092.
Vincelli, F. e. a. (2001). Virtual reality as clinical tool: immersion and threedimensionality in the relationship between patient and therapist. Stud Health Technol
Inform, 81, 551-553.
Waterworth, E. L., & Waterworth, J. A. (2001). Focus, locus, and sensus: the three
dimensions of virtual experience. Cyberpsychol Behav, 4(2), 203-213.
Zhang, L., Abreu, B. C., Masel, B., Scheibel, R. S., Christiansen, C. H., Huddleston, N.,
et al. (2001). Virtual reality in the assessment of selected cognitive function after
brain injury. Am J Phys Med Rehabil, 80(8), 597-604; quiz 605.
325