Psychological Medicine
cambridge.org/psm
Review Article
Cite this article: Horigome T, Kurokawa S,
Sawada K, Kudo S, Shiga K, Mimura M,
Kishimoto T (2020). Virtual reality exposure
therapy for social anxiety disorder: a
systematic review and meta-analysis.
Psychological Medicine 50, 2487–2497. https://
doi.org/10.1017/S0033291720003785
Received: 20 July 2020
Revised: 17 September 2020
Accepted: 22 September 2020
First published online: 19 October 2020
Key words:
Dropout rate; in vivo exposure; long-term
efficacy; social anxiety disorder; virtual reality
exposure
Author for correspondence:
Taishiro Kishimoto,
E-mail: taishiro-k@mti.biglobe.ne.jp
Virtual reality exposure therapy for social
anxiety disorder: a systematic review and
meta-analysis
Toshiro Horigome1, Shunya Kurokawa1, Kyosuke Sawada2, Shun Kudo3,
Kiko Shiga1, Masaru Mimura1 and Taishiro Kishimoto1,4
1
Department of Neuropsychiatry, Keio University School of Medicine, Tokyo, Japan; 2Asaka Hospital, Fukushima,
Japan; 3Department of Neuropsychiatry, Japanese Red Cross Ashikaga Hospital, Tochigi, Japan and 4Psychiatry at
Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, New York, New York, USA
Abstract
Background. Virtual reality exposure therapy (VRET) is currently being used to treat social
anxiety disorder (SAD); however, VRET’s magnitude of efficacy, duration of efficacy, and
impact on treatment discontinuation are still unclear.
Methods. We conducted a meta-analysis of studies that investigated the efficacy of VRET for
SAD. The search strategy and analysis method are registered at PROSPERO
(#CRD42019121097). Inclusion criteria were: (1) studies that targeted patients with SAD or
related phobias; (2) studies where VRET was conducted for at least three sessions; (3) studies
that included at least 10 participants. The primary outcome was social anxiety evaluation score
change. Hedges’ g and its 95% confidence intervals were calculated using random-effect models. The secondary outcome was the risk ratio for treatment discontinuation.
Results. Twenty-two studies (n = 703) met the inclusion criteria and were analyzed. The efficacy of VRET for SAD was significant and continued over a long-term follow-up period:
Hedges’ g for effect size at post-intervention, −0.86 (−1.04 to −0.68); three months post-intervention, −1.03 (−1.35 to −0.72); 6 months post-intervention, −1.14 (−1.39 to −0.89); and 12
months post-intervention, −0.74 (−1.05 to −0.43). When compared to in vivo exposure, the
efficacy of VRET was similar at post-intervention but became inferior at later follow-up
points. Participant dropout rates showed no significant difference compared to in vivo
exposure.
Conclusion. VRET is an acceptable treatment for SAD patients that has significant, long-lasting efficacy, although it is possible that during long-term follow-up, VRET efficacy lessens as
compared to in vivo exposure.
Introduction
© The Author(s) 2020. Published by Cambridge
University Press
Social anxiety disorder (SAD) is one of the most common psychiatric disorders (Kessler et al.,
2005) and many patients with SAD are refractory to pharmacotherapy (Keller, 2006; Yonkers,
Dyck, & Keller, 2001). Among other treatment methods, exposure therapy has been identified
as effective (Heimberg, Becker, Goldfinger, & Vermilyea, 1985; Ponniah & Hollon, 2008).
However, when performing exposure therapy for SAD patients, it takes a great deal of effort
and time to accurately recreate interpersonal situations that will provoke the appropriate level
of fear response in patients (e.g. in order to conduct exposure therapy for a patient with a fear
of public speaking, one would need to gather an audience and regulate their reactions). Virtual
reality (VR) has made large advances and has allowed for experimenting with the use of VR in
exposure therapy (Virtual reality exposure therapy: VRET) to treat phobias and anxiety disorders (Mishkind, Norr, Katz, & Reger, 2017). There is currently research being done on the efficacy of VRET for treating SAD, public speaking anxiety (PSA), fear of public speaking (FPS),
and similar conditions, and several meta-analyses of such studies already exist. In all these
meta-analyses, the effectiveness shown in pre-post and waiting list comparisons was the
same. When comparing in vivo exposure, Wechsler, Mühlberger, and Kümpers (2019)
reported that in vivo exposure had greater efficacy than VRET, but meta-analyses that included
larger numbers of studies, such as those by Chesham, Malouff, and Schutte (2018) and Carl
et al. (2019) reported that evidence suggests VRET may have the similar efficacy as in vivo
exposure. However, because other types of studies, such as single-arm studies and randomized
controlled trials (RCTs) that had control groups other than in vivo exposure, were not included
in these previous meta-analyses, there are many more studies on the efficacy of VRET for SAD
that should be considered. Therefore, these should also be examined to confirm whether
results so far may change with more information. Also, only Kampmann, Emmelkamp, and
Morina (2016b) conducted meta-analyses of follow-up studies and the number of studies
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
2488
included was small. It is important to look at the long-term effects
of a large number of studies. This is because evaluations done directly following the intervention period will reflect a decrease in
the fear targeted by the VRET experience, and there is a possibility that anxiety felt in real-life social situations is not reflected; or
that after VR intervention has finished, once the subject is placed
in real-life social situations, it may be easier for symptoms to
reoccur as compared to conventional therapy. Furthermore, as
far as we know, there have been no meta-analyses done for dropout rates for SAD patients. Also, methods for treating SAD are
becoming more diverse, which means that VRET need not consist
purely of VR exposure; in fact, there are multiple studies that
combine VRET with additional treatment methods such as cognitive behavioral therapy (CBT) and other psychotherapy practices.
This diversification of methodology makes it more difficult to
fully understand which methods are effective.
In this study, our aim was to conduct a meta-analysis of
research that has used VRET as an intervention for SAD, and
to comprehensively evaluate the efficacy of VRET. To accomplish
this, we investigated the following three points of inquiry in our
analysis: (1) How do symptoms change with the use of VRET,
and especially, how do they change over a long-term period?
(2) When VRET is compared with waiting lists or
treatment-as-usual (TAU), what are the effect size and dropout
rates? (3) When VRET is compared with in vivo exposure, are
there differences in efficacy (over both short and long terms) or
in dropout rates?
Toshiro Horigome et al.
majority of RCTs for VR intervention have small case sizes.
This means that, in order to conduct a meta-analysis of all the
research and participants, we considered it is best to also include
single-arm studies. However, case reports with small numbers of
cases are more likely to be influenced by reporting bias, and in
order to avoid this, we established the inclusion criteria of 10 or
more participants. For both the primary and secondary screenings
of the articles, two of the five investigators looked at each article,
and in cases where investigators disagreed on the status of an article, a third investigator was asked to assist in the decision-making
process.
Data extraction
Studies that met all the eligibility criteria outlined above were each
reviewed by two investigators who extracted the data from each
study. The primary outcome was the change in social anxiety
score from the baseline point. In cases where social anxiety was
measured with more than one method, we used the Liebowitz
Social Anxiety Scale (LSAS) score (Liebowitz, 1987). For missing
data (e.g. missing standard deviations), we asked for data and
clarification by sending an email directly to the relevant study
authors. If the standard deviation value of the outcome was not
available, we estimated the value based on other research that
used the same outcome (Furukawa, Barbui, Cipriani, Brambilla,
& Watanabe, 2006). The secondary outcome was the dropout rate.
Comparisons
Methods
Search strategy
This systematic review and meta-analysis followed the standards
set forth in the Preferred Reporting Items for Systematic
Reviews and Meta-Analyses (PRISMA) guidelines (Moher,
Liberati, Tetzlaff, & Altman, 2009). The search strategy and analysis method are registered at PROSPERO (#CRD42019121097).
The systematic literature search was done by five investigators
using PubMed, PsycINFO, and Scopus databases. The search
terms used were [SAD OR ‘social anxiety’ OR ‘social fear’ OR
‘social phobia’ OR ((speak* OR speech) AND (anxiety OR
fear))] AND [(virtual OR augmented OR artificial OR mixed)
AND (environment OR reality OR audience)]. The last search
was conducted in May 2020. There was no lower publication
date limit to the searches.
Study selection
A primary screening of the studies identified in the literature
search was done by reviewing the titles and abstracts for those
items that appeared to focus on intervention methods using
VR. For studies that were selected via this screening method, as
well as studies that had been used in the previous meta-analyses
on this topic, we obtained full versions of the articles, and, in a
secondary screening process, determined if the articles met the
following eligibility criteria: (1) study participants were diagnosed
with one of the following: SAD, FPS, or PSA; (2) VRET was conducted for three or more sessions; (3) study participants numbered 10 or more people. This meta-analysis included not only
RCTs, but also single-arm studies where all subjects were given
VR intervention. Research using VR has so far been experimental
and focused mainly on smaller numbers of cases, so even the
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
Improvement of symptoms and duration thereof following VRET
In order to investigate how effective VRET is, and how long the
effects will continue, we first compared symptom severity at baseline
v. the following five-time points, respectively: post-intervention, 3
months post-intervention, 6 months post-intervention, 12 months
post-intervention, and 6 years post-intervention.
Comparison with waiting list or treatment-as-usual groups
In order to examine the effect size of VRET as compared to
untreated controls and standard treatments, the change in symptoms from baseline to post-intervention was compared between
the VRET group and waiting list/TAU group. The dropout rate
for the VRET and waiting list/TAU groups was also compared.
Comparison with in vivo exposure
To investigate the differences in efficacy between VRET and in
vivo exposure therapy, the change in symptoms from baseline
to post-intervention was compared between the VRET group
and in vivo exposure group. The long-term effect was also compared based on symptom differences between the baseline and
follow-up time points. Additionally, the dropout rate for VRET
and in vivo exposure was compared.
Sensitivity analysis and moderator analysis
We conducted a sensitivity analysis and moderator analysis on the
results from baseline and post-intervention comparisons. To
investigate whether or not the treatment efficacy changed depending on differences in intervention methods, study protocol, and
subjects’ diagnoses, we conducted sensitivity analyses for the following: (1) whether VRET was used with or without structured
psychotherapy; (2) whether the study protocol was an RCT or
Psychological Medicine
not; (3) whether participants were diagnosed with SAD or other
disorders (i.e. PSA and FPS).
Next, with the goal of determining which factors are important
for VRET to be effective, we conducted a meta-regression analysis
of the following continuous moderator variables: (1) number of
VR sessions; (2) number of total sessions; (3) ratio of VR sessions
to total sessions; (4) time of total VR sessions; (5) time of total
sessions; (6) ratio of VR sessions’ time to total sessions’ time;
(7) baseline LSAS-total score; (8) publication year. However, we
did not conduct multiple meta-regression analyses because we
considered there were too few studies to do so.
Meta-analytic procedures
Hedges’ g ± 95% confidence intervals (CIs) were calculated for
continuous outcomes (i.e. social anxiety score) using
random-effect models. For dichotomous outcome, the risk ratio
was calculated using random-effect models. To evaluate heterogeneity, I 2, Q, and p values were reported. When there were
three or more studies, we also calculated the prediction interval.
Comprehensive Meta-analysis Version 3 was used for the analysis.
Risk of bias assessment
For all studies that met the eligibility criteria, two investigators
evaluated each study for risk of bias. These assessments were
done in accordance with the Cochrane Collaboration’s tool for
assessing the risk of bias in RCTs (Higgins et al., 2011).
Publication bias was assessed via visual inspection of funnel
plots using Egger’s intercept test (Egger, Smith, Schneider, &
Minder, 1997) and trim and fill procedure (Duval & Tweedie,
2000).
Results
Search results
Out of 478 articles that were screened in the primary and secondary processes, 22 articles met the eligibility criteria for this
meta-analysis (online Supplementary Fig. S1). The 78 articles
that were excluded during the secondary screening were rejected
for the following reasons: 14 studies had less than three intervention sessions; four studies had too few participants; 32 studies did
not measure efficacy; six studies did not actually utilize VR in
their intervention methods; two studies did not investigate SAD,
PSA, and/or FPS; six were review articles; 13 studies examined
the exact same cohort; and lastly, one study looked at the efficacy
of virtual space in a social game as an intervention method (Yuen
et al., 2013), and it was felt that the heterogeneity of this study was
too high compared to the others, so it was excluded.
Study characteristics
Of the 22 studies selected for our meta-analysis, 14 of those studies investigated patients with SAD, and eight studies investigated
FPS or PSA. Eleven studies were RCTs. Seven studies reported
data for waiting list/TAU groups as control groups. Ten studies
had in vivo exposure (including group exposure, imaginary exposure, etc.) comparison groups. Twelve studies used a combination
of VRET and psychotherapy, nine studies used only VRET, and
one study used both VRET and psychotherapy separately.
Within the studies that used a combination of VRET and
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
2489
psychotherapy, CBT was the most-used psychotherapy method,
appearing in eight studies (Table 1).
The number of participants included in the meta-analysis was
586 [VRET 325 (VRET + psychotherapy 163, VRET-only 162), in
vivo exposure therapy 138, waiting list/TAU 123], and the average
age was 33.5 years.
Improvement of symptoms and duration thereof following
VRET
There was a significant decrease in symptom evaluation scores
from baseline to post-intervention [studies = 20, n = 299,
Hedges’ g = −0.86 (−1.04 to −0.68), p < 0.001; heterogeneity I 2
= 77.6%, Q = 84.9, p < 0.001; prediction interval = −0.80 (−1.54
to −0.06)] (Fig. 1). Each follow-up time points also displayed a
significant decrease in symptom scores from baseline (Fig. 1).
Effect sizes at each follow-up time points are as follows: 3 months
post-intervention [studies = 5, n = 80, Hedges’ g = −1.03 (−1.35 to
−0.72), p < 0.001; heterogeneity I 2 = 70.0%, Q = 13.3, p = 0.01;
prediction interval = −0.96 (−1.94 to 0.03)], 6 months postintervention [studies = 2, n = 38, Hedges’ g = −1.14 (−1.39 to
−0.89), p < 0.001; heterogeneity I 2 = 0%, Q < 0.01, p = 0.94)], 12
months post-intervention [studies = 2, n = 55, Hedges’ g = −0.74
(−1.05 to −0.43), p < 0.001; heterogeneity I 2 = 64.5%, Q = 2.8, p
= 0.09], and 6 years post-intervention [studies = 1, n = 13,
Hedges’ g = −0.43 (−0.77 to −0.09), p = 0.01; heterogeneity
I 2 = 0%, Q < 0.01, p < 0.001].
VRET v. waiting list or treatment-as-usual
VRET was significantly more effective compared with waiting list/
TAU [studies = 7, n = 273, Hedges’ g = −1.23 (−1.70 to −0.76),
p < 0.001; heterogeneity I 2 = 67.6%, Q = 18.5, p = 0.005; prediction
interval = −1.23 (−2.71 to 0.25)] (Fig. 2).
There was no difference in dropout rates between VRET
groups and waiting list/TAU groups [studies = 6, n = 118,
Risk ratio = 1.25 (0.66 to 2.37), p = 0.49; heterogeneity I 2 = 0%,
Q = 4.70, p = 0.45; prediction interval = 1.25 (−0.33 to 2.84)]
(online Supplementary Fig. S2).
VRET v. in vivo exposure
There was no significant difference between the effect size of
VRET and in vivo exposure at post-intervention [studies = 10, n
= 355, Hedges’ g = 0.07 (−0.41 to 0.55), p = 0.78; heterogeneity
I 2 = 78.6%, Q = 42.0, p < 0.001; prediction interval = 0.07 (−1.57
to 1.71)] (Fig. 3).
However, at 3 months post-intervention, in vivo exposure
showed greater efficacy [studies = 3, n = 129, Hedges’ g = 0.81
(0.01 to 1.61), p = 0.047; heterogeneity I 2 = 76.8%, Q = 8.6, p =
0.01; prediction interval = 0.81 (−8.57 to 10.19)], whereas at 6
months post-intervention, VRET showed greater efficacy [studies
= 1, n = 39, Hedges’ g = −0.86 (−1.51 to −0.21), p = 0.10]. Then at
12 months post-intervention [studies = 2, n = 118, Hedges’
g = 0.46 (0.09–0.82), p = 0.01; heterogeneity I 2 = 0%, Q = 0.32,
p = 0.57] and 6 years post-intervention [studies = 1, n = 28,
Hedges’ g = 1.12 (0.34–1.90), p = 0.005], in vivo exposure, again,
showed greater efficacy compared to VRET (Fig. 3).
There was no significant difference in dropout rates between
VRET and in vivo exposure [studies = 4, n = 126, risk ratio =
0.63 (0.37–1.06), p = 0.08; heterogeneity I 2 = 0%, Q = 2.79,
p = 0.43; prediction interval = −0.33 (−1.17 to 0.51)] (Fig. 4).
2490
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
Table 1. Characteristics of studies and patients
Study
Diagnosis
Design
VR intervention method
Comparison
N
Mean
age
No. of
total
sessions
No. of
exposure
sessions
Long-term
follow-up
(months)
SAD (DSM-IV)
Single-arm study
Psychoeducation and cognitive
therapy + VRET
First four sessions:
psychoeducation and cognitive
therapy; second four sessions:
patient manipulates exposure
software and proceeds through
session. Questions for therapist
possible.
–
11
44.6
8
4
3
Anderson et al. (2013)
SAD (DSM-IV)
RCT
Original protocol using CBT +
VRET
VRET conducted 4 times.
Therapist controls the reactions
of the virtual humans.
Homework given at each session.
EGT/waiting list
58
39
8
4
3/12
Anderson, Edwards,
and Goodnight (2017)
SAD (DSM-IV)
RCT
Original protocol using CBT +
VRET
Follow-up study of Anderson et
al. (2013).
EGT
28
42
8
4
72
Bouchard et al. (2017)
SAD (DSM-5)
RCT
CBT + VRET
Incorporated VR into existing CBT
protocol (Clark & Wells, 1995).
CBT + in vivo
exposure/waiting list
59
34.5
14
8
6
Denizci Nazligul et al.
(2019)
PSA (LSAS
fear and
avoidance
each ⩾20)
RCT
VRET
Session 1: assessment by
therapist and focus on
psychoeducation. Sessions 2–4:
VRET wherein therapist controls
the reactions of the virtual
humans; after VRET, discussion
with therapist about cognition.
Psychoeducationa
14
21.4
4
3
–
Gebara, Barros-Neto,
Gertsenchtein, and
Lotufo-Neto (2016)
SAD (DSM-IV)
Single-arm study
VRET
Six types of VR exposure
scenarios that are repeated until
anxiety is reduced. Therapist
with CBT training controls the
reactions of the virtual humans.
–
25
39
12
7
6
Geraets et al. (2019)
SAD
Single-arm study
CBT + VRET
Incorporated VR into existing CBT
protocol (Pot-Kolder et al., 2018).
Therapist controls the number of
audience members and their eye
lines.
–
15
34.9
16
14
3
Grillon, Riquier,
Herbelin, and
Thalmann (2006)
SAD (DSM-IV)
Single-arm study
VRET
Protocol based on Hofmann’s
model with added use of VR.
Homework given at each session.
–
10
–
11
8
–
Toshiro Horigome et al.
Anderson, Zimand,
Schmertz, and Ferrer
(2007)
RCT
VRET
VR exposure for public speaking
scenarios. Therapist controls
number of audience members
and their behavior; also uses
audience applause to encourage
the patient and draw patient’s
line of sight to the audience.
Waiting list
17
–
4
4
–
Kampmann et al.
(2016a)
SAD (DSM-IV)
RCT
VRET
Protocol using only behavioral
exposure elements (Hofmann &
Otto, 2008; Scholing&
Emmelkamp, 1993) in
conjunction with VR. Therapist
communicated with patient
before and after treatment.
In vivo exposure /
waiting list
60
36.9
10
7
3
Kim et al. (2017)
SAD (DSM-IV)
Controlled clinical
trial (demographic
data used for
matching)
VRET
Treatment was VRET-only.
Normal controla
48
23
8
8
–
Klinger et al. (2005)
SAD (DSM-IV)
Controlled clinical
trial (demographic
data used for
matching)
Original psychotherapy + VRET
While using VRET, therapist
supports patient in learning
about adapted cognition and
behavior to help them lessen
anxiety in real-life settings.
CBGT
36
31.6
12
12
–
Kovar (2018)
SAD
Non-randomized
parallel comparison
trial
Psychotherapy + VRET
Intervention combining
psychotherapy and VRET.
Psychotherapya
10
34.6
10
8
–
Lister et al. (2010)
FPS (PRCS
⩾21)
RCT
VRET
VRET consisting of a scenario
where patient reads a children’s
book to a virtual audience.
Waiting listb
20
–
4
4
–
North, North, and
Coble (1998)
FPS (DSM-IV)
Controlled clinical
trial (demographic
data used for
matching)
VRET
VR exposure for public speaking
scenarios. Therapist controls
number of audience members
and their behavior.
Placebo (exposed to
trivial VR scenes)a
16
–
5
5
–
Robillard, Bouchard,
Dumoulin, Guitard,
and Klinger (2010)
SAD (DSM-IV)
RCT
CBT + VRET
Intervention combining CBT and
VRET.
CBT + in vivo
exposure/waiting list
45
34.9
–
16
–
Roy et al. (2003)
SAD (DSM-IV)
No information on
randomization
Original psychotherapy + VRET
While using VRET, therapist
supports patient in learning
about adapted cognition and
behavior to help them lessen
anxiety in real-life settings.
CBGT
10
36.1
12
12
–
Safir, Wallach, and
Bar-Zvi (2012)
PSA
RCT
CBT + VRET
Follow-up study of Wallach, Safir,
and Bar-Zvi (2009).
CBT + imaginary
exposure
49
27
12
–
12
(Continued )
2491
FPS (PRCS
⩾16)
Psychological Medicine
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
Harris, Kemmerling, &
North (2002)
Study
Diagnosis
Design
VR intervention method
Comparison
Long-term
follow-up
(months)
N
Mean
age
No. of
total
sessions
No. of
exposure
sessions
38
–
8
8
–
112
27
12
8
–
12
–
–
6
4
3
Stupar-Rutenfrans
et al. (2017)
PSA
(PRCA ⩾ 55)
Single-arm study
VRET
VRET with public speaking
scenarios only. Number of virtual
audience members was
controlled.
–
Wallach et al. (2009)
PSA
RCT
CBT + VRET
Original CBT protocol combined
with VRET.
CBT + imaginary
exposure/waiting list
Wallach, Safir, and
Bar-Zvi (2011)
PSA
RCT
VRET / CBT + VRETc
VRET group was given exposure
therapy based on the behavioral
aspects of a treatment outlined
by Heimberg and Becker (2002).
Cognitive aspects were not
touched on. CBT + VRET group
referenced data from Wallach
et al. (2009).
CTa/waiting listc
78
27
Yuen et al. (2019)
SAD (DSM-IV)
Non-randomized
parallel comparison
trial
ACT + VRET
ACT program members using a
video conferencing system were
given additional homework to
participate in 4 sessions of VRET.
ACT +
videoconferencing
exposure
26
43.5
2492
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
Table 1. (Continued.)
ACT, acceptance and commitment therapy; CBT, cognitive behavioral therapy; CBGT, cognitive behavioral group therapy; CT, cognitive therapy; DSM, Diagnostic and Statistical Manual of Mental Disorders; EGT, exposure group therapy; FPS, fear of
public speaking; LSAS, Liebowitz Social Anxiety Scale; PRCA, Personal Report of Communication Apprehension; PRCS, Personal Report of Confidence as a Speaker; PSA, public speaking anxiety; SAD, social anxiety disorder; VRET, virtual reality exposure
therapy.
a
This comparison is different from that of in vivo exposure and waiting list groups, and was not included in our analysis.
b
Due to inadequacies in the data, we did not include them in our analysis.
c
These data are duplicated from other studies, and so were not used in our analysis.
Toshiro Horigome et al.
Psychological Medicine
2493
Fig. 1. Symptom reduction compared to baseline score at each time point. Negative scores indicate treatment efficacy.
Fig. 2. Comparison of treatment efficacy for VRET and waiting list/treatment-as-usual groups at treatment end-point.
Sensitivity analysis and moderator analysis
Regardless of whether VRET was combined with psychotherapy
or not, there was significant improvement after intervention for
combination VRET and psychotherapy [studies = 10, n = 163,
Hedges’ g = −1.05 (−1.25 to −0.84), p < 0.001; heterogeneity
I 2 = 64.1%, Q = 25.0, p = 0.003; prediction interval = −1.01
(−1.63 to −0.39)), and for VRET only (studies = 10, n = 162,
Hedges’ g = −0.67 (−0.90 to −0.44), p < 0.001; heterogeneity
I 2 = 75.7%, Q = 37.0, p < 0.001; prediction interval = −0.62
(−1.36 to 0.13)] (online Supplementary Fig. S3).
Similarly, both RCT and non-RCT studies showed significant
treatment effect: RCTs [studies = 9, n = 142, Hedges’ g = −0.73
(−0.93 to −0.52), p < 0.001; heterogeneity I 2 = 66.6%, Q = 23.9, p
= 0.002; prediction interval = −0.75 (−1.36 to −0.15)]; non-RCTs
[studies = 11, n = 157, Hedges’ g = −1.00 (−1.30 to −0.71), p <
0.001; heterogeneity I 2 = 83.3%, Q = 60.0, p < 0.001; prediction
interval = −0.84 (−1.85 to 0.18)] (online Supplementary Fig. S4).
Likewise, patients with SAD as well as those with FPS or PSA
responded to VRET treatment: SAD [studies = 13, n = 196,
Hedges’ g = −1.01 (−1.21 to −0.81), p < 0.001; heterogeneity
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
I 2 = 71.3%, Q = 41.8, p < 0.001; prediction interval = −0.96
(−1.65 to −0.28)]; FPS or PSA [studies = 7, n = 103, Hedges’
g = −0.58 (−0.82 to −0.33), p < 0.001; heterogeneity I 2 = 69.0%,
Q = 19.3, p = 0.004; prediction interval = −0.55 (−1.26 to 0.15)]
(online Supplementary Fig. S5).
A meta-regression analysis showed that none of the individual
moderators had a statistically significant effect on the effect size of
VR (online Supplementary Table S1). Although there were no significant effects, we found that there was a trend for effect size to increase
as the number of total sessions increased (β = −0.05, p = 0.06).
Study quality and publication bias
For many studies, there was an inadequate amount of detail
necessary to determine whether they contained information that
should be considered for bias risk (online Supplementary
Fig. S6). Only three studies mentioned random sequence generation, and two studies mentioned allocation concealment
(Anderson et al., 2013; Kampmann et al., 2016a), but one study
did not have enough information about concealment (Bouchard
2494
Toshiro Horigome et al.
Fig. 3. Comparison of treatment efficacy for VRET and in vivo exposure groups at each post-treatment time point.
Fig. 4. Dropout comparison between VRET and in vivo exposure groups.
et al., 2017). There were no studies that mentioned blinding for
data analysts. The analyses in three studies were considered to
have attrition bias (Anderson et al., 2013; Bouchard et al., 2017;
Kampmann et al., 2016a). For reporting bias, study protocols
were available for only two studies (Bouchard et al., 2017;
Kampmann et al., 2016a) and all studies reported their predicted
outcomes. For other bias, one study used multiple recruiting
methods, which consequently made the randomization of those
methods unequal (Harris, Kemmerling, & North, 2002).
A funnel plot appeared to show asymmetry in the effect size of
the standard error and Egger’s test showed significance ( p = 0.03)
(online Supplementary Fig. S7). Applying Duval and Tweedie’s
trim and fill procedure did not alter results.
Discussion
Summary of results
We conducted a systematic review and meta-analysis targeting 22
studies that examined the efficacy of VRET for SAD, PSA, and
FPS. Compared to previous meta-analyses on this topic, this
meta-analysis comprises the largest number of articles to date.
The results of our analysis showed that the efficacy of VRET intervention for SAD had a large effect size and that throughout the
follow-up time points of 3 months to 6 years, there was a longlasting effect. Even when compared with waiting list/TAU groups,
there was a large effect size and dropout rates were similar.
Compared to in vivo exposure intervention, the efficacy of
VRET at the post-intervention time points was similar.
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
However, although VRET was shown to be significantly more
effective at 6 months post-intervention, in vivo exposure was
shown to be more effective at 3 months, 12 months, and 6
years post-intervention. There was no difference in dropout
rates between VRET and in vivo exposure.
Efficacy of VRET at post-intervention
VRET intervention for SAD showed a large effect size for symptom evaluation scores at post-intervention and also showed a
large effect size when compared with waiting list/TAU groups.
In prior meta-analyses, conclusions regarding the efficacy of
VRET when compared with in vivo exposure differed across studies (Carl et al., 2019; Chesham et al., 2018; Kampmann et al.,
2016b; Wechsler et al., 2019), but we found that VRET had the
similar efficacy as in vivo exposure. Even when we examined
the efficacy of VRET for FPS/PSA and SAD separately in our analysis, we did not find a significant difference in efficacy.
Additionally, there was no significant difference in efficacy
between intervention methods that combined VRET with psychotherapy and VRET-only methods.
Long-term efficacy of VRET
When compared to baseline data, the efficacy of VRET intervention continued for a long-term period following treatment and
had a large effect size at time points of 3 months and 6 months
post-intervention. Based on these findings, we believe that the
changes caused by VRET intervention allowed for a decrease in
Psychological Medicine
fear felt in real-life situations and prevented relapses following the
intervention. However, the effect size was smaller at 12 months
and 6 years post-intervention. VRET was more effective than in
vivo exposure at 6 months post-intervention, but for all other
follow-up time points, in vivo exposure showed more efficacy. It
was reported that methods such as traditional CBT and exposure
therapy using in vivo exposure had continued efficacy for more
than 12 months (Ponniah & Hollon, 2008; van Dis et al., 2020).
Compared to that, although it is difficult to make a definitive
statement since only a few studies have compared the long-term
efficacy of in vivo exposure, the long-term effects of VRET intervention may be inferior.
Influence of VRET on dropout rates
There were no significant differences in dropout rates between
VRET and in vivo exposure or waiting list/TAU groups. While
there may be a perception that dropout rates for VRET are
high, the truth may actually be counterintuitive, as it has been
pointed out that dropout rates for in vivo exposure are high due
to patients’ strong reluctance to undergo the treatment (Choy,
Fyer, & Lipsitz, 2007) and one of the merits of VRET is that
patients have a less psychological aversion to participating in it
(Garcia-Palacios, Botella, Hoffman, & Fabregat, 2007). While
there is still room for research on why dropout rate for VRET
is low, if VRET treatment technology was to advance even further
than it has now, VRET could provide a more gentle introduction
to exposure therapy than in vivo methods, possibly leading to a
lower dropout rate.
It is also important to note that the working alliance of a
patient and their healthcare provider is a major variable in psychotherapy (Flückiger, Del Re, Wampold, & Horvath, 2018;
Sharf, Primavera, & Diener, 2010) and the working alliance has
been shown to have an effect on the treatment efficacy and dropout rates for SAD patients when treated with CBT (Haug et al.,
2016). While we do not yet know how the working alliance
may influence the outcome of VRET (Meyerbröker &
Emmelkamp, 2010), two of the studies included in this
meta-analysis evaluated the working alliance as part of their
research. Anderson et al. (2013) stated that there was no difference in the working alliance between VRET and group exposure
therapy. Bouchard et al. (2017) reported that there was no statistical difference in the working alliance between a CBT + VRET
group and CBT + in vivo exposure group and that the working
alliance data were a predictor of symptom improvement for
both groups. In light of this, it would appear that the working alliance would be a major factor when utilizing VRET to treat SAD
and it is possible that the use of VR technology would not cause
dropout rates to increase by way of negatively impacting the working alliance.
Quality of VR technology
In the case of VRET, the quality of the VR technology used can
influence the efficacy of the treatment. When using VRET to
treat anxiety disorder, the feeling of presence created by VR technology is thought to influence subjects’ anxiety levels and this is
an important factor in making the treatment effective (Price &
Anderson, 2007; Robillard, Bouchard, Fournier, & Renaud,
2003). Additionally, it was reported that when the levels of feeling
of presence and anxiety produced via the VR technology are too
low, there is an increase in the dropout rate (Krijn et al., 2004).
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
2495
The results of this meta-analysis show that: dropout rates are
similar for VRET and in vivo exposure interventions; efficacy is
also similar for at least the post-intervention time point; and
that the VR technology used in research is at a level that allows
for practical clinical application.
It should be noted that it is still unclear what role feeling of
presence plays in using VRET to treat SAD and there are even
reports that feeling of presence in a VR environment does not
actually affect the anxiety levels of SAD patients (Ling, Nefs,
Morina, Heynderickx, & Brinkman, 2014; Morina, Brinkman,
Hartanto, & Emmelkamp, 2014). However, Price, Mehta, Tone,
and Anderson (2011) analyzed the relationship between the treatment effect of VRET for SAD and feeling of presence at each subscale. As a result, they found that the realness subscale (the degree
to which the virtual stimulus matches the expected value of the
real stimulus) was connected to anxiety levels and that the
involvement subscale (the degree to which a subject stays focused
on the virtual stimulus) was associated with symptom improvement. Further investigation is needed to determine what virtual
environment/stimulus is most appropriate for treating SAD.
Recommendations for future research
There were significant differences regarding VRET content, number of VRET sessions, the proportion of VRET sessions to the
total number of sessions, and use or non-use of psychotherapy
among the studies that were included in this meta-analysis.
Further research may uncover the ideal way to utilize VRET.
However, we also anticipate that there will be individual differences
between subjects as to what situations cause the greatest anxiety,
their degree of symptom severity, etc. Therefore, the ideal treatment
would be one that can be optimized for each patient by providing
the ability to access a diversity of content and adjust the
anxiety-inducing stimuli. VRET is suitable for this purpose and
thus could be an important tool in exposure therapy.
There are already some approaches that can be used to adjust
VRET to create a tailor-made treatment for each SAD patient. To
find the best exposure environment for each SAD patient, various
scenarios are used in VRET, such as speeches, meetings, meals,
and ordering in shops. There are also a few methods to regulate
anxiety. For example, patients with SAD are more likely to be anxious about the negative expressions of others (Phan, Fitzgerald,
Nathan, & Tancer, 2006; Stein, Goldin, Sareen, Zorrilla, &
Brown, 2002; Straube, Mentzel, & Miltner, 2005), so patients’ anxiety levels can be controlled by varying the facial expressions on
VR avatars used in VRET (Qu, Brinkman, Ling, Wiggers, &
Heynderickx, 2014). Additionally, in public speaking settings,
there have been attempts to control patients’ anxiety levels by
adjusting the number of audience members they see
(Kampmann et al., 2016a; North, North, & Coble, 1998;
Stupar-Rutenfrans, Ketelaars, & van Gisbergen, 2017). In the
future, creating these controlled stimuli using VR will become
increasingly easier than preparing real-life situations. It is also
possible to use VR technology to create even stronger trigger
situations that would be hard to produce in real-life situations,
such as creating an audience of 100 000 people in a stadium for
a public speaking situation. Furthermore, among SAD patients,
there are those who have a strong fear of speaking to authority figures, and with VR technology it is possible to put patients
face-to-face with great historical figures, like past presidents,
and have them converse or have a meal together.
2496
The other method for diversifying treatment is to conduct
exposure therapy while monitoring patients’ autonomic nervous
systems and adjusting exposure until the ideal anxiety level is
achieved. It is thought that skin conductance response and
heart rate fluctuate based on changes in anxiety levels are triggered by VRET. Several studies included in our meta-analysis
monitored such biological signals during treatments but were
unable to clarify exactly how those signals are related to the treatments (Harris et al., 2002; Kim et al., 2017; Lister, Piercey, &
Joordens, 2010; North et al., 1998). It would be advantageous to
develop an algorithm for selecting the most appropriate virtual
stimulus based on such biological information.
Limitations
There were several limitations for this meta-analysis study. First,
there were large differences in how VR technology was used in
each study and there was also a broad range among the studies
of how much time was provided for VR exposure at each session
and how many sessions were done in total. Second, it is likely that
there were differences in the ‘reality’ created by the different VR
software used among the studies. Third, few studies had sufficient
study quality, which highlights the need for more high-quality,
large-scale RCTs. In particular, there were very few studies that
did long-term evaluations of efficacy and it is possible that conclusions put forth thus far may change based on the results of
future long-term research.
Conclusion
VRET is an acceptable treatment for SAD patients that has significant, long-lasting efficacy, although it is possible that during long
term follow-up, VRET efficacy lessens as compared to in vivo
exposure. While VRET offers flexibility in how it can be tailored
to the individual situation of each patient with SAD, there is still
much that is unclear about which intervention method is most
appropriate. More high-quality RCTs that examine the long-term
effects of VRET are needed.
Supplementary material. The supplementary material for this article can
be found at https://doi.org/10.1017/S0033291720003785.
Acknowledgements. We would like to thank Dr Hélène Wallach for providing additional data and Dr Max North for his advice on our analysis. This
research received no specific grant from any funding agency, commercial or
not-for-profit sectors.
Authors contributions. TH, TK, and MM contributed to designing the
study, analyzing the data, and writing the manuscript. The systematic literature
search and data extraction were done by TH, S Kurokawa, K Sawada, S Kudo,
K Shiga, and TK.
Conflict of interest. None.
References
Anderson, P. L., Edwards, S. M., & Goodnight, J. R. (2017). Virtual reality and
exposure group therapy for social anxiety disorder: Results from a 4–6 year
follow-up. Cognitive Therapy and Research, 41(2), 230–236.
Anderson, P. L., Price, M., Edwards, S. M., Obasaju, M. A., Schmertz, S. K.,
Zimand, E., & Calamaras, M. R. (2013). Virtual reality exposure therapy
for social anxiety disorder: A randomized controlled trial. Journal of
Consulting and Clinical Psychology, 81(5), 751.
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
Toshiro Horigome et al.
Anderson, P., Zimand, E., Schmertz, S. K., & Ferrer, M. (2007). Usability and
utility of a computerized cognitive-behavioral self-help program for public
speaking anxiety. Cognitive and Behavioral Practice, 14(2), 198–207.
Bouchard, S., Dumoulin, S., Robillard, G., Guitard, T., Klinger, E., Forget, H.,
… Roucaut, F. X. (2017). Virtual reality compared with in vivo exposure in
the treatment of social anxiety disorder: A three-arm randomised controlled
trial. The British Journal of Psychiatry, 210(4), 276–283.
Carl, E., Stein, A. T., Levihn-Coon, A., Pogue, J. R., Rothbaum, B.,
Emmelkamp, P., … Powers, M. B. (2019). Virtual reality exposure therapy
for anxiety and related disorders: A meta-analysis of randomized controlled
trials. Journal of Anxiety Disorders, 61, 27–36.
Chesham, R. K., Malouff, J. M., & Schutte, N. S. (2018). Meta-analysis of the
efficacy of virtual reality exposure therapy for social anxiety. Behaviour
Change, 35(3), 152–166.
Choy, Y., Fyer, A. J., & Lipsitz, J. D. (2007). Treatment of specific phobia in
adults. Clinical Psychology Review, 27(3), 266–286.
Clark, D. M., & Wells, A. (1995). A cognitive model of social phobia. In
Heimberg, R. G., Liebowitz, M. R., Hope, D. A., & Schneier, F. R. (Eds.),
Social phobia: Diagnosis, assessment and treatment (pp. 69–93).
New York: Guilford Press.
Denizci Nazligul, M., Yilmaz, M., Gulec, U., Yilmaz, A. E., Isler, V., O’Connor,
R. V., … Clarke, P. (2019). An interactive 3d virtual environment to reduce
the public speaking anxiety levels of novice software engineers. IET
Software, 13(2), 152–158.
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based
method of testing and adjusting for publication bias in meta-analysis.
Biometrics, 56(2), 455–463.
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in
meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634.
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018). The
alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy,
55(4), 316.
Furukawa, T. A., Barbui, C., Cipriani, A., Brambilla, P., & Watanabe, N. (2006).
Imputing missing standard deviations in meta-analyses can provide accurate results. Journal of Clinical Epidemiology, 59(1), 7–10.
Garcia-Palacios, A., Botella, C., Hoffman, H., & Fabregat, S. (2007). Comparing
acceptance and refusal rates of virtual reality exposure vs. in vivo exposure
by patients with specific phobias. Cyberpsychology & Behavior, 10(5), 722–
724.
Gebara, C. M., Barros-Neto, T. P. D., Gertsenchtein, L., & Lotufo-Neto, F.
(2016). Virtual reality exposure using three-dimensional images for the
treatment of social phobia. Brazilian Journal of Psychiatry, 38(1), 24–29.
Geraets, C. N., Veling, W., Witlox, M., Staring, A. B., Matthijssen, S. J., & Cath,
D. (2019). Virtual reality-based cognitive behavioural therapy for patients
with generalized social anxiety disorder: A pilot study. Behavioural and
Cognitive Psychotherapy, 47(6), 745–750.
Grillon, H., Riquier, F., Herbelin, B., & Thalmann, D. (2006). Virtual reality as
therapeutic tool in the confines of social anxiety disorder treatment.
International Journal in Disability and Human Development, 5(3), 243–250.
Harris, S. R., Kemmerling, R. L., & North, M. M. (2002). Brief virtual reality
therapy for public speaking anxiety. Cyberpsychology & Behavior, 5(6),
543–550.
Haug, T., Nordgreen, T., Öst, L. G., Tangen, T., Kvale, G., Hovland, O. J., …
Havik, O. E. (2016). Working alliance and competence as predictors of outcome in cognitive behavioral therapy for social anxiety and panic disorder
in adults. Behaviour Research and Therapy, 77, 40–51.
Heimberg, R. G., & Becker, R. E. (2002). Cognitive-Behavioral group therapy
for social phobia: Basic mechanisms and clinical strategies. New York:
Guilford Press.
Heimberg, R. G., Becker, R. E., Goldfinger, K., & Vermilyea, J. A. (1985).
Treatment of social phobia by exposure, cognitive restructuring and homework assignments. Journal of Nervous and Mental Disease, 173(4), 236–245.
Higgins, J. P., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A.
D., … Sterne, J. A. (2011). The Cochrane collaboration’s tool for assessing
risk of bias in randomised trials. BMJ, 343, d5928.
Hofmann, S. G., & Otto, M. W. (2008). Cognitive-behavior therapy of social
anxiety disorder: Evidence-based and disorder specific treatment techniques.
New York: Routledge.
Psychological Medicine
Kampmann, I. L., Emmelkamp, P. M., Hartanto, D., Brinkman, W. P., Zijlstra,
B. J., & Morina, N. (2016a). Exposure to virtual social interactions in the
treatment of social anxiety disorder: A randomized controlled trial.
Behaviour Research and Therapy, 77, 147–156.
Kampmann, I. L., Emmelkamp, P. M., & Morina, N. (2016b). Meta-analysis of
technology-assisted interventions for social anxiety disorder. Journal of
Anxiety Disorders, 42, 71–84.
Keller, M. B. (2006). Social anxiety disorder clinical course and outcome:
Review of Harvard/Brown Anxiety Research Project (HARP) findings.
The Journal of Clinical Psychiatry, 67, 14–19.
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters,
E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV
disorders in the National Comorbidity Survey Replication. Archives of
General Psychiatry, 62(6), 593–602.
Kim, H. E., Hong, Y. J., Kim, M. K., Jung, Y. H., Kyeong, S., & Kim, J. J. (2017).
Effectiveness of self-training using the mobile-based virtual reality program
in patients with social anxiety disorder. Computers in Human Behavior, 73,
614–619.
Klinger, E., Bouchard, S., Légeron, P., Roy, S., Lauer, F., Chemin, I., & Nugues,
P. (2005). Virtual reality therapy versus cognitive behavior therapy for social
phobia: A preliminary controlled study. Cyberpsychology & Behavior, 8(1),
76–88.
Kovar, I. (2018). Virtual reality as support of cognitive behavioral therapy in
social anxiety disorder. International Journal on Advanced Science,
Engineering and Information Technology, 8(4), 1343–1349.
Krijn, M., Emmelkamp, P. M., Biemond, R., de Ligny, C. D. W., Schuemie, M. J., &
van der Mast, C. A. (2004). Treatment of acrophobia in virtual reality:
The role of immersion and presence. Behaviour Research and Therapy, 42(2),
229–239.
Liebowitz, M. R. (1987). Social phobia. Modern Problems of Pharmacopsychiatry,
22, 141–173.
Ling, Y., Nefs, H. T., Morina, N., Heynderickx, I., & Brinkman, W. P. (2014). A
meta-analysis on the relationship between self-reported presence and anxiety
in virtual reality exposure therapy for anxiety disorders. PLoS ONE, 9(5),
e96144.
Lister, H. A., Piercey, C. D., & Joordens, C. (2010). The effectiveness of 3-D
video virtual reality for the treatment of fear of public speaking. Journal
of CyberTherapy and Rehabilitation, 3(4), 375–381.
Meyerbröker, K., & Emmelkamp, P. M. (2010). Virtual reality exposure therapy in anxiety disorders: A systematic review of process-and-outcome studies. Depression and Anxiety, 27(10), 933–944.
Mishkind, M. C., Norr, A. M., Katz, A. C., & Reger, G. M. (2017). Review of
virtual reality treatment in psychiatry: Evidence versus current diffusion and
use. Current Psychiatry Reports, 19(11), 80.
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting
items for systematic reviews and meta-analyses: The PRISMA statement.
Annals of Internal Medicine, 151(4), 264–269.
Morina, N., Brinkman, W. P., Hartanto, D., & Emmelkamp, P. M. (2014).
Sense of presence and anxiety during virtual social interactions between a
human and virtual humans. PeerJournal, 2, e337.
North, M. M., North, S. M., & Coble, J. R. (1998). Virtual reality therapy: An
effective treatment for the fear of public speaking. International Journal of
Virtual Reality, 3(3), 1–6.
Phan, K. L., Fitzgerald, D. A., Nathan, P. J., & Tancer, M. E. (2006).
Association between amygdala hyperactivity to harsh faces and severity of
social anxiety in generalized social phobia. Biological Psychiatry, 59(5),
424–429.
Ponniah, K., & Hollon, S. D. (2008). Empirically supported psychological
interventions for social phobia in adults: A qualitative review of randomized
controlled trials. Psychological Medicine, 38(1), 3–14.
Pot-Kolder, R. M., Geraets, C. N., Veling, W., van Beilen, M., Staring, A. B.,
Gijsman, H. J., … van der Gaag, M. (2018). Virtual-reality-based cognitive
behavioural therapy versus waiting list control for paranoid ideation and
social avoidance in patients with psychotic disorders: A single-blind randomised controlled trial. The Lancet Psychiatry, 5(3), 217–226.
https://doi.org/10.1017/S0033291720003785 Published online by Cambridge University Press
2497
Price, M., & Anderson, P. (2007). The role of presence in virtual reality exposure therapy. Journal of Anxiety Disorders, 21(5), 742–751.
Price, M., Mehta, N., Tone, E. B., & Anderson, P. L. (2011). Does engagement
with exposure yield better outcomes? Components of presence as a predictor of treatment response for virtual reality exposure therapy for social
phobia. Journal of Anxiety Disorders, 25(6), 763–770.
Qu, C., Brinkman, W. P., Ling, Y., Wiggers, P., & Heynderickx, I. (2014).
Conversations with a virtual human: Synthetic emotions and human
responses. Computers in Human Behavior, 34, 58–68.
Robillard, G., Bouchard, S., Dumoulin, S., Guitard, T., & Klinger, E. (2010).
Using virtual humans to alleviate social anxiety: Preliminary report from
a comparative outcome study. Studies in Health Technology and
Informatics, 154, 57–60.
Robillard, G., Bouchard, S., Fournier, T., & Renaud, P. (2003). Anxiety and
presence during VR immersion: A comparative study of the reactions of
phobic and non-phobic participants in therapeutic virtual environments
derived from computer games. CyberPsychology & Behavior, 6(5), 467–476.
Roy, S., Klinger, E., Légeron, P., Lauer, F., Chemin, I., & Nugues, P. (2003).
Definition of a VR-based protocol to treat social phobia. Cyberpsychology
& Behavior, 6(4), 411–420.
Safir, M. P., Wallach, H. S., & Bar-Zvi, M. (2012). Virtual reality cognitivebehavior therapy for public speaking anxiety: One-year follow-up.
Behavior Modification, 36(2), 235–246.
Scholing, A., & Emmelkamp, P. M. (1993). Exposure with and without cognitive therapy for generalized social phobia: Effects of individual and group
treatment. Behaviour Research and Therapy, 31(7), 667–681.
Sharf, J., Primavera, L. H., & Diener, M. J. (2010). Dropout and therapeutic alliance: A meta-analysis of adult individual psychotherapy. Psychotherapy:
Theory, Research, Practice, Training, 47(4), 637.
Stein, M. B., Goldin, P. R., Sareen, J., Zorrilla, L. T. E., & Brown, G. G. (2002).
Increased amygdala activation to angry and contemptuous faces in generalized social phobia. Archives of General Psychiatry, 59(11), 1027–1034.
Straube, T., Mentzel, H. J., & Miltner, W. H. (2005). Common and distinct
brain activation to threat and safety signals in social phobia.
Neuropsychobiology, 52(3), 163–168.
Stupar-Rutenfrans, S., Ketelaars, L. E., & van Gisbergen, M. S. (2017). Beat the
fear of public speaking: Mobile 360 video virtual reality exposure training in
home environment reduces public speaking anxiety. Cyberpsychology,
Behavior, and Social Networking, 20(10), 624–633.
van Dis, E. A., van Veen, S. C., Hagenaars, M. A., Batelaan, N. M., Bockting, C.
L., van den Heuvel, R. M., … Engelhard, I. M. (2020). Long-term outcomes
of cognitive behavioral therapy for anxiety-related disorders: A systematic
review and meta-analysis. JAMA Psychiatry, 77(3), 265–273.
Wallach, H. S., Safir, M. P., & Bar-Zvi, M. (2009). Virtual reality cognitive
behavior therapy for public speaking anxiety: A randomized clinical trial.
Behavior Modification, 33(3), 314–338.
Wallach, H. S., Safir, M. P., & Bar-Zvi, M. (2011). Virtual reality exposure versus cognitive restructuring for treatment of public speaking anxiety: A pilot
study. The Israel Journal of Psychiatry and Related Sciences, 48(2), 91–97.
Wechsler, T. F., Mühlberger, A., & Kümpers, F. (2019). Inferiority or even
superiority of virtual reality exposure therapy in phobias? A systematic
review and quantitative meta-analysis on randomized controlled trials specifically comparing the efficacy of virtual reality exposure to gold standard
In vivo exposure in agoraphobia, specific phobia and social phobia.
Frontiers in Psychology, 10, 1758.
Yonkers, K. A., Dyck, I. R., & Keller, M. B. (2001). An eight-year longitudinal
comparison of clinical course and characteristics of social phobia among
men and women. Psychiatric Services, 52(5), 637–643.
Yuen, E. K., Goetter, E. M., Stasio, M. J., Ash, P., Mansour, B., McNally, E., …
Watkins, J. (2019). A pilot of acceptance and commitment therapy for public speaking anxiety delivered with group videoconferencing and virtual
reality exposure. Journal of Contextual Behavioral Science, 12, 47–54.
Yuen, E. K., Herbert, J. D., Forman, E. M., Goetter, E. M., Comer, R., &
Bradley, J. C. (2013). Treatment of social anxiety disorder using online virtual environments in second life. Behavior Therapy, 44(1), 51–61.