The Design, Development, and Evaluation of an Immersive Collaborative Learning
Platform Incorporating Extended Reality (XR) Technology to Improve EFL
Interlanguage Pragmatics (ILP) Competence for Vocational College Students
以延展實境(XR)跨越真實與虛擬的互動科技探討科大生英文語用學發展
1. Project’s Background and its Significance (計畫的背景及重要性)
Introduction
Within EFL studies, pragmatics involves the organization and management of
communication, embracing appropriate speech acts, conversational structure, conversational
implicature, discourse organization, and sociolinguistic aspects of languag use, such as choice
of address forms (Bardovi-Harlig, 2001; Kasper & Schmidt, 1996; and Kasper & Rose, 2001).
Bardovi-Harlig and Mahan-Taylor (2003) summoned pragmatics as the rule of English, that
language users are required to match utterances with contexts and rules to use language
appropriately. Stalnaker (1972) conied pragmatics as “the study of languistic acts and the
contexts in whcih they are performed” (p. 383). Traditionally, curriculums and instructional
materials in language learning have seldom been pragmatics-focused, instead centering on
language skills (reading, writing, speaking and listening), resulting in gaps in practical
communication (Huang, Grant & Pasfield-Neofitou, 2013). Words do not only have meaning
by themselves, but also produce a speech act which is a way of performing an action through
words. Pragmatic development reflects how form, function and context are related to one
another, which is the ability to communicate effectively and involves knowledge beyond the
level of grammar (Austin, 1962; Searle, 1969; Yule, 1996; Leech & Thomas, 1983). CelceMurcia, Dörnyei, & Thurrell (1995) explained that effective oral communication requires
maintaining a balance in linguistic accuracy and social function. Kasper and Dahl (1991)
stated that pragmatics results in a fuller, deeper, and generally more reasonable account of
human language behaviour, comprising of three components: pragmalinguistics (form),
sociopragmatics (function) and psycholinguistics (context) (Fig. 1).
Fig. 1 Characteristics of Pragmatics Competence
Inter-language pragmatics (ILP) as identified by Kasper and Dahl (1991), involves how L2
learners acquire the ability to understand and produce speech acts in the second language.
ILP concerns the dynamic and complex processes involved in L2 acquisition and interaction,
and the effects their use of language has on other participants in the act of communication
socially (Bachman & Palmer, 2010; Kasper & Rose, 2002). ILP competence is vital for L2
learners to become successful communicators in a second language, which involves
pragmatic transfers in pragmalinguistic, sociopragmatics and psycholingustics. L2 learners
needs to perform strategies for pragmatic transfers, since their L1 in linguistic function is
often significantly different in the L2. This study is focus on examining the development of
ILP competence, how higher education students feel secure in every situation “how-to-say1
what-to-whom-when” (Bardovi-Harlig, 2013, p.68) they may face when communicating in
the target language with the aid of the latest technology.
Background of Study
The Taiwanese government acknowledged that English proficiency is a vital skill in
promoting Taiwan’s internationalization and to become more competitive in today’s global
economy. However, English education across all levels of formal schooling in Taiwan has
failed to develop students’ functional abilities in English communication (Chou, Woodbine &
Su, 2006; Horner, Lu, Royster & Trimbur, 2011; Hsieh, 2011). Despite efforts made at the
ministerial level to change how students’ English proficiency is measured, the traditional
school-based English exam continues to be used, applying a scored item format (i.e. multiplechoice questions) to answer grammar questions and fill in vocabularies, rather than testing for
practical usage in writing and oral communication. While many students are able to score
highly on traditional exams, they are often incapable of having an actual authentic English
conversation (Lin, 2018). An overemphasis on learning grammar in the classroom, and the
lack of opportunity for students to practice communication effectively, finds students often
misusing vocabulary and neglecting content causes. Moreover, Chen (2013) pointed out there
is a shortage of qualified local English teachers available to conduct effective English
instruction. In fact, many teachers themselves often lack training in their own communicative
language ability. Therefore, the Ministry of Education (MOE) has introduced a scheme aimed
at hiring qualified English teachers from native English-speaking countries to serve as seed
teachers at elementary, junior and high school levels under the Cabinet’s vision of turning
Taiwan into a Mandarin-English bilingual country by 2030 (Nguyen, 2018). However, the
ratio of native speaking teachers to students is far too low in order to perform effective
teaching.
Research Gap in Pragmatics Development with Latest Technologies
Children learning English as a first language (L1), obtain pragmatic skills through
engagement in daily communicative activities with adults. However, opportunities for EFL
learners to communicate in English outside the classroom are limited, and instructional
activities that constitute their classroom experiences are also regulated by the EFL teachers’
own pragmatic knowledge and skills (Crystal, 1997; Taguchi & Roever, 2017). Thus, the lack
of any systematic appearance of pragmatic features and practices in fully English-speaking
environments, lead to EFL learners’ lack of opportunities to fully develop their pragmatic
competence (Bardovi-Harling, 2001).
Since it is difficult for traditional EFL classrooms to afford resources in providing authentic
input and varied social contexts for developing ILP competence, technology offers great
potential for pragmatics learning (Sykes, 2017; Taguchi, 2015; Taguchi & Roever, 2017;
Taguchi & Sykes, 2013). Integrating digital resources and maximizing the benefits of online
learning environments to improve EFL learners’ communication skills is a main aim in the
Taiwan MOE’s 2030 bilingual policy. Digital technology has become a part of our daily lives,
and so too is having an impact on the world of education. Education is undergoing a massive
transformation as a result of the digital revolution that new technologies are creating, and is
challenging the traditional modes of learning in schools and in colleges (Collins & Halverson,
2018). According to the Ambient Report (2016), a flood of new highly-advanced and lowcost technologies are becoming available, offering adaptive and individualized education
programs to suit the changing learning habits of the next generation (Fig. 2). These new
learning technology innovations are heavily concentrated in four learning product types:
Simulation-based learning, game-based learning, cognitive learning, and mobile learning.
They involve the latest technologies that are the major market trends for new education
learning, in particular the products that integrate with virtual, augmented and mixed realities
(VR, AR and MR), and artificial intelligence (AI) technologies.
2
Fig. 2 2016-2021 Global Five-year Growth Rates for Learning Technology Product Types
More recently, some research studies have suggested that extended reality (XR) will be the
next step in reality-environment technologies including real-time augmented reality decision
support (Jowallah, Bauer, Futch & Gunter, 2019; Leal, Chapman & Westerlund, 2019; Joibi,
Yong, Gee, Lee & Myung, 2019; Sheshagiri, Baheti, Gute & Lakshmikantha, 2019). Since
XR is receiving more recognition across a variety of industries, its growth is expected to
grow eightfold, reaching an estimated market size of more than US$209 billion by 2022 and
to US$900 billion by 2025 (Marr, 2019; Scribani, 2019), which will become the mainstream
in the next five years (Fig. 3). These new learning niches are increasingly focussing on the
subjects of adaptive and individualized education which enable people of all ages to pursue
learning on their own, where they want to learn, what they want to learn, and how they want
to learn, in an entirely new digital world.
Fig. 3 The Trends of Extended Reality (XR) Source: https://www.visualcapitalist.com/extended-reality-xr/
The Transformation of Educational Technologies
Technology continues transform peoples’ interests, habits, and how time is spent. It also
applies to education, becoming central to the way people read, write, think, and learn. For
example, Technavio Report (2018) indicates that the global K-12 online tutoring market is
projected to grow at a compound annual growth rate (CAGR) of almost 12% from 2018 to
2023, with incremental growth to US$60.03 billion by 2023 (Fig. 4). The report has also
identified the top three emerging marketing trends expected to impact the future K-12
education beyond 2018:
1. Student support beyond the classroom
To provide customized intelligent tutoring systems using AI tools beyond the classroom
hours of instruction.
3
2. Automation of admin tasks with AI
To provide automated education solutions including administration and admission
processes, also able to provide in-depth reports on errors made by students, leaving more
time for teachers to plan lessons and conduct one-to-one interactions with students.
3. Bringing immersive technology to the classroom
AI in education, facilitates a more practical learning experience for students through
immersive technology.
Fig. 4 Global K-12 Online Tutoring Market 2019-2023 by Technavio
Source: https://www.technavio.com/report/global-k-12-online-tutoring-market-industry-analysis
Within the rapidly expanding digital education industry, many various new AI-based
education products are now hitting the market. Fastest growth is coming from intelligent
tutoring systems (ITS) which are the advanced version of online tutoring services. ITS
provides one-on-one teaching and training by AI technologies and cognitive psychology
models, providing experienced instructors, automatically and cost-effectively, while offering
adaptive teaching for L2 learners. Statistics have shown that learning retention rates of faceto-face training is only 8% to 10%, but e-learning is between 25% to 60% (Gutierrez, 2016)
as students have more control over the learning process and they can revisit the materials
whenever they need to and provide adaptive learning. For example, in China, the online
education market has taken off rapidly with an estimated revenue of US$44 billion in 2019,
up 24.5% from the previous year and projected to reach US$77 billion in 2022 (iResearch,
2019) (Fig. 5). Higher education and vocational training make up about 80% of the online
education market in China (Fig. 6). Students and parents have expressed that online education
provides the flexibility in scheduling when lessons are conducted, as well as the ability to
customize the content of the lessons, leveraging various technologies, to suit the learning
pace and prior knowledge of learners.
Fig. 5 Revenue of Online Education in China Fig. 6 Shares of China’s Online Education
2012-2022 by iResearch
Market Segmentations in H1 2019
4
Since education trends are on the precipice of major change, integrating technology and
transforming students’ motivations, numerous companies such as Apple, Google, Facebook,
Microsoft, Sony and others, have been researching and investing in new forms of humancomputer interaction (HCI). Many institutions of higher education are adapting and adopting
their products to deliver new learning experiences. New forms of HCI such as AR, VR, MR
and XR, although varying in their technology and performance, all involve some similar
technology, such as 3D objects combining with cognitive computing and AI. Computers now
can create virtual learning environments and customize education to the particular needs and
abilities of individual learners, while educators are attracted by the potential of using 3D
virtual environments for language teaching and learning (Zheng, Newgarden, & Young,
2012).
Today, foreign language learners are faced with a pragmatic revolution in digital technology
the way in which they can feel immersed in the authentic nature of a virtual world. These
forms of HCI have led to an increase, not only in the use of two-dimensional virtual
environments, but also three-dimensional multi-user virtual environments (3D MUVEs)
which are increasingly being seen as the future trend in creating authentic learning
environments (Doğan, Çınar, & Tüzün, 2018). In 3D virtual learning environments (VLE)
where students learn in situated contexts, it allows them to experience realities unavailable
within the classroom. These strategic planning VLEs are effective educational methodologies
and offer the potential to enhance students’ motivation to learn through their simulated gamelike experience. It also supports the students in developing a sense of empowerment, as well
as, provides collaborative and interactive opportunities with their peers and other users
(Nonis, 2005). For instance, virtual communities provide great opportunities for L2 learners
to easily use language in real contexts during work or leisure related settings, and remotely
communicate with other English users enhancing pragmatics skills.
The Horizon Report 2019, states that MR is expected be a commonly adopted technology in
education in the next two to three years, with its interactivity will confer significantly adding
the potential for learning and assessment. MR covers the continuum from AR to AV and aims
at blending the real and virtual environments in different ways (Fig. 7). According to
Microsoft (2019), MR is a spectrum, the result of blending the physical world with the digital
world. Nevertheless, reality technologies have gone further in transforming our environments
introducing a relatively new development in most circles – called the extended reality (XR).
XR covers the full spectrum of all real-and-virtual combined environments and humanmachine interactions generated by computer technology and wearables, merging the physical
and virtual, bringing all realities (AR, VR and MR) together under one term. In this study, we
will develop an immersive 3D MUVE platform adopting the XR technology to nurture the
cognitive learning of interlanguage pragmatics (ILP) development.
5
Fig. 7 The XR portrayal and all reality technologies between the real and virtual
environments
XR generation and presentation tools define an extended reality environment based on certain
elements of digital content, such as captured digital video, digital images, digital audio
content, or synthetic audio-visual content (i.e. computer-generated images and animated
content). The tools can deploy the elements of digital content on a mobile device for
presentation to a user through a display, such as a head-mountable display (HMD),
incorporated within an extended, virtual, or augmented reality headset. The user of the mobile
device may also explore and interact with the extended reality environment (i.e. as presented
by the HMD) via gestural or spoken input to the mobile device. For example, the mobile
device can apply gesture-recognition tools to the gestural input to determine a context of that
gestural input and perform additional operations corresponding to the determined context.
The extended reality computing system or the mobile device can also determine a position
and orientation of the mobile device (i.e. a position and orientation of the HMD or the
augmented reality eyewear) within the extended reality environment, and can further obtain
data indicating a position and orientation of one or more other users within the extended
reality environment.
MUVE platform’s incorporation with the XR technology is a meta-cognition technology that
enable users to modify cognitive behaviour (learn) by understanding and manipulating the
learning process itself (Poppe, Brown, Recker, Johnson, & Vandeerfeesten, 2017; Ibáñez,
Rudea, Maroto, & Kloos, 2013). MUVE activities provide different features for people to
interact by presenting as avatars and enable distance members of a group to collaborate in
project studies. According to Salmon (2004), collaboration is only possible when users feel at
ease and are provided with the opportunity for online socialization. The virtual platform
generates a feeling of presence and provides social interaction which can affect group
performance positively (Bosch-Sijtsema & Haapamaki, 2014). Social interaction provides
communication and collaboration within a group which is a crucial process for pragmatics
development. The reality conforms to the lack of an English environment, and integration
with native speakers. L2 learners feel anxious and less willing to communicate in English,
and so usually switch back to their native language after leaving the English classroom
(Jauregi, 2016; Lan, Fang, Legault & Li, 2014). Learning in virtual environments represented
by avatars, can provide safer settings to practice the target language. 3D VLEs and XR
technology are frontier educational methodologies, so there is still limited research on EFL
education purposes.
6
At the present time, the Government of Taiwan is promoting the Digital Nation and
Innovative Economic Development Program (DIGI+) and by 2025, Taiwan’s digital
economy is expected to expand to US$213.73 billion. In addition, the government has
allocated an annual budget of NT$10 billion, to be carried out over four years, to transform
Taiwan into the best possible environment for AI innovation and applications, in their Taiwan
AI Action Plan (2018). One of the five key initiatives is the AI Talent program, which
promotes innovative clusters of AI talents and provides a convenient living and educational
environment in Taiwan. This can encourage researchers to conduct studies on frontier
educational technologies to enhance new teaching methods.
This research study aims to examine interlanguage pragmatics (ILP) development by
activating metacognitive skills with the efficacy of a new immersive 3D MUVEs learning
platform, integrating a consortium of XR technologies.
2. Relevant Research, Literature Review and Educational Theory (相關研究、參考文獻
及教育與心理的理論基礎)
L2 Inter-Language Pragmatics Competence
Pragmatics was originally developed as a subfield of linguistics, coined by Morris (1938),
that deals with the origins, uses, and effects of movements, gestures, tone of voice, and signs,
within the total behaviour of interpreters, and often accompanies speech. Pragmatics research
is widely studied in the fields of philosophy, sociology, linguistics, and anthropology. It is
concerned with the way people produce and comprehend meaning through language, and
how others interpret their utterances in social contexts (Nordquist, 2019). Fig. 8 indicates the
development process of pragmatic competence for L2 learners, and comprises not only
linguistic and grammatical usage, but also considers the appropriateness of language,
embracing the language producing process and its producers, with a focus on more authentic
language use.
Fig. 8 Pragmatic Competence Development Process for L2 Learners
Source: Cork English College, 2015
Kasper and Dahl (1991) revealed that ILP involves comprehension and production of speech
acts including pragmalinguistic, sociopragmatics and psycholinguistic aspects of language
use. This pragmatic development is the reflection of the relationship between form and
function associated with context. In other words, the performance of communicative acts by
L2 learners has to assess linguistics choices (pragmalinguistics) and aspects for social support
(sociopragmatics). The importance of the association between form and functions of language
has been revealed in many studies. Learners do not only need exposure to linguistic choices,
but also to mastering a balance between linguistically accurate and socially functional
utterances (Celce-Murcia et al., 1995). Thus, the systematic, formulaic, and interactional
7
aspects of language, must all be addressed in effective language instruction for L2 learners to
understand that they can be significantly different to their L1 pragmatics, and required to
perform a linguistic function to transfer (Barron 2006; DuFon & Churchill, 2006).

Pragmalinguistics (Form)
Pramalinguistics is the integration of grammar with pragmatics to find the most
appropriate and practical structures for utterances in a language.

Sociopragmatics (Function)
Sociopragmatics is defined as the sociological interface of pragmatics as the social
perceptions that underlie participants’ interpretation and performance of communicative
action. The appropriateness of the social behaviour of the language culture is corelated
with the outcome of making pragmatic choices by the language learner. Once the
development and achievement of pragmatic competence is linked to both linguistic and
non-linguistic signals, in relation to social organized activities, it is believed that learners
would develop their sociopragmatics sensitivity and knowledge once they experienced
diverse patters of communication in the target culture (Shimizu 2009; Taguchi 2008).

Psycholinguistics (Context)
Pragmatics takes context into account to complement the contributions that semantics and
grammar make to meaning. The context, concerns the social signs, gestures, and tone of
voice, and what makes utterances clear or unclear to the speaker or listener. (Brisard,
Ostman, & Verschueren, 2009). It follows that psycholinguistics deals with the linguistic
environment of thought influencing thought, the mental use of language (Takahashi, 2001;
Kasper & Rose, 2002).
All L2 learners, at the beginner level, possess basic communicative competence, which may
be exhibited by communicating without a grammatical structure, i.e. ‘Pen!’ for asking for
someone’s pen. Once some grammatical knowledge begins to be developed and some more
vocabulary is acquired, then the utterance may be able to be established, i.e. “Give me your
pen, please!”. However, while this sentence is grammatically correct, the imperative
statement may be generated as being impolite, even though the L2 speaker is not being rude,
but rather she/he is merely pragmatically incompetent. Unsworth (2004) studied the
development of the pragmatics-syntax interface in L2 acquisition and deduced that mapping
between pragmatics and syntax is problematic for both L1 and L2 learners. Bos, Baars, &
Raaij (2004) argued that L2 learners’ knowledge of pragmatics is largely a function of their
L1 pragmatics, or semantics that could transfer to their L2 syntax. Furthermore, Jernigan
(2007) displayed an integrative model for the acquisition of pragmatics competence (Fig. 9).
Fig. 9 Theoretical Model of Pragmatic Competence Acquisition
8
In the model, the coherence in placement and co-ordination is important between each step of
the pragmatic acquisition. From pragmalinguistic input to output, linguistic data interacts
with sociopragmatic signals, and learners become socioculturally and pragmatically
conditioned, before entering the interlanguage system. Upon entering the dynamic field of
interlanguage, the data passes through a pragmatic filter that modifies pragmalinguistic input
to account for the sociocultural context in which the input was produced before converting to
intake. The next stage for the linguistic input is to get into the “Halo”, in which the grammar
component first runs its normal grammatical analysis and then selects a set of utterances for
the final pragmalinguistic form. As the pragmatic acquisition system is an intellectually
iterative process, the grammar component is involved in processing both the input and the
output. The pattern of the grammar and pragmatic is mutually complementary in that the
grammar module can alert the pragmatic component regarding the need for intra-sentential
anaphora, and the pragmatic component can update the grammar component on the items in
the linguistic data. With the pragmatic component at the core of the model, the task is to
receive the processed linguistic data that is needed to refer to outside concepts. Prior to the
output, the explicit knowledge and metapragmatic awareness are in contact with final
pragmalinguistic forms, which might be considered a natural outgrowth of the learners’ use
of language to articulate the intended speech acts. The output helps the learner use
interlanguage to test his or her hypotheses about pragmalinguistic production possibilities and
determine whether they are accepTable Theoretically, the output can be compared with
authentic pragmalinguistic forms in the input for L2 development.
Some significant research studies have been conducted to adopt the integrative model on
output and developing L2 pragmatics (Eslami & Liu, 2013; Martinez-Flor & Fukuya, 2005;
Jernigan, 2007, 2012; Piri, Pishghadam, & Dixon, & Rasekh, 2018). Furthermore, there are
studies attempting to investigate the effective combination of technology and pragmatics. For
example, the University of Maryland Centre of Advanced Study of Language (CASL)
undertook a research project in 2008, examining an online group that completed a yearlong
task-based Chinese course, and a control group, who only completed the pre/post course
proficiency test. The online course that students completed was based on audio-visual input
from real people performing transactions, where role-plays were performed by using
computer-mediated communication and online conversations with feedback in real time. The
results concluded that the online students were outperformed due to the fact that they spent so
much time studying materials that were individualized for each of them by the computer,
meeting the needs of different types of learners.
In the same vein, Takamiya (2008) examined the promotion of pragmatic awareness
regarding culture through blogging. Intermediate and advanced US learners of Japanese,
conducted individual research through interviews and online readings regarding topics from
the course. Students were asked to post their findings on their blogs, from which they should
receive feedback from partners and Japanese speakers in the university. By being engaged in
an online virtual platform, learners experienced enhanced pragmatic awareness in their
understanding of given issues, such as humility in gift giving customs in the target culture.
The researchers supported that blog-based interaction as a potential tool in promoting L2
pragmatic development.
Adaptive Educational Criteria for Cognitive Learning
Much research has reported the combination of pragmatics and technology enhanced
pragmatics awareness (Barak & Levenber, 2016; Lewin & Lundie, 2016; Sykes, 2017;
Taguchi & Roever, 2017). Adaptive learning environments can be developed to be
cognitively flexible according to users’ specific criteria, to infer users’ preferences, and
consequentially to facilitate learning. Technology affordance encourages adaptive learning by
social interaction, communication, and collaboration. Adaptive education interfaces, such as
intelligent tutoring systems (ITS) and adaptive educational hypermedia, can be modified to to
9
accommodate individual differences for different educational contexts and student needs. In
order to facilitate learning for students of various cognitive cultural, and socioeconomic
backgrounds, Table 1 presents some adaptive ideas for creating an adaptive learning
environment for pragmatics development (Wang & Tsao, 2015):
Table 1 Adaptive ideas for an adaptive learning environment for pragmatics development
There are more and more advanced technologies innovated to help students develop their
metacognitive skills by modelling their thinking (Normann & Morina, 2018). The
metacognition technologies available enable students to modify their learning behaviour by
understanding and manipulating the learning process itself. There are three primary types of
cognitive learning products on the market (Ambient Report, 2018):



Cognitive assessments: evaluate and measure the spatial perceptions, verbal abilities,
memory, problem-solving skills, temperament, and ways of thinking of users;
Cognitive and intelligent tutors: is a metacognition technology that acts like a human
mentor and provides personalized responses, remediation, and intervention, in real
time, for a particular user.
Brain training and brain fitness products: these include new suppliers bringing
innovative products to market, such as various reality products, game-based
simulation and neurotechnology, that assess and train the cognitive states of users.
MUVEs have been characterized by a situated, socio-constructivist (Duffy & Kirkley, 2004)
and adaptive (Wang, 1984; Santoianni, 2010a) approach to learning, enhancing motivation to
learn through the a simulated game-like experience. Moreover, in 3D VLEs, students learn in
situated contexts, which allow them to have experiences unavailable within the classroom,
through simulated what-if scenarios. XR technologies apply to the 3D VLEs, and constitute a
metacognition process that draws on the ability to reflect on the knowledge and process,
which can be evaluated for further learning. The metacognitive technologies help students to
exert their knowledge and strategies to solve problems more efficiently. When students need
to learn new concepts, solve problems, make connections or create relationships between
ideas and concepts, they require self-control and self-regulation to perform these tasks by reusing and adjusting their learning strategies consciously.
For conducive L2 pragmatics awareness, it is important for learners to be exposed to
authentic and rich language input of the target culture. Nevertheless, the reality is that L2
learners are not always able to travel and experience the target culture in real life. A lack of
10
regular interaction with native speakers, lead most of the EFL students to develop learning
anxiety in communication and makes them more reluctant to communicate in English
(Reinders & Wattana, 2014). For example, Martinez (2018) conducted a study and
highlighted a teaching proposal for 10 and 11-year-old English language learners to achieve
pragmatic fluency in apologizing by a great variety of computer-assisted activities and tools
that teachers can use to teach pragmatics. Results in the implementation of this teaching
intervention have shown that the use of tasks and technology in order to teach the speech act
of apologies, is really positive for the primary learners. The participants improved their
pragmatic competence and they enjoyed the learning activities as they demonstrated their
newly-acquired skills.
In view of these, Second Life (SL) offers features such as task simulation, real-time
collaboration, identity exploration, and flexible multimodality, which simulate as real-world
tasks to make English learning more experiential, and reinforce engagement for L2 learners
(Chun, Smith, & Kern, 2016; Gonzalez-Lloret & Ortega, 2014). SL is an immersive learning
environment that has also drawn attention to many SLA researchers and has revealed
positive findings in L2 acquisitions.
Jauregi and Canto (2012) examined Spanish learners' interaction patterns in intercultural
communication tasks with native speakers or peers in three settings: Adobe Connect, SL, and
a traditional class. The research results indicate that SL stimulates more negotiation patterns
than the other two settings, and allows learners to produce more communication acts. It also
suggests that SL provides a greater platform for interaction with native speakers, and learners
report having benefitted from interacting through avatar forms to perform tasks. In a similar
vein, Lan (2014) found that SL provides an immersive environment which enables Chinese
language learners to perform tasks in the virtual world, and resulted in oral communication
skills more significantly than those in a traditional class. Students participating through the
avatar forms enabled them to become more proactive and able to keep the conversation
flowing. Hence, SL is perceived as a unique learning environment that allows for simulation
of real-world scenarios for foreign language acquisition and collaboration.
Extended Reality (XR) Collaboration with MUVE Online Learning Platform
MUVEs provide an extremely natural medium for three-dimensional computer supported
collaborative work (CSCW). This setting provides the same type of collaborative information
that people have in face-to-face interactions, including object manipulation (O’Malley,
Langton, Anderson, Sneddon & Bruce, 1996). Gesture, voice, and graphical information can
all be communicated seamlessly between participants, and all of these are important in
providing information for pragmatics awareness. Functional and cognitive seams in
collaborative interfaces (such as video-mediated conversation) do not produce the same
conversation style as face-to-face interaction. This occurs because video cannot adequately
convey the nonverbal signals so vital in face-to-face communication, introducing a functional
seam between the participants. Thus, participants sharing the same physical space positively
affects conversation in ways to convey the nonverbal signals for a functional seam between
communications.
XR interfaces are ideal for CSCW because they meet the needs of collaboration and face-toface communication, as it can overlay the virtual and real world in the MUVE platform in
real time. This allows for the creation of XR interfaces that combine the advantages of both
virtual environments and seamless collaboration. XR technologies enable information overlay,
where the virtual and real-world may be used by remote collaborators to annotate the user's
view, or may enhance face-to-face conversation by producing shared interactive virtual
models. In this way, XR techniques can be used to enhance communication, regardless of
proximity, and supports seamless collaboration with the real world, reducing the functional
and cognitive seams between participants. These attributes imply that XR approaches would
be ideal for many CSCW applications.
11
Being a fairly new term, XR, not yet prevalent in the literature, conveys the idea of extending
human experiences and the breadth of technologies necessary for implementing such
experiences (Johnson, 2019; Smith & Cousins, 2019). At a minimum, developing XR
experiences requires a display device, an input device, and a virtual environment (VE). Each
of these components utilizes a range of technologies covering the entire real and virtual
environment. Prior to the development of XR, VR and AR, were two common categories of
hardware technologies that enabled the 3D graphics, spatialized sound, and motion tracking
necessary for simulated and augmented user experiences. The main difference between these
technologies is the level in which the XR user is immersed into a simulated environment. As
far as XR development goes, XR devices create the illusion to make people feel as if they are
in an entirely new digital world. Such technologies have been praised for their ability to
create virtual tours in stores and destinations, rehabilitate brain injuries, and virtually inspect
the interior and exterior design of a car which no existing technology could do. Given its
enormous potential, XR technology has been increasingly applied and studied in a plethora of
fields, ranging from tourism, education, retailing, gaming and healthcare, to manufacturing
(Fig. 10) (Mordor Intelligence, 2018).
Fig. 10 Industry Application Consumer XR, in %, 2018
Currently, in the academic area, there are only a few studies related to the use of XR for
training in various fields, however, there are no studies on its effectiveness for language
learning yet. Through virtual simulation or augmentation, XR has the capability to enhance
existing training techniques or to provide new ones in areas where traditional methods are
expensive, impractical, or are otherwise infeasible. Various fields have utilized these
capabilities to improve training processes. Research has shown the potential for effective XR
training environments in surgery and medicine (John, Phillips, Cenydd, Coope, Souza, and
Watt, 2016), production assembly and maintenance (MurciaLopez & Steed, 2018; Werrlich,
Nguyen & Notni, 2018), spatial navigation (Sharma, Mehra, Kaulgud & Podder, 2019),
neurorehabilitation (John, Pop, Ritsos, & Headleand, 2018), and sports (Miles, Pop,
Lawrence, & John, 2012).
Recently, there have been numerous studies that have shown positive results in XR training.
Gonzalez-Franco, Pizarro, Cermeron, Thorn, Hutabarat, Tiwari, and Germell-Garcia (2017)
performed a study evaluating the learning transfer of complex manufacturing training in XR
compared to traditional face-to-face training. In their work, the trainer and the trainee wore
XR HMDs to perform collaborative training and interacted with a virtual model of the
manufacturing elements; whereas, face-to-face training employed a scaled physical model
during the training session. Their results supported the use of collaborative XR training was
more effective. While these results are positive for observation-based training, the benefits of
XR lie with its potential for experiential training. Borsci, Lawson, Jha, Burges, and Salanitri
(2016) compare two XR based experiential training environments with video based
observational training. One XR environment employed a CAVE for immersive training and
the other used a holographic 3D Table All groups received similar instructions about how to
complete the maintenance process, but the XR groups had additional training through
performing trial procedures in the XR learning environments. Post tests revealed that both
12
XR trained groups outperformed the video trained groups and there was no significant
difference between the two XR groups.
More recently, Thomsen, Bach-Holm, Subhi, Saleh, Park, and Cour (2017) investigated
learning transfer with a stereoscopic training simulator for cataract surgery in the operating
room. In this work, the authors concluded that surgical skills learned in the XR simulator
were more effective. This positive result provides evidence that XR training has the ability to
improve learning in complicated surgical operations.
Kunnen (2015) reports on several advanced virtual assistance technologies that allow realtime collaboration in virtual reality: Oculus Rift, wearable computing with Google Glass; a
telepresence-based robot from Double Robotics; gesture computing with Leap Motion;
mobile devices such as the Swivel personal video “capture” solution. Leap Motions creates
simulations where users interact with virtual objects in a game environment. Swivel collects
and delivers content, such as turning a mobile device into a presentation delivery tool.
The XR market is moderately competitive and consists of a few players. In terms of market
share, some of the players are currently expanding their services across the emerging market.
However, with the advancement in the immersive technology trend across the virtual
platform, new players are increasing their market presence, thereby enhancing their services
as well as expanding their business footprint across the emerging economies, such as
Softserve, Qualcomm Technologies, Accenture and Northern Digital. Microsoft has invented
the application of holograms to business and education in the MR experience, offering an
immersive mixed reality experience. Its HoloLens headset enables high-definition holograms
to come to life in the real world, seamlessly integrating with physical places, spaces, and
things (Microsoft, 2015).
HoloLens 2 (Fig. 11) is the new version MR headset by Microsoft, which will enter the
market by end of 2019. The new version of the headset will be enhanced by the reliability,
security, and scalability of cloud and AI services from Microsoft (Microsoft, 2019).
HoloLens 2 allows users instinctually to interact, touch, grasp, move, read text and see
intricate details on 3D images, overlapping the virtual and real environments. Users can build
holograms, which are objects made of light and sound that appear in the world around us, just
as if they were real objects (Fig. 12 & 13). Holograms can also respond to users’ gaze,
gestures and voice commands, and can interact with real-world surfaces in surroundings.
These developments have the potential to revolutionize the education field. In this
educational study, HoloLens 2 is proposed as the MR headset incorporating with the new XR
system for the research.
Fig. 11 The new HoloLens 2 by Microsoft – a mixed reality headset
13
Fig. 12 Example of reading text and details on
3D image with HoloLens
Fig. 13 Example of interaction with
hologram instinctually
Pedagogy and Strategy for Pragmatics Teaching
The main concern of language educators is how to develop an accurate pragmatic behaviour
in instructional settings. However, there is an additional difficulty in the teaching of
pragmatics in foreign languages: there is limited exposure and interaction with speakers of
the target language in order to experience these societal conventions in L2 teaching.
Therefore, there are different ways in which pragmatic transfer can have a profound impact
on students’ experiences within English speaking environments and on their interlanguage
progress. Fig.14 shows the devastating process that pragmatic failure can trigger for a L2
learner and teachers needs to intervene in the breakdown.
Fig. 14 The devastating process that pragmatic failure can trigger for a L2 learner
Creating a classroom atmosphere where students can have opportunities to make the most of
the instruction available is a demanding process. Trying to make students communicate via
the World Wide Web is not enough to promote their receptivity and reduce the anxiety that
learning L2 entails. As an example, students are often fearful of speaking or expressing their
doubts in front of their classmates, because they do not know if the question is appropriate or
not. However, it is necessary for students to be comfortable while working with the language
in order to promote their understanding as well as to promote self-confidence. In this regard,
implementing individualization should be enhanced for teachers to be aware of the
differences in their students' learning needs. Since public schools are overcrowded with
students, students are often squeezed in one single classroom. It is thus important for teachers
to understand that each learner has their own needs and learning requirements, that can be
different from their peers. For this reason, this pedagogical proposal was designed where
individualized instruction allows teachers to understand and be aware of the wants and needs
of each student.
This affirms the usefulness of autonomous learning for pragmatics, and prompts us to
consider strategy instruction as an alternative approach to pragmatics teaching. Therefore,
technologies play an important role allowing learners to have out-of-class learning
14
independently. Strategy instruction could offer solutions to two existing problems in
pragmatics instruction. One problem is that classroom learning is poor in opportunities for
pragmatics learning due to the paucity of pragmatics-focused input and opportunities for
pragmatic practice in a classroom (Bardovi-Harlig, 2001; Diepenbroek & Derwing, 2014).
In recent years, researchers have understood the positive effects it may have on the
teachability of pragmatics and a great deal of investigation has been carried out. Individuals
make choices and build strategies based on some of the unique properties of pragmatic
competence, such as the following (Balconi & Amenta, 2010):
 variability: the range of communicative possibilities to formulating communicative
choices;
 negotiability: the possibility of making choices based on flexible strategies;
 adaptability: the ability to modulate and regulate communicative choices in relation to
the communicative context;
 salience: the degree of awareness reached by communicative choices;
 indeterminacy: the rearrangement of pragmatic choices during interactions in order to
fulfill communicative intentions;
 dynamicity: development of the communicative interaction in time.
Theoretical Framework: Strategic Self-Regulation Model (S2R)
The underpinning of this research project is to sustain users’ self-efficacy through a discovery
learning process regulated by users’ personal experiences by participating in different scenes
and doing different activities combining the virtual and real world. Students who participate
and use the new XR system should develop their metacognitive skills in order to become
autonomous and effective, and reach their full potential. Since the system provides automatic
feedback on the work accomplished, students are able to foster their metacognition skills and
also become more active learners when using the assistive technology.
The learning environment of the new system is designed to be modified and customized by
users through the chance of moving 3D objects to modify the learning environment in a
structural coupling way. In language learning, a learner manages strategies by selecting the
tactics most suitable to the situation and makes it effectively communicated across the target
audience. In this sense, learning is conceived during performing the task and overseeing the
learning process. Therefore, learners can manage their own learning process by selfregulation strategies (Oxford, 2011, 2016). The strategic self-regulation model (S2R), is a
model in which learners can actively and constructively use strategies to manage their own
learning process by self-regulation. Self-regulated learning strategies involve conscious,
deliberate, and goal-oriented attempts to manage and control efforts to learn the target
language (Afflerbach, Pearson & Paris, 2008).
The S2R model presents strategies in three dimensions: cognitive, affective, and
sociocultural-interactive strategies. The cognitive dimension deals with the form of
pragmalinguistics which involves the process of constructing, transforming, and applying L2
knowledge. The other dimension is affective dimension which deals with the content of
psycholinguistics, in relation to the mental aspect of learning, such as creating positive
emotions, attitudes, and motivation. The last form is sociopragmatics, which can be analysed
by the sociocultural-interactive dimension, which is dedicated to the areas of communication
and sociocultural contexts, and facilitate learners’ interaction with people and communities
that are in target cultures. Each of these dimensions comprises a set of meta-strategies and
specific strategies. Meta-strategies help manage and control L2 learning in the executivecontrol functions, such as planning, obtaining resources, implementing plans, monitoring, and
evaluating. The structure of the S2R model with example tactics is displayed in Fig. 15
(adapted from Oxford, 2011, p. 24).
15
Fig. 15 Structure of meta-strategies and strategies in the S2R model
3. Research Objective and Framework of the Three Year Project
Corresponding to 405.3 (新興技術與人機互動科技) and 409.4 (電腦輔助外語教學), this
three-year project aims to explore and examine the interlanguage pragmatics (ILP)
competence development by activating students' metacognitive skills, with the efficacy of a
new immersive 3D MUVEs learning platform integrating the consortium of XR technologies
with the aid of Microsoft HoloLens 2 device. This research will engage vocational college
students in accomplishing various projects to implement the XR system through synchronous
multimodality. With all these aspects combined, we propose the focus of this 3-year project
as shown in Fig.16.
Fig. 16 Focus of the three-year project
The study will span the course of three years, where each year represents a significant stage
of development that ultimately leads to vocational college students’ communicative skills and
interlanguage pragmatic competencies. In the first-year, the project aims to develop
vocational college students’ pragmalinguistics skills with cognitive strategies through
adaptive 3D VLE one-to-one tutoring and conversational coaching and examine its
effectiveness. In the second year, the project will then investigate vocational college students’
perceived benefits and motivation cooperating with peers in an XR learning platform to
develop psycholinguistics competence. In the third-year project, Taiwanese students will
collaborate with native English-speaker students through the XR system with Microsoft
HoloLens. The goal is to examine how vocational college students overcome knowledge gaps
16
in communication and deal with sociocultural contexts with international students for
interlanguage sociopragmatics competence development. The aims of this 3-year project are
detailed in Table 2.
Table 2. The Aims of Three-Year Project
Year
Participants
First year
(20192020):
First-year
students from a
two-credit
required course
“Aural and
Speaking”





Second
year
(20202021):
Second-year
students from a
two-credit
required course
“Multimedia
English”




Third year
(20212022):
Third-year

students from a
three-credit
required course

“Intercultural
Communication” 

Research Focus
Cognitive strategies of
pragmalinguistics (form)
development
In-class individual project
Adaptive 3D VLE one-to-one
intelligent tutoring and
conversational coaching
Implement with Cognii and ELSA
Speak applications in the XR
learning platform
Explore the variance between
intelligent tutoring on XR platform
and ordinary classroom tutoring;
evaluations on practicing oral
English and development of
pragmatic competence by using XR
Affective strategies of
psycholinguistics competence
(context) development
Between peers in the same class and
form groups cross-over, peers’
collaboration
Second live software integrates with
the new XR system
Investigate vocational college
students’ psycholinguistics
competence and the effectiveness of
collaboration activities, also review
the perceived benefits and
motivation of the XR platform
Sociocultural-interactive strategies
of ILP competence
(Sociopragmatics – Function)
Interact with English native speaker
college students
Examine how the authentic virtual
environment on the XR platform
facilitate students’ interaction with
foreigners and intercultural
exchange
Investigate how to overcome
knowledge gaps in communication
and deal with sociocultural contexts
for interlanguage sociopragmatics
competence development
System Development
Build-in essential
aspects of augmented, virtual-, and mixedreality application
including:
 Overlapping real
and virtual world
 Tracking and
registration
 Virtual environment
modelling
 Computers, display,
and devices for input
and tracking
 Interaction
interfaces
 Adaptive 3D VLE
one-to-one tutoring
and conversational
coaching
 Immersive real-time
collaboration
MUVE platform
17
Design and Development of the Immersive XR Learning Platform (XR 系統設計及開發)
The new XR system can be deployed by computer or mobile device, incorporative with the
mixed reality headset – HoloLens 2 for displaying 3D elements of the digital content. The
headset can display the graphical elements as augmented reality layers superimposed over
real-world objects. Users can access or establish the deployed portions of the digital content
using their respective mobile devices to explore the augmented or other extended reality
environment, and also can be deployed to other mobile devices. In face-to-face conversation,
speech, gesture, body language and other non-verbal cues are all combined to show attention
and interest and convey meanings. However, the absence of spatial cues in most video
conferencing systems means that users often find it difficult to know when people are paying
attention to them, to hold side conversations, and to establish eye contact (Sellen, 1992 &
1995). Although the online tutoring industries have been well-received in the past few years,
the teaching mechanism can be improved by the XR technology providing immersive
experience overlapping the virtual and real-world as same as face-to-face teaching
communication to match individual needs.
In the XR environments, spatial, visual and audio cues can combine in natural ways to aid
communication. Users can freely move through the space setting their own viewpoints and
spatial relationships; enabling people in different locations remotely to inhabit the same
environment and interact. The well-known "cocktail-party" effect shows that people can
easily monitor several spatialized audio streams at once, selectively focusing on those of
interest (Schmalsteig, Fugrmann, Szalavári, & Gervautz, 1996). Therefore, users are able to
distinguish between multiple speakers even in the virtual avatar representation. These results
suggest that an ideal XR interface for remote collaboration should have high quality audio
communication, visual representations of the collaborators and an underlying spatial
metaphor, which can be afforded by HoloLens 2 (Microsoft, 2019).
User’s cognitive state in HoloLens 2 may be achievable through head and hand tracking, eye
tracking instantly with Windows Hello, and voice commands through smart microphones and
natural language speech processing (Fig. 17). HoloLens 2 is built with four visible light
cameras for head tracking and two infra-red cameras for eye tracking, also with five channels
of microphone array. It is important to build a consistent and accurate model of a user’s
current understanding of a language by detecting his/her body and eyes movements. For the
environmental tracking, it provides 6DoF tracking for world-scale positional tracking and
spatial mapping for real-time environment mesh. With such a model, the goal is to implement
a practical system that can be used for language learning applications, and in particular the
development of pragmatics competence.
Fig. 17 Input Paradigm of HoloLens
Elliott, Peiris & Parnin (2015) explore the affordances of VR specific to software engineering
research and introduce three general categories of affordances: spatial cognition,
manipulation and motion, and feedback. Johnson (2019) then suggests the following
affordances of XR:


a digital 3D visual layer generated by a display device allowing for the simulation of
virtual worlds or augmentation of the real world;
spatial representation of virtual objects allowing users to perceive objects similar to how
18




they would in the real world;
registration and localization of real-world objects to add new interactions to existing
physical objects through visual overlays;
reality based interactions provided by natural input devices and realistic physics engines,
allowing users to interact with objects using real world like interactions;
non-reality based interactions, using simulation, computer generated graphics, and the
ability to defy the laws of physics, allowing users to interact in ways not possible in the
real world, and
enhanced modes of collaboration allowing users in distributed locations to more easily
work together.
The new XR learning system for this research, is motivated by the affordances as mentionedabove, will create a multi-purpose experience that adapts to changes in the user’s context. We
think this can be interesting for language learning, as it provides opportunities for group
collaboration, language immersion by sensing and translating the user’s environment. It shifts
the method of classroom instruction to a self-learning model by participating in activities in
the digital world individually or with other learners. The developed XR system can design
different 3DVLEs space which are internet-hosted environments; they can be a sense of realtime presence or immersion, or graphically replicate real-world places or scenarios. Users can
manipulate an 3D avatar, which can be a replication of the user or completely a new creation,
to interact and communicate with other user avatars in different environments by using the
HoloLens. Interactions in the virtual world play an important role in user engagement and
desire learning outcomes. The types of interactions are classified into four categories
(Christopoulos, Conrad & Shukla, 2018):




Content exploration: User can explore the virtual world which may be overlapping with
the real-world, and perform assignments or activities, interact and play as student-toworld or between student-to-student, in different scenarios.
Use of scripting tools and 3D object creation: The 3D design and scripting of object
creation will turn coding into a stimulating activity. Students are able to design and create
3D objects performing actions, which can be visualized in 3D representations.
Participants can interact with the content of the world with these 3D objects; hence,
students are motivated and resulted in higher levels of engagement.
Avatar appearance modification: Students can create and edit their avatars’ appearance,
such as they can replicate their real identity, as considering the virtual world to be an
extension of the physical classroom. Students can modify the body shape, skin/hair color
and clothing, etc. to resemble the avatar’s appearance for different roleplay activities.
Communication with others: Students can share with each other their knowledge and
thoughts (peer-tutoring) when requests for feedback and suggestions are made (peerlearning). They can use the chat tool (verbal or writing), or face-to-face conversation, or
use the social media platforms to interact with others in English privately or open to
public. Working with groups or individually affects students’ engagement and emotion.
Work in groups with peers encourage students helping and influencing each other
positively.
Fig. 18 illustrates the interconnecting and synchronisation between physical objects and
virtual objects, which are transformed and exist in both real and virtual worlds
simultaneously. Furthermore, multiple distant learning environments between different users
can be interconnected, allowing bidirectional communication and interactions between
different environments. HoloLens is the inter-reality portal, which receives and process the
physical world, and also can be transformed to virtual counterpart. It can detect changes in
one environment and translate them into appropriate actions within the other environment.
19
Fig. 18 Distributed XR environment interactions (Pena-Rios, 2016)
The user may also explore and interact with the extended reality environment via gestural or
spoken input by the recognition sensors of the HoloLens to determine a context of that
gestural input and perform additional operations corresponding to the determined context.
HoloLens can also apply speech recognition tools and natural language processing algorithms
to the captured utterance to determine a context of the spoken utterance and perform
additional operations corresponding to the determined context.
The XR system can apply various image processing tools to generate data that establishes and
applies various semantic scene analysis processes to the visible portions of the extended
reality environment and the scene depth map to identify and characterize objects disposed
within the extended reality environment and to map the identified objects to positions within
the extended reality environment and the scene depth map. The XR system can also define a
position and orientation of a user within the extended reality environment, and can further
obtain data indicating a position and orientation of one or more other users within the
extended reality environment, by the HoloLens worn by the user and determined by the
user’s eyes. Furthermore, the XR system can dispose to other users within the extended
reality environment, a determined visibility of a face of each of the other users, their positions,
and interactions between all or some of the users.
In addition, the XR system can generate the item of virtual content through animation and
speech by synthesis tools. The system can generate instructions that cause the HoloLens to
present the item of virtual content at the corresponding position within the augmented reality
environment. It can modify one or more visual characteristics of the item of virtual content in
response to additional gestural or spoken input, such as that representing user interaction with
the virtual assistant.
The new immersive XR learning platform integrates various education applications as
partners such as Cognii, ELSA Speak and Second Life (SL). Each application has its
exclusive features and useful for the implementation in the new XR learning platform. In the
new XR learning platform, students can learn, explore, search, and manipulate knowledge in
a 3D virtual space and learn a wide variety of knowledge through AR recognition and AI
response. It supports touch interaction augmented virtual reality technology with HoloLens
headsets, providing students with a dynamic and immersive learning model that increases
student motivation and effectiveness. Table 3 indicates the targeted partnering applications
and their implementation to the new XR learning platform system.
20
Table 3 Targeted Partnering Applications and Implementations (夥伴軟體融入系統平台)
Instructor-student
interactions
Cognii assessment platform
Fig. 19 Examples of Cognii features
21
Fig. 20 Screen Capture of ELSA Speak for Conversational Coaching
Fig. 21 Screen captures of Avatars Collaboration in Second Life
The features and implementations of the new XR learning platform are highlighted in Fig. 22
and Table 4. With the use of Microsoft HoloLens 2 device, the platform provides a dynamic
real-virtual environment for interaction of digital and real objects, where users feel immersed
and their perception of the real world is enhanced.
Fig. 22 Features and implementations of the new XR Learning Platform (系統特色及架構)
22
Table 4 Characteristics of the new XR Learning Platform
For creating the prototype of the MR environments and holograms models in various
scenarios, we will use Unity, Ray Casting, Python and C++ in our programming team. By
adopting the HoloLens 2, this XR learning platform will detect users’ interactions
environment sensing (through object detection and environmental tracking), and attentionbased interaction (through eye tracking). Attention-based interaction is expected to be
implemented as a personalized learning model that tracks the user’s current understanding of
a language by their body language and eye-contact. Fig. 23 shows an architecture diagram of
23
the new XR learning platform indicating the processing between computing device and
HoloLens. Fig. 24 indicates some prototypes of the new system.
Fig. 23 The architecture diagram of the new XR learning platform (algorithms and system in
blue; data flow in green)
(a) User can experience different virtual scenes
(b) Menu of the log records for evaluation
(c) Users are able to search multi-layers of information on
internet in the real-world situation
(d) Interactions with other avatars and 3D objects
in the real-world
(e) Users can replicate their real identity represent as (f) Collaboration with multi-users
their avatar
Fig. 24 The prototypes of the new XR system with the aid of HoloLens (系統示意圖)
24
The design development of the XR learning platform will span over three years in parallel
with the investigation of ILP skills development (Fig. 25). In the first-year, Cognii and ELSA
Speak will be incorporated into the XR platform. Their AI intelligent tutoring system and
conversational coaching system aim to promote student-teacher interactions through the
adaptive 3D VLE one-to-one learning environment. In the second year, the XR platform will
be improved for multi-players. Student-student, group collaboration task-based activities are
conducted in the same real and virtual environments to investigate students’ perceived
benefits and motivation cooperating with peers in the XR learning platform. In the third-year
stage, the XR platform will be developed for remote collaboration in different locations.
Taiwanese non-native English-speaker students (NNS) will collaborate with native Englishspeaker students (NS) through the XR system with Microsoft HoloLens, to overcome
knowledge gaps in communication and deal with sociocultural contexts.
Fig. 25 Three-year system development of XR Learning Platform
The First Year Project – Development of Pramalinguistics Competence (the Form) by
Cognitive Strategies
The aim of first-year project is to implement the 3D VLE intelligent tutoring and
conversational coaching system on the new XR platform, and investigate the effectiveness on
communication and pragmalinguistics competence development. The guiding RQs are:
RQ1: Do the adaptive 3D VLE intelligent tutoring on the XR learning platform and ordinary
face-to-face instructions result in varied pragmalinguistics knowledge gain?
RQ2: What are L2 learners' development of English oral communication and pragmatic
competence by the XR tutoring and conversation coaching system?
The participants will be first-year students from a two-credit required course “Aural and
Speaking” in a Taiwan university, approximately 50 students in this course for 12 weeks.
These students are EFL learners at intermediate levels. The students will be divided into two
groups: 25 students will have one-to-one intelligent tutoring course by the XR learning
system (Group 1) and the other 25 students will have ordinary face-to-face teaching (Group
2).
At first, all students will conduct a pre-test on the pragmatics discourse in relation to apology
acts. Then students to receive metapragmatic instruction, Group 1 will attend the computer
centre equipped with individual access to PCs. The explicit instruction in the second week
will centre on introduction of the XR platform and HoloLens, based on the cognitive
strategies for pragmalinguistics development. Following the metapragmatic instruction, eight
25
video clips are uploaded which are all in different scenarios about reasoning and making
apologies, and to be played to the students consecutively for eight weeks. A flow chat
demonstrates the data collection process of year one (Fig. 26).
Fig. 26 A flow chat of data collection process for year one
The video clips for this exercise depict the reasoning and apology acts performed by native
English tutors in authentic situations, representing by avatars and showing with subtitles of
the dialogues in the videos to make sure students can all understand. Subtitles could help
learners extract language and other non-verbal cues (e.g. facial expressions, gaze, gestures)
used to convey intentions. The video clips are based on different avatar creations appear on
the screen explaining a situation they regret or look for a reason for explanation (Table 5).
The videos will be shown in the XR learning platform for group 1 and on the classroom
projector's screen for group 2. The students can replay each video as many times as they
wished.
Having watched a video clip each week, students will conduct role-plays and answering some
prescribed questions about the weekly topic as same as the video clips. Each student will take
turn using the HoloLens. The role-plays are in hypothetical situations for which the students
are required to create their own dialogue. Students will be asked during the role-plays for
advice in order to apologize and give reasons, so they have to record themselves saying the
best apology they can think of. The reason for one-to-one tutoring rather than saying the
solutions aloud is that some students do not want to talk in front of the class because they feel
anxious or they get frustrated and embarrassed.
Group 1 students will conduct the role-plays on the XR platform by 3D VLE one-to-one with
the intelligent tutoring and conversational coaching system. Group 2 will perform the roleplays individually with the class tutor and worksheets will be distributed for answering
questions. Dialogue construction tasks in role-plays engage learners a deeper level of
processing in pragmatics, grammatic, and lexicosyntactic features.
Oxford (2011) distinguishes two types of reasoning, deductive and inductive. When explicit
information about pragmatics are provided to learners and then they can apply the knowledge
to analyse information, deductive reasoning occurs. Learners can apply learned pragmatics in
real-life communication or collecting samples of the same expressions through observation.
26
On the other hand, inductive reasoning occurs when learners discover rules and norms of
pragmatic behavior on their own by analyzing communicative acts. Learners are able to
identify the contextual factors and to convey the speaker’s intentions and why by analyzing
the pragmatics rules.
Table 5 Examples of role play conversation to deal with reasoning and apologies
Automatic feedback and adaptive assessment will be provided by the XR system for group 1,
while teacher will comment and provide feedback for group 2. For RQ1, a pragmatic
discourse test is given before and after the course, relates to implicatures and speech acts on
reasoning and making apologies, separately to the two groups (Appendix A&B). Thus,
learners’ outcomes are analysed by comparing the pre- and post-tests results on pairedsamples t-test to investigate the effectiveness on pragmalinguistics knowledge gain between
the two groups, and examine if there is a significant gain by the XR learning platform.
For RQ2, a reflective questionnaire will be given to students to evaluate the XR intelligent
tutoring and conversation coaching system for practicing oral English communication and
pragmatic competence development. This refers to the last group of metacognitive strategies
27
on monitoring and evaluating performance. The students are required to answer the items
using a 5-point Likert scale and Table 6 shows a sample of the questionnaire.
These questions can help learners stay focused on the key concepts of pragmatics (form,
function, context) and reflect on their own communicative acts from the point of
appropriateness and acceptability. Finally, the reliability of the questionnaire is evaluated by
Cronbach’s alpha coefficient and the descriptive statistics are input and calculated using
SPSS. Other process data recorded in the system will also serve as supplementary data to help
answer both research questions.
Table 6 Sample of some questions of questionnaire
The Second Year Project – Development of Psycholinguistics Competence (the Context)
by Affective Strategies
The aim of the study for year two is to investigate psycholinguistics competence development
with affective strategies, and the effectiveness on collaboration activities by the XR system,
hence, to review the benefits and motivation of the system. The guiding RQs are:
RQ1: (a) How do students develop their psycholinguistics competence by affective strategies
on the XR platform? (b) What is the relative effectiveness on collaboration activities by the
XR learning system?
RQ2: What are the perceived benefits and motivation in the task-based learning collaborative
activities on the XR platform?
The participants will be second-year students from a two-credit required course “Multimedia
English” in the same university as year 1, approximately 70 EFL students in this course for
12 weeks. Creating positive emotion is one of the affective strategies and politeness theory is
included in the instructional design which involves a number of social and situational factors,
including the relative power relationship between the interlocutors, the social distance, and
the degree of request’s imposition affect the realization of requests (Brown, Penelope,
Levinson, & Stephen, 1987). Table 7 indicates some examples of request strategies which
will be delivered to students as teaching materials to increase their pragmatics awareness.
Table 7 Example of request strategies (adapted from Blum-Kulka, Shoshana, House, &
Kasper., 1989)
28
At the beginning of the course, students are introduced to the XR system and HoloLens. Then
they are asked to build their own avatar (they can replicate their existing appearance or
completely imaginary) in the system. The affordance of the rich fidelity of space on XR
platform gives users a sense of real-time presence and immersion. Students are able to
personalize their own avatar appearance, perform gestures, and even make facial expressions.
Presenting as avatars likely to increase students’ self-efficacy and willingness to
communicate, increase motivation, and increase risk-taking (Peterson, 2010, 2012). Fig. 27
indicates a flow chat of the data collection process of year-2.
Fig. 27 Flow chat of data collection process for year two
Students are divided into ten groups, seven people each and to participate group learning
activities in four different scenes presenting as 3D VLEs on the XR platform overlapping
with the real-world by the HoloLens. Students can mediate not just by the space, but through
virtually rendered 3D objects and non-player avatars which are purposefully designed for
each activity. The immersion of real and virtual spaces provides synchronous interactions
which are significant catalysts for learning and leads to a true perception of immersion and
belongings (Dalgarno & Lee, 2010). Each scene will conduct two real-life situations for
students to practice English communication, in particular making requests by role-plays
(Table 8). The role-play activities will be carried out total for eight consecutive weeks. For
each situation, students are required to role-play particular roles in their group and with a
native English-speaking instructor. All situations in the four scenes require students to
produce requests speech acts by performing prescribed tasks to access the psycholinguistics
accuracy between peers and between instructor.
Table 8 The proposed design of role-play scenes and situations
Scene / Task
Scene 1 – A downtown with
shops, a café, a library, a sports
centre and a high school
Task: To request and collect
Situation 1
Student 1 – Student
Student 2 – Teacher
Student 3&4 – Café operator
Student 5&6 – Shop operator
Situation 2
Student 1 – Café operator
Student 2&3 – Shop operator
Student 4 – Teacher
Student 5 – Sports centre
29
three items from each character.
Scene 2 – Career Island
Task: Students to explore
various professional roles and
earn 1000 points.
Scene 3 – Sustainable Village
and Market
Task: Students work in groups,
to resolve the environmental
issues in the village and teach
the people there a healthy diet.
Scene 4 – An Eco-MUVE Forest
Task: Students work in groups,
to take role of a scientist each
and investigate the reasons of
the destroyed eco-system, and
how can help to resume a
healthy eco life.
Student 7 – Sports centre
operator
Instructor – Principle of high
school
Student 1&2 – a patient in a
clinic
Student 3 – a student in a
classroom
Student 4 – a vet in a pet clinic
Student 5 – a policeman
Student 6&7 – a customer in a
supermarket
Instructor – Career advisor
All students act as a visitor to
the village.
operator
Student 6&7 – Student
Instructor – Librarian
Student 1 – Water chemist
Student 2 – Microscopic
specialist
Student 3&4 – Naturalist
Student 5 – Private investigator
Student 6&7 – Bird watcher
Instructor – Public health
worker
Student 1 – Private investigator
Student 2&3 – Water chemist
Student 4 – Bird watcher
Student 5&6 – Naturalist
Student 7 – Microscopic
specialist
Instructor – Public health
worker
Student 1 – a policeman
Student 2 – a patient in a clinic
Student 3&4 – a customer in a
supermarket
Student 5 – a vet in a pet clinic
Student 6&7 – a student in a
classroom
Instructor – Career advisor
All students act as a citizen
living in the village.
Prior to each situation, students will be given role cards describing the situations and their
roles respectively, and are required to complete prescribed tasks. Role-play activities will be
conducted on the XR platform and each student is equipped a computer at the computer
centre. Students are required to start the role-play in the form of avatar, rotating with the use
of HoloLens, by asking for something from their interlocutors and they will have 15 mins
preparing for each role-play before participate in the situation. The objective of the learning
activities is to illustrate concepts of psycholinguistics competence using a co-creative
approach to encourage students to create as many behavioural rules as possible based on the
hypothetical situations using the XR platform.
Students will participate in each situation and perform tasks individually or in groups,
depends on the tasks’ requirements. The task scenario matches the environment and students
are instructed to explore the scenes in their given character. They are required to ask
questions, make requests for information and search for clues about the prescribed tasks.
Some tasks involve students working in groups for collaboration. All the tasks are designed
to allow students to interact with each other to encourage pragmatics behaviours, which
consists of five stages:
1. Brainstorming with others what resources are needed to perform in the scene;
2. Work together with peers to familiarize with the environment and to locate key
landmarks;
3. Exploring the environment in each scene to search for information and clues;
4. Putting together a plan for how to complete the prescribed tasks;
5. Sharing the plan to the team members.
The exercise of the role-plays is recorded by the XR system and student’s individual
performance is rated for appropriateness of request forms on a five-point Likert scale score
by two native English-speaking teachers. The score sheet contains 20 items with a maximum
30
score being 100. The rating scale employed is to assess the psycholinguistics accuracy based
on the rubrics of affective strategies (Appendix C). Linguistic accuracy is assessed by
allocating five points if the response to a speech act is lexically and grammatically accurate,
one to four points if it is partially correct, and no point if it is grammatically or lexically
inaccurate. The scores are qualitatively analysed using multivariate variate of analysis
(MANOVA) between situations and start-to-finish respectively, to examine for RQ1: (a)
students’ performance on psycholinguistics competence between different situations; and (b)
the effectiveness on collaboration activities in the XR system.
At the end of the course, a survey in multiple-choice questions are completed by the students
to evaluate their own performance, perceived benefits and motivation after the role-play
activities (Appendix D). Specifically, questions regarding perceived learning effectiveness
are adapted from the corresponding questions in the Computer Attitude Scale (Selwyn, 1997)
measuring perceived usefulness on the XR platform. In addition, the Intrinsic Motivation
Inventor is used which is a multidimensional measurement for assessing participants’
subjective experience of enjoyment and motivation when using the XR system (Tamborini,
Bowman, Eden Grizzard, & Organ, 2010).
At the end, the surveys are analysed by factorial analysis and compare the reliability scores
using Cronbach’s alpha, also to perform Confirmatory Factor Analysis (CFA). The factorial
analysis will have seven factors, named as follows: enjoyment, perceived ease of use,
perceived learning effectiveness, presence, realism, motivation, and collaboration. Finally,
the CFA scores will be analysed to investigate the perceived benefits and motivation in the
task-based learning collaborative activities on the XR platform (RQ2).
The Third Year Project – Development of Sociopragmatics (the Function) by
Sociocultural-interactive Strategies
The third-year project aims to enhance vocational college students’ sociopragmatics
competence through intercultural exchanges with native Australia students.
RQ1: What are the effects of sociopragmatics competence upon students’ interaction with
English native speakers on XR learning platform?
RQ2: How do students use the XR system to interact and communicate the cultural
information with foreigners on the XR platform?
This project will engage 48 Taiwanese EFL students (non-native speakers (NNS)) studying in
a Taiwan university of Science and Technology and 48 native-speaking (NS) students who
learns Chinese from a university in Australia. The research study is conducted online with
students from two different countries to participate activities for cultural exchange for 16weeks.
The project involves the depiction of student as an avatar through which he or she is able to
develop and project an online identity. The avatar can replicate their real identities or as a
new creation on the XR platform. The avatar helps to create a sense of individuality that aides
in co-presence with the other geographically dispersed user in the remote environment, and
enriches the social interactions occurring (Schroeder, 2002). Students perceive or feel that
other users are also present in the virtual environments as well as themselves, co-presence
occurs as a social, task-related, or physiological response to embodied agents (Bailenson &
Yee, 2005). Users interacting as avatars can socially and psychologically feel connected to
others, resulting socio-emotional processes which are crucial for building effective structures
and communities,
In the beginning, all students are briefed on the XR system. During the first three weeks, each
student is requested to compose a video of at least three tourist sites introducing the cultural
significance of their own country and present on the XR platform to their assigned partner
31
later. The video can consist of web information, you-tube videos, or student can film by
themselves. Then the intercultural communication will take place through the XR system
presenting Taiwan and Australia tourist places and sharing activities that will run for ten
weeks. The outline of data collection process is shown in Fig. 28.
Fig. 28 Outline of data collection process for Year Three
During the ten-weeks interactions and activities, one Taiwanese student will be paired up
with one Australian student and they will interact with each other for the first five weeks.
Then students will swap partners and interact for another five weeks. Each session of the
activity allows six paired of students to conduct activities on the XR platform for 20-minutes
respectively, total spent time for each pair is 100-minutes. The combinations of the
participants are shown in Table 9 below.
Table9 Combinations of the participants
Students
NS
Pair 1-6
Pair 25-30
NS
Pair 7-12
Pair 31-36
NS
Pair 13-18
Pair 37-42
NS
Pair 19-24
Pair 43-48
NNS
Pair 1-6
Pair 25-30
Week 5-9
5 sessions
(20-mins each)
NNS
Pair 7-12
Pair 31-36
NNS
Pair 13-18
Pair 37-42
Week 10-14
5 sessions
(20-mins each)
Week 5-9
5 sessions
(20-mins each)
Week 10-14
5 sessions
(20-mins each)
NNS
Pair 19-24
Pair 43-48
Week 10-14
5 sessions
(20-mins each)
Week 5-9
5 sessions
(20-mins each)
Week 10-14
5 sessions
(20-mins each)
Week 5-9
5 sessions
(20-mins each)
During the five-week sessions period, the paired NNS and NS participants log in to the XR
platform at the same time. Australia is only three hours faster than Taiwan, therefore students
are able to participate during class time. Within the XR environment, students can enter into
various virtual scenarios or overlap with the real-life environment, or explore any places on
Google Maps. The pair meets up face-to-face in their avatar forms at the assigned meeting
place, they first introduce themselves and exchange basic information. The interactions
include basic in-world operations, such as walking, sitting on a chair, teleporting, and talking.
In the XR platform setting, the participates are free to do anything and present their prepared
cultural videos to each other.
32
Fig. 29 Instruments employed in Year Three
The pair is required to exchange their interpersonal and intercultural information, and
participate activities together during their session on the XR platform. Some prescribed tasks
using authentic materials and contexts are designed for students to explore and execute, in
order to teach language skills, encourage communication, and motivate students to interact.
Table 10 indicates two goal-based scenarios using the Google Maps street view panoramas,
students can freely explore, discuss, role-play, navigate, and interact through their avatars on
the XR platform. The contextualized curriculum design process is driven by Schank’s goalbased scenarios (1996) and Shih’s research study (2018).
Table 10 Detailed information about two goal-based scenarios
All interactions are recorded by the system (process data) and students are required to
complete a questionnaire after each session to ask their impressions on the sessions for
33
analysis. During the last two weeks of this project, the participants are asked to review their
peers’ works and complete the final reflective essays.
The data for this study will include scores of Taiwanese and Australian students on the
Intercultural Sensitivity Scale, scores on questionnaires for each session, and students’ final
reflective essays and interviews on the process of using the XR platform, and their
discussions on respective county and their cultural differences. The rating scale employed is
to assess the sociopragmatics accuracy based on the rubrics of sociocultural-interactive
strategies on the questionnaires (Appendix E). Finally, the researcher and her team will
analyze the reflective essays and interviews through content analysis (Patton, 2002) and the
guiding questions in reflective essays are included in Appendix F. Content analysis will
include four stages: coding, categorization, description and interpretation, which will be
adopted to identify codes and categorize the patterns or themes that emerge from the data and
use it to search for the relationships between those categories (Weber, 1990; Patton, 2002).
Anticipated Accomplishments and Potential Contributions
The project will exemplify the importance of interlanguage pragmatics competence for EFL
learners and the research gaps are addressed in the existing literature, including the design
and development of a XR learning platform. This research project aims to demonstrate that
3D VLE in the latest XR technology is an optimal platform for online interaction,
communication, and collaboration beyond classroom instruction teaching. It holds great
potential for language learners by engaging them in the overlapping virtual and real-life
environments with authentic contexts and materials for social interaction and group
collaborations. This project also examines how the XR platform affects students’ perceptions
of interpersonal and emotional connections with peers in the learning environment. Moreover,
this project strives to equip EFL learners with necessary skill sets and practical training that
connects classrooms to the real world, thereby raising EFL students to a level to contribute to
Taiwan and the global community through communication and pragmatics competence.
About 8-12 SSCI papers will be published and our team will apply for four patents for the
XR system which will be implemented in different settings and classes. Approximately, 1200
students will benefit from using this system. In addition, this study will provide a coherent
line of research from educational practice, through integration of different modalities, such as
Cognii, ELSA Speaking, and Second Life, to educational evaluation that is missing in a lot of
research and development papers. Both educational problems and its practices will be well
defined and the use of XR technology can be seen as a viable solution that allows particular
problems to be addressed, resolved and discussed.
Yearly Plan
Year One: This year project will complete the design and development of first stage of the
XR learning platform incorporation of Cognii and ELSA Speak applications to provide an
immersive one-to-one tutoring and conversational coaching.
(a) Requirement gathering of what is needed and what can be done
Our team will work closely together to understand what is required for the design of the
learning activities and gather relative information for the video clips into numbers of
scenarios, for providing one-to-one learning experience to the students. Eight different
content creation videos on pragmatics to be produced, including the after role-plays and
questions for teacher-student interaction. This will involve researching, many meetings,
interviewing between different parties (i.e. teachers) so that everyone shares the comparable
understanding of what is required for developing the video and the scripts.
(b) Design and plan the learning activities for each theme
34
The videos are related to pragmatics skills on apologize and reasoning, then the role-plays are
in hypothetical situations for which the students are able to create their own dialogue. The
team will plan and create the relevant learning activities for each of the eight themes. The
learning activities planned should not be overlapped but complimentary in nature so that
teams can cover as much of situations for apologize and reasoning as possible.
(c) Design and development of XR system for intelligent tutoring and conversational
coaching
The team will concurrently work on the design and development of the XR system. In the
first year, Cognii and ELSA Speak applications will be incorporated into the XR system for
the benefits of the AI intelligent tutoring system and conversational coaching. C++ and Unity
will be used to facilitate the creation the XR system.
(d) Video making
During this stage, the production team will prepare the videos based on the relevant media
contents. Teacher and student will be in avatar forms which will also be designed and created
by the team. Testing of the XR system together with the video scripts will be done in this
stage, and rehearsals will also be done to make sure its deployment will be smooth.
(e) Deployment of after roleplays and answer questions
The after roleplays scenarios and the relative questions will input to the XR system.
Automate feedback, clues and analysis will be generated instantly with the students.
(f) Data collection and analysis
Students from both experimental and control groups are required to complete a pre-test on the
pragmatic discourse before the experimentation and a post-test after the treatments to
compare the effectiveness on pragmalinguistics knowledge gain between the XR system and
conventional teaching. At the end of the experimentation, the experimental group is required
to complete a questionnaire to examine the perceived values on the use of the XR system.
During the learning activities with the XR system in the classrooms, video recording will be
made and also all the logs will be recorded by the XR system. Statistical analysis will be done
on the data collected.
(g) Construct design principles and development guidelines
Based on the experiences of both the content creation and system production, and the
feedback gathered from the users (teacher and student), an initial set of design principles and
development guidelines is set. This design principles and development guidelines will be the
basis for improving the XR system in Year Two.
Year Two
(a) Improvement of XR system
Using design principles and development guidelines produced in Year One, the team will
improve the system for multi-players collaboration, and for better learning activity design and
learning results.
(b) Requirement gathering for customization
According to the feedback from teacher and students, also from the observations by the team,
we will have experience to know how well these activities fit the needs of their students. A
list of requirements and comments will be compiled for the later customization.
(c) Design and plan the learning activities for each scenario
35
The team will gather information to design and plan for four different scenes in virtual
environments. Each scene will conduct two real-life situations for students to practice English
communication, in relation to making requests speech acts by performing prescribed tasks
between peers and between instructor.
(d) Design and development of XR system for multi-players collaboration
In parallel with the design of the learning activities, the team will concurrently work on the
design and development of the XR system for multi-players collaboration. In the second year,
Second Life application will be incorporated into the XR system for the benefits of the multiplayers collaboration in virtual avatars form. However, the new XR system will be developed
not only allowing users to participate in the virtual environment, but immersive with the realworld. Unity will be programmed to facilitate the creation the XR system through object
detection, and attention-based interaction (through eye tracking) by HoloLens.
(e) Creation of 3D objects and avatars
Programs need to be created by Unity to allow students to create their own 3D objects and
avatars. Students can mediate not just by the space, but through virtually rendered 3D objects
and non-player avatars which are purposefully designed by the team for each activity. They
can replicate their existing appearance or completely new imaginary for their avatar
representation in the system, such as personalize their own avatar appearance, perform
gestures, and even make facial expressions.
(f) Data collection and analysis
The exercise of the role-plays is recorded by the XR system and student’s individual
performance on psycholinguistics accuracy is rated for appropriateness of request forms on a
five-point Likert scale score sheet assessed by two native English-speaking teachers. At the
end of the semester, students are required to fill up questionnaires to gather data about the
students’ perceived benefits and motivation on the task-based learning collaborative activities
by the XR platform. Statistical analysis will be done on the data collected. Log files of the
XR system will be analysed too. This information will be useful in Year 3 to aid in creating
adaptive learning activities.
(g) Refine design principles and development guidelines
Based on the experiences of both the content creation and content production teams, and the
feedback gathered from the users, design principles and development guidelines will be
further revised and refined for Year Three.
Year Three
(a) Improvement of XR system
Using design principles and development guidelines produced in Year Two, the team will
improve the system for multi-players collaboration in remote distance, and for better learning
activity design and learning results.
(b) Requirement gathering for customization
According to the feedback from teacher and students, also from the observations by the team,
we will have experience to know how well these activities fit the needs of their students. A
list of requirements and comments resulted from Year Two will be compiled for the later
customization.
(c) Design and plan the learning activities in the immersive virtual and real worlds
The team will gather information to design and plan for various virtual scenarios for different
learning activities. These virtual scenes may also overlap with the real-life environment in
36
different remote locations. Prescribed tasks are designed in goal-based scenarios using the
Google Maps street view panoramas, students can freely explore, discuss, role-play, navigate,
and interact through their avatars on the XR platform. Testing of the XR system together with
the activities will be done at this stage, and rehearsals will also be done to make sure its
deployment will be smooth.
(d) Design and development of XR system for multi-players collaboration in remote distances
In parallel with the design of the learning activities, the team will concurrently work on the
design and development of the XR system for multi-players collaboration in remote distances.
Users interactions include basic in-world operations, such as walking, sitting on a chair,
teleporting, and talking. In the XR platform setting, the users are free to do anything and
students can present their prepared cultural videos. The system can detect users’ interactions
environment sensing (through object detection and environmental tracking), and attentionbased interaction (through eye tracking). Attention-based interaction is expected to be
implemented as a personalized learning model that tracks the user’s current understanding of
a language by their body language and eye-contact.
(e) Data collection and analysis
The collaboration activities are recorded by the XR system and students are required to
complete a questionnaire after each session to ask their impressions on the sessions for
analysis which will be rated to assess the sociopragmatics accuracy. At the end of the
semester, the participants are asked to review their peers’ works and complete the final
reflective essays. Statistical analysis will be done on the data collected. Log files of the XR
system will be analysed too.
(f) Finalize design principles and development guidelines
After the iterations of revising and improving, and based on the experiences of both the
content creation and content production, together with the feedback gathered from different
users, this project will produce design principles and development guidelines for the new XR
learning platform.
References:
Afflerbach, P., Pearson, P. D., & Paris, S. G. (2008). Clarifying differences between reading skills and
reading strategies. The reading teacher, 61(5), 364-373.
Alcaya, C., Lybeck, K., & Mougel, P. (1994). Speaking strategies: A list compiled by teachers in the
Experimental Sections of the Speaking Strategies Experiment. Unpublished document, National
Language Resource Center, University of Minnesota, Minneapolis.
Austin, J. (1962). How to do things with words. Oxford, Basil Blackwell.
Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language
assessments and justifying their use in the real world. Oxford University Press.
Barak, M., & Levenberg, A. (2016). Flexible thinking in learning: An individual difference measure
for learning in technology-enhanced environments. Computers & Education, 99, 39-52.
Bardovi-Harlig, K., & Salsbury, T. (2004). The organization of turns in the disagreements of L2
learners: A longitudinal perspective. Studying speaking to inform second language learning, 199-227.
Bardovi & Harlig, K. (2013). Developing L2 pragmatics. Language Learning, 63, 68-86.
Belz, J. A., & Kinginger, C. (2003). Discourse options and the development of pragmatic competence
by classroom learners of German: The case of address forms. Language learning, 53(4), 591-647.
37
Blas, N.D., Buccieero, A., Mainetti, L., & Paolini, P. (2012). Multi-User Virtual Environments for
Learning: Experience and Technology Design. IEEE Transactions on learning technologies, 5(4).
Bosch-Sijtsema, P. M., & Haapamäki, J. (2014). Perceived enablers of 3D virtual environments for
virtual team learning and innovation. Computers in Human Behavior, 37, 395-401.
Campbell, C., & Sarac, B. (2018). The role of technology in language learning in the twenty-first
century: Perspectives from academe, government, and the private sector. Hispania, 100(5), 77-84.
Canto, S., de Graaff, R., & Jauregi, K. (2014). Collaborative tasks for negotiation of intercultural
meaning in virtual worlds and video-web communication. In Technology-mediated TBLT (pp. 183212). John Benjamins.
Celce-Murcia, M., Dörnyei, Z., & Thurrell, S. (1995). Communicative competence: A pedagogically
motivated model with content specifications. Issues in Applied linguistics, 6(2), 5-35.
Chen (2013). An Evaluation on Primary English Education in Taiwan: From the Perspective of
Language Policy. English Language teaching, 6(10).
Chun, D., Kern, R., & Smith, B. (2016). Technology in language use, language teaching, and
language learning. The Modern Language Journal, 100(S1), 64-80.
Collins, A., & Halverson, R. (2018). Rethinking education in the age of technology: The digital
revolution and schooling in America. Teachers College Press.
Crystal, D. (1997). A dictionary of linguistics and phonetics. 4th edition. Cambridge, MA: Blackwell.
Diepenbroek, L. G., & Derwing, T. M. (2014). To what extent do popular ESL textbooks incorporate
oral fluency and pragmatic development. TESL Canada Journal, 1-1.
Doğan, D., Çınar, M., & Tüzün, H. (2018). Multi-user virtual environments for education. In: N. Lee
(Ed.) Encyclopedia of Computer Graphics and Games. Springer International Publishing. Retrieved
from http://yunus.hacettepe.edu.tr/~htuzun/html/academic/2018-Dogan-Cinar-Tuzun.pdf
Eslami, Z., & Liu, C. N. (2013). Learning pragmatics through computer-mediated communication in
Taiwan. International Journal of Society, Culture & Language, 1(1), 52-73.
Finch, G. (2000). Semantics and Pragmatics. In Linguistic Terms and Concepts (pp. 142-187).
Palgrave, London.
Gonzalez-Franco, M., Pizarro, R., Cermeron, J., Li, K., Thorn, J., Hutabarat, W., Tiwari, A., BermellGarcia, P. (2017) Immersive mixed reality for manufacturing training. Front Robot AI 4:3
Gutierrez, K. (2016). Facts and Stats That Reveal The Power Of eLearning [Infographic]. SH!FT.
https://www.shiftelearning.com/blog/bid/301248/15-facts-and-stats-that-reveal-the-power-ofelearning
Hartwick, P. (2018). Investigating research approaches: Classroom-based interaction studies in
physical and virtual contexts. Cambridge University Press, 30(2), pp.161-176.
Horner, B., Lu, M.Z., Royster, J.J., & Trimbur, J. (2011). Language diff Language difference in
writing. ThinkIR: The University of Louisville’s Institutional Repository.
Ibáñez, M. B., Rueda, J. J. G., Maroto, D., & Kloos, C. D. (2013). Collaborative learning in multi-user
virtual environments. Journal of Network and Computer Applications, 36(6), 1566-1576.
Jauregi, K. (2016). Researching telecollaboration processes in foreign language education: Challenges
and achievements. In Technology implementation in second language teaching and translation
studies (pp. 155-178). Springer, Singapore.
Johnson, D. (2019). MusE-XR: musical experiences in extended reality to enhance learning and
performance. University of Victoria.
38
Joibi, J. C., Kim, Y. M., Shin, G. W., Lee, K. J., & Yun, M. H. (2019). 1H3-3 A Systematic Review
of Extended Reality (XR)’s Head-Mounted Display (HMD) Hardware Design Factors from the
Perspective of Usability. 人間工学, 55(Supplement), 1H3-3.
Jowallah, R., Bauer, S., Futch, L., & Gunter, G. (2019, November). Developing, Designing, and
Implementing Extended Reality within Learning Environments: In E-Learn: World Conference on ELearning in Corporate, Government, Healthcare, and Higher Education (pp. 553-556).
Kasper, G., & Rose, K. R. (2002). Pragmatic Development in a Second Language. Language
Learning: A Journal of Research in Language Studies, 52, 1.
Lan, Y.J. (2014). Does second life improve mandarin learning by overseas Chinese students?
Language Learning & Technology, 18(2), p.36-56.
Leal, L. E., Chapman, A., & Westerlund, M. (2019). Reinforcement Learning for Extended Reality:
Designing Self-Play Scenarios. In HICSS (pp. 1-8).
Lewin, D., & Lundie, D. (2016). Philosophies of digital pedagogy. Studies in Philosophy and
Education, 35(3), 235-240.
Lin, I. 2018. Why English education in public schools is useless. Taipei Teen Tribune.
https://taipeiteentribune.com/english-education/
Marr B. (2019). What is extended reality technology? A simple explanation for everyone. Forbes.
https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simpleexplanation-for-anyone/#ebd8a6e72498
Martínez-Flor, A., & Fukuya, Y. J. (2005). The effects of instruction on learners’ production of
appropriate and accurate suggestions. System, 33(3), 463-480.
Martinez, A.H. (2018). Technology and task-based language teaching (TBLT): Exploring pragmatics.
IJEDICT, 14(2), p.38-61.
Microsoft (2015, 2019). https://www.microsoft.com/zh-tw/education/mixed-reality
Miles, H. C., Pop, S. R., Watt, S. J., Lawrence, G. P., & John, N. W. (2012). A review of virtual
environments for training in ball sports. Computers & Graphics, 36(6), 714-726.
Montello, D. R., Grossner, K. E., & Janelle, D. G. (Eds.). (2014). Space in mind: Concepts for spatial
learning and education. Mit Press.
Morris, C. W. (1938). Foundations of the Theory of Signs.
Murcia-Lopez, M., Steed, A. (2018) A comparison of virtual and physical training transfer of
Nordquist, R. (2019). Pragmatics gives context to language. ThoughtCo.
https://www.thoughtco.com/pragmatics-language-1691654
Normann, N., & Morina, N. (2018). The Efficacy of Metacognitive Therapy: A Systematic Review
and Meta-Analysis. Systematic Review Article.
https://www.frontiersin.org/articles/10.3389/fpsyg.2018.02211/full
Oxford, R.L. (2011). Strategies for learning a second or foreign language. Cambridge University
Press, 44(2), pp.167-180.
Oxford, R.L. (2016). Teaching and researching language learning strategies: Self-regulation in context.
Routledge.
Patton, M.Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage.
Peterson, M. (2012). EFL learner collaborative interaction in Second Life. ReCALL, 24(1), 20-39.
Pena-Rios, A. (2016). Exploring mixed reality in distributed collaborative learning environments
(Doctoral dissertation). University of Essex, UK.
39
Piri, S., Pishghadam, R., & Dixon, L.Q., & Rasekh, Z.E. (2018). Predictors of L2 achievement:
Testing a model based on EFL learners’ emotional, social, and cultural capitals. Issues in Educational
Research, 28(3).
Scribani, J. (2019). What is extended reality (XR). Visual Capitalist, 16, 2019.
Sellen, A. (1995). Remote Conversations: The effects of mediating talk with technology. Human
Computer Interaction, Vol. 10, No. 4, pp. 401-444.
Sharma, V.S., Mehra, R., Kaulgud, V., & Podder, S. (2019). An extended reality approach for
creating immersive software project workspaces. Proceeding in CHASE, p.27-30.
Sheshagiri P.G., Baheti P.K., Gute A.D., & Lakshmikanth S.K. (2019). Extended reality virtual
assistant. US Patent US15/655,762.
Shih, Y.C. (2018). Contextualizing Language Learning With Street View Panoramas. IGI Global,
p.18.
Smith, C.H. & Cousins, J.F. (2019). XR for XR – Context Craft: An extended reality platform that
transforms local environments through play. Proceedings of EVA London, p.218-225.
Sundstrom, E. (2019). Shared Augmented Reality: Developing a Multiplayer AR Mobile Game to
Study Playability. Luleå University of Technology.
Sykes, J.M. (2017). Technologies for Teaching and Learning Intercultural Competence and
Interlanguage Pragmatics. The handbook of technology and second language teaching and learning.
Chapter 9.
Taguchi, N. (2015). Instructed pragmatics at a glance: Where instructional studies were, are, and
should be going. Language Teaching, 48(1), 1-50.
Taguchi, N., & Roever, C. (2017). Second language pragmatics. Oxford University Press.
Tyre, M. J., & Von Hippel, E. (1997). The situated nature of adaptive learning in
organizations. Organization science, 8(1), 71-83.
Van Den Branden, K., Bygate, M., & Norris, J. (2009). Task-based language teaching: Introducing the
reader.
Wang, J.E., & Tsao, F.M. (2015). Emotional prosody perception and its association with pragmatic
language in school-aged children with high-function autism. ScienceDirect, Vol. 37, p.162-170.
Wang, M.C. (1984). Report. The Adaptive Learning Environments Model: Design, Implementation,
and Effects. Washington: National Institute of Education.
Zheng, D., Newgarden, K., & Young, M. F. (2012). Multimodal analysis of language learning in
World of Warcraft play: Language as values-realizing. ReCALL, 24(3), 339-360.
Appendix A: Year One – Example of Pre-test on Pragmatic Discourse
Test conducted by participants before performing the activities.
1. What is an apology?
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––.
2. Put the correct words in the correct order to build up utterances.
A. accept apologies sincerest our
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
40
B. sorry I very am
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
C. fault it my was
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
D. me excuse
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
E. wait I really am for sorry you having
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
F. your I sorry loss am hear so to about
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
3. There are three different situations where we can apologize. Choose the most appropriate apology
for each context.
A. You just bump into an old lady on the street. What would you say?
1. Sorry.
2. It was my fault.
3. I am very sorry. I didn’t mean it. Are you okay?
B. You forgot to do your homework and your teacher is asking you why. What would you say
in order for her not to be mad at you?
1. I am sorry but they were very difficult.
2. It was my fault.
3. I am very sorry. They were a little bit demanding and I couldn’t figure them out.
C. You were supposed to pick your best friend at the airport, but it completely slipped your
mind. How would you apologize to her or him?
1. Ever sorry. I was so excited for this day to come, but it completely slipped my
mind.
2. Can you wait a little bit? I will be there in a while.
3. I am sorry. Do you mind to take the bus? Talk to you later.
Appendix B: Year One – Example of Post-test on Pragmatic Discourse
Test conducted by participants after the activities were performed.
1. What is an apology?
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––– –––––––––––––––
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––.
2. Put the correct words in the correct order to build up utterances.
A. sorry I I did mean not am very it
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
B. was it who broke me I did see them not glasses I the sorry totally
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
41
C. care I I wish know had just the words right
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
D. for mistake we our apologize E. cannot imagine you how am I sorry
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
F. me pardon
–––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
3. There are three different situations where we can apologize. Choose the most appropriate apology
for each context.
A. Your dog peed on someone’s foot, and you did not realize. What would you say to that
person?
1. It is not my fault. I did not see it.
2. I am very embarrassed. I am sorry. I was not paying attention.
3. I didn’t mean it. Are you okay?
B. You sent a text message to the wrong friend, and he replied that you woke him/her up in a
bad mood. How can you fix the situation?
1. Excuse me.
2. Sorry. I made a mistake when sending the message. Sorry again.
3. Sorry but it is time to get up!
C. You were invited to your boyfriend’s / girlfriend’s birthday party, but you fell asleep while
watching your favorite movie and you were late.
1. Sorry. I fell asleep and I just got up!
2. I am very sorry. I fell asleep while watching a movie. Every sorry.
3. You know how I am. Sorry but I was really tired honey!
42
Appendix C: Year Two - The Rubric for Assessing Students’ Psycholinguistics
Accuracy Adapted from Alcaya, Lybeck, & Mougel (1994)
43
Appendix D: Year Two – Questionnaire on perceived benefits and motivation
Factor
Description
Scores (10=highest;
1=lowest)
1. Perceived learning
effectiveness
a. I feel that XR platform can ease the way I learn.
b. XR platform is a much easier way to learn compared to
the usual teaching.
c. Why use XR platform? There are easier ways to learn
what I want to learn.*
d. XR platform can make learning more interesting.
2. Perceived ease of use
a. Learning to use the XR platform was easy for me.
b. I found the XR platform easy to use.
c. Whenever I used the XR platform I needed help
because it was not easy for me to use.*
d. It was easy for me to become skilful at using the XR
platform.
3. Presence
a. I forgot about my immediate surroundings when I used
the XR platform.
b. When I used the XR platform I often forgot where I
am.
c. When I used the XR platform, the virtual world was
more real than the real world.
d. When I used the XR platform, I felt that my body was
in the room, but my mind was inside the world created by
the system.
4. Enjoyment
a. My experience in the XR platform was quite enjoyable.
b. I would describe my experience in the XR platform as
very interesting.
c. The experience in the XR platform was fun.
d. I enjoyed experiencing the immersive virtual and real
worlds in the XR platform very much.
5. Motivation
a. When using the XR platform, I had the impulse to
learn more about space exploration.
b. I tried to explore all the XR platform because
everything was so interesting.
c. I wasn’t interested in learning using this type of
computer program.*
d. This type of computer program did not hold my
attention.*
6. Collaboration
a. I enjoyed to collaborate with others in the XR platform.
b. It was interesting that in the XR platform I was doing
things together with my peers.
c. With my fellow students, we were able to jointly
decide where to go and what to do in the XR platform.
7. Realism
a. The visual display quality of the XR platform
distracted me from doing other things.
b. When interacting with the virtual 3D objects, these
interactions seemed like real.
c. There were times when the virtual 3D objects became
more real and present for me compared to the real ones.
d. The virtual 3D objects seemed like the real objects to
me.
Note. * = Item for which scoring is reversed.
44
Appendix E: Year Three - Questionnaire
Examples of questions:
1.
2.
3.
4.
5.
Teaching and learning activities
a. Did the XR platform teaching methods give you the same amount of information, as you would get in
classical lectures?
b. Describe the activities that are most beneficial to your learning? What were the effective skills that
were demonstrated in your group?
c. Were there other skills that would have also been more appropriate to use? (If so, what were they?)
d. Would you like to use XR platform for learning activities in the future?
e. Did you find it was easier to communicate with Native speaker through the XR platform?
XR Platform environment
a. How long did it take you to feel comfortable with the environment?
b. Have you experienced any technical difficulties with the system?
c. According to you, what are the main differences between learning in XR platform vs. learning in real
life?
d. Describe the activities that differ the most and the least between XR platform and real-life learning.
Social interaction and communication
a. Which methods did you use to communicate with other users?
b. Did you think XR platform makes social interaction easy or difficult?
c. Did you find any difficulties to interact and socialize within the group?
d. Which differences can you identify between communication in XR platform and in real-world?
Collaborative activities
a. How was your collaborative activity organized?
b. Do you feel that you learned more working with peers than you would have working alone?
c. Compared to the real-world, in which ways did XR platform support your collaborative activities?
d. Do you think it was interested and motivated to learn through the collaborative activities in the XR
platform?
3D Artifacts creation
a. What artifacts were involved in collaborative activities?
b. Are the virtual buildings similar to the real ones?
c. What artifacts allowed you to practice the skills needed for the activities?
d. How were your 3D objects created? (individually or collaboratively)
e. How did these artifacts facilitate the roleplay activities and collaboration among the groups?
Appendix F: Year Three - Reflective Essay
Each reflection entry is designed to help you reflect what you have learned from multimodal
blogging project on Taiwan Culture this semester. Please compose a 500 – 750 word reflective
essay upon each presentation, and thoroughly explains each of the questions in your essay. Please
post your reflection entry one week after your presentation entry.
-
What is unique about your topic that made you choose it over other topics?
How can your project help to preserve Taiwan’s culture, heritage or traditions?
How do you see your project influencing the current and future generations in a beneficial way? (i.e.
helping future young adults appreciate cultural traditions)
Why do you think it is important for people to know the “authentic” Taiwan versus the “Tourist” version
of Taiwan?
What are the most valuable values or cultural traditions of Taiwan you want to present to the world? Why?
How can your project be different from others? How can you not repeat the same information or
presentation of Taiwan that exists?
45