Federated Learning for Metaverse: A Survey
Yao Chen∗
Jinan University
Guangzhou, China
csyaochen@gmail.com
Shan Huang∗
Jinan University
Guangzhou, China
shuang9901@gmail.com
arXiv:2303.17987v1 [cs.CR] 23 Mar 2023
Gengsen Huang
The metaverse, which is at the stage of innovation and exploration,
faces the dilemma of data collection and the problem of private data
leakage in the process of development. This can seriously hinder the
widespread deployment of the metaverse. Fortunately, federated
learning (FL) is a solution to the above problems. FL is a distributed
machine learning paradigm with privacy-preserving features designed for a large number of edge devices. Federated learning for
metaverse (FL4M) will be a powerful tool. Because FL allows edge
devices to participate in training tasks locally using their own data,
computational power, and model-building capabilities. Applying
FL to the metaverse not only protects the data privacy of participants but also reduces the need for high computing power and
high memory on servers. Until now, there have been many studies
about FL and the metaverse, respectively. In this paper, we review
some of the early advances of FL4M, which will be a research direction with unlimited development potential. We first introduce the
concepts of metaverse and FL, respectively. Besides, we discuss the
convergence of key metaverse technologies and FL in detail, such as
big data, communication technology, the Internet of Things, edge
computing, blockchain, and extended reality. Finally, we discuss
some key challenges and promising directions of FL4M in detail.
In summary, we hope that our up-to-date brief survey can help
people better understand FL4M and build a fair, open, and secure
metaverse.
CCS CONCEPTS
• Computing methodologies → Federated learning.
KEYWORDS
Metaverse, federated learning, intelligent, applications, survey.
∗ Both
authors contributed equally to this work.
author, also with Pazhou Lab, Guangzhou 510330, China
† Corresponding
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-9419-2/23/04. . . $15.00
https://doi.org/10.1145/3543873.3587584
Jinan University
Guangzhou, China
wsgan001@gmail.com
Yongdong Wu
Jinan University
Guangzhou, China
hgengsen@gmail.com
ABSTRACT
Wensheng Gan†
Jinan University
Guangzhou, China
wuyd007@qq.com
ACM Reference Format:
Yao Chen, Shan Huang, Wensheng Gan, Gengsen Huang, and Yongdong Wu.
2023. Federated Learning for Metaverse: A Survey. In Companion Proceedings
of the ACM Web Conference 2023 (WWW ’23 Companion), April 30-May 4,
2023, Austin, TX, USA. ACM, New York, NY, USA, 10 pages. https://doi.org/
10.1145/3543873.3587584
1
INTRODUCTION
The meaning of the word "metaverse" [1–3] is a world beyond reality. At the technical level, the metaverse can be regarded as the
integration mechanism or carrier of big data and information technology [4]. The metaverse combines various technologies and hardware, such as virtual reality, digital twins, the Internet of Things,
blockchain, digital collections, and so on [5]. The main goal of
the metaverse is to build a new world of interaction between the
real and the virtual. All things in the real world can be digitally
replicated in the metaverse and interact with and influence the real
world [6]. The metaverse is the human imagination of digital life. It
also inspires the imagination about the future development of the
Internet. From the perspective of digital civilization, the metaverse
will drive the development of digital civilization.
In recent years, driven by practical needs and the feasibility
of the construction of the metaverse, the metaverse has attracted
worldwide attention. The metaverse has aroused much discussion
and research in various fields. Many technology companies are
also attracted by the metaverse [7]. Figure 1 lists some companies
that have invested in the metaverse field. Analyze the reasons why
the metaverse is important from the perspective of practical needs.
People want to work more efficiently under any circumstances and
want social and entertainment methods to become more diverse.
From the perspective of construction feasibility, the rapid developments of the Internet of Things, artificial intelligence, virtual reality,
edge computing, blockchain, and other technologies provide the
feasibility of the metaverse’s construction [8].
At present, the metaverse is in a stage of innovation and exploration, and it is full of chaos in terms of privacy data disclosure,
which needs to be regulated and managed. The metaverse can be
seen as the integration mechanism or carrier of big data and information technology. In general, the metaverse needs a lot of data
for support. However, these data are often difficult to obtain. On
the one hand, data privacy and security issues have attracted much
attention. Governments have enacted many laws to protect user
privacy, such as the GDPR [9] issued by the European Union and the
CCPA [10] issued by the United States. On the other hand, there is
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
In 2021, Facebook announced that
it would change its name to
"Meta" and was committed to
building the future metaverse.
In September 2022, NVDIA launched
the Omniverse Cloud, which is used to
build and run industrial metaverse
applications. The metaverse is one of
the key areas of NVIDIA.
Epic Games is committed to
developing a metaverse
programming language called Verse.
Facebook
NVDIA
Microsoft
Enterprises
enter the
metaverse
Roblox
Epic games UnitySoftware
Inc
Chen et al.
Microsoft is also an active player in
the field of metaverse, and is
investing and deploying in metaverse.
Roblox is the first company to
add the word "metaverse" in the
prospectus, and put forward the
key features of metaverse.
UnitySoftwareInc creates and
operates an interactive realtime 3D content platform.
Figure 1: Enterprises enter the metaverse.
competition in the industry. Sharing data may harm the company’s
own interests. There is no doubt that the metaverse is facing the
challenges of data privacy and security and the difficulties of data
collection at the data level.
Federated learning (FL) [11] is a powerful technique that can
solve the metaverse dilemma at the data level. FL is a distributed
machine learning paradigm with a privacy protection function,
designed for a large number of edge devices [12]. FL allows the
original data to be retained on each edge device and uses the data,
computing power, and model-building capabilities of each edge
device to perform tasks [13, 14]. The server in FL collects and
aggregates the parameters or models sent by the edge devices. The
operation of FL training models is an iterative process of model
distribution, collection, aggregation, and update [15]. In FL, each
round of iteration can be divided into three steps. First, the central
server generates a global model using the existing data or updates
the global model using the collected parameters. Then, this global
model is sent to all participants. Each participant trains this global
model using the local data to obtain some parameters. Finally, the
participants send the parameters to the central server. The central
server uses these parameters to update the global model. Until the
model reaches convergence, FL stops training the model. In order to
further improve the data privacy protection ability of the metaverse
and help collect data, this paper discusses the integration of the
metaverse and FL.
Federated learning for the metaverse (FL4M) is a promising research direction for the following reasons. First, FL ensures that
the original data does not leave the user. This is because the user
only needs to transmit parameters or models to third parties. It
guarantees the privacy and security of the user’s data. Therefore,
combining FL into the metaverse improves data security. Secondly,
FL is a distributed machine learning method. FL trains the model by
using each user’s data, computing ability, and model building ability.
Therefore, combining FL into the metaverse reduces the demands
on the memory and computing capacity of the central server. In
addition, FL can introduce contribution measurement mechanisms
and incentive strategies. They come into play when dealing with
cross-silos settings. FL evaluates the contribution value of each user
based on the contribution measurement mechanism and provides
incentives. In this way, more users will be attracted to participate
in the model training. Therefore, integrating FL’s contribution measurement mechanism and incentive strategy into the metaverse
will increase the interest of users and the accuracy of the model.
Contributions: The research in this paper focuses on the advances and opportunities presented by FL4M. The main contributions of this paper are as follows.
• To the best of our knowledge, this is the first paper discussing
the advances and opportunities of FL for the metaverse. We
explain why the FL should be integrated into the metaverse.
In many scenarios, FL has the advantage of effectively helping the metaverse. Besides, we analyze the reasons why
FL4M is a promising research direction.
• We discuss for the first time the convergence of key metaverse technologies and FL, such as big data, communication technologies, the Internet of Things, edge computing,
blockchain, and extended reality.
• We highlight some key challenges and promising directions
in FL4M and try to put forward some enlightening ideas for
this research direction. In addition, our survey aims to help
researchers understand the up-to-date development of FL4M.
We hope that our survey can help researchers understand
and establish a fair, open, and safe metaverse.
Organization: The rest of the paper is organized as follows.
In Section 2, we discuss related concepts, including metaverse,
artificial intelligence, FL, and artificial intelligence in the metaverse.
Then, in Section 3, we describe FL for key metaverse technologies
in detail. Section 4 and Section 5 respectively discuss the challenges
and promising directions of FL4M. Finally, Section 6 summarizes
this paper. The important acronyms are listed in Table 1.
Table 1: Summary of important abbreviations
Abbr.
FL
AI
NNs
DL
ML
IoT
MEC
XR
VR
AR
MR
HCI
Definition
Federated Learning
Artificial Intelligence
Neural Networks
Deep Learning
Machine Learning
Internet of Things
Mobile Edge Computing
Extended Reality
Virtual Reality
Augmented Reality
Mixed Reality
Human-Computer Interaction
2 RELATED CONCEPTS
2.1 Metaverse
The term "metaverse" originated from the American science fiction
novel Snow Crash [16, 17], which constructed a virtual space parallel to and interacting with real space. Unlike mobile terminals such
as cell phones and computers, which are only in flat space, the metaverse is a three-dimensional space-time Internet [18]. By merging
network technology and extended reality, the metaverse achieves
the goal of integrating virtual life with natural life and machine life
[19]. 2021 is the first year of the rapid development of the metaverse,
Federated Learning for Metaverse: A Survey
and academic and media circles all have deep expectations for the
realization of the metaverse [1]. In fact, the metaverse is not a new
concept or technology but an integration of existing technologies,
linking networks, hardware terminals, and users, thus forming a
virtual display system that is closely connected to the real world
yet highly independent [20]. Key technologies in the metaverse
include data science [21, 22], big data [2, 23], Internet of behavior
[24], artificial intelligence [25], interactivity [26], communication
technology [27], edge computing [28], IoT [29], blockchain [30],
and networking [31]. Better than the traditional Internet, the realistic simulated world created by the metaverse provides a more
real-time and immersive experience for users. The realization of
the metaverse requires three steps, including digital twins [32],
virtual natives, and virtual-real fusion [4]. The term "digital twins"
refers to the mapping of the real world by replicating it in the metaverse. For example, in Snow Crash, each user has incarnations in
the metaverse, i.e., a precise copy is created in the virtual world.
Virtual world scenes and users are also able to interact like in the
real world, thus achieving "virtual natives" status. The third realm
is an integrated symbiotic system where virtual and reality interact
and influence each other.
2.2
Artificial Intelligence
Artificial intelligence (AI) [33, 34] is intelligence exhibited by machines with the ability to perceive, synthesize, and infer information.
For example, the application scenarios of AI in pattern recognition
[35] include computer vision [36], machine translation [37], etc. In
addition, the cognitive functions of AI can be used for predictive
judgment, such as in autonomous driving [25], AlphaGo [38], etc. AI
maximizes efficient and successful responses or decisions by sensing
the external environment and imitating the way humans think. AI
is divided into two main categories: weak artificial intelligence and
strong artificial intelligence. Weak artificial intelligence refers to AI
that only focuses on one area without an independent will, such as
voice assistants and face recognition. Strong artificial intelligence,
on the other hand, is capable of acting autonomously outside of a
set program and can even surpass humans in some respects, such as
with driverless cars and intelligent robots. Research areas of AI include machine learning (ML) [39], artificial neural networks (NNs)
[40], deep learning (DL) [41], etc. Among them, machine learning,
as a method to achieve artificial intelligence, generally employs
algorithms to parse and learn from data to accomplish decisionmaking or prediction tasks. As an implementation technique in
machine learning, deep learning has enabled greater breakthroughs
in machine learning with applications in a wider range of fields
[37]. It is mentioned in [7] that AI can provide product inspection
and fault diagnosis in the metaverse. A number of CNN-based and
RNN-based frameworks have been proposed to build intelligent
fault detection systems in manufacturing. Examples include the
data-driven learning model on CNN architecture with a transfer
learning mechanism [42] and the deep encoder-decoder framework
on RNN architecture with an attention mechanism [43].
2.3
Federated learning
Federated learning was originally proposed by Google to update
models locally for Android phones [44]. The purpose of federated
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
learning is to carry out efficient ML while safeguarding the privacy
of the user and terminal data [11]. Thus, essentially, federated
learning is a distributed machine learning technique [45] used to
provide data privacy protection. Depending on the distribution
of data among multiple participants, federated learning is divided
into three main categories: horizontal federated learning, vertical
federated learning, and federated transfer learning [11].
• In this case, horizontal federated learning is to distribute the
sample data to each machine, and then these machines download
the model from the server separately for training, and finally, the
parameters will be fed back to each machine. This type of federated
learning is suitable for scenarios where the samples are different,
but the features between the samples are similar.
• The applicable scenario of vertical federated learning is the opposite of horizontal federated learning, i.e., the union of features of
cross-users under different businesses. During the learning process,
each participant is unaware of the existence of other parties and
only uploads their model parameters, but the prediction requires
joint modeling and collaboration to complete the task.
• Federated transfer learning can be considered useful when
there is no significant overlap between either the samples or their
features. The key problem is to identify similarities between the
source and target domains.
Federated learning [46] takes the distributed data and trains the
model locally for data security. With a trusted channel established
by secure algorithms such as homomorphic encryption, the results
of the model can be aggregated to a central node for training to
obtain the final training model. Federated learning ensures both
parity and independence among the participants. Moreover, the
entire process of learning and training achieves data isolation to
prevent data leakage, which well protects the privacy of users [45].
In such a distributed machine learning framework, the various participants can be coordinated to build more accurate global models.
Until now, there are many studies about federated learning.
2.4
Artificial Intelligence in the Metaverse
Artificial intelligence is an essential technological support for the
realization of the metaverse platform. On the one hand, the real
world is mapped to the virtual world through AI technologies such
as visual recognition. On the other hand, with the help of AI algorithms such as image generation, the virtual world is also able
to influence the real world. The essence of the metaverse is the
combination of various technologies to create a virtual world that
interacts with reality. Therefore, the combination of AI with VR,
AR, and other technologies is important for creating a reliable and
efficient metaverse. AI technology can not only improve the performance of the facilities in the metaverse but also provide security for
their infrastructure [7, 47]. For example, the algorithm proposed
by Yampolskiy et al. [48] is able to accurately identify and verify
virtual characters. In the metaverse, law enforcement agencies also
need this type of technology to track a character who may have
committed a criminal act [8]. In addition, the metaverse needs to be
supported by communication technologies, and there are already
many AI technologies [49–51] that are widely used in network
architecture applications that provide reliable and low-latency network access and security.
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
3
KEY FL4M TECHNOLOGIES
As shown in Figure A1, FL4M mainly has the following six key
techniques. The combination of FL and big data, communication
technology, IoT, edge computing, blockchain, and extended reality
technology can help alleviate the privacy leakage problem in the
metaverse and facilitate the sharing of data resources.
3.1
Big Data of FL4M
Big data is the foundation for metaverse applications.
• Purposes. Big data refers to data that contains many types and
whose volume is large and requires fast processing speed. Therefore, the acquisition, storage, analysis, and protection of big data is
a crucial and challenging task. In the metaverse, the constructed
digital space is essentially data. In other words, the metaverse can
be regarded as a fusion mechanism for big data and various information technologies. When building a bridge between virtual
and reality, real-time data gathering and processing capabilities of
sensors and smart devices are the cornerstones for supporting large
numbers of users online at once.
• Challenges. In the midst of increased privacy protection and
data oversight, the phenomenon of data existing as isolated islands
has emerged [11]. The rich data of multiple business departments
in an enterprise is defined by each other independently. Data from
a single department is similar to an isolated island that is incapable
of interacting with other rich data [46]. In the era of the metaverse,
data is undoubtedly one of the most important assets of an enterprise. Therefore, it is vital to shatter the isolated island of data
in order to actualize information exchange and the hidden value
underlying big data [2].
• Methods. In compliance with data security management, each
participant in the federated model is able to use its own data for
joint training with others in a safe and fair manner without causing
data leakage. In the metaverse, each user can be abstracted as a
data owner. Utilizing all the potential value that may be found in
data and fully optimizing business technology and services are both
possible with the help of big data and FL technology.
• Applications. FL, a new generation of technology, is applied
in finance, healthcare, and security to address isolated data islands
and privacy protection issues [52]. FATE is the first industrial-level
federated learning open-source framework developed by WeBank,
which can build federated models of general business scenarios.
3.2
Communication Technology of FL4M
• Purposes. It can be expected that the metaverse will generate
huge data throughput. When it comes to human-computer interaction, a huge user community demands lower latency. Therefore,
the interaction between the virtual and real worlds in the metaverse is based on high-performance communication technologies.
Wired and wireless communication are the two main categories of
communication. Without taking into account the distance between
computers and communication devices, wireless communication is
more flexible, making it more appropriate for the metaverse.
• Challenges. Wireless communication technology has evolved
to 5G at present. However, its transmission speed is still some way
from the communication goal of the metaverse. In this case, future
Chen et al.
6G wireless networks need to address high throughput, low latency,
and the security of data transmission in the metaverse.
• Methods. Yang et al. [53] proposed the use of federated learning for 6G networks. Federated learning is able to conserve wireless
resources and reduce transmission latency with distributed management. Moreover, sharing predictive models and collaborative
learning are needed for federated learning among the devices. The
model parameters are transferred from each terminal device in
the metaverse to the server more quickly thanks to the effectively
developed network transmission technology. The combination of
communication technology and FL will realize the comprehensive
promotion of intelligence anytime and anywhere in the metaverse.
• Applications. Federated supervised learning algorithms can
provide network environment analysis and intrusion prediction. A
prediction model is a crucial tool for determining how to distribute
wireless resources. Convex optimization, for instance, can be solved
using the federated reinforcement learning technique. It breaks
down the difficult issue into manageable components like resource
management and network control. Therefore, federated learning
can improve the intelligence of wireless networks, which is the
goal pursued in the metaverse [54]. Unlike traditional machine
learning, federated learning is able to avoid direct exposure of data
to third parties at each terminal and has natural protection for data.
The application of federated learning to wireless communication in
the metaverse has great potential. For example, in [55], federated
learning is used for the estimation of key parameters in vehicular
communication. Experiments demonstrate the scheme significantly
preserves privacy while conserving transmission resources.
3.3
IoT of FL4M
• Purposes. The Internet of Things (IoT) is an extended and expanded network based on the Internet that extends the interaction
between users and objects. The IoT generates data through a large
number of sensors and then uses artificial intelligence technology
to accomplish various tasks. In the metaverse, the ways to link
virtual and physical spaces are diverse. Therefore, IoT technology
assists the metaverse in achieving data exchange and easy access.
• Challenges. However, it is obvious that safely offloading the
explosive data generated by each edge device to the server is a
difficult problem. Furthermore, in the IoT, each user’s data is heterogeneous [56]. It is unrealistic to apply traditional ML models to
complex application scenarios in the metaverse.
• Methods. Recently, FL, a distributed collaborative AI approach,
has been proposed to build an intelligent and privacy-enhanced
IoT system [57]. The combination of FL and IoT technology in the
metaverse not only provides privacy protection for distributed IoT
systems but also reduces the communication delays caused by data
offloading. In other words, in the metaverse, the combination of IoT
and FL completes the transformation from physics to data while
ensuring the privacy of users.
• Applications. Recent studies [58, 59] have shown that FL can
be broadly applicable in the IoT industry, such as in smart healthcare and smart cities, etc. In addition, FL also helps detect malicious
attacks in federated IoT systems. The TensorFlow federated project
(TFF) is an open-source federated learning platform developed by
Google, and it contains two layers of interfaces: the federated layer
Federated Learning for Metaverse: A Survey
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
API and the federated core (FC) API. Besides, synchronously optimized modeling algorithms like FedAvg [44], personalized federation learning schemes [60, 61] in the metaverse are also applicable
to build models adapted to different clients and scenarios.
3.4
3.5
Lo
Blockchain of FL4M
• Purposes. Different from other technologies used in the metaverse, blockchain not only establishes the connection between these
technologies but also creates a complete closed loop with scenarios,
behaviors, and actions. The metaverse constructs a distributed economic system based on two essential characteristics of blockchain,
digital identity, and personal data rights confirmation.
• Challenges. The traditional centralized federated learning
framework relies on the security of the centralized nodes, while
one of the core requirements of blockchain in the metaverse is
ca
od
lm
el
b
Glo
al
d
mo
el
Gl
ob
Lo
ca
al
Offloading stage
Learning stage
Personalization stage
mo
lm
od
de
l
el
Edge
Edge Computing of FL4M
• Purposes. Edge computing [62] refers to computing generated
near the device to provide low-latency services and more rapid
response. In contrast, cloud computing [63] is processed by transferring data to a cloud computing center. Applications such as live
video platforms and e-commerce platforms are inherently dependent on cloud computing. Each IoT device can select to transfer
its intensive computing tasks to the edge devices via wireless connectivity, thereby reducing communication overhead. However, in
some areas that require high real-time performance, cloud computing may have an incalculable impact on users due to high latency
and network lag due to resource constraints. For example, if a selfdriving car suffers a delay in its direct command during driving,
failing the system to respond in a timely manner, it could lead to
a traffic accident. Therefore, in the metaverse, a huge amount of
data generated by massive IoT device connections requires more
real-time and secure computing technology [64].
• Challenges. Edge computing meets this requirement, offering computation and storage to users close to edge devices while
lowering the bandwidth pressure of data transmission to the cloud
center server [28]. However, it still leaves the problem of user data
being shared externally. As a result, some businesses are reluctant
to share their private datasets with edge servers for fear that rivals
will steal and use the information against them.
• Methods. FL was introduced into mobile edge computing
(MEC) technology to ensure that the user’s sample data is always
stored locally on the device [65]. Collaboration among users requires only uploading the locally trained model parameters to the
central server. A more desirable global model can be achieved with
the collaboration of all participants. In addition, even if a small
number of edge devices are under attack or fail, it will not lead
to problems of massive data privacy exposure or service interruption. Consequently, the integration of edge computing enhances
the security and reliability of federated learning [66].
• Applications. Figure 2 shows that the PerFit framework [61]
provides the necessary edge computing power for IoT devices. IoT
devices are ubiquitous in the metaverse, which makes cyberattacks
more varied and sophisticated. Combining edge computing with FL,
network attack detection models guarantee users’ privacy while
collaborating with multiple devices for optimization.
Global model is learned
by aggregating and
averaging local models
Cloud
Computing
tasks
IoT devices
Integrated
communication and
computing
Computing
tasks
Personalized
model
Global
model
Personal
information
Figure 2: Edge computing in FL for IoT applications [61].
Computing tasks
decentralization. Therefore, how to combine federated learning
with blockchain is a challenging problem.
• Methods. At present, there have been many studies [66–68]
combining FL with blockchain, replacing the central aggregator
with peer-to-peer blockchain, and utilizing blockchain nodes to
aggregate the federated learning model. In this way, the security
and privacy of data in distributed machine learning are guaranteed.
Such distributed storage ensures the identity authentication of users
and the security of digital assets. In particular, economic activities
in the metaverse, such as real estate development and game rewards,
are inseparable from cryptocurrencies and permanent records of
digital transactions. Digital identities in the metaverse can be traced
to some underlying data in such a transparent and open paradigm.
• Applications. Integrating blockchain and FL can prevent
model poisoning attacks and form a more stealthy defense mechanism [69]. Kang et al. [5] proposed the application of combining
blockchain and FL in the industrial metaverse. By establishing a
cross-chain authorized federated learning framework, it is possible
to conduct safe and protected data training in virtual space and
real space. In the metaverse, blockchain and federated learning can
complement each other. Blockchain facilitates federated learning to
establish a learning incentive mechanism and realize the automatic
distribution of benefits.
3.6
Extended Reality of FL4M
• Purposes. Extended Reality (XR) is essential in the metaverse to
achieve the interaction between the virtual and real worlds. XR is
widely considered to be the key to opening the door to the metaverse. Extended reality includes VR (virtual reality), AR (augmented
reality), and MR (mixed reality). VR employs devices to simulate a
virtual environment to assist users in fully immersing themselves
in the virtual world. On the other hand, AR draws virtual images
in the real world by using imaging technology, allowing users to
experience the intersection of the real and virtual worlds. MR incorporates the technical features of VR and AR.
• Challenges. Of course, hardware equipment like wearable
headsets or smart terminals is necessary for the technological realization of XR. The scene construction of these devices is inseparable from the recognition and target-tracking algorithms. However,
those target detection algorithms require a lot of data training to
obtain an accurate model while only a limited amount of data can
be stored on mobile devices.
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
• Methods. An illustration of XR of FL4M is shown in Figure
A2. Users in the real world interact with the virtual world through
XR technology. Each user has an avatar and a large amount of data
in the metaverse. These users can perform federated learning in
the metaverse. That is, these users use their own devices and data
to jointly participate in a machine learning task.
• Applications. The study [70] applied FL with mobile augmented reality to address the issues of data storage and model
accuracy. It also discusses the feasibility of the FL-MAR system and
proposes an optimized resource allocation scheme for the FL-MAR
system. As a result, the metaverse leverages the combination of XR
and FL more effectively in fields such as education, shopping, and
autonomous driving. Everyone has varied shopping preferences
and lifestyles, for instance. Combining FL and XR provides customers with better virtual shopping experiences and more accurate
shopping recommendations by meeting the local training models
of different mobile devices.
4
KEY CHALLENGES
Federated learning and the metaverse are two promising concepts.
The former can address the issue of security in model transmission and sharing to an extent. The metaverse aims to improve the
interactivity between the physical and virtual worlds, as well as
to improve the user experience. The convergence of their uses is
not a simple combination of technological solutions. That is, it is
not just AI research with privacy protection in various areas of the
metaverse. Therefore, in our opinion, the following challenging
issues are worth exploring and discussing in detail. Figure 3 lists
the keywords of each challenge to give an overview of the FL4M
challenges. Details are described below.
Unbalanced network,
computing power, and
bandwidth.
Ex
i
and sting
per defi
son cien
a liz c ie
a tio s
n.
Ot h
cha
er
lle n
ges
nd
ya
vac ty
Pri ecuri
s
and
ent
nm
iro data.
nv
d e ve
ste ensiti
s
T ru
Limited
resources
te r
He
ht
e ig
htw s
Lig ethod
m
L
ht F
eig ight
htw twe
Lig Ligh rse.
e
and etav
m
og
da eneo
ta
us
M
ult
ipl
e
div se
ers rvi
e d ces
a ta a n
d
.
Challenge
Resource
allocation
Model accuracy, system
security, and energy
consumption.
Figure 3: The keywords of each challenge.
• Limited resources. For a system or framework, resources are
often limited and need to be scheduled and allocated appropriately.
The problem of limited resources may be caused by various reasons,
such as the heterogeneity of data. For applications and deployments
with multiple unbalanced network nodes and metaverse use, a lot
of content is difficult to accomplish without a cluster of computing
Chen et al.
networks [71]. When there is a shortage of some important computing resources, the whole system may quickly reach a bottleneck
or fail to complete the task. Some problems involving communication and computation also need to be solved. Meanwhile, a large
amount of data would be generated. The data may be valuable but
difficult to organize. This may make the entire FL process more
burdensome. The first issue is how to handle the data generated
by different devices with limited resources. Another fundamental
issue is the trade-off between data utility and system efficiency.
With fewer data available, there is a need to ensure that the whole
system operates efficiently and accurately. For the accuracy of the
model, this may cause additional overhead. In addition, interactions
at the metaverse level need to be taken into account. In a real-world
society, it is not reasonable to make users perform long waits or
complex behaviors. The design of the entire application requires
the integration of many process details.
• Privacy and security. In the applications of the metaverse,
users often generate a variety of large amounts of data in their
activities. At the same time, various devices that connect the virtual
and real worlds also collect data related to users. This raises the
issue of users’ privacy protection. In an untrustworthy metaverse
environment, this may lead to a waste of resources or deterioration
of model performance in the training of FL. Although privacy protection is a key concern of FL techniques, there is still the risk of
private data leakage in the metaverse. The data in the collection of
devices, the interaction between the real and virtual worlds, and
the transmission of the FL framework may have the possibility of
leakage. The metaverse is the end closer to the data and requires
greater responsibility for the supervision and storage of the data.
Once data is at risk here, it makes no sense, even if FL strictly protects it. Besides, handling large data in the metaverse for FL requires
an efficient and lightweight security and privacy solution. Security
and efficiency need to be considered simultaneously. Some security
mechanisms, such as the RSA encryption method, while reliable
enough, are also not applicable to scenarios with large data. FL
needs to focus on what perspective and in what way to protect the
data. For example, the random response approach in FL is efficient,
but it is not always able to achieve the complex data generated by
the devices of the metaverse.
• Heterogeneous data. To maintain data utility, the metaverse
must handle multi-source heterogeneous data. The quality of metaverse’s work is influenced by the source and completeness of the
data. Especially in predictive applications, if the data is collected
by inaccurate devices, the overall performance may be poor. The
problem of data heterogeneity in FL is mainly caused by the diverse
data of each client involved in the training. These data are nonindependent and identically distributed (non-IID), often leading to
severe degradation of model accuracy. Effective and comprehensive
segmentation strategies and datasets can cover different non-IID
scenarios. There is a need to devise efficient methods for FL to
address the redundancy and unrealistic nature of heterogeneous
data in the metaverse. In the industrial metaverse, HFedMS [58]
reduces the data heterogeneity of FL by using dynamic grouping
and training mode conversion. Historical data semantics are compressed to compensate for forgotten knowledge. However, how
can we solve this challenge in other scenarios? It is worth noting
that the multiple devices of the meta-universe also introduce other
Federated Learning for Metaverse: A Survey
aspects of heterogeneity. It is difficult to ensure that the protocols,
formats, and workflows are consistent across devices. Although
complex settings are freer, they also add a certain degree of burden
to a system.
• Resource allocation. There is a crucial challenge in allocating resources in a rational and effective way. The limited resources
make it necessary for FL to run in a better-planned way in the
metaverse to avoid some extreme cases, such as too much running
time and poorly trained models. Multiple metrics, including total
energy consumption, system security, and model accuracy, form a
weighted optimization problem. Low bandwidth makes the training
of FL models or the response of metaverse applications too slow.
Devices with different computing powers also cause the overall
performance of the training to be uneven or users to become biased.
It is difficult to consider how to design FL solutions that can run
stably with low or unbalanced computing resources according to
the scenario requirements. Some applications seek greater system
security, while others seek better model accuracy. Although there
are some algorithms and schemes [6, 59] proposed to solve the load
and resource allocation problems of the metaverse systems, they
do not take into account the dynamic cases. The number of system
nodes is usually not determined at the time of application deployment, and the allocation of resources should change dynamically
as nodes are added or removed. In addition, the robustness of the
resource allocation scheme needs to be considered in order to cope
with extreme situations.
• Lightweight methods. Metaverse applications can be accessed and interacted with at any time through IoT devices. The
highly immersive virtual experience provided to users is served
by a large amount of human-computer interaction (HCI) [72]. The
large number of devices in FL also brings expensive computation
costs. With limited computing and communication resources, some
traditional large-scale deployment schemes are not suitable for IoT
devices. The design of lightweight models not only simplifies the
process of accessing applications of the metaverse but also reduces
the burden of training in FL. Existing metaverse applications still
lack lightweight AI techniques. The relevant definitions are not
very clear in the industry. MetaAID [47] is a flexible framework
designed to support the language and semantic technologies in
digital twins and virtual humans. There are also many lightweight
federated schemes. A lightweight privacy-preserving scheme of FL
is proposed by Wei et al. [73] to protect the security of all local data.
To deal with large-scale FL tasks, a secure mask-reusing mechanism is also designed. In the research of federated recommender
systems, Zhang et al. [74] proposed a model called LightFR that is
able to perform tasks well in the condition of devices with limited
computing resources.
• Other challenges. In addition to the challenges mentioned
above, there are others that deserve the attention of researchers.
For example, the personalization of FL in metaverse applications is
an act that allows specific services and content empowerment for
different users. One of the emerging challenges in the metaverse
is how to perform personalized model learning or personalized
solutions. To prevent the overall system from failing, the issue of
FL attacks and defenses should also be addressed in the metaverse.
Besides, some of the deficiencies of FL can also be solved or minimized in future studies using metaverse techniques. It will welcome
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
both practical and theoretical contributions for solutions to the
interpretability/explainability problems in FL4M.
5
PROMISING DIRECTIONS
In our opinion, integrating FL into the metaverse is conducive to
building an open, fair, and safe metaverse. At present, FL4M is still
in its initial stage, and there are many research problems. Therefore,
in this section, we discuss several potential research directions of
FL4M, including the incentive mechanism of FL4M, intellectual
property right protection in FL4M, the model fairness of FL4M, the
privacy-utility trade-off in FL4M, catastrophic forgetting in FL4M,
and the application of FL4M.
• Incentive mechanism of FL4M. In the future, people will
also live in the metaverse when they are in the virtual world. Everyone’s information and activity data are recorded as data. Therefore,
each user in the metaverse has a large number of digital assets.
However, we cannot ignore two important issues. How to encourage people to contribute data to the construction of the metaverse?
How do we encourage people to use their own data and equipment
to participate in machine learning tasks in the metaverse? A reasonable incentive mechanism is a solution to these problems. The
design goal of the incentive mechanism is to distribute rewards
fairly according to everyone’s contribution, so as to encourage
more people to join the machine learning tasks in the metaverse.
A reasonable incentive mechanism can increase the enthusiasm
of data asset owners. Furthermore, the incentive mechanism has
the potential to reduce the behavior of malicious users who intentionally send incorrect data. Therefore, designing a reasonable
incentive mechanism in the field of FL4M is a promising research
direction. We believe that a reasonable incentive mechanism has
at least several characteristics, such as trusted, fairness, credibility,
accuracy, and timeliness.
• Intellectual property right protection in FL4M. In the field
of FL4M, the protection of model intellectual property rights is an
important problem. Training a federated learning model successfully usually requires a lot of resources. The computational and
storage resources of each participant are consumed. The trained
model should be considered the product of the labor of all participants. Therefore, participants should own the corresponding
intellectual property rights. The protection of model intellectual
property rights should not only ensure that the participants obtain
the corresponding model intellectual property rights but also protect the model from illegal copying, tampering, redistribution, and
abuse. Some hackers may steal the model through illegal means
and then tamper with the model or claim ownership of the model.
These behaviors will damage the interests and intellectual property
rights of model owners. In this case, verifying the ownership of
the model and thus protecting the model owner’s intellectual property is a matter of great concern. The issue of model intellectual
property protection is an interdisciplinary and comprehensive topic
involving computer security, intellectual property protection, law,
and many other aspects.
• Model fairness of FL4M. FL4M is easy to have the problem of
model unfairness. Model unfairness occurs in model training. The
distribution of training data or the deviation in the model makes
the model perform unevenly on some sensitive features. A common
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
phenomenon is that individual feature data are underrepresented
in the total dataset, so the weights obtained after the model has
been trained are not representative of the problem. For example,
we use gender as a sensitive feature. If the ratio of male to female
participants is 100:1, then samples containing males will dominate
the model training phase. Thereby, the model may not learn accurately for the female sample. Unbalanced data distribution is a very
common phenomenon in the FL4M field. Since FL training models require multiple participants, each participant is a data owner.
The participants have different characteristics and behaviors. The
data distribution among participants may be different in different
regions. Therefore, in the field of FL4M, it is easy to have model
unfairness due to unbalanced data distribution. Model unfairness
can seriously damage the accuracy of the model. How to introduce
fairness into the model is an important research topic. This allows
the model to produce fair output even if it is trained on unfair data.
This research direction is very important because unbalanced data
distribution is common.
• Privacy-utility trade-off in FL4M. One of the core issues
of FL4M is how to achieve a balance between privacy and utility.
Privacy can be quantified through information privacy [75]. Accuracy can be used to quantify utility. Quantifying the privacy utility,
however, is a difficult problem. We are aware that security and
availability are opposing factors. Based on practical requirements,
we should consider security and availability together in order to
maximize availability while meeting security requirements. This
frequently necessitates a great deal of knowledge and effort. As a
result, we wonder if there is a unified standard or framework for
calculating privacy utility. Furthermore, whether there are algorithms that optimize security, model performance, and accuracy all
at the same time is an important research question. Privacy and
utility are frequently required in practical application scenarios.
There is no doubt that the FL4M algorithms should strike a balance
between privacy and utility.
• Catastrophic forgetting in FL4M. Catastrophic forgetting
refers to an artificial intelligence system (such as a deep learning
model) forgetting or losing some of its previously acquired abilities
while learning a new task or adapting to a new environment. In
the FL4M domain, the following scenarios can cause or exacerbate
the phenomenon of catastrophic forgetting. i) When data are not
distributed independently and identically across different clients,
the model may forget previous knowledge gained by training with
other clients’ data. This phenomenon arises because of the discrepancy between the local and global data distributions. ii) In the
federated category incremental learning problem, participants may
often collect new categories. Since each participant’s device has
very limited storage space, it is difficult to keep a sufficient amount
of data for all the categories. In this case, the performance of the
model on the old category data is likely to suffer from severe catastrophic forgetting. iii) When new participants are added during the
model training, as these new participants often have a large amount
of new data. The catastrophic forgetting of the global model can
be further exacerbated in this case. Catastrophic forgetting leads
to a higher rate of loss of training data and a longer time to train
the model. Therefore, how to solve the problem of catastrophic
forgetting in FL4M is a promising research direction.
Chen et al.
• Application for FL4M. Finding FL4M application scenarios
and using FL4M to solve problems is an important direction for
research. Until now, there have been a lot of studies that focus on
how the metaverse can be used in situations like education, games,
healthcare, social networks, advertising, marketing, etc. In particular, some literature discusses the application scenarios of FL4M
in detail. For example, Kang et al. [5] used FL to analyze sensing
data in the industrial metaverse. In addition, some studies focus on
the combination of FL and other technologies in the metaverse to
solve practical problems. For example, Zhou et al. [70] discussed
the necessity and rationality of the combination of FL and MAR
in the metaverse and studied two cases of the metaverse FL-MAR
system. When developing technology, we must consider not only
the technology itself but also the technology’s application scenarios. When technology is thought to add significant value to certain
fields, it will naturally advance. As a result, it is critical to investigate FL4M application scenarios (e.g., FL4M for healthcare, edge
devices, advertising, social networks, blockchain, web search, etc.).
The investigation of FL4M application scenarios may attract additional research and expose the shortcomings of the technologies
when FL4M is used.
6
CONCLUSION
The metaverse, which is still in its early stages, is facing data challenges that will stymie its development. With increasing user privacy protection awareness and competition among enterprises or
institutions, data acquisition becomes increasingly difficult. Data
acquisition can be made easier and more legal in a FL-enabled
metaverse. In this paper, we provide an introduction to FL4M and
examine the necessity and rationality of studying FL4M in depth.
FL4M not only protects users’ data privacy to some extent, but
it also makes use of each user’s data, computational power, and
model-building abilities. Further research on FL4M contributes to
the creation of a future metaverse that is open, fair, and secure.
The combination of some key metaverse and FL technologies is
discussed in this paper. Finally, we investigate FL4M’s challenges
and promising directions. Remember that FL4M is likely to receive
more algorithmic and theoretical improvements in the future. We
hope that this article will stimulate the interest of more researchers
in FL4M and inspire them to conduct more innovative research.
ACKNOWLEDGMENTS
This research was supported in part by the National Natural Science Foundation of China (Nos. 62002136, 62272196, and 61932011),
Fundamental Research Funds for the Central Universities of Jinan
University (No. 21622416), Guangzhou Basic and Applied Basic Research Foundation (No. 202102020277), Natural Science Foundation
of Guangdong Province (Nos. 2022A1515011861 and 2019B1515120010),
Guangdong Key R&D Plan2020 (No. 2020B0101090002), National
Key R&D Plan of China (No. 2020YFB1005600), the Young Scholar
Program of Pazhou Lab (No. PZL2021KF0023), National Joint Engineering Research Center for Network Security Detection and
Protection Technology, and Guangdong Key Laboratory for Data
Security and Privacy Preserving. Dr. Wensheng Gan is the corresponding author of this paper.
Federated Learning for Metaverse: A Survey
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
REFERENCES
[1] Jiayi Sun, Wensheng Gan, Han-Chieh Chao, and Philip S Yu. 2022. Metaverse:
survey, applications, security, and opportunities. arXiv:2210.07990 (2022).
[2] Jiayi Sun, Wensheng Gan, Zefeng Chen, Junhui Li, and Philip S Yu. 2022. Big
data meets metaverse: A survey. arXiv preprint, arXiv:2210.16282 (2022).
[3] Zefeng Chen, Jiayang Wu, Wensheng Gan, and Zhenlian Qi. 2022. Metaverse
security and privacy: An overview. In IEEE International Conference on Big Data.
IEEE, 2950–2959.
[4] Yuntao Wang, Zhou Su, Ning Zhang, Rui Xing, Dongxiao Liu, Tom H Luan, and
Xuemin Shen. 2022. A survey on metaverse: Fundamentals, security, and privacy.
IEEE Communications Surveys & Tutorials (2022).
[5] Jiawen Kang, Dongdong Ye, Jiangtian Nie, Jiang Xiao, Xianjun Deng, Siming
Wang, Zehui Xiong, Rong Yu, and Dusit Niyato. 2022. Blockchain-based federated
learning for industrial metaverses: Incentive scheme with optimal aoi. In IEEE
International Conference on Blockchain. IEEE, 71–78.
[6] Yitong Wang and Jun Zhao. 2022. A survey of mobile edge computing for the Metaverse: Architectures, applications, and challenges. arXiv preprint, arXiv:2212.00481
(2022).
[7] Quoc Viet Pham, Xuan Qui Pham, Thanh Thi Nguyen, Zhu Han, Dong Seong Kim,
et al. 2022. Artificial intelligence for the metaverse: A survey. arXiv: 2202.10336
(2022).
[8] Qinglin Yang, Yetong Zhao, Huawei Huang, Zehui Xiong, Jiawen Kang, and Zibin
Zheng. 2022. Fusing blockchain and AI with metaverse: A survey. IEEE Open
Journal of the Computer Society 3 (2022), 122–136.
[9] Paul Voigt and Axel Von dem Bussche. 2017. The EU general data protection
regulation (GDPR). A Practical Guide, 1st Ed., Cham: Springer International
Publishing 10, 3152676 (2017), 10–5555.
[10] Stuart L Pardau. 2018. The California consumer privacy act: Towards a Europeanstyle privacy regime in the United States. Journal of Technology Law and Policy
23 (2018), 68.
[11] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine
learning: Concept and applications. ACM Transactions on Intelligent Systems and
Technology 10, 2 (2019), 1–19.
[12] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode,
Rachel Cummings, et al. 2021. Advances and open problems in federated learning.
Foundations and Trends® in Machine Learning 14, 1–2 (2021), 1–210.
[13] Dun Zeng, Siqi Liang, Xiangjing Hu, Hui Wang, and Zenglin Xu. 2021. FedLab:
A flexible federated learning framework. arXiv preprint, arXiv:2107.11621 (2021).
[14] Yao Chen, Wensheng Gan, Yongdong Wu, and Philip S Yu. 2023. Privacypreserving federated mining of frequent itemsets. Information Sciences 625
(2023), 504–520.
[15] Yao Chen, Yijie Gui, Hong Lin, Wensheng Gan, and Yongdong Wu. 2022. Federated
learning attacks and defenses: A survey. (2022), 4256–4265.
[16] Judy Joshua. 2017. Information bodies: Computational anxiety in Neal Stephenson’s snow crash. Interdisciplinary Literary Studies 19, 1 (2017), 17–47.
[17] Lik-Hang Lee, Tristan Braud, Pengyuan Zhou, Lin Wang, Dianlei Xu, Zijun Lin,
Abhishek Kumar, Carlos Bermejo, and Pan Hui. 2021. All one needs to know about
metaverse: A complete survey on technological singularity, virtual ecosystem,
and research agenda. arXiv:2110.05352 (2021).
[18] Huansheng Ning, Hang Wang, Yujia Lin, Wenxi Wang, Sahraoui Dhelim, Fadi
Farha, Jianguo Ding, and Mahmoud Daneshmand. 2021. A survey on metaverse:
the state-of-the-art, technologies, applications, and challenges. arXiv:2111.09673
(2021).
[19] Dawn Owens, Alanah Mitchell, Deepak Khazanchi, and Ilze Zigurs. 2011. An
empirical investigation of virtual world projects and metaverse technology capabilities. ACM SIGMIS Database: the Database for Advances in Information Systems
42, 1 (2011), 74–101.
[20] Stylianos Mystakidis. 2022. Metaverse. Encyclopedia 2, 1 (2022), 486–497.
[21] Wensheng Gan, Jerry Chun-Wei Lin, Philippe Fournier-Viger, Han-Chieh Chao,
and Philip S Yu. 2019. A survey of parallel sequential pattern mining. ACM
Transactions on Knowledge Discovery from Data 13, 3 (2019), 1–34.
[22] Wensheng Gan, Jerry Chun-Wei Lin, Philippe Fournier-Viger, Han-Chieh Chao,
Vincent S Tseng, and Philip S Yu. 2021. A survey of utility-oriented pattern
mining. IEEE Transactions on Knowledge and Data Engineering 33, 4 (2021),
1306–1327.
[23] Wensheng Gan, Jerry Chun-Wei Lin, Han-Chieh Chao, and Justin Zhan. 2017.
Data mining in distributed environment: a survey. Wiley Interdisciplinary Reviews:
Data Mining and Knowledge Discovery 7, 6 (2017), e1216.
[24] Jiayi Sun, Wensheng Gan, Han-Chieh Chao, Philip S Yu, and Weiping Ding. 2023.
Internet of Behaviors: A Survey. IEEE Internet of Things Journal (2023), 1–18.
[25] Zixuan Zhang, Feng Wen, Zhongda Sun, Xinge Guo, Tianyiyi He, and Chengkuo
Lee. 2022. Artificial intelligence-enabled sensing technologies in the 5G/Internet
of things era: From virtual reality/augmented reality to the digital twin. Advanced
Intelligent Systems (2022), 2100228.
[26] Anna Zauskova, Renata Miklencicova, and Gheorghe H Popescu. 2022. Visual
imagery and geospatial mapping tools, virtual simulation algorithms, and deep
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
[45]
[46]
[47]
[48]
[49]
[50]
[51]
[52]
learningbased sensing technologies in the metaverse interactive environment.
Review of Contemporary Philosophy 21 (2022), 122–137.
Jiadong Yu, Ahmad Alhilal, Pan Hui, and Danny HK Tsang. 2022. 6G mobile-edge
empowered metaverse: Requirements, technologies, challenges and research
directions. arXiv:2211.04854 (2022).
Yushan Siriwardhana, Pawani Porambage, Madhusanka Liyanage, and Mika
Ylianttila. 2021. A survey on mobile augmented reality with 5G mobile edge computing: architectures, applications, and technical aspects. IEEE Communications
Surveys & Tutorials 23, 2 (2021), 1160–1192.
Kinza Shafique, Bilal A Khawaja, Farah Sabir, Sameer Qazi, and Muhammad
Mustaqim. 2020. Internet of things (IoT) for next-generation smart systems: A
review of current challenges, future trends and prospects for emerging 5G-IoT
scenarios. IEEE Access 8 (2020), 23022–23040.
Bektur Ryskeldiev, Yoichi Ochiai, Michael Cohen, and Jens Herder. 2018. Distributed metaverse: creating decentralized blockchain-based model for peer-topeer sharing of virtual spaces for mixed reality applications. In The 9th Augmented
Human International Conference. 1–3.
Fengxiao Tang, Xuehan Chen, Ming Zhao, and Nei Kato. 2022. The roadmap of
communication and networking in 6G for the metaverse. IEEE Wireless Communications (2022).
Mengnan Liu, Shuiliang Fang, Huiyue Dong, and Cunzhi Xu. 2021. Review of
digital twin about concepts, technologies, and industrial applications. Journal of
Manufacturing Systems 58 (2021), 346–361.
Sotiris B. Kotsiantis. 2007. Supervised Machine Learning: A Review of classification techniques. Informatica (Slovenia) 31, 3 (2007), 249–268.
Stuart J Russell. 2010. Artificial intelligence: A modern approach. Pearson Education, Inc.
Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and
machine learning. Number 4. Springer.
Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. 2018. Deep learning for computer vision: A brief review.
Computational Intelligence and Neuroscience 2018 (2018).
Yang Lu. 2019. Artificial intelligence: a survey on evolution, models, applications
and future trends. Journal of Management Analytics 6, 1 (2019), 1–29.
AI Strong. 2016. Applications of artificial intelligence & associated technologies.
Science 5, 6 (2016).
Tom Michael Mitchell et al. 2007. Machine learning. McGraw-hill New York.
Bayya Yegnanarayana. 2009. Artificial neural networks. PHI Learning Pvt. Ltd.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature
521, 7553 (2015), 436–444.
Liang Guo, Yaguo Lei, Saibo Xing, Tao Yan, and Naipeng Li. 2018. Deep convolutional transfer learning network: A new method for intelligent fault diagnosis of
machines with unlabeled data. IEEE Transactions on Industrial Electronics 66, 9
(2018), 7316–7325.
Hojin Lee, Hyeyun Jeong, Gyogwon Koo, Jaepil Ban, and Sang Woo Kim. 2020.
Attention recurrent neural network-based severity estimation method for interturn short-circuit fault in permanent magnet synchronous machines. IEEE
Transactions on Industrial Electronics 68, 4 (2020), 3445–3453.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and
Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. PMLR,
1273–1282.
Jakub Konečnỳ, H Brendan McMahan, Felix X Yu, Peter Richtárik, and et al.
2016. Federated learning: Strategies for improving communication efficiency.
arXiv:1610.05492 (2016).
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated
learning: Challenges, methods, and future directions. IEEE Signal Processing
Magazine 37, 3 (2020), 50–60.
Hongyin Zhu. 2022. MetaAID: A flexible framework for developing metaverse applications via AI technology and human editing. arXiv preprint, arXiv:2204.01614
(2022).
Roman V Yampolskiy, Brendan Klare, and Anil K Jain. 2012. Face recognition in
the virtual world: recognizing avatar faces. In 11th International Conference on
Machine Learning and Applications, Vol. 1. IEEE, 40–45.
Mingzhe Chen, Ursula Challita, Walid Saad, Changchuan Yin, and Mérouane
Debbah. 2019. Artificial neural networks-based machine learning for wireless
networks: A tutorial. IEEE Communications Surveys & Tutorials 21, 4 (2019),
3039–3071.
Changyang She, Rui Dong, Zhouyou Gu, Zhanwei Hou, Yonghui Li, Wibowo
Hardjawana, Chenyang Yang, Lingyang Song, and Branka Vucetic. 2020. Deep
learning for ultra-reliable and low-latency communications in 6G networks. IEEE
Network 34, 5 (2020), 219–225.
Madyan Alsenwi, Nguyen H Tran, Mehdi Bennis, Shashi Raj Pandey, Anupam Kumar Bairagi, and Choong Seon Hong. 2021. Intelligent resource slicing for eMBB
and URLLC coexistence in 5G and beyond: A deep reinforcement learning based
approach. IEEE Transactions on Wireless Communications 20, 7 (2021), 4585–4600.
Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and
Bingsheng He. 2023. A survey on federated learning systems: vision, hype and
WWW ’23 Companion, April 30-May 4, 2023, Austin, TX, USA
[53]
[54]
[55]
[56]
[57]
[58]
[59]
[60]
[61]
[62]
[63]
[64]
[65]
[66]
[67]
[68]
[69]
[70]
[71]
[72]
[73]
[74]
[75]
reality for data privacy and protection. IEEE Transactions on Knowledge and Data
Engineering 35, 4 (2023), 3347–3366.
Zhaohui Yang, Mingzhe Chen, Kai-Kit Wong, H Vincent Poor, and Shuguang
Cui. 2022. Federated learning for 6G: Applications, challenges, and opportunities.
Engineering 8 (2022), 33–41.
Zhijin Qin, Geoffrey Ye Li, and Hao Ye. 2021. Federated learning and wireless
communications. IEEE Wireless Communications 28, 5 (2021), 134–140.
Sumudu Samarakoon, Mehdi Bennis, Walid Saad, and Mérouane Debbah. 2019.
Distributed federated learning for ultra-reliable low-latency vehicular communications. IEEE Transactions on Communications 68, 2 (2019), 1146–1159.
Anudit Nagar. 2019. Privacy-preserving blockchain based federated learning
with differential data sharing. arXiv:1912.04859 (2019).
Dinh C Nguyen, Ming Ding, Pubudu N Pathirana, Aruna Seneviratne, Jun Li, and
H Vincent Poor. 2021. Federated learning for internet of things: A comprehensive
survey. IEEE Communications Surveys & Tutorials 23, 3 (2021), 1622–1658.
Shenglai Zeng, Zonghang Li, Hongfang Yu, Zhihao Zhang, Long Luo, Bo Li, and
Dusit Niyato. 2022. HFedMS: Heterogeneous federated learning with memorable
data semantics in industrial metaverse. arXiv preprint, arXiv:2211.03300 (2022).
Xinyu Zhou, Chang Liu, and Jun Zhao. 2022. Resource allocation of federated learning for the metaverse with mobile augmented reality. arXiv preprint,
arXiv:2211.08705 (2022).
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. 2022. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning
Systems. DOI: 10.1109/TNNLS.2022.3160699 (2022), 1–17.
Qiong Wu, Kaiwen He, and Xu Chen. 2020. Personalized federated learning for
intelligent IoT applications: A cloud-edge based framework. IEEE Open Journal
of the Computer Society 1 (2020), 35–44.
Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge
computing: Vision and challenges. IEEE Internet of Things Journal 3, 5 (2016),
637–646.
Tharam Dillon, Chen Wu, and Elizabeth Chang. 2010. Cloud computing: issues
and challenges. In 24th IEEE International Conference on Advanced Information
Networking and Applications. IEEE, 27–33.
He Li, Kaoru Ota, and Mianxiong Dong. 2018. Learning IoT in edge: Deep
learning for the Internet of Things with edge computing. IEEE Network 32, 1
(2018), 96–101.
Wei Yang Bryan Lim, Nguyen Cong Luong, Dinh Thai Hoang, Yutao Jiao, YingChang Liang, Qiang Yang, Dusit Niyato, and Chunyan Miao. 2020. Federated
learning in mobile edge networks: A comprehensive survey. IEEE Communications
Surveys & Tutorials 22, 3 (2020), 2031–2063.
Yichen Wan, Youyang Qu, Longxiang Gao, and Yong Xiang. 2022. Privacypreserving blockchain-enabled federated learning for B5G-Driven edge computing. Computer Networks 204 (2022), 108671.
Zhilin Wang and Qin Hu. 2021. Blockchain-based federated learning: A comprehensive survey. arXiv:2110.02182 (2021).
Thar Baker, Muhammad Asim, Hezekiah Samwini, Nauman Shamim, Mohammed M Alani, and Rajkumar Buyya. 2022. A blockchain-based Fog-oriented
lightweight framework for smart public vehicular transportation systems. Computer Networks 203 (2022), 108676.
Devrim Unal, Mohammad Hammoudeh, Muhammad Asif Khan, Abdelrahman
Abuarqoub, Gregory Epiphaniou, and Ridha Hamila. 2021. Integration of federated machine learning and blockchain for the provision of secure big data
analytics for Internet of Things. Computers & Security 109 (2021), 102393.
Xinyu Zhou and Jun Zhao. 2022. Mobile augmented reality with federated
learning in the metaverse. arXiv:2212.08324 (2022).
Ahmed Imteaj, Urmish Thakker, Shiqiang Wang, Jian Li, and M Hadi Amini.
2021. A survey on federated learning for resource-constrained IoT devices. IEEE
Internet of Things Journal 9, 1 (2021), 1–24.
Fakhreddine Karray, Milad Alemzadeh, Jamil Abou Saleh, and Mo Nours Arab.
2008. Human-computer interaction: Overview on state of the art. International
Journal on Smart Sensing and Intelligent Systems 1, 1 (2008), 137–159.
Zhaohui Wei, Qingqi Pei, Ning Zhang, Xuefeng Liu, Celimuge Wu, and Amirhosein Taherkordi. 2023. Lightweight federated learning for large-scale IoT devices
with privacy guarantee. IEEE Internet of Things Journal 10, 4 (2023), 3179–3191.
Honglei Zhang, Fangyuan Luo, Jun Wu, Xiangnan He, and Yidong Li. 2022.
LightFR: Lightweight federated recommendation with privacy-preserving matrix
factorization. ACM Transactions on Information Systems (2022), 1–28.
Xiaojin Zhang, Hanlin Gu, Lixin Fan, Kai Chen, and Qiang Yang. 2022. No free
lunch theorem for security and utility in federated learning. ACM Transactions
on Intelligent Systems and Technology 14, 1 (2022), 1–35.
Chen et al.
A
PICTURE APPENDIX
D
Big
ata
Co
Te mmu
c hn ni
olo c a ti
gy on
Extended
Reality
Blo
IoT
c kc
ha i
n
ge g
Ed ut i n
mp
o
C
Figure A1: Six key techniques in FL4M.
Metaverse
FL server
Data
Avatar
Real world
Extended Reality
User
Figure A2: Illustration of extended reality of FL4M.