See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/348262202
On the Potential of Rocket League for Driving Team AI Development
Conference Paper · December 2020
DOI: 10.1109/SSCI47803.2020.9308248
CITATIONS
READS
2
413
2 authors:
Yannick Verhoeven
Mike Preuss
University of Münster
Leiden University
3 PUBLICATIONS 5 CITATIONS
266 PUBLICATIONS 5,585 CITATIONS
SEE PROFILE
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
General Intelligence on Games View project
PropStop: Identification, detection, and combating of hidden propaganda attacks via online media View project
All content following this page was uploaded by Mike Preuss on 11 January 2022.
The user has requested enhancement of the downloaded file.
On the Potential of Rocket League
for Driving Team AI Development
Yannick Verhoeven
Mike Preuss
Department of Information Systems
Westfälische Wilhelms-Universität Münster, Germany
yannick.verhoeven@uni-muenster.de
Leiden Institute of Advanced Computer Science
Universiteit Leiden, The Netherlands
m.preuss@liacs.leidenuniv.nl
Abstract—Several Games have been used as test cases for new
AI developments in the last years, e.g. the games of the Atari
Learning Environment, Go, and also StarCraft. As many single
player problems are treated successfully now, research is turning
more to Team AI problems: AI/human and AI/AI coordinated
decision making in mixed cooperative/competitive environments.
OpenAI Five and RoboCup and related environments as MuJoCo
soccer are pushing into this direction. Rocket League is an
interesting alternative as it has quite different game mechanics
compared to soccer, and an API with a number of different bots
already exists. We summarize the current state, relate it to the
situation in the fields that may provide useful approaches as e.g.
sports analysis, and argue that it makes sense to do more research
on Rocket League bot control and coordination.
Index Terms—Rocket League, multi-agent control, Team AI
as its e-sport variants. However, there are certain aspects that
also limit agent coordination research, e.g., it features a strong
positional system. We therefore argue that Rocket League is
a good addition to soccer for research because of its inherent
differences:
•
•
•
I. I NTRODUCTION
Computer games have played a very important role in recent
AI research by providing relatively simple but well structured
and scalable decision problems [1]. After Go [2], even the
much more complicated realtime strategy game StarCraft [3]
has been mastered successfully by means of specifically created AIs. The next complexity level for artificial agents is
arguably multi-agent coordination, or more general, team AI,
including agent/agent as well as human/agent collaboration.
This is not to say that there is no research in that direction
yet. Next to an unmanageable size of publications on multiagent systems that have been produced over the last decades,
there are at least three notable developments that are related
to team AI in games:
• Robocup features robots playing soccer, in hardware or
in simulation only. It exists since 1995 and has been
extended to a lot of leagues and tackled a lot of scientific
challenges, especially in the area of agent coordination [4].
• OpenAI Five [5] is an approach that is able to control
5 individual Dota2 players on human competitive level.
It is related to the emergent tool use work of [6] that is
much more general. Both use long-term, very resource
intense deep learning.
• MuJoCo soccer [7] is a multi-agent simulation based
framework for team coordination research.
Soccer is definitively a good test setting for many types of
team AI problems, and it has the advantage that it can draw
large public interest from its popularity as a real game as well
•
In Rocket League, player roles are not fixed, instead they
are rotated steadily, such that every player is changing
from offense to defense and back regularly.
Orientation towards team members is important at all
time, uncoordinated movement often provides scoring
opportunities to the opponent team.
Conservation of momentum is a crucial aspect of competitive play, which forbids idling on a position as the
default state.
Movement patterns for ball and players are more complex
than e.g. in soccer as they rely much more on the 3dimensional space. The ball is rarely moving flat on the
ground.
The requirement that every player/bot has to be able to play
a bit of every role means that there no strong specialization,
as opposed to soccer. Together with the low number of players
and the relatively simple interface this leads to a comparably
low entry barrier. We shall later on look at the currently
available bots in order to see how good they are. However,
we can already state now that they are not very good at
coordinated play, meaning that the entry barrier for bots and
bot teams is also rather low.
Next to full bot teams, Rocket League also poses an
interesting test case for human/AI mixed team coordination.
Whereas humans tend to intuitively extrapolate the behavior
of their team mates in order to anticipate movements and
tactics, this is not so easy for bots. However, these can
easily scan the whole playing field and know all positions
of teammates and opponents, even if these are placed behind
them. Humans cannot do this without turning their camera due
to field of view limitations . Of course, there is a potential for
collaboration here as the abilities of humans and bots are partly
complementary.
As for other e-sports, the extensive research on team behavior in real sports due to the growing availability of spatiotemporal data [8] can only benefit team behavior analysis in
video games. For Rocket League, such data is available as
well, and methods shall be at least partially transferrable.
Another important aspect that may be studied utilizing
Rocket League as a test case is ad-hoc teamwork between
bots and in mixed human/bot teams with and without communication. Rocket League teams are ad-hoc by default and
draw on predetermined vocabularies for communication, such
that human-agent cooperation is already possible. However,
we are not aware that it is already explicitly used.
Summarizing, Rocket League is a readily available candidate for team AI research that could clearly enrich the current
zoo of utilized games with some unique features.
In the following, we first explain the game mechanics of
Rocket League (Sect. II) before we assess the potential of
spatio-temporal data analysis in Sect. III. Sect. IV deals with
the intricacies of the tactical system in Rocket League that are
somewhat different from many team sports, and in Sect. V
we investigate team communication in Rocket League. We
continue with reporting on a tournament with Rocket League
bots we have conducted in Sect. VI before specifying different
ways for assessing team behavior in Sect. VII, followed by
conclusion and outlook. More detailed results can be looked
up in the Master’s thesis this work is based on [9].
II. ROCKET L EAGUE
Rocket League is “a high-powered hybrid of arcade-style
soccer and vehicular mayhem with easy-to-understand controls
and fluid, physics-driven competition” 1 . The aim of the game
is simple: To shoot the ball into the opposing team’s goal.
Towards that end, players control their cars and try to make
contact with the ball, transferring part of their momentum onto
the ball according to rules of physics dictated by the game
engine. The field is bounded by walls and a ceiling, similar to
cage soccer. Consequently, the ball cannot go out of bounds.
Furthermore, there are no rules, and therefore no fouls, in
Rocket League, meaning that play is never interrupted unless
a goal is scored. Figure Fig. 1 is a screenshot, taken from a
3v3 bot game, that shows how cars, the ball, the goal, and
the walls look in an actual game. Fig. 3 As can be seen from
the screenshot, cars are able to achieve lift in Rocket League,
adding a third dimension to the potential movement options.
This is done by jumping and/or utilizing boost (a limited
resource collected from pads on the field), which grants thrust
acceleration. When pointed downwards, this can sustain flight
for a limited duration. The features of games discussed by the
group working on the question “Which games should we (AI)
explore next?” during an NII Shonan meeting on game AI in
2019 [10, p.17] are used for the characterization of the game.
These are formulated from the perspective of how an artificial
intelligence can play the game. Rocket league is a multiagent game with a team structure of either 1v1, 2v2, or 3v3.
The team structure discussed in this thesis is predominately
3v3. It is a mechanically challenging game based on physics,
and bots work by relying on position data for all players
and the ball provided at each time cycle. As is typical
Fig. 1: Rocket League Bot Gameplay. Two teams of 3 cars
each try to put the ball into the goal of the opposite team.
for team sports games, the environment is continuous, and
all agents can act simultaneously and independently. Rocket
League is characterized by sparse rewards and a long timehorizon, because goals are scored infrequently during play. It
requires several cognitive capabilities, including memorizing
the spatio-temporal information gained through perception of
the environment, communicating this information for a shared
team awareness, planning based on this awareness, and being
attentive to actions of the opposing players [10, p.19].
There have been some successful machine learning/AI applications in Rocket League, but these have been reported in
informal literature. We list some highlights in the following:
Harry Xie [12], a member of the Rocket Bot community,
successfully trained an agent to air dribble (a maneuver that
keeps the ball balanced on top of the car by continuously
adjusting the car position to stay under the ball’s center
of gravity) in place with a randomly generated initial state
with fixed car and ball positions and randomized yaw and
roll parameters. Using deep deterministic policy gradients,
the maximum carry duration reached was roughly 30s, with
averages around the 7s mark. This is already longer than
the overwhelming majority of players can manage. With twin
delayed deep deterministic policy gradients (TD3), evaluation
episodes averaged two minutes, with some episode being far
longer. A clip of a very successful episode using TD3 that is
competitive with the best human air dribblers, who dedicate
a lot of time to this skill and are far better than professional
players in that regard, is available online.2
The same author also made progress on even more challenging shots using ghost controls of a human completing the task
for denser rewards. Also, a Rocket League positioning neural
network was created and trained on professional replays3 .
It predicts the likely position of the sixth car based on the
positions of the other five cars and the ball. This is currently
not useful for positioning of bots, because the situations arising
in the professional replays are not replicated in bot games.
However, it could be valuable for the assessment of adherence
2 https://www.twitch.tv/videos/479182384
1 from
Rocket League® - official site: https://www.rocketleague.com/
3 https://www.youtube.com/watch?v=-VE4iYkmd8g
Fig. 2: Movement Model Creation Steps (left to right), standardization, example data set, resulting movement model after
smoothing with kernel density estimation, Brefeld et al. 2019, p.132 [11]
to the tactical system in the future, when bots are closer to the
team formations exhibited by the players the neural network
is trained on.
III. S PATIO -T EMPORAL DATA A NALYSIS
Team sports games, such as soccer, basketball, ice hockey
or handball have been analyzed by coaches, sports analysts,
television commentators, and viewers with differing degrees of
precision for decades [13]. However, detailed game analyses
through observational approaches are highly time consuming
and, due to the highly dynamic and continuously changing
situations, the reliability of such analysis is debatable, creating demand for quantitative approaches [14]. Lately, the
availability of reliable, automatically generated data, such as
positions of the ball and the players for each time frame,
has motivated intensive research into such data-driven sports
game analysis [8]. In contrast to traditional analysis, the
automated analysis of spatio-temporal data yields objective
and comparable results, provided the difficulty of associating
the continuous nature of this data with events [15] is managed
appropriately. The two major strands of research attempting
to leverage this data are the analysis of spatial arrangements
of teams, and the use and control of space by teams [8].
Even though this research is based mostly in real-life sports,
several concepts can be adapted to e-sports, and specifically
Rocket League. For example, inferring information about the
opposing teams strategy is possible by analysing ball movements during each teams possession periods [15]. Adapting the
automated determination of team formations by Bialkowski
et al. [16] - which is based on player positions relative
to static space defined through iterative minimum entropy
data positioning - to dynamic roles based on ball position
and possession can show agents’ tendencies and differences
in tactical formations between teams. Both approaches have
potential for opponent modelling and self-improvement by
quantifying the bots’ team behavior. Movement models describe the positions players can reach on the field in a given
time based on their current position, orientation and velocity.
First introduced by Taki et al. [17], and improved in [18],
movement models are an important part of sports analysis
and were since revisited and further refined [19], [20]. The
growing availability of precise and continuous position data
through tracking techniques enabled Brefeld et al. [11] to
create a method for the automated generation of probabilistic
individual movement models (Fig. 2). The method represents
the positional data points as triplets (ps , pt , pu ) including a
starting position, present position, and future position. This
data is first standardized through translation and rotation so
that the movement vector ps pt is in alignment with the x-axis
and pt is in the origin. The point pu then describes the point
reached by the player from the origin given the initial velocity
and orientation defined through the vector ps pt in the triplet.
By calculating the distance from the origin to pu and the
angle between ps pt and pu for all available data triples for the
player, the ability of the player to move in different directions
with a given initial speed is described. Though individual
differences in physical performance are irrelevant in virtual
games because the maximum acceleration, velocity, and turn
radius are constants defined by the game engine, differences
in performance of the controlling agent, human or artificial,
lead to different movement models. Therefore, adapting the
method of Brefeld et al. [11] to Rocket League can provide
insights into individual agent performance and reveal zones
of control. The zone of control is the zone on the field that
an agent can reach before any other agent [18] and is crucial
for passing algorithms [21], while the sum of agent’s zones of
control is an important measurement for effective use of space
and an indicator for team behavior. Due to Rocket Leagues
allowing for movement in three dimensions the results would
show an ellipsoid of control rather than a zone, which becomes
increasingly more spread out for higher initial velocities.
The analysis, interpretation, and abstracted description of
one or multiple adversary’s actions with the goal of formulating a counter-strategy is called opponent modelling [22]–
[24]. Opponent models can be created during competition
or based on historical data for pre-game preparation of a
targeted strategy. Though humans tend to create opponent
models subconsciously, and analyzing soccer teams for apriori tactical decisions by coaches is an established practice,
modelling three-dimensional autonomous multi-agent environ-
ments is a relatively new and challenging concept [14], [22],
[23]. The first problem in more complex three-dimensional
environments, such as the RoboCup 3D Simulation Leagues or
Rocket League, is the unavailability of annotations for actions
(e.g. pass, shot, dribble). Following the research of Iglesias
et al. [22], targeted towards the RoboCup Coach Competition
held in 2006, sequence tries for Rocket League can be created.
The added complexity stemming from Rocket Leagues threedimensionality can be managed by introducing ranges for
the height of players during atomic actions such as shots,
differentiating ground, jump, and aerial shots. The resulting
tries can be used for opponent modelling and the formulation
of counter strategies. Abusing individual movement models
is a valid strategy especially on lower levels and can be as
simple as tracking the average height of the opponent’s car,
which, when low, may indicate vulnerabilities to aerial play.
This potential vulnerability can then be exploited by relying
more on chips and high passes. If the average height does not
rise in response, or goals are conceded, the vulnerability is
confirmed.
IV. ROCKET L EAGUE TACTICAL S YSTEM
A problem for the creation of a computational team behavior
model for use with bots in a specific domain, such as RoboCup
or Rocket League, is that an accurate model requires the collaboration of domain experts, analysts and software developers
[25]. These groups rely on different ways of presentation,
ranging from natural language to formal modelling languages.
For Rocket League team behavior, inputs from sports analysis
research, RoboCup research, and domain knowledge of players
are relevant. The domain knowledge that all players implicitly
possess to some degree has to be expatiated and aggregated to
become accessible, which has been done by content creators
over the last years. By combining these inputs, a tactical
system for Rocket League, which can form the basis for bot
behavior modelling was created. This system is applicable
for the standard format of competitive play, 3v3, and relies
on continuous reassignment of roles based on rotation. The
roles are not defined through static space on the field, but
dynamically in relation to the ball position.
The roles, numbered from 1 to 3, are the attacker, receiver
and defender respectively as shown in Fig. 3. The attacking
role, determined by lowest travel time to the ball combined
with proper orientation, is tasked with challenging, dribbling,
shooting or passing the ball. The receiver, usually comparably
far up the field but on the different side to the attacker, exerts
pressure by threatening a passing play via the passing lanes
(yellow). Lastly, the defender stays behind and between the
two attacking players and acts as backup. After a shot or
pass, role 1 transitions to role 3 by rotating behind player
3 while collecting boost on the field. Role 2 becomes the
new attacker, provided he is able to reach the ball coming
from role 1, and role 3 becomes the new role 2 by moving
up the field. Adhering to this tactical system, characterized
by continuous reassessment of the game state and dynamic
assignment of roles, allows for all players to retain their
Fig. 3: Rotation scheme (roles) in Rocket League with role
numbers. Each number represents a car, the grey circle
represents the ball. Orange dots represent boosts, red lines
are meaningful move alternatives for all roles, yellow lines
represent possible passes.
momentum while maintaining adequate spacing relative to
their teammates. This in turn maximized the team’s zone of
control while simultaneously ensuring good resource (boost)
management.
V. ROCKET L EAGUE C OMMUNICATION
Communication is defined as the process by which information is clearly and accurately exchanged between individuals in
a prescribed manner through use of proper terminology [26].
It is a fundamental part in scenarios focused on collaboration
and teamwork and should be as natural and intuitive as
possible [27]. In team games such as soccer or Rocket League,
players create their own spatial mental models which are
representations of their environment [28]. These models, with
weights given to features pertaining to current goals, form
the basis of human decision making in complex environments
[28], [29]. Concepts such as zones of control, pressure or team
formations are essentially formally expatiated representations
of the mental models that players build for their respective
needs, like identifying potential receivers for a pass. Because
individuals base their decision making on their spatial mental
models, successful operation as a team requires a shared model
[30]. Without a shared understanding of positions, responsibilities and plans, each individual makes decisions based
only on his or her interpretation of the information available,
making good team behavior difficult. A shared understanding
can be achieved through regular communication between team
members with the purpose of aggregating each individual’s
situation awareness and plans [30], [31]. Therefore, good
communication is a major driver of team performance. The
difficulty lies in finding the right channel, recipients, content,
volume, and time for communication. Whereas a lack of
communication leads to mistakes in decision making due to
differing individual mental models [27], over-communicating
carries high coordination costs, meaning that the overall taskrelated performance could be worse due to excessive time
spend on communication [31] Three general types of communication, human to human, human to bot, and bot to bot
communication are interesting for team behavior in games.
•
•
•
Human to human communication can serve as the foundation by giving insights into which players want to know
what, how much information they want, and when they
need that information (recipients, content, volume, time).
Human to bot communication, often called human-agent
communication, is the most complex due to the different
needs, strength and weaknesses of humans and machines.
It is a hot topic in current research for diverse humanagent teaming applications [27]–[33].
Lastly, bot to bot communication can become a driver for
team performance in premade and ad-hoc bot teams and
therefore advance team behavior of artificial intelligence
bots in games.
For the analysis of human communication in Rocket League,
we investigated the voice communication of professional players during games. Through transcription and color coding
based on the message content and intent, five communication categories were identified. Frequency analysis showed
that information pertaining to the sending players situation
(e.g. position, boost-level) and intention (e.g. going for the
ball, falling back) were communicated most frequently with
more than half of the messages falling in either category,
followed by information about opposing players situation or
anticipated intention. Estimations about future game states,
tactical directives, confirmation and feedback were present,
but less frequent (< 5% of total messages). Since ad-hoc
teams usually do not utilize voice communication, quick chat
commands, which form a domain specific fixed-vocabulary
system, can be used to communicate information. Mapping
the categories identified in the voice communication analysis
to the available quick chats showed that almost everything
can be communicated, albeit with reduced precision. These
quick chats are thus usable for human-bot communication.
The communication volume in pure bot games can exceed
what is possible in human communication because the cost
for communication is low and information processing speed
is much higher [30].
Bot communication within the current framework can be
done via sending and receiving quick chat messages4 or via
Matchcomms5 . The quick chat is comparable to the limited
standard communication protocol for the RoboCup drop-in
league, where robots are randomly put into ad-hoc teams and
do not know the identities of their teammates [34].
4 see
5 see
https://github.com/RLBot/RLBotPythonExample/wiki/Quickchat
https://github.com/RLBot/RLBot/wiki/Matchcomms
VI. B OT T OURNAMENT
In order to assess the current level of Rocket League bots
in terms of mechanical skill and team behavior, the 21 bots
developed and made available in the framework by the Rocket
Bot community were first put against each other to determine
the fittest bots. The match settings were set to the default
state used in competitive play. Because the client currently
does not support simultaneous matches, and tests showed
that speeding the game up resulted in performance drops for
some of the bots (measured by packet loss), the game could
not be sped up beyond real time. Consequently, quantitative
testing with large sample sizes, which would lead to more
robust results, was not feasible. Instead, the validity of the
results was increased by using a competition format that yields
good placement results for relatively small samples for the
team competitions [35]. This competitive format consists of
a round-robin with one game played for each pair of teams
and can include placement matches between the pairs directly
adjacent on the resulting leader-board. For the initial screening
and preliminary placement of the bots, each custom bot was
made to play three matches against the official bot used in
the client by Psyonix, set to the highest skill level (Allstar).
This approach has two advantages. First, it ensures a fair
comparison for the initial test, as each bot plays the same
opponent. Second, it provides a good first insight into the level
of the bot, because the in-game bot is familiar to most players
and forms a good baseline. During this process, 3 bots were
excluded from further consideration due to malfunctioning
or cheating by ignoring physics rules. For the next stage,
three-player teams made up of equal bots were formed and
set against each other in the round-robin format suggested
by Budden et al. [35]. Using the placement results from the
three versus three equal team tournament, the top five bots
were combined into mixed three player teams, leading to ten
teams which were again set against each other in a round-robin
tournament. Finally, the top performing equal bot teams were
set against the top performing mixed bot teams. The results
show that mixed teams are slightly superior to equal teams
with current bots. Presumably, this is due to less overlap in
the team’s zone of control and fewer double commits (two
teammates challenging for the ball at the same time) for bots
that have different decision-making processes. Especially for
bots that appear to have little or no consideration for their
teammates, a team of three bots that make decisions based
only on their own position and the ball position would be
expected to perform worse in equal teams when compared
to mixed teams. Arguably, a team with three players who all
make egoistical decisions is better off if those three players
are as different as possible. Even in teams that have bots with
team-behavior, having bots with different decision making can
improve the team. For example, an equal bot team with only
defensive bots is unlikely to beat a mixed team with two
defensive and one offensive bot.
VII. A SSESSMENT OF T EAM B EHAVIOR
Team performance is relatively easy to assess by looking
at match results and using simple metrics such as win rate
and goal difference. Assessing team behavior is much more
difficult, because it requires determining how much of the
team performance is actually due to team behavior and how
much is just the individual skill of the players. In RoboCup,
MacAlpine and Stone [34] found a high correlation between
the performance of a team in the regular competition and
in drop-in leagues, which could also be observed in the bot
tournament. This suggest that the high win rate and goal
difference of teams with skilled bots is - to a non-trivial
extend - due to its individual mechanical skill and not its
team behavior. There are several potential ways to assess team
behavior.
First, based on observation. The straightforward way is to
observe replays and judge team behavior manually, which is
time consuming and subjective [14]. Alternatively, to observe
replays and judge team behavior based on predefined attributes
or conditions and utilizing checklists and counting occurrences
is still time consuming but more objective. The next step could
be to automate this process.
Second, to find metrics that can be extracted from replays
and assess team behavior. Promising metrics for team behavior
in Rocket League are assisted goal percentage, total boost
acquired by the team and demolitions (short-term disablement
of opposing cars via crashes), among others. These were
identified by comparing the relative performances of players
across two different 3v3 playlists, the 3v3 standard and the 3v3
solo standard leagues. 3v3 standard allows for teams of up to
three players to play together, while solo standard is only open
to individuals. Therefore, all teams in solo standard are ad-hoc
teams, while the standard playlist is usually played by premade
teams of three. It stands to reason that the team behavior in
a premade team, which is also more likely to utilize voice
communication, is better than the team behavior of ad-hoc
teams restricted to quick chat communication. This should be
reflected in the data, meaning that metrics that show distinct
differences for the two leagues are likely metrics linked to
team behavior. Another advantage gained from the availability
of human replays is that the performance metrics of bots can
be put into perspective by comparing them to the performance
of human players of differing skill levels. The data for the bots
was generated and averaged over the several stages of roundrobin competition, while the human player data was extracted
from replays available online 6 . The data is based on roughly
80 hours of competitive play in 3v3 solo standard and 100
hours of competitive play in 3v3 standard across the different
skill groups, assuming an average match length of 7min.
The performance of bots compared to human players, averaged over hundreds of games for different 3v3 playlists and the
several competitive matchmaking ranks, are shown in Fig. 4
for the metric assisted goal percentage. The best three equal
bot and the best three mixed bot teams, determined by win
6 see
ballchasing.com
Fig. 4: Assisted goal percentage. Solo Standard and Standard
values are based on human player replays across the 6 skill
groups, SDC, Relief and Botimus are the best equal bot teams,
Teams 2, 4, 8 are the best mixed bot teams based on the bot
tournament
rate and goal difference through round robin tournaments, are
chosen here. Compared to humans, the equal bot teams showed
values for assisted goal percentage that are slightly below the
lowest skill level (bronze) on average, while two of the most
successful mixed teams approach the performance of players
at the silver or gold ranks. This is consistent with the observed
trend that mixed teams exhibit superior team behavior.
Lastly, the code the bots run on can be analyzed, if available, as it will clearly show whether the bot considers his
teammates, and in which way. By comparing this with domain
knowledge on tactical formation and team behavior, statements
regarding the bot’s fitness for teamwork are possible. Investigating the top-performing bots, for which the code was openly
available, showed, that rudimentary team behavior, in the form
of adherence to the rotational system, is implemented. More
advanced algorithms, for example for passing, are missing,
but are believed to hold the potential to drastically increase
competitiveness when implemented.
VIII. C ONCLUSION / O UTLOOK
Several game analysis concepts from real life sports can be
adapted to the Rocket League domain and used in combination
with domain knowledge for the formulation of a well-founded
team-oriented tactical system for Rocket League. This tactical
system forms the basis for the current and future development
and assessment of artificial intelligence team behavior in
Rocket League. It can be used as input for the development of
better behavior trees for coded game playing agents or aid in
the formulation of shaped reward functions for reinforcement
learning techniques. Additionally, the team behavior of human
or artificial players can be assessed based on their adherence
to the tactical system, through manual observation along
predefined criteria. However, additional research is required to
develop and implement systems that can automatically assess
the team behavior of an individual agent or a team. This
could include the use of entropy maps for the assessment
of passing variability, or an adaption of the automatic team
formation identification by Bialkowski et al. [16] to dynamic
zones defined relative to the ball position. Especially the ability
to automatically infer the current role of a bot from spatial data
could greatly enhance team behavior assessment, by giving
better insights into bot decision making processes.
The analysis of professional player communication logs
yields interesting insights into the content, volume, and timing
of messages on the highest level of competition. These insights
are valuable for the eventual design of human-agent communication in Rocket League and can also be used to derive
information needs in bot games. The assessment of existing
bots, through observation, different metrics, and code analysis
reveals that even comparatively well-performing bots do not
show sophisticated team behavior. Most notably, intentional
passing behavior, which is one of the most basic forms of
teamwork in any ball game, is not observable in existing
bots. This is evident from the relatively low assisted goal
percentage, the lack of passing behaviors in the analyzed code,
and the under-representation of role 2 (pass receiver) during
experimental analysis. Furthermore, no bot was observed to
use communication for its decision-making process. Though
some bots show promise, especially in defensive behavior,
boost collection behavior, and demolition tactics, the team
behavior of current bots can only be described as lacking.
On account of this, the development of a bot that fixes
these shortcomings is the logical next step for Rocket League
team behavior research. The development can draw on existing
bot’s code for physical calculations, and the Rocket Bot
framework supports python, java, .NET, Scratch, and Rust,
thus allowing researchers to get started with relative ease.
In this line of research, the development of a skills, tactics,
and plays architecture for distributed multi-agent teams is
an interesting prospect. Optionally, skills could be learned
using reinforcement learning, which was successfully done
in Rocket League for the comparatively difficult mechanical
skill air-dribbling regardless of current limitations. Due to
Rocket Leagues advantage of accommodating both artificial
and human players, the comparison of bots to human players
based on observation could show interesting results. In that
direction, a Rocket League Turing test could be administered
by letting humans guess which of the players on the field is
a bot. Another interesting experiment could be to let humans
guess the rank of an agent without telling them that the agent
is a bot and comparing the guessed ranks to the results gained
from ranking the bot based on quantitative metrics gained from
replays.
Although Rocket League has some inhibitors for machine
learning, most notably that the game cannot be sped up
significantly, it is in our view more suited for human-agent
teaming and communication research than RoboCup. The
availability of human replay data and the possibility to test
bots with and against humans, while still retaining the ability
to draw on sports analysis, work on RoboCup, and general
artificial intelligence research, is unique to Rocket League.
Furthermore, even the existing bots, which are almost completely lacking team behavior and mechanically behave around
the level of an average player, are much closer to beating the
RLCS champions than RoboCup teams are to beating the FIFA
champions, which is their long-term goal. Rocket League bot
developers have the advantage of being able to test their bots
against humans early and repeatedly. Insights gained from such
tests are meaningful for bot development, because humans
tend to be very good at finding and exploiting weaknesses,
thus uncovering potential for improvement. Due to the ability
to draw on differing fields of research, the low entry barriers
for bot development, and the game’s inherent advantage of
providing an even playing field for human players and bots in
ad-hoc teams, Rocket League is an interesting game for future
research and has great potential for driving all aspects of team
AI development.
R EFERENCES
[1] S. Risi and M. Preuss, “From chess and atari to starcraft and
beyond: How game AI is driving the world of AI,” Künstliche
Intell., vol. 34, no. 1, pp. 7–17, 2020. [Online]. Available:
https://doi.org/10.1007/s13218-020-00647-w
[2] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van
Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam,
M. Lanctot et al., “Mastering the game of go with deep neural networks
and tree search,” nature, vol. 529, no. 7587, p. 484, 2016.
[3] O. Vinyals, I. Babuschkin, W. Czarnecki, M. Mathieu, A. Dudzik,
J. Chung, D. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan,
M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. Agapiou,
M. Jaderberg, and D. Silver, “Grandmaster level in starcraft ii using
multi-agent reinforcement learning,” Nature, vol. 575, 11 2019.
[4] M. Asada and O. von Stryk, “Scientific and technological challenges
in robocup,” Annual Review of Control, Robotics, and Autonomous
Systems, vol. 3, no. 1, pp. 441–471, 2020. [Online]. Available:
https://doi.org/10.1146/annurev-control-100719-064806
[5] J. Raiman, S. Zhang, and F. Wolski, “Long-term planning and situational
awareness in openai five,” 2019.
[6] B. Baker, I. Kanitscheider, T. Markov, Y. Wu, G. Powell, B. McGrew,
and I. Mordatch, “Emergent tool use from multi-agent autocurricula,”
2019.
[7] S. Liu, G. Lever, J. Merel, S. Tunyasuvunakool, N. Heess, and
T. Graepel, “Emergent coordination through competition,” in 7th
International Conference on Learning Representations, ICLR 2019,
New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
[Online]. Available: https://openreview.net/forum?id=BkG8sjR5Km
[8] G. Andrienko, N. Andrienko, G. Budziak, J. Dykes, G. Fuchs,
T. von Landesberger, and H. Weber, “Visual analysis of pressure in
football,” Data Mining and Knowledge Discovery, vol. 31, no. 6, pp.
1793–1839, nov 2017. [Online]. Available: http://link.springer.com/10.
1007/s10618-017-0513-2
[9] Y. Verhoeven, “Team Behavior of Artificial Intelligence Bots in Games,”
Master’s thesis, Rijksuniversiteit Groningen, Westfälische WilhelmsUniversität, Münster, Germany, 2020.
[10] R. Thawonmas, J. Togelius, and G. N. Yannakakis, “Artificial
General Intelligence in Games : Where Play Meets Design and
User Experience,” in NII Shonan Meeting, no. 130, 2019. [Online].
Available: https://shonan.nii.ac.jp/seminars/130/
[11] U. Brefeld, J. Lasek, and S. Mair, “Probabilistic movement models
and zones of control,” Machine Learning, vol. 108, no. 1, pp.
127–147, jan 2019. [Online]. Available: http://link.springer.com/10.
1007/s10994-018-5725-1
[12] H. Xie, “Achieving Human-Level Results in a Physics-Based Game
Environment Using Reinforcement Learning in Rocket League,” vol. 0,
pp. 1–10, 2019.
[13] M. Clements and D. Polani, “Empowerment as a generic utility
function for agents in a simple team sport simulation,” in Lecture
Notes in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics), vol.
10459 LNAI. Springer, Cham, 2017, pp. 37–49. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-66471-2 5
[14] R. Rein and D. Memmert, “Big data and tactical analysis in elite
soccer: future challenges and opportunities for sports science,” p.
1410, dec 2016. [Online]. Available: http://springerplus.springeropen.
com/articles/10.1186/s40064-016-3108-2
[15] P. Lucey, D. Oliver, P. Carr, J. Roth, and I. Matthews, “Assessing team
strategy using spatiotemporal data,” in Proceedings of the ACM SIGKDD
International Conference on Knowledge Discovery and Data Mining,
vol. Part F1288. Association for Computing Machinery, aug 2013, pp.
1366–1374.
[16] A. Bialkowski, P. Lucey, P. Carr, Y. Yue, S. Sridharan, and I. Matthews,
“Large-Scale Analysis of Soccer Matches Using Spatiotemporal Tracking Data,” in Proceedings - IEEE International Conference on Data
Mining, ICDM, vol. 2015-Janua, no. January. Institute of Electrical
and Electronics Engineers Inc., jan 2014, pp. 725–730.
[17] T. Taki, J. ichi Hasegawa, and T. Fukumura, “Development of motion
analysis system for quantitative evaluation of teamwork in soccer
games,” in IEEE International Conference on Image Processing, vol. 3.
IEEE, 1996, pp. 815–818.
[18] T. Taki and J. ichi Hasegawa, “Visualization of dominant region in
team games and its application to teamwork analysis,” in Proceedings
of Computer Graphics International Conference, CGI. IEEE, 2000, pp.
227–235.
[19] A. Fujimura and K. Sugihara, “Geometric analysis and quantitative
evaluation of sport teamwork,” Systems and Computers in Japan, vol. 36,
no. 6, pp. 49–58, jun 2005.
[20] J. Gudmundsson and T. Wolle, “Football analysis using spatio-temporal
tools,” Computers, Environment and Urban Systems, vol. 47, pp. 16–27,
2014.
[21] Z. Chen, H. Zhang, D. Guo, S. Jia, X. Fang, Z. Huang, Y. Wang,
P. Hu, L. Wen, L. Chen, Z. Li, and R. Xiong, “Champion Team
Paper: Dynamic Passing-Shooting Algorithm Based on CUDA of The
RoboCup SSL 2019 Champion,” Robocup 2019: Robot World Cup
XXI, pp. 479–490, 2019. [Online]. Available: http://link.springer.com/
10.1007/978-3-030-35699-6 39http://arxiv.org/abs/1909.07717
[22] J. A. Iglesias, A. Ledezma, and A. Sanchis, “Opponent Modeling
in RoboCup Soccer Simulation,” in Advances in Physical Agents.
WAF 2018. Advances in Intelligent Systems and Computing, vol
855. Springer, Cham, nov 2019, pp. 303–316. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-99885-5 21
[23] A. S. Larik and S. Haider, “Opponent classification in robot soccer,”
in Lecture Notes in Computer Science (including subseries Lecture
Notes in Artificial Intelligence and Lecture Notes in Bioinformatics),
vol. 9101. Springer, Cham, 2015, pp. 478–487. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-19066-2 46
[24] M. Shvo, S. Sohrabi, and S. A. McIlraith, “An AI planning-based
approach to the multi-agent plan recognition problem,” in Lecture
Notes in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10832
LNAI. Springer, Cham, 2018, pp. 253–258. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-89656-4 23
[25] R. Evertsz, J. Thangarajah, and M. Papasimeon, “The conceptual
modelling of dynamic teams for autonomous systems,” in Lecture
Notes in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10650
LNCS. Springer, Cham, nov 2017, pp. 311–324. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-69904-2 25
[26] S. I. Cannon-Bowers, J. A. Tannenbaum, E. Salas, and C. E. Volpe,
“Defining competencies and establishing team training requirements,”
in Team effectiveness and decision making in organizations.
San
Francisco, CA: Jossey-Bass, 1995, pp. 333–380.
[27] C. Pons Lelardeux, D. Panzoli, V. Lubrano, V. Minville, P. Lagarrigue,
and J. P. Jessel, “Communication system and team situation awareness
in a multiplayer real-time learning environment: application to a
virtual operating room,” Visual Computer, vol. 33, no. 4, pp. 489–
515, apr 2017. [Online]. Available: http://link.springer.com/10.1007/
s00371-016-1280-6
[28] K. E. Schaefer, B. S. Perelman, R. W. Brewer, J. L. Wright, N. Roy,
and D. Aksaray, “Quantifying human decision-making: Implications
for bidirectional communication in human-robot teams,” in Lecture
Notes in Computer Science (including subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10909
LNCS. Springer, Cham, jul 2018, pp. 361–379. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-91581-4 27
View publication stats
[29] A. R. Marathe, K. E. Schaefer, A. W. Evans, and J. S. Metcalfe,
“Bidirectional communication for effective human-agent teaming,” in
Lecture Notes in Computer Science (including subseries Lecture Notes
in Artificial Intelligence and Lecture Notes in Bioinformatics), vol.
10909 LNCS. Springer, Cham, jul 2018, pp. 338–350. [Online].
Available: http://link.springer.com/10.1007/978-3-319-91581-4 25
[30] S. Li, W. Sun, and T. Miller, “Communication in human-agent teams for
tasks with joint action,” in Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics), vol. 9628. Springer Verlag, 2016, pp. 224–241.
[31] F. Gao, M. L. Cummings, and E. Solovey, “Designing for robust and
effective teamwork in human-agent teams,” in Robust Intelligence and
Trust in Autonomous Systems. Springer US, jan 2016, pp. 167–190.
[32] K. Gillespie, M. W. Floyd, M. Molineaux, S. S. Vattam, and D. W.
Aha, “Semantic classification of utterances in a language-driven game,”
in Communications in Computer and Information Science, vol. 705.
Springer Verlag, 2017, pp. 116–129.
[33] M. Zuparic, V. Jauregui, M. Prokopenko, and Y. Yue, “Quantifying
the impact of communication on performance in multi-agent teams,”
Artificial Life and Robotics, vol. 22, no. 3, pp. 357–373, sep 2017.
[34] P. MacAlpine and P. Stone, “Evaluating Ad Hoc Teamwork Performance
in Drop-In Player Challenges,” in Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture
Notes in Bioinformatics), vol. 10642 LNAI. Springer Verlag, 2017, pp.
168–186.
[35] D. Budden, P. Wang, O. Obst, and M. Prokopenko, “Simulation
leagues: Analysis of competition formats,” in Lecture Notes in Artificial
Intelligence (Subseries of Lecture Notes in Computer Science), vol.
8992. Springer, Cham, 2015, pp. 183–194. [Online]. Available:
http://link.springer.com/10.1007/978-3-319-18615-3 15