Natural Language Processing for Computer Games
Marc Cavazza
Natural language understanding has always had a particular position within Artificial
Intelligence, being at the same time a display of machine intelligence, most
impressive to human users and, from a more practical perspective, the ideal interface
for exchanging complex information with a computer.
This is why in the future, language technologies could add significant value to
computer games, such as i) increased realism (for instance, for the game’s characters),
ii) a better integration of user commands with the gameplay itself and iii) new
paradigms for interactivity.
The techniques of Natural Language Processing (NLP) are varied and complex. The
general principle consists in using linguistic knowledge on the structure of sentences
and their content (words’ meaning) to convert the sentence into a formal
representation the computer can use. This transformation is called parsing and, in
most recent approaches, largely integrates syntactic and semantic processing. An NLP
module is thus based on linguistic resources (vocabulary and grammar) and specific
algorithms that embed the processing strategy. Parsing has to recognise the various
meaning units, solve ambiguities and identify the conceptual relations between these
units. This will for instance distinguish between the roles introduced by “with” in
commands such as “open the door with the red card” and “open the door with a skull
on it”. Further, there is an obvious continuity between NLP and the other AI
techniques used in the game, as the output of the language processing step is fed into
other AI functions, such as decision making, agent behaviour, path planning, etc.
The inclusion of language technologies in computer games is best discussed from the
gameplay perspective. This will help introducing the problems, the techniques that
can be used and their potential results.
In terms of gameplay, two main paradigms for natural language interfaces can be
envisioned. The first one is to issue commands or instructions to game agents (e.g.
NPC). Imagine for instance the player as the coach of his virtual football or basketball
team, shouting advice to them, which would influence the way the team performs.
The second one consists in acquiring information from the game characters through
dialogue, for instance in adventure games where the user would interact with
characters providing him with information needed to progress in the game level.
To illustrate the use of natural language instructions, let us consider the case of
military simulation games, such as air combat or various tactical simulations. These,
like their real-world counterparts, involve a rich background of radio
communications, which make use of well-defined “sub-languages” (a sub-language is
a consistent subset of language with restricted vocabulary and regular structures used
in a specific professional activity, such as battlefield radio transmissions). The
structure of these sub-languages can be used to simplify their automatic processing:
they can be described with limited linguistic resources while the regularities in their
structure would facilitate parsing. In this way, tactical orders or short messages could
be converted into appropriate game instructions in real time.
This approach would actually just downsize to computer games the mechanisms
which are being developed in real-world military simulators to allow human users to
communicate realistically with synthetic forces.
In most of the above examples, the realistic use of natural language instructions
requires speech recognition, which introduces further complexity and the problem of
speech recognition errors. Speech recognition is likely to be part of the next
generation of operating systems in PCs. Though its inclusion in gaming consoles is
probably a more distant perspective, the availability of built-in speech recognition in
hardware platforms will boost the demand for language understanding, as the use of
keyword patterns without further language processing quickly faces limitations.
The other paradigm for language interfaces in computer games is that of humancomputer dialogue. The constant improvement of visually realistic characters creates
expectations on their intelligence, one of which is certainly the ability to understand
language and engage in conversation. Strictly speaking, dialogue is an exchange
between partners that includes the construction of a common knowledge background.
It follows specific conversational rules, different across dialogue genres (e.g.,
instruction, negotiation, etc.). Human-computer dialogue technologies are a superset
of sentence processing techniques, adding formalisms that process the conversational
phenomena ensuring the continuity of dialogue. These conversational aspects are
sometimes called “dialogue acts” and describe the many conventions through which
implicit information in dialogue can constitute common grounds for the speakers. In
many game genres, especially in adventure games, dialogue can be conceived of as a
way for the user to acquire information from game characters. Once again, an
important advantage is that through dialogue, this exchange of information is totally
integrated to the gameplay. Human-Computer conversation can be used to solve
riddles, to negotiate information (rules of negotiation becoming part of the gameplay
itself).
Chatterbots have been the first pragmatic response of game developers to the problem
of language understanding. They are simple programs based on pattern matching,
which associate patterns of keywords to canned sentences forming the bot’s reply.
One recent example has been the chatterbot embedded in the latest Quake™ releases.
Its purpose is to re-create the fun of communicating with the other contenders, even in
the case where all the other players are actually bots. User communications during
deathmatches is usually not very informational, it is more atmospheric, and is heavily
based on challenging the other players. As such, it is a good target for the patternbased approaches of chatterbots, which can directly associate replies to patterns of
keywords detected in the user input. The result is more impressive than one would
expect considering the elementary technologies used. There have been mixed
comments on the user acceptance of the Quake chatterbot function: however, these
may be due to the relatively small scale of the current implementation rather than to
the real limitations of the approach.
Chatterbots can create the illusion of communication, often a pleasant one, which can
support games of their kind. However, chatterbots techniques are not sufficient when
the dialogue involves more complex information processing such as those involved in
negotiation or in the acquisition of information, for which it is necessary to model the
dialogue acts themselves to be able to follow the constant shift in dialogue focus.
We have discussed so far the inclusion of language technologies in current game
genres. To a large extent, existing game genres are constrained by the limited
interaction modalities offered by current technology. For instance, communicating in
natural language with virtual actors would open the way to new game genres based on
interactive storytelling. Figure 1 illustrates the use of speech recognition in an
interactive storytelling system we are developing. In this system, the user can issue
advice to the autonomous characters while the story is in progress: this advice is
processed so that its consequences update the plans which support real-time character
behaviour and generate the plot. This kind of “narrative” games, making more
extensive use of language technologies could potentially attract a new audience to
computer games, not least in terms of age group and gender.
As a conclusion, the use of language technologies has a significsnt potential both in
terms of realism and gameplay. It could also support entirely new game genres with a
stronger narrative component.
However, language technologies are more complex to develop and maintain than
other AI techniques, due to the intertwining of linguistic knowledge and AI
algorithms that characterise them. The challenge for games developers will thus
consist in adapting the many NLP techniques and formalisms to the specific
requirements of computer games.
Marc Cavazza is Professor of Intelligent Virtual Environments at the University of
Teesside. His research interests include language-enabled virtual agents, interactive
storytelling and conversational characters.
Figure 1. Speech understanding in Interactive Storytelling (environment developed
with the Unreal Tournament™ engine; character design by “Austin” and Brian
Collins.).