Qualitative Representations for Robots: Papers from the AAAI Spring Symposium
A Qualitative Representation of Social Conventions for Application in Robotics
Diedrich Wolter
Frank Dylla and Arne Kreutzmann
Smart Environments,
Information Systems and Applied CS
University of Bamberg, Germany
Cognitive Systems, SFB/TR8 Spatial Cognition
University of Bremen, Germany
Abstract
presented that are capable of interacting with individuals or
groups of people, putting forward the prominent example of
the museum tour guide robot RHINO (Burgard et al. 1999).
Systems were designed to pass people (Pacchierotti, Christensen, and Jensfelt 2005) or approaching groups of people
(Althaus et al. 2004) in an adequate manner. In (Shi et al.
2008) the change of velocity near other pedestrians is considered, whereas in the Companion framework more general
joint human-robot navigation is considered (Kirby 2010).
Taking a closer look at these existing social robot systems,
they all have in common a behavior that is directly based on
quantitative parameters tailored to specific tasks, i.e., they
are not aware of their spatial context but their actions are
hardwired to the control system. By contrast, social behavior as described in the social sciences involves abstract concepts of space and time that vary across social status, culture,
etc. It requires observation and interpretation to align oneself
with the current context. Not until awareness of conventions
is tackled, a robot will be able to recognize that it is interfering with human activity unintentionally. Viewed from the
perspective of intelligent agent design, abstract declarative
representations of social conventions could offer appropriate
means for such reasoning. Linking such abstract representations to robot control is a difficult endeavor though. With
our work we aim to bridge the gap between abstract descriptions of spatio-temporal patterns and robot navigation
techniques. The contribution of this paper is to describe how
patterns of socially acceptable behavior can be represented
using qualitative concepts of space and time. The representation obtained can then provide the basis on which a robot
can achieve awareness of social context.
We first give a system overview of how the approach integrates with a robotic software architecture. Section 3 reviews the concept of spaces (proxemics) and navigation conventions which are assumed to be socially acceptable. Section 4 shows how spatial knowledge present in the navigation conventions can be captured with qualitative representations. Using these spatial primitives we construct the new
logic QLTL (Linear Temporal Logic with Qualitative Spatial Primitives), discuss required reasoning techniques and
their realization. Section 5 presents a modeling of the socially acceptable navigation behaviors presented earlier and
how they are processed in Section 6. Finally, Section 7 outlines handling of real-world situations on a robotic platform.
Acceptance of everyday robots will largely depend on
their ability of social interaction. Patterns of socially acceptable behavior have been characterized in social science by means of abstract concepts of space and time.
This makes integration with robot navigation a challenging problem. Moreover, an integration should allow
the robot to build awareness of these patterns as in reality there will be misunderstandings a robot should be
able to respond to. The contribution of this paper is a
representation of navigation patterns that is based on
qualitative representations of space. We present a logic
with clear semantics for specification of navigation patterns and show how it allows several conventions of social interactions to be represented. We also outline integration with robot navigation and give examples with
real world situations.
1
Introduction
Robots becoming part of our everyday life is a vision that
is widely shared among researchers in the robotics community. Aside from technical challenges in design and development of such robots, we also need to acknowledge that
such a system would be part of our human society. Acceptance of robots is henceforth also depending on their social
acceptance. How does the robot interact with humans?
Social interaction goes beyond goal-directed human-robot
interaction, it also comprises casual or emergent interaction,
for example, how a robot traverses a populated area. Shall
it take advantage of its advanced motion and path-planning
capabilities to rush through even the most crowded places?
We are motivated by the assumption that acceptability of
robots will depend on acknowledging established patterns
of social interaction. Lindner and Eschenbach claim that
people would feel offended if a robot cuts path in front of
them (Lindner and Eschenbach 2011). Other patterns may
be more imperative. For example, humans recognize and acknowledge that others have queued up in a waiting area and
they line up as well, not trying to overtake and press forward if space permits. Programmers realizing a robot’s navigation component need thus to include patterns of socially
acceptable behavior. In recent years several robots have been
c 2014, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
34
proxemics or social spaces (Hall 1966; Hall et al. 1968).
high-level controller
3.1
Social Spaces
Hall distinguishes four different kinds of spaces:
social
rule base
KB
Kinect
LRF
local pp
motor
global pp
localize
intimate space touching, hugging, close conversation,
personal space interaction with well known people,
social space interaction with people, and
public space people in this region are ignored in general.
features
Additionally, for each region Hall defines a close and far
sub-region to distinguish social interaction on a finer level
of granularity. The nested structure of the spaces is depicted
in Figure 2. Hall does not limit public space in its extend and
the 7.6m relates to the close public space. Based on his work
further investigations were carried out, e.g. asymmetries of
personal spaces (Ashton and Shaw 1980), intercultural differences of spaces (Burgess 1983), or on which side people
may pass in case walking paths are intersecting (Bitgood and
Dukes 2006). Furthermore, Frith and Frith (2006) considered how these findings of social conventions help to predict
other people’s behavior. Nevertheless, not much work has
been presented on what people exactly do and why they behave like that. That is, the connection between qualitative
descriptions and actual behavior remains unclear.
In addition to Hall’s view other kinds of spaces which influence social behavior can be distinguished (Lindner and
Eschenbach 2011). Affordance Spaces are based on possible
actions an agent could perform in that space (see e.g. (Galton 2010), e.g. a park affords to play football in. In contrast activity spaces are defined on the basis of which actions
are actually performed in them, for example a space a football game is actually played (see e.g. (Ostermann and Timpf
2007; Kendon 1990). Territory Spaces are regions which are
claimed by a single or group of agents. In general, boundaries or extent of territories are marked with markers that are
perceivable by potential intruders (e.g. (Lawson 2001)). The
last space to mention is Penetrated Space describing spaces
which are influenced by other activities, for example, the
space one’s voice is perceivable (auditory) or the olfactory
penetrated space of a barbecue. Although, the discrimination of all kinds of spaces is important, we restrict ourselves
in this work to social spaces as defined by Hall for reasons
of simplicity of the examples. In general, our approach is
capable to deal with all kinds.
A first formalization of social spaces for robotic systems has been presented in Lindner and Eschenbach (2011),
purely based on mereo-topological knowledge that is axiomatized within the highly expressive Situation Calculus. In
contrast to this approach, we make use of qualitative concepts to link robotic perception with logic primitives. Also,
we opt for a more restricted logic that nicely integrates with
robotic architectures.
...
Figure 1: System overview, showing where social conventions could be integrated into a layered architecture.
2
Approach
On the technical level, we develop a new representation language that combines the temporal logic LTL (Pnueli 1977)
with qualitative spatial calculi (see Section 4). Based on our
representation language, we formalize several conventions
of socially acceptable navigation. We discuss how the new
logic can easily be integrated with existing navigation components to ensure socially acceptable navigation behavior.
For conceiving system integration we adapt the approach
of a layered architecture as used successfully in many
robotic systems, e.g. for museum tour guide robots (Burgard
et al. 1999; Thrun et al. 1999). In the most general interpretation it consists of three layers (cp. Figure 1): 1) the
low-level hardware control layer for any kind of sensor or
actuator (e.g. wheels or camera), 2) the intermediate navigation and (pre-)processing layer (e.g. path planning and feature detection) and 3) the high-level reasoning and learning
layer (e.g. Golog as in case of (Burgard et al. 1999)). The
main organizational criteria is that on the lowest level data
is completely quantitative and on the highest level abstract
symbolic representations are predominant.
We adapt this kind of architecture. Our robot setup involves a laser range finder (LRF) for obstacle detection and
a Kinect camera for people detection. On the intermediate
layer we consider the standard navigation modules. Socially
aware navigation mainly affects the local path planning (’local pp’ Figure 1), but for example due to unavailability of
socially acceptable paths, different routes on the global level
may need to be considered. In this work, we focus on the
interaction with local path-planning only.
Within the navigation layer we propose adding a new
module, which we call social (behavior) module, consisting of a declarative rule base of social conventions, means
to keep track of rules applicable in the current situation, and
interaction to the path-planning module. The declarative nature of the knowledge eases integration with the high-level
controller to foster awareness of the social context.
3
3.2
Classification of Social Conventions
In (Dylla, Coors, and Bhatt 2012) a classification of social
navigation conventions is presented. It comprises five (not
necessarily disjunctive) categories mainly discriminated on
the basis of spatial configurations and the number of people involved. The authors don’t claim this taxonomy to be
Social Conventions in Human Navigation
Human navigation is spatial interaction of individuals based
on social conventions (Kirby 2010). Hall described this interaction on the basis of spaces around individuals called
35
Personal Space < 1.2m (4 ft)
Intimate Space < 0.45m (1.5 ft)
Social Space
< 3.6m (12ft)
(a) in general moving on the right of a walkway, or
(b) no running in a library
These conventions may have to be adapted regarding cultural background. For example, in Great Britain pedestrians
would evade to the left and overtaking should take place on
the right side. Although not complete these kinds of conventions must be formalized in order to enable robots to be
aware of these conventions and behave accordingly.
(close)
Public Space
< 7.6m (25ft)
Figure 2: Social Spaces as described by Hall (1966)
4
Representation of Coarse Knowledge
Social conventions are given in natural language and thus
in a vague, coarse, and imprecise manner. By means of
qualitative descriptions one can focus on distinctions between objects that make an important and relevant difference with respect to a given task (Kuipers 1994). The field
of Qualitative Reasoning (QR) is concerned with capturing
such distinctions about objects in the real world, also considered as commonsense knowledge, with a limited set of
symbols, i.e. without numerical values (Cohn and Hazarika
2001). These distinctions are captured by relations, which
summarize indistinguishable cases into a single symbol. For
example, in most cases it is sufficient to refer to the color
‘red’ as it is not of importance whether it is slightly lighter
or darker than some prototypical red. Qualitative SpatioTemporal Representation and Reasoning (QSTR) is a subfield of QR, where the underlying physical structure of the
domain can be exploited for performing well defined reasoning. In general, topological (e.g. part of) and positional
relations can be distinguished (Freksa and Röhrig 1993). Positional relations can be subdivided into orientation (relative:
e.g. left, right; and absolute: e.g. south, north) and distance
calculi (e.g. close, far). A set of relations together with operations on them is called a qualitative calculus.
Qualitative calculi are based on partition schemes of the
underlying domain. The set of all possible relations between
two spatial entities (or three in case of ternary calculi) is
categorized into a set of atomic relationships called base relations (BR) which either represent themselves meaningful
relations for the task at hand or which allow these relations
to be obtained by means of union of base relations. Since relations are ordinary sets, set-theoretic operations are applicable to qualitative relations. Algebraically, qualitative calculi
are related to relation algebras in the sense of Tarski (Dylla
et al. 2013). For the purpose of this paper it is sufficient that
a qualitative calculus allows us to model binary (or ternary)
relations between spatial entities using unions of base relations. Since base relations are defined by a partition scheme,
they are naturally disjoint and thus conjunction would be
useless. The most widely considered knowledge representation for qualitative calculi is constraint-based. Given a set
of variables X = {x1 , . . . , xn } and a set of base relations
BR = {b1 , . . . , bm }, a knowledge base consists of constraints (xi {bi1 , . . . , bik } xj ) which say that spatial entities
xi and xj are in relation bi1 ∪ . . . ∪ bik . Qualitative spatial
reasoning then provides us with (calculi-specific) algorithms
to decide whether a constraint-satisfaction problem (CSP)
consisting of such constraints is satisfiable or not (Renz and
Nebel 2007). The test of satisfiability also allows new con-
(a) pedestrians approaching (b) overtaking a with the same
head-on evading to the right
direction on the left
(c) following another pedes- (d) yield for another pedestrian
trian in a narrow passage
near a passage entrance
Figure 3: Four examplary conventions of pedestrian navigation. The triangle depicts the robot and the circle some other
agent.
complete as, e.g., non-spatial discriminators like gender and
cooperative behaviors like ‘ladies first’ are not considered.
The five categories and some of the conventions are:
1. approaching head-on or from behind
(a) pedestrians approaching in head-on both have to evade
to the right (Figure 3(a))
(b) pedestrians moving in the same direction have to be
overtaken on the left side (Figure 3(b))
2. crossing situations
(a) other (not necessarily moving) pedestrians on the left or
right may be passed in their front with some distance,
(b) may be passed in their back (with smaller distance), or
(c) the agent lets the other pedestrians pass (by stopping).
Here the other agent has to move.
3. bottlenecks or narrow passages
(a) follow other pedestrians in a narrow passage (no overtaking possible) (Figure 3(c))
(b) ’crossing’ other pedestrian at narrow passage
(c) yield to others near passage entrance (Figure 3(d))
4. interaction with groups
(a) evading or passing a group on the outside
(b) crossing large groups (if passing is inappropriate or not
possible at all)
5. individual constraints , i.e. conventions very dependent on
the context an agent is in, e.g.
36
0
0
disconnected
DC
partial
externaly
connected EC overlap PO
2
1
3
equal EQ
A
6
8
10
9
7
15
12
B
8
0
14
13
11
10
9
2
3
4
1
B
A
15
12
11
10
5
6
14
13
7
8
9
~ 4∠5 B
~ and A
~ 4∠3 B.
~
Figure 5: OPRA4 relations A
13
tangential non-tangential non-tangential
proper
proper part
proper part
inverse TPPI part NTPP inverse NTPPI
provide the means to represent a configuration, the representation of a convention also involves inscribed behavior. Conventions inscribe behavior within a certain context, typically
triggered by an event or associated with a process (e.g., keep
left when standing on an escalator). Among the different options to represent behavior, we choose to apply a temporal
logic that links spatial configuration knowledge (snapshots)
to temporal sequence. This approach has several nice implications:
Figure 4: The eight base relations of RCC-8.
straints that follow from a given set of constraints to be inferred, similar to how resolution of logic formulas allows
for deduction. An implementation of the methods described
above is available via the qualitative spatial reasoning toolbox SparQ (Wolter and Wallgrün 2011)1 .
In our formalizations we apply the topological Region
Connection Calculus (RCC-8) and the Oriented Point Reasoning Algebra (OPRAm ) (Moratz 2006).
• Employing a temporal logic to connect qualitative representations of snapshots allows static as well as dynamic
spatial knowledge to be represented with the same vocabulary of spatial concepts.
• Like any standard logic, it allows non-spatial and nontemporal knowledge to be represented aside the spatiotemporal knowledge using propositional symbols.
RCC-8: The Region Connection Calculus
Topological distinctions are inherently qualitative in nature
and they also represent one of the most general and cognitively adequate ways for the representation of spatial information (Renz, Rauh, and Knauff 2000). Based on this inherent qualitative nature different qualitative calculi were developed, among them the Region Connection Calculus (RCC-8)
(Cohn et al. 1997) which is based on a binary connection relation C(a, b) denoting that region a is connected to region
b. Exploiting the connectivity of regions eight base relations
are defined (see Figure 4).
• Last but not least, temporal logic has been integrated with
motion planning and robot control (Kress-Gazit and Pappas 2010; Ding et al. 2011; Kloetzer and Belta 2010) and
recognition of processes (Kreutzmann et al. 2013).
Of course, this approach has limitations. A technical limitation are missing quantifiers, thus the approach requires a
predetermined maximum number of distinct objects to consider. A potential knowledge modeling limitation can be
seen in binary truth evaluation of classic logics, e.g., if handling graduated, fine-grained concepts.
OPRAm : A Relative Orientation Calculus
In OPRAm relative orientation is partitioned into 2m
equidistant angular cones and their separating lines with an
intrinsic orientation. Within this work we apply granularity m = 4. For simplicity, these are numbered from 0 (intrinsic orientation) to 15. The relation is formed by a tuple
of the relative orientation i of object B wrt. object A and
vice versa (j), usually written as A 4 ∠ji B. In Figure 5
(left) relation A 4 ∠513 B is depicted. This directly relates
to a linguistic expression like ”B is ahead to the right of A
moving in the same direction”. If point positions coincide
relations are only determined by the segment number s of
A the orientation of B falls in, e.g. A 4 ∠3 B in Figure 5
−jm
(right). We abbreviate complex relations by m ∠ji11−i
=
n
jm
j
in
∨i=i1 ∨j=j1 m ∠i with i, j ∈ Z4m , ∗ abbreviates 0 − 4m.
5
7
1
5
13
4
5
tangential
proper
part TPP
15
2
5.1
QLTL: Linear Temporal Logic with
Qualitative Spatial Primitives
For representing social conventions we require temporal sequence knowledge, i.e. which situation occurs before/after
another. This can be achieved using a lean logic like Linear
Temporal Logic (LTL) (Pnueli 1977). LTL is a modal logic
that extends propositional logic with modal operators which
connect statements about snapshots (called worlds in modal
logic) using dedicated modal operators. In order to integrate
this logic with spatial primitives, we choose to encode qualitative spatial relations in terms of propositional symbols and
we extend the semantics to also include a spatial semantic.
Since conventions also depend on the types of objects involved in a certain configuration (person, obstacle, robot,
etc.), we also introduce sorts to retain category information
in logic formulae. This leads to the following syntax definition for our logic QLTL. The ingredients are:
Formalization of Social Conventions
We have argued that a representation of socially adequate
behavior is essentially based on qualitative concepts of spatial configurations. While qualitative constraint networks
• a set of spatial symbols S. Let s be a number of sorts,
then Si = {si,1 , si,2
S, . . .}, i = 1, . . . , s are sets of spatial
symbols and S := i=1,...,s Si
1
available from http://www.sfbtr8.uni-bremen.de/project/r3/
sparq/
• R = {r1 , . . . , rn } is a set of qualitative relation symbols
37
In this work, we define the set of qualitative relation symbols Q to be the union of OPRA4 and RCC-8 relations.
This allows us to represent the social conventions. Since we
will obtain an interpretation from the perception of the robot,
we can assume it to be spatially consistent.
• F = {f1 , . . . , fl } be a set of function symbols
• G = {g1 , g2 , . . .} is a set of propositional symbols for
representing general, non-spatial knowledge
• The set of propositions P is defined as P := G ∪
{r(s, t)|r ∈ R, s, t ∈ (S ∪ {f (si )|f ∈ F, si ∈ S})}.
The idea underlying this definition is to use natural notation of qualitative relations so that it is possible to represent
spatial knowledge by a single propositional symbol. Propositions are either describing non-spatial facts (G) or some
spatial relation r between two objects s and t which can either be sorts or some sort dependent aspect f (si ). For example, NTPP(h, sec(r)) that a human h is standing in the
security range sec of the robot r.
Formulae in QLTL are then defined recursively:
• p is a formula for every p ∈ P
• If φ is a formula, so is ¬φ
• If φ, ψ are formulae, so is φ ⊗ ψ with ⊗ ∈ {∧, ∨, →, ↔}
• If φ is a formula, so is M φ with M ∈ {◦, 2, }
• If φ, ψ are formulae, so is φ N ψ with N ∈ {U, R}
The semantics of QLTL are similar to LTL, i.e., an interpretation establishes an ordered sequence of worlds. Within
each single world, all propositional symbols are mapped to
truth values true and false, inducing the interpretation of formulae composed with logic conjuncts (∧, ∨, . . .). For convenience of notation QLTL includes function symbols for
relating the individual regions established by an agent, e.g.,
social space, personal space, etc. The semantics of a function
f is a mapping S → S of spatial symbols. In other words,
functions are used as shorthand notations for the respective
spatial symbols. In QLTL we further require interpretations
within all worlds to be spatially consistent, i.e., sensor interpretations must not be contradictory. This defines the spatial
semantics of QLTL. Within one world, the interpretation of
all (spatial) propositions r(s1 , s2 ) with si ∈ S or si = fj (s)
for some fj ∈ F, s ∈ S induces a qualitative constraint
network with variables S and according constraints ri (x, y)
where x, y are either the spatial symbols s1 , s2 or the symbols obtained by application of fj (si ). The spatial semantics
of a relation r is defined by the respective qualitative calculus. Functions F are also assigned with a respective spatial
semantic, e.g., mapping the position of a human to its estimated personal space. We say that an interpretation is spatially consistent if, first, all induced constraint networks are
consistent and, second, if all mappings in F respect their
spatial semantics, e.g., personal(h) is the region that determines the personal space of h as defined by the function
personal.
The semantics of modal operations connect distinct
worlds within a sequence, identical to the sematics of LTL:
◦φ (next) φ holds in the following world
2φ (always) φ holds in the current and in all future worlds
φ (eventually) φ holds in a future world, (φ ↔ ¬2¬φ)
φ U ψ (until) φ holds at least until eventually ψ holds, but
they dont have to hold at the same time
φ R ψ (release) ψ holds until and including the world in
which φ first becomes true
5.2
Conventions as QLTL formulae
We introduce a convenient notation for representing conventions as QLTL formulae. Conventions considered in this
work essentially come in an “if-then-until” flavor in the
sense that observing a certain event or process triggers (start)
a sequence of required configurations (effects) to achieve a
behavior in accordance with the regulation until an end state
or a break criteria is reached. The end criteria is reached if
everybody behaves as expected while the break criteria prevents the system from getting stuck in a rule if something
unexpected happens, e.g. a person involved does not behave
as expected by turning around and moving away. Break criteria might be a timeout or if involved persons leave the public space of the agent. For reasons of space we do not consider break criteria in further detail. Conventions can easily
be represented as QLTL formula if we pursue a declarative
description of regulation-compliant behavior using the pattern
φstart → ◦ φeffect U (φend ∨ φbreak )
(1)
We note that QLTL is expressive enough to allow conventions to be represented which are not effective. If, for
example, the sub-formula specifying the trigger condition
would refer to a future situation, than it may not be possible
to decide whether the trigger condition is satisfied. Roughly
saying, we want to exclude conventions of the kind “if this
will turn out to be wrong, don’t do it”. Deciding effectiveness of a convention is beyond the scope of this paper—we
assume that the conventions are stated in a way that allows
trigger conditions to be detected at the time they apply.
We point out an inconvenience of directly using modal
logic for knowledge representation, namely its lack of variables. The pattern introduced in Equation 1 can involve
propositional symbols only. As a consequence, for a convention that says how to avoid an obstacle, we require separate
formulae, one for each individual obstacle. To this end, we
introduce a shorthand notation for conventions that supports
variables. In the following we write x1 : s1 , . . . , xn : sn :
φ with variables
x1 , . . . , xn of respective sorts s1 , . . . , sn ,
V
meaning xi ∈si ,i=1,...,n φ0 with φ0 obtained by substitution
of xi for the respective spatial or propositional symbol.
5.3
QLTL Representation of Social Conventions
We now exemplarily formalize social conventions from
Class 1 and 2 (see Section 3.2) in QLTL. Throughout
this section we use r to denote the robot (r : robot),
and h for humans (h : human). In the following we
use x and y to denote objects of any of these sorts.2
2
Remark: these symbols need different interpretations regarding the relation they are used for. For RCC-8 they need to be interpreted as regions, e.g. the space covered by an object (obj(x)), and
for OPRA4 as oriented point (opos(x)). For brevity we omit this
distinction in the presented formalizations.
38
To refer to the social spaces constituted by a human h
we use the functional notation intimate(h), personal(h),
social(h), public(h) respectively. On the syntactical level
these functions denote special symbols for referring to the
regions, whereas on the semantic level the specific region
must be interpreted based on the physical object specified
by h. In order to improve readability of formal conventions we define some macro relations to abbreviate complex relations. First, describing that two agents are in a
head on situation: HEAD ON(x, y) := x 4 ∠15−1
15−1 y or
are moving in the same direction: SAME DIR(x, y) :=
7−9
x 4 ∠15−1
7−9 y ∨ x 4 ∠15−1 y. Next we define x being on
the left or right side of y: ON LEFT(x, y) := y 4 ∠∗2−6 x,
ON RIGHT (x, y) := y 4 ∠∗
11−13 x and x being in front or behind y: IN FRONT(x, y) := y 4 ∠∗15−1 x, BEHIND(x, y) :=
y 4 ∠∗5−11 x. Finally, by OVERLAP we define that there is a
partial or complete overlap: OVERLAP(x, y) := PO(x, y) ∨
TPP (x, y) ∨ NTPP(x, y) ∨ TPPI (x, y) ∨ NTPPI (x, y). Using
these primitives we can formalize convention 1a as follows:
φ1a
start
φ1a
effect
:=
:=
6
ON
6.1
:=
(2)
LEFT (h, r) R BEHIND (h, r)
BEHIND (h, r)
observations |= φstart ⇔ φeffect U φend becomes active (4)
As discussed in (Kreutzmann et al. 2013) this task of model
checking can be accomplished by first assigning propositional symbols to truth values according to the robot’s observation and then applying an ASP solver (answer set programming) to search a model for a given precondition formula. In this work we are however involved with a more
complex logic that additionally involves object sorts. During model checking one needs to ensure that a model also
respects the correct association of object sorts, i.e., humans
in the formula can only be matched to humans, obstacles
to obstacles, etc. A straightforward yet sufficient solution
can be realized in ASP-based model checking. ASP supports relational knowledge that allows sort knowledge like
“person(x)” to be denoted. Doing so for convention formulae as well as for observation ensures correct association.
However, we are only interested in spatially consistent
interpretations of the propositional symbols and filter out
all spatially inconsistent models, as provided by SparQ
(Wolter and Wallgrün 2011). Filtering out spatially inconsistent models in a subsequent step to model checking can
lead to large amounts of models that need to be rejected.
Therefore, we propose to introduce an intermediate step
that responds to partial observations. Prior to grounding
propositional symbols with observations, QSR is applied for
constraint propagation in order to make implicit knowledge
explicit. For example, we might observe two facts: two persons are standing in front, one to the left looking to the
right and one to right looking left. Here, constraint propagation in OPRAm would reveal the relation between both
persons would be looking at one another. Enriching observations prior to model checking can thus help recognizing
whether a convention’s precondition is met. We are aware
OVERLAP (r, social(h)) ∧
(ON
φ2a
effect
φ2b
effect
φ2c
effect
2
φeffect
φ2end
LEFT (h, r) ∨ ON RIGHT (h, r))
:= ¬PO(r, personal(h)) ∧ IN FRONT (r, h)
:= ¬PO(r, intimate(h)) ∧ BEHIND (r, h)
:= stop(r) U
(3)
BEHIND (r, h)
:=
φ2a
effect
:=
BEHIND (h, r)
∨
φ2b
effect
Detecting Applicability
Testing convention applicability requires perception of the
robot to be matched against the precondition of a convention
φstart . The convention is applicable if the observations allow
for a model of φstart . We are thus confronted with the task of
model checking where observations provide (partial) knowledge and the task is to decide whether this partial model
can be extended to a spatially consistent interpretation that
makes φstart come true:
This means, if the robot enters or is in the social space of another agent h and they are head on, the robot must not move
into the personal space of h. Furthermore, h has to be on the
left of r, thus r needs to turn right until r has passed h, i.e.,
r is behind h. Convention 1b can be modeled similar except
that they are in SAME DIR(r, h) instead of HEAD ON(r, h)
and r has to overtake on the left (ON RIGHT(h, r)).
All conventions of class 2 are covered by the following:
φ2start
Processing QLTL Conventions
Given a set of QLTL formulae representing the social conventions known to the robot, the two tasks are to (1) identify that the preconditions of a pattern are met and (2) to
determine actions which are admissible with respect to the
pattern. Although the two tasks seem different at first sight,
they both can be tackled with the method of testing convention applicability.
OVERLAP (r, social(h)) ∧ HEAD ON (r, h)
:= ¬PO(r, personal(h)) ∧
φ1a
end
Dealing with class 5 is straightforward. If the robot is in a
specific context, e.g. a library l (φstart = OVERLAP(r, l)),
he has to adapt his behavior, e.g., switch to quiet mode
(φeffect = q mode(r)), until he is not in the context anymore
(φend = ¬φstart ).
∨ φ2c
effect
If the robot enters the social space of h on the left or right,
r has three options. Either he passes h in the front with not
entering the personal space, pass behind h with a smaller
distance, i.e. it is allowed to enter personal but not the intimate space, or he can stop until the human has passed.
For brevity we only sketch the main aspects to consider formalizing the three remaining classes. In case of
a narrow passage (class 3) we need to represent an overlap of obstacles with a space of the agent to interact
with, e.g. a wall to the left in the social space of h:
OVERLAP (w, personal(h)) ∧ ON LEFT (w, h). For dealing
with conventions regarding groups (class 4) we need to redefine the spaces with respect to the individuals involved.
One approach is to define the group space as the union
of all individual spaces, e.g. social(g) := ∪h∈g social(h).
39
Kinect
Laser Range Finder
Fused View
Situation Description
NTPP(obj(robot), social(person 1))
OPRA(opos(robot), opos(person 1),
{0, 1}, {0, 1, 15})
=⇒ head on(robot, person 1)
Figure 6: From the sensor readings to the knowledgebase. Using a kinect and a laser range finder a human is detected and the
positions and extends of his social spaces are determined, resulting in the knowledge base on the lower right. Due to uncertainty
of the sensor readings, the OPRA4 relation of robot 4 ∠11 person is coarsened to robot 4 ∠15−1
0−1 person and returned by our
system as OPRA(opos(robot), opos(person 1){0, 1}, {0, 1, 15}).
that due to noisy sensor data qualitative relations might be
computed which do not map reality exactly, e.g., jitter at relation transition. For reasons of space we neglect a detailed
consideration here and refer to work like (Dubba, Cohn, and
Hogg 2010) where sensor data is interpreted robustly by a
pre-computation step.
6.2
gers Convention 1a, but it cannot be executed since there
exists no path in the free space, that would not cross the persons personal space. Thus the robot will stop until the world
changes such that the convention is not applicable any more,
i.e., the start condition is not met any more.
8
Planning Admissible Actions
As mentioned earlier we reduce action planning to checking
convention admissibility. The idea, as discussed in (Wolter,
Dylla, and Kreutzmann 2011), is to link the applicability
check to a planner. Whenever the planner generates or extends a partial plan, these plans are checked for admissibility, discarding them if they do not match the convention.
This kind of integration is easy with many planners like randomized road map planners or lattice-based planners. In order to check that a plan satisfies the inscribed effects of convention φstart → ◦ φeffect U φend , we simply need to modelcheck φeffect U φend with the plan generated by the planner,
treating the intermediate configurations of the plan like observations when checking convention applicability. Essentially, this step is the same as process recognition with LTL
formulae as described in (Kreutzmann et al. 2013).
7
Conclusions and Future Work
The ability to respect social conventions is key for public acceptance of shared human-robot environments. Ultimately,
the robot requires the ability to reflect on the social conventions, e.g., enabling it to decide on applicability of certain
conventions. A promising approach towards such awareness
is to pursue a declarative, abstract representation of conventions that supports abstract deliberation as well as integration with navigation components of a robotic system. By
proposing the qualitative spatio-temporal logic QLTL we indicate how such abstract representation can be constructed.
Qualitative primitives in the representation provide the important link between robot navigation and abstract logic. By
adjoining qualitative spatial reasoning techniques from the
SparQ toolbox with answer set programming (ASP) we obtain effective means to reason about QLTL formulae, in particular to recognize applicability of conventions. In future
work we aim to exploit the flexibility of declarative convention specification in order to allow the robot to adapt to situations dynamically, in particular to handle necessary convention violations adequately.
Examplary Case Study
In this section we outline a robot implementation of our
approach using a SICK LMS200 laser range finder and a
Microsoft Kinect RGBD camera mounted to an Active Media Pioneer 3-AT mobile robot. The Kinect sensors allows
recognition of humans and estimating their position and orientation. The orientation estimate in our implementation is
not very crisp, but as can be seen in Figure 6 our approach
can handle such uncertainty. The laser scanner is used to detect obstacles and determine the free space. Currently, only
local reactive behavior is employed.
In the example shown in Figure 6 a person is about to
enter an office, which the robot is trying to leave. This trig-
Acknowledgments
Financial support by the Deutsche Forschungsgemeinschaft
in the Transregional Collaborative Research Center SFB/TR
8 Spatial Cognition project R3-[Q-Shape] is gratefully acknowledged.
References
Althaus, P.; Ishiguro, H.; Kanda, T.; Miyashita, T.; and Christensen, H. I. 2004. Navigation for human-robot interaction
40
Kirby, R. 2010. Social Robot Navigation. Ph.D. Dissertation, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.
Kloetzer, M., and Belta, C. 2010. Automatic deployment of
distributed teams of robots from temporal logic motion specifications. IEEE Transactions on Robotics 26(1):48–61.
Kress-Gazit, H., and Pappas, G. J. 2010. Automatic synthesis
of robot controllers for tasks with locative prepositions. In
IEEE International Conference on Robotics and Automation,
3215–3220.
Kreutzmann, A.; Colonius, I.; Wolter, D.; Dylla, F.; Frommberger, L.; and Freksa, C. 2013. Temporal logic for process
specification and recognition. Intelligent Service Robotics
6(1):5–18.
Kuipers, B. 1994. Qualitative Reasoning: Modeling and Simulation with Incomplete Knowledge. The MIT Press.
Lawson, B. 2001. Language of Space. Taylor & Francis.
Lindner, F., and Eschenbach, C. 2011. Towards a formalization of social spaces for socially aware robots. In Egenhofer,
M. J., and et al., eds., COSIT, volume 6899 of LNCS, 283–
303. Springer.
Moratz, R. 2006. Representing relative direction as a binary
relation of oriented points. In Brewka, G., and et al., eds., 17th
European Conference on AI, 407–411. Italy: IOS Press.
Ostermann, F., and Timpf, S. 2007. Modelling space appropriation in public parks. Proceedings of the 10th AGILE International Conference on Geographic Information Science.
Pacchierotti, E.; Christensen, H. I.; and Jensfelt, P. 2005. Embodied social interaction for service robots in hallway environments. In Corke, P. I., and Sukkarieh, S., eds., FSR, volume 25 of Springer Tracts in Advanced Robotics, 293–304.
Springer.
Pnueli, A. 1977. The temporal logic of programs. In Proceedings of the 18th annual symposium on foundations of computer science (FOCS), 4657.
Renz, J., and Nebel, B. 2007. Qualitative spatial reasoning
using constraint calculi. In Aiello, M.; Pratt-Hartmann, I.; and
van Benthem, J., eds., Handbook of Spatial Logics. Springer.
161–215.
Renz, J.; Rauh, R.; and Knauff, M. 2000. Towards cognitive
adequacy of topological spatial relations. In Proc. of Spatial
Cognition II, 184–197. Springer.
Shi, D.; Jr., E. G. C.; Donate, A.; Liu, X.; Goldiez, B.; and
Dunlap, D. D. 2008. Human-aware robot motion planning
with velocity constraints. In McQuay, W. K., and Smari,
W. W., eds., CTS, 490–497. IEEE.
Thrun, S.; Bennewitz, M.; Burgard, W.; Cremers, A. B.; Dellaert, F.; Fox, D.; Hähnel, D.; Rosenberg, C. R.; Roy, N.;
Schulte, J.; and Schulz, D. 1999. Minerva: A secondgeneration museum tour-guide robot. In ICRA, 1999–2005.
IEEE Robotics and Automation Society.
Wolter, D., and Wallgrün, J. O. 2011. Qualitative spatial reasoning for applications: New challenges and the SparQ toolbox. In Hazarika, S. M., ed., Qualitative Spatio-Temporal
Representation and Reasoning: Trends and Future Directions.
Hershey (PA), USA: IGI Global.
Wolter, D.; Dylla, F.; and Kreutzmann, A. 2011. Rulecompliant navigation with qualitative spatial reasoning. In
Proceedings of the 4th Intl. Robotic Sailing Conference.
Springer.
tasks. In IEEE International Conference on Robotics and Automation, 1894–1900.
Ashton, N. L., and Shaw, M. E. 1980. Empirical investigations of a reconceptualized personal space. Bulletin of the
Psychonomic Society 15(5):309–312.
Bitgood, S., and Dukes, S. 2006. Not another step! economy
of movement and pedestrian choice point behavior in shopping malls. Environment and Behavior 38(3):394–405.
Burgard, W.; Cremers, A. B.; Fox, D.; Hähnel, D.; Lakemeyer,
G.; Schulz, D.; Steiner, W.; and Thrun, S. 1999. Experiences
with an interactive museum tour-guide robot. Artificial Intelligence 114(1-2):3 – 55.
Burgess, J. W. 1983. Interpersonal spacing behavior between
surrounding nearest neighbors reflects both familiarity and environmental density. Ethology and Sociobiology 4(1):11–17.
Cohn, A. G., and Hazarika, S. M. 2001. Qualitative spatial
representation and reasoning: An overview. Fundamenta Informaticae 46(1-2):1–29.
Cohn, A. G.; Bennett, B.; Gooday, J. M.; and Gotts, N. 1997.
RCC: A calculus for region based qualitative spatial reasoning. GeoInformatica 1:275–316.
Ding, X. C.; Smith, S. L.; Belta, C.; and Rus, D. 2011. LTL
control in uncertain environments with probabilistic satisfaction guarantees. arXiv:1104.1159v2 [math.OC]. Technical
Report accompanying IFAC 2011.
Dubba, K. S. R.; Cohn, A. G.; and Hogg, D. C. 2010. Event
model learning from complex videos using ilp. In Coelho, H.;
Studer, R.; and Wooldridge, M., eds., 19th European Conference on AI, volume 215, 93–98. IOS Press.
Dylla, F.; Mossakowski, T.; Schneider, T.; and Wolter, D.
2013. Algebraic properties of qualitative spatio-temporal calculi. In Conference On Spatial Information Theory (COSIT13). North Yorkshire, UK: Springer.
Dylla, F.; Coors, M.; and Bhatt, M. 2012. Socially compliant
navigation in crowded environments. In Spatial and Temporal
Dynamics Workshop at ECAI 2012.
Freksa, C., and Röhrig, R. 1993. Dimensions of qualitative spatial reasoning. In Carreté, N. P., and Singh,
M. G., eds., Proceedings of the III IMACS International Workshop on Qualitative Reasoning and Decision Technologies –
QUARDET’93, 483–492. CIMNE Barcelona.
Frith, C. D., and Frith, U. 2006. How we predict what other
people are going to do. Brain Research 1079(1):36–46.
Galton, A. 2010. The formalities of affordance. In Spatial
and Temporal Dynamics Workshop at ECAI 2010, 1–6.
Hall, E. T.; Birdwhistell, R. L.; Bock, B.; Bohannan, P.;
Diebold, A. Richard, J.; Durbin, M.; Edmonson, M. S.; Fischer, J. L.; Hymes, D.; Kimball, S. T.; Barre, W. L.; Lynch,
Frank, S. J.; McClellan, J. E.; Marshall, D. S.; Milner, G. B.;
Sarles, H. B.; Trager, G. L.; and Vayda, A. P. 1968. Proxemics
[and comments and replies]. Current Anthropology 9(2/3):pp.
83–108.
Hall, E. 1966. The Hidden Dimension, Man’s Use of Space
in Public and Private. The Bodley Head Ltd.
Kendon, A. 1990. Conducting Interaction: Patterns of Behavior in Focused Encounters. Studies in interactional sociolinguistics. Cambridge University Press.
41