Industrial Leadership: Research & Innovation Actions
ICT 2014 - Information and Communications Technologies
H2020-ICT-2014-1
AutoMat: Autonomous Material Handling
Acronym: AutoMat
Date of Preparation: March 24, 2014
Work program topic addressed by AutoMat: Robotics
Coordinator: Libor Preučil
e-mail: preucil@labe.felk.cvut.cz
tel/fax: +420 224 357 290/224
#
Participant organisation name
Short name
Country
1
2
Czech Technical University in Prague
Institut National de Recherche en Informatique et en
Automatique FR
Aalto Korkeakoulusaatio
Almende B.V.
University of Lincoln
VOP CZ, s.p.
CTU
INRIA
CZ
FR
AALTO
ALMENDE
UoL
VOP
FI
NL
UK
CZ
3
4
5
6
AutoMat
page 2 of 37
page 2 of 37
AutoMat
Abstract
The project AutoMat targets research and development of scientific and technical principles focusing applications of
ad vanced auto mation and aut ono my i n th e do mai n of const ru ction machin es. Primary aim of the there under
suggested project is to r e s e a r c h a n d develop methods leading to practically usable and marketable unmanned systems.
The ultimate goal is to provide these with the necessary capabilities to propose and f u l f i l l autonomous modifications to its
workspace in material handling, construction and landscape forming tasks not for large devices only, but specifically for smaller
machines not addressed previously, and which are diverse in size and in the art of their use.. The keypoint herein stands in expected
substantial increase of the use efficiency of robotics methods with construction machines in accomplishing the required
tasks autonomously, making these adaptive, autonomous and independent on environmental infrastructure than ever
before, as well as delivering a scalable and bringing up a truly functional solutions. This will be achieved hand in
hand with creating a technological demonstrator at TRL8 in the project runtime followed by a real product in the
domain of small and inexpensive DAPPER construction machine, that will easily penetrate the European market.
The project focuses three core scientific challenges, are based on advices from, and expertise of construction machine
manufacturers, their users and i n i t i a l m a r k e t s t u d i e , w h a t d enotes the key enabling technologies
needed for application of autonomy and robotic technologies in construction machines domain:
(1) Autonomous modification of the given machine activity plan, including proactive robot-habitat interaction towards
improvement of the robots’ mo bi li t y and e n v i r o n m e n t maintenance, and further extension of capabilities of active
workspace modification to facilitate the robots’ localization and mapping (a concept we refer to as “Bulldozer SLAM”).
(2) Introduction of p erformance op ti mizat ion and safety into the machine controller learning. The proposed approach
is inspired by ontogenesis and enables expertise transfer among various types and sizes of, even coo p erat i n g ,
co n s t ru ct i on mach i n es , to assure their optimal safety and performace.
(3) 3D mapping and semantic segmentation of the highly dynamic 3D workspace providing techniques necessary to
consistently represent the environment over time, regardless of the changes imposed by the system itself, and to further
facilitate planning and cognitive control of the robots towards achieving the tasks of the target application.
The project aims at resolution of the principal and outstanding problems achieving highest possible robot autonomy in real,
uncertain and highly structured environments, with specific focus onto the project outcome application in the product. The
project’s achievements will be highlighted in a real-world experiment s c e n a r io , w h e r e a s t h e e x p e c te d im p a c t w i l l
s h o w u p in m a r k e t d i s s e m in a t i o n o f th e e l a b o r a t e d a u t o n o m y s o lu t io n s f o r s m a l l - s c a le c o n s t r u c t io n
m a c h in e s .
Contents
2
2
AutoMat
page 3 of 37
page 3 of 37
AutoMat
B.1 Excellence
3
B.1.1 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
B.1.1.1 Target application and verification scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
B.1.1.2 Scientific challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5
B.1.1.3 Introducing safety into the controller learnin g . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
B.1.2 Relation to the work programme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
B.1.3 Concept and approach
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
B.1.4 Ambition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
12
B.1.4.1 Perception and Distributed 3D mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13
B.1.4.2 Control of learning and operating risks
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
B.1.4.3 “Bulldozer SLAM”- System for Simultaneous Localization and Active Mapping with Environment
Modification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15
B.1.4.4 Autonomous material handling
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
B.1.4.5 Relation to other EU projects
B.2 Impact
20
B.3 Implementation
21
B.3.1 Work plan Work packages, deliverables and milestones . . . . . . . . . . . . . . . . . . . . . . . . .
21
B.3.1.1 Work package list and Summary of effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
B.3.1.2 List of Deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
22
3
3
AutoMat
page 4 of 37
page 4 of 37
AutoMat
Chapter B.1
Excellence
B.1.1
Objectives
A system for an autonomous manipulation with material in waste repositories and bulk materials storages is proposed
in this project called AutoMat (Autonomous Material Handling). The project aims at development of autonomous functionalities for robotic construction machines with focus on specific demands of end-users of these devices. Even thought,
alike solutions have already been introduced with large construction machines before (Catepillar, Komatsu, etc.), these approaches targeted
mainly dully automation of certain repetitive activities. As good examples may be i.e. automated transportation of ore and spoil in mines, or base
and causeway forming for highway construction sites, etc. The major and the most common constraints in these use cases are requirements for
well established infrastructure as i.e. precise (D)GPS availability, many assumptions on the workspace occupancy (typically a single-machine
operation only), machine activity following pre-programmed and rigid plan, all leading to a common de-numerator of these approaches – weak or
very limited capability to handle uncertainty through the given task accomplishment. Besides, the aforementioned solutions are offered
exclusively with very large and costly devices, what draws back the possibilities to further wider dissemination of these applications.
Therefore, to compensate for the previous drawbacks, a primary objective herein is to reach a higher usability of the
autonomy in these applications, which remains necessary for the system commercialization. The key idea is, the project
addresses research o f u s a b l e principles for autonomous manipulation with material according to humandesigned mission objectives, being capable to operate robots in varying scenes, under uncertainty and even more –
taking the advantage of intentional physical modifying of the scene towards improvement of the autonomous
operation. The herein presented approach counts with the requirements and constraints imposed by i n d i v i d u a l
r o b o t i c s y s t e m s , a n d e v e n f l e e t s o f i n v o l v e d r o b o t s , and optimizes coupling of each component of the problem to
the others (the workspace shaping and the robot operational capabilities). Usage of machine learning principles denotes a powerful tool to
achieve the optimization across various types of robotic construction machines, in diverse situations and tasks. Such an approach
of intentional active adaptation of the workspace by robots for their needs, and needs of the task given, stands for the
essence of the herein suggested advance towards efficient deployment of the robotic system in unstructured real world
workspace.
“The project AutoMat is devoted to development of a robotic system capable of cognitive material handling to achieve
the given task and to increase the robotic construction machines autonomy and performance.”
Particular requirements for the effective utilization of t h e s e robots are identified by the robotic system itself during
the mission accomplishment. As a simple example: the system may find out that a new road going from a
material storage would speed up the material displacement, what may invoke building of this road. Such, or similar
cognitive judgments are essential as the useful and important properties of the workspace can rarely be
specified
prior to the mission. These are dependent upon the characteristics of the workspace and can be obtained during the
robot deployment in the particular scenario first. The ability to handle all the aspects of the autonomous system
activity in real-world environment i s crucial for its deployment without necessity of presence and advice of robotic
experts. As the latter stands for one of the main current bottlenecks of commercial application of autonomous
construction machines for material handling and manipulation tasks, it has been selected as the main targeting of the
activities within the AutoMat project. Moreover, to reflect the needs of the future exploitation activities concerning
this technology, all the solutions will target the domain of small size construction machine in tight cooperation with
its’ manufacturer. The expectation is, the systems’ moderate price and wide variety of possible applications will affect
the dissemination activities and final impact in a positive way.
4
4
AutoMat
B.1.1.1
page 5 of 37
page 5 of 37
AutoMat
Target application and verification scenario
To be more specific, let us mention three scenarios, where we want to advance the development of recent technologies
for autonomous driving and operation of mobile machines to b r i n g t h es e t o sufficient technology readiness level for their
commercialization. These three scenarios include most of the typical functionalities demanded by end-users o f
s m a l l -size construction machines (see the attached market study), which has become one of the key products in the
VOP partner portfolio.
Scenario 1: The waste repository.
In the first scenario, a metropolitan waste repository area, which consists of a sealed ground pool on which the waste is
stored, is considered. To reuse efficiently the area, the waste is piled up into a hill or stored into layers for further reclamation. Fetching the new coming waste, the loading machines (in our scenario autonomous loaders and bulldozers)
shall drive up the hill and store the waist in a proper place. As the environment (the waste pile) rapidly changes with
the amount of stored waste over time, the machines have to build/re-build their environmental models and temporal
roads to the storage area (typically up the hill) to assure delivery of new waste. Similarly, a request for steady shaping
the waste hill to the requirements on the repository exists, as well as a request to cover the waste by a layer of regular
soil to reclaim the environment may arise. Again, further safe access roads for these activities are needed to be built
for
temporal
use.
5
5
AutoMat
page 6 of 37
page 6 of 37
AutoMat
The challenging task in this scenario is the autonomous control of robots in changing workspace (the workspace is
changed by the robotic systems itself). In addition, the autonomous system has to be capable of autonomous planning
of workspace modifications (to build the temporary access roads) with the aim of accomplishing the given plan (to
store the waste into hills or layers for the reclamation). These changes depends on the properties of the environment
(various types of waste etc.) and therefore cannot be planned prior the system deployment.
Scenario 2: Bulk materials storage and manipulation.
In the second scenario, a bulk material storage area (i.e. construction material retailer, or a factory using bulk raw
materials as inputs for their production) as sand, gravel or similar repositories is considered. The task for the system
is to keep the storage in terms of delivering required type and amount of the material to certain unloading area (i.e. to
load a truck) while not mixing these and keeping the storage clean and tidy. In the case of delivery of materials to be
stored, the loader shall fetch the material to a proper place of the storage (including carrying and unloading the delivery
on the top of the storage pile). As for The waste repository, the loaders shall build the environmental model, suggest
certain temporal changes (building temporal roads) to fulfill the fetch and carry tasks, etc. Specifically, it is assumed
the whole scenario may be executed in GPS-dark areas (inside a production hall, under metal roofing, etc.) so the
GNSS (i.e. GSP) systems cannot be relied on.
Scenario 3: Landscape reclamation activities.
The third scenario dealing with landscape reclamation is the most challenging one, but it brings also the biggest
possible income in future. Nowadays, there are many lands in technologically advanced as well as developing countries
that are burdened by the legacy of industrial activities like exhausted coal mines, dumps, or old and unused industrial
complexes. Across Europe more than 300,000 locations are waiting for the reclamation (see section ?? for details).
The proposed robotic landscape reclamation is a tool that can be used to recultivate such lands and to return them back
to the nature or to make them usable and comfortable for humans. The reclamation does not include only the initial
workspace restoration, but also a continuous long-term maintenance to provide the sustainable environment growth.
Landscape reclamation activities typically focus shaping of the land combined with soil redistribution. This typically
comprises removal of tailing materials followed by targeted shaping of the underlying layers and spreading of fresh
quality soils, sand, or peat, etc. which are bulk materials. For the reclamation performed in purely natural conditions
(i.e. in woods) the soil forming machines cannot rely purely on GNSS positioning as shielding and dispersion of the RF
signals (i.e. by tree canopies) decreases precision to unusable levels for this purpose. To assure sufficient precision
of the land forming actions and to guarantee fine localization of the machine (herein autonomous robot) relatively
to the ground it is useful to rely on existing landmarks on the processed ground. Artificial building of some other
additional/temporal landmarks during the machine work time (the previously mentioned bulldozer SLAM) will enable
and improve the machine activity efficiency towards the given goal. Besides, creation of temporary roads to fetch and
carry processed materials, remove tail materials and bring new ones seems to be inseparable part of this scenario as
well.
For testing the Scenario 1, a bordered workspace will be prepared to enable experimental verification of all scientific
challenges being tackled in the AutoMat project. In the environment, areas with heterogeneous surface (sand, gravel,
small stones, bigger stones...; see Fig. ?? for example) will be created. A testing plan of storing of construction waste
(crushed bricks) will be given to the system by human supervisors. The surface of the selected part of the industrial
waste dump will be appropriately modified to highlight the key functionalities of the system, which is the cognition and
adaptation to environment changes and the safe machine learning. The aim of the experiments, which is the verification
of developed methodologies, does not require utilization of real size excavators and smaller mobile robots of particular
partners may be used in this phase of testing. Therefore, the experimental site needs to be suited for utilization of
available robotic platforms. The robots workspace will be adapted prior the autonomous system deployment to enable
movement of employed small-size robots.
The robotic system equipped with necessary actuators for the material carrying and handling (shoveling) and sensors
for outdoor sensing will be capable of the plan execution and its modification towards better performance of robots.
Based on the evaluation of the short experimental trials and information obtained from the sensors, the system can
design new auxiliary roads to simplify movement of robots. The road building will be demonstrated by shoveling of the
sand from a harder surface. This action can be realized with small size robots and it provides measurable improvement
of the robot’s movement. Besides, the system can propose a creation of new localization landmarks (e.g. a small
sand hill or a group of stones) in locations where the localization system suffers from high uncertainty (the proposed
Bulldozer SLAM).
6
6
AutoMat
page 7 of 37
page 7 of 37
AutoMat
Thanks to the Destro company, being part of the AutoMat advisory board, permission to realize the autonomous wast
7
7
AutoMat
page 8 of 37
page 8 of 37
(a) Outdoor bulk materials storage.
AutoMat
(b) Indoor bulk materials storage.
(c) The industrial waste repository - overview.
(d) The industrial waste repository - detail.
Figure B.1.1: Examples of workspaces used for final demonstration (scenario 1 and scenario 2). Source: Destro and
VOP companies.
handling task in the workspace of industrial waste dumps has been obtained. The provided area is large enough to
prepare various real-world experiments in such a scale, which corresponds to the size of the utilized experimental
robots (see Fig. ?? for examples of available robotic platforms of university partners). Besides, the consortium may
take the advantage of various environments presented in the area (see Fig. ?? for examples). The experimental site
is part of the protected areas of the Destro company with limited access, which makes the experiments reasonable
and with minimal costs (there will be no charge required by the Destro company for these services).
The second scenario will be tested at the test-field of VOP partner, built for this purpose and using its small-size
construction machines (DAPPER). The purpose of this endeavour is t o c r e a t e c o n d i t i o n s f o r transfer of the
technology and skills gathered previously with the smaller laboratory robots o nto th e ta rg et D AP P E R product.
Afterwards, a selected set of particular functionalities creating autonomy of the controlled machine, specifically imposed by
material storage cases and material manipulation scenarios, will be encapsulated and provided to p r o s p e c t i v e f u t u r e
u s e r s o f t h e s ys t e m – D A P P E R customers. In addition, the complete integrated system will be tested in its final
configuration and for the expected range of environmental conditions for use. The aim of this experimentation is to
verify, that completed software system together with the VOP’s product and given use scenarios has qualified for
commercialization. The final system is expected to satisfy safety conditions and to achieve reliability and performance
required by TRL8 standard.
B.1.1.2 Scientific challenges
8
8
AutoMat
page 9 of 37
page 9 of 37
AutoMat
In addition to the technological challenges required by the scenarios motivated by typical applications of smallsize construction machines, three main scientific challenges arise from the proposed concept: The first challenge
is the above mentioned possibility of the autonomous modification of the given material handling plan,
which is the key idea of the project. This application requires to equip the employed robots with a set
9
9
AutoMat
page 10 of 37
page 10 of 37
AutoMat
of skills for material handling, whereas two m a j o r bottlenecks were identified across roboticists and scientists and
manufacturers of construction machines communities:
• The real-world material handling at construction sites and waste repositories may involve a large fleet of
various machines with diverse end-effectors (attached tools).
• Development of controllers, their tuning for the task , robustness and safety tests, etc. directly with the large and
heavy machines may become too risky and dangerous for the workspace as well as for the u s e d robotic
machines themselves.
Therefore, the project aims at development of an approach, which introduces setup procedures and safety into the
controller learning stepwise with smaller and safer robots, and provides experience/skills transfer between
different robotic platforms subsequently.
The third scientific challenge is related to perception and representations for distributed mapping of dynamic 3D
workspace. The material handling in the above mentioned applications focus the following aspects of perception
and mapping:
• The robotic system itself can intentionally and markedly change the workspace, what forms the outdoor mapping
problem highly dynamic.
• A semantic segmentation of the 3D map is required to efficiently realize the material handling task.
• Various construction machines may change and perceive the workspace simultaneously.
All these afore listed challenges pave the research road far beyond the current state-of-the-art in robotics and machine
learning, and t h e a t t a i n e d s o l u t i o n s w i l l significantly extend the existing methodologies.
Proactive robot-environment interaction
The h e r e i n a d d r e s s e d robotic systems purposely modify its workspace to enable more efficient deployment for
the material handling, either via hierarchical segmentation of the material manipulation task into multiple levels, or by
building dedicated structures, navigation landmarks to adjust the workspace to its’ needs. In that aspect, certain
preliminary changes/adjustments of the workspace may help to fulfill much more demanding tasks consequently as:
• Autonomously proposed m o d i f i c a t i o n s t o
the workspace enable/simplify operations of robots
• The plan for material handling is purposely modified to couple better to the robots’ needs (constraints,
intentions)
Let us refer to the possibility of autonomous modification of the given material handling plan towards improving the
operational performance of robots as “proactive robot-environment interaction” in this project. This idea goes beyond
the regular understanding of autonomous systems operation in outdoor workspace – classically trying to
accommodate the robot to the environment. Here, the workspace is not considered as a passive invariant but it
serves as a vehicle that facilitates accomplishment of the desired task by robots. The robots have the c a p ability to
suggest changes to the workspace that are not explicitly included in the desired task. The modifications a r e
suggested t o influence the desired task of the robots in positive ways. These can be by increasing the efficiency of
the robots’ deployment, or by removing existing constraints or obstacles to accomplish successfully the task at all.
This brings intelligent robotics yet another step closer to human thinking.
In common scenarios, human actors tend prepare their working site to facilitate their foreseen activities. Construction
workers build scaffolding, auxiliary footpaths and other auxiliary structures even if these are not directly included in
their tasks. Besides, the human actors propose improvements based on their previous experience gained in similar
tasks before. We propose to design an intelligent robotic system with similar abilities in the AutoMat project.
In the material handling, the system is expected to suggest changes of the early plan for the given task during
its’ execution, r e s p o n d i n g t o v a r i a t i o n s o f t h e e n v i r o n m e n t s h a p e , p r o p e r t i e s , e t c . to enable effective
10
10
AutoMat
page 11 of 37
page 11 of 37
AutoMat
completion of the given task. T h e r e f o r e , t h e robots may on the fly propose new roads to optimize their motions, new
landmarks to serve their better localization and many further aspects.
All the afore represents a set of important steps towards effective and usable autonomy, that allows
commercialization of the autonomous system in question in broad field of construction works, material manipulation
and reclamation activities applications. Through the obtained level of autonomy a given plan may be adjusted for
realization by robots in a way as it would be done by robotic experts, if present.
In the proposed concept, the improvements to the current state-of-the-art in respective fields will be targeted using a
series of short autonomously designed partial and simplified attempts to solutions evaluated i n test trials.
Nevertheless, the beauty of segmentation of the target problem is, robots change the surrounding workspace i n a
m o d e r a t e w a y o n l y in these separate trials. T h a t a l l o w s t h e m t o measure and evaluate performance
in the key aspects of localization, motion performance, possibility to move the material, etc.
Based on
11
11
AutoMat
page 12 of 37
page 12 of 37
AutoMat
the obtained rewards, the successfully evaluated trials/actions are subject for further evolution and generalization. The
generalized actions are integrated into the proposed large-scale changes of the material handling plan.
An example of such a proactive robot-environment interaction might be an effort to adjust the workspace in order to
improve the localization of robots. This leads to a technique that we call “Bulldozer SLAM”.
Bulldozer SLAM
The robotic system autonomously adapts the given plan of the material handling to achieve more efficient and precise
localization of the robots in the workspace over time.
• The workspace is naturally modified to decrease the localization uncertainty and increase the map quality/precision.
• The robots learn the environmental changes that positively affect the localization uncertainty.
• The learning is realized during the robots’ deployment executing particular scenario.
The possibility of the workspace modifications towards better localization of robots exceeds the state-of-the-art of
SLAM approaches. Let us sort the SLAM methods according the process of the mapping and localization as follows:
(Passive) SLAM → Active SLAM → Bulldozer SLAM
The traditional SLAM approaches “passively” observe the working environment with the aim to determine robot position
and they build a model of the workspace, while the robot is controlled through some other means. The Active SLAM
techniques (also called SPLAM - Simultaneous Planning, Localization and Mapping) control actively the robot in order
to best minimal localization error, e.g. by avoiding places providing poor contribution to localization. The Bulldozer SLAM
goes even beyond as it controls not only the robot’s motion, but also other robot’s actuators leading to workspace
modifications.
There are robotic systems, that build artificial landmarks in the areas where the information from the obtained map is
not sufficient for the localization. These artificial landmarks can even be purposely built with the only aim for the
localization improvement. In the proposed Bulldozer SLAM, the landmarks arise as a side product of the material
handling. O f t e n , e ven a slight change of shapes of objects in the workspace may significantly decrease the
uncertainty of the localization with a minimal effect to the targeting of the given task.
B.1.1.3
Introducing safety into the controller learning
The active modification of the workspace by autonomous heavy Bulldozers and construction robots introduces a high
risk of damage of the robots, or hardly reversible and unwanted changes to the surrounding workspace, etc. More
generally, a key challenge for autonomous cognitive robots is to acquire the desired skills and thereafter
operate without bringing into danger themselves, or others, or the workspace.
Thus, dangerous situations must be thoroughly prevented during the robot training, and avoided during the course of
the robot missions. Two complementary approaches will be designed and deployed to address the safe learning and
operating challenges. The first one is inspired from the biological developmental process. The second one, detailed
below, proceeds by transfer of expertise between different robotic platforms.
B.1.1.3.1
Developmental Learning
The developmental process is remotely inspired by the learning of motion primitives by infants. The idea is that natural
limitations, e.g. their limited strength, prevent infants from endangering their body when exploring their sensori-motor
abilities. As children grow, their musculoskeletal system evolves and becomes stronger. Adults, who can operate with
full strength, are able to seriously injure themselves; but they are assumedly skilled enough to avoid injuries with high
probability.
This analogy leads to define a robotic developmental process as follows. The developmental learning will be implemented as the actuator ranges, initially very limited, are gradually increased along the learning process. The primary
12
12
AutoMat
page 13 of 37
page 13 of 37
AutoMat
stage relies on using randomly generated actions with very limited intensity, albeit sufficient to build some forward
models. This stage will be used to build a reward function. The progress beyond the state of the art, e.g. the design of
13
13
AutoMat
page 14 of 37
page 14 of 37
AutoMat
intrinsic, curiosity-driven motivations, relies on the use of the scarce feedback from the human teacher (“it’s better; it’s
not better”). These rewards will be used to determine when and how the robot actuator range can be extended, akin to
the process of musculoskeletal growth. Then, the intensity of the robot actions will increase as its skills improve.
The process will be iterated until the robot is able to deploy the desired skills with full strength.
In summary, the natural growth process is emulated by controlling and gradually extending the actuator ranges. The
teacher’s feedback allows the robot to simultaneously shape its exploration of its sensori-motor abilities (what is allowed) and to improve its skills in situation (what is appropriate).
Expertise transfer among various robotic platforms
Along the same line, the musculoskeletal growth of the robot can be emulated by considering different robotic platforms,
with increased motor abilities. The growth of robot bodies and “muscles” can be realized by a utilization of bigger
and more powerful vehicles. The proposed approach is inspired by the success story of NASA’s Mars exploration by
bigger and bigger mobile platforms. Taking the knowledge and experience obtained by several generations of vehicles
experimentally employed in the task of Mars exploration, the scientists developed a robust large-scale machine.
The progress beyond the state of the art is to achieve an automatic transfer of expertise among robotic systems,
combining the fast and cheap acquisition of primary skills on usually smaller and inexpensive robotic platforms, and transferring these skills to more powerful robots thereafter. The mechanism of expertise transfer is
the second scientific challenge related to the machine learning in AutoMat. It tackles the following objective:
• To achieve a skill (policy) transfer between different robotic platforms.
The experience transfer between different robotic platforms of the team is useful in general, not only for the robot’s
“growth”. Let consider a situation in which a robot learns to move in a sandy part of the site. By the short experimental
trials, it discovers that stopping in this area may be dangerous, slowing down can cause unwanted wheel slippage, fast
turning may cause skidding, etc. Another robot with a smaller size can combine this experience with its own knowledge.
It can learn that sandy areas have to be passed quickly even without an experience from such a kind of area before.
Without this knowledge, visiting this area could be fatal for the smaller robot. Contrariwise, a bigger robot can learn
that it is better to go slower to avoid the skidding that could cause serious collisions and so on.
This cognitive approach is crucial for robust utilization of robots in the afore mentioned application scenarios. It
simplifies installation of the autonomous system on the site of a new customer. Besides, the cognitive approach
allows adaptation of the system and enables to deal with unforeseen events and slight variations of t h e given task.
This also reduces service costs and reduces time needed for adaptation of the system.
The utilization of the proposed machine learning techniques to obtain the controllers and skills for the material handling
is a very demanding process. As it is difficult to forecast if the required skills can be gained by these techniques in a
sufficient manner at the time when these are needed by depending research activities (e.g. the mentioned proactive
robot-environment interaction requires the learnt skills as one of its inputs), the learning procedures design remains a real
challenge. Nevertheless, the benefit of obtaining safe controller/skills learning and the skills transfer between
different platforms together is so attractive for end-users of the t e c h n o l o g y i n q u e s t i o n that the AutoMat
consortium has decided to spent part of the planned efforts on investigation of these methods. Also some manufacturers
of large construction, or agricultural machines (Liebherr Group, Zetor tractors - members of the AutoMat’s Scientific
Advisory Board), indicated the needfulness of a mechanism for controller/skills transfer as the key property of the
autonomous a c t i v i t i e s o f t h e i r p r o s p e c t i v e s o l u t i o n s i n i . e . material manipulation and agricultural
activities due to i n v o l v e m e n t o f e v e n a fleet of diverse machines.
AutoMat consortium has a long-term expertise in predictive control of outdoor robots in tasks of workspace maintenance (the Snow Shoveling project at CTU [100, 99, 98, 47] or the Soil Moving at AALTO [41]). These control methods
of material pushing can be r e used for the preliminary phases of the project while the herein investigated novel machine
learning techniques will still be under development (and without any possibility of the skills transfer as proposed in this
section). Details on management of the skills learning activity can be found in section ??, Significant risks and
contingency plans.
B.1.1.3.2
Perception and representations for distributed mapping of dynamic 3D workspace
14
14
AutoMat
page 15 of 37
page 15 of 37
AutoMat
AutoMat addresses the problem, in which an autonomous system performs workspace modification task (material
handling) based on an a-priori plan. The plan and its’ goal in AutoMat describes a sequence of actions and the desired
high-level target state of the workspace. This plan is further conveyed to actions of robots by planning the sequential
tasks
that
have
to
be
performed
in
order
15
15
AutoMat
page 16 of 37
page 16 of 37
AutoMat
Figure B.1.2: Motivation for the learning of large and heavy construction machines through the expertise transfer among
various robotic platforms.
to reach the desired goal. A vital prerequisite for this process is t o enable recovery of the workspace state as a
spatial representation (or simply a map). Real-world workspaces are by nature 3D and therefore in order to truly
represent its state a 3D representation is needed. Moreover, by active modifying of the workspace the robots
continuously change the state of the workspace. Thus, a central property of the map should be that it can consistently
represent the workspace regardless of the changes.
16
16
AutoMat
page 17 of 37
page 17 of 37
Finally, AutoMat addresses also ouotlook to multiple robots working for a common goal.
modification
17
17
AutoMat
Generally, the workspace
AutoMat
page 18 of 37
page 18 of 37
AutoMat
application requires various construction machines and the proposed machine learning approach requires vehicles of
different sizes. It is important to point out, AutoMat project is not focused on the multi-robot cooperation and
coordination, but more than one robot can occur on work site at the same time solving either the same, or different tasks.
Imagine loaders transporting sand near to a s t o r a g e a r e a and our DAPPER machine is used for storing the
material at the dedicated place, i.e. into a heap. The loaders change part of the machine workspace which has to be
taken into account if planning tasks for DAPPER. The preceeding means that the workspace representation should
allow real-time updates by multiple robots/vehicles.
To sum up, in AutoMat we aim for a 3D model of the workspace, that is kept and updated in real-time, by
possibly multiple robots, or by a combination of robots and human-driven vehicles, T h e m o d e l i s
m a d e capable of representing the dynamics of the workspace consistently.
• Spatial representation for mapping in dynamic 3D workspace
We plan to build upon an existing spatial representation — the Normal Distribution Transform (NDT). The NDT was
originally developed in the context of 2D laser scan registration [17]. The central idea is to represent the observed
range points as a set of Gaussian probability distributions. NDT has later been extended to three dimensions in the
context of 3D r a n g e scan registration [64]. Prior work has shown that the NDT is an accurate [105] representation for
building of 3D maps, however, the primary use of the representation has so far been in modeling of a single (or
few) scan(s). In our recent contributions [96, 95], we presented an extension of the NDT framework to explicitly
model free space, introducing a novel 3D representation — the Normal Distribution Transform Occupancy Map (NDTOM). The proposed framework natively supports multi-resolution maps, as well as consistent probabilistic updates of
re-observed portions of the workspace. The resulting algorithm provides the only current solution capable of online,
real-time modeling of large scale, dynamic 3D workspace.
In AutoMat our key contributions will be to extend the NDT-OM to consistently represent different timescales
of dynamics in long-term operation.
• Distributed mapping in 3D
The second part of our main contribution is to enable efficient update of NDT-OM in the context of distributed
sensing. Having multiple robots observing the workspace using 3D sensors produces the need for communicating
the information between the entities. However, 3D sensors produce a vast amount of information, and to communicate
this information is not feasible. In AutoMat we propose to approach this by sending this information encoded as
normal distributions. We already demonstrated in [96, 95] that a NDT map can be built recursively by combining
arbitrary size sample sets. The challenge of this task is that low-resolution gaussian components cannot be fused
with high-resolution ones in a trivial manner. Thus, we propose to recreate the measurement from the low-resolution
representation by predicting the maximum likelihood measurement based on the model. This is a similar approach as
presented in [19], however, we aim to incorporate the learned model of dynamics into the sensor model. Using this
formulation the observations, coming from different sources and with different resolutions, can be consistently updated
to a global model.
• 3D Semantic Segmentation and geometrical/topological mapping
This task is to extract relevant structure from the acquired 3D maps by segmentation of the acquired 3D representations
into semantic categories including:
1) Landmarks (meaning distinctive 3D features that
mapping),
ensure
2) “Diggable” regions - places where the m a n i p u l a t e d
material can be loaded,
high-quality results in localization and
bulky
3) Paths already prepared for efficient movement of machinery,
4)Traversable terrain,
18
18
AutoMat
page 19 of 37
page 19 of 37
AutoMat
5) Other “obstacles”, etc.
The initial models used for the semantic segmentation will be acquired by a process of unsupervised learning based on
autonomously designed testing trials. Unsupervised learning of the 3D semantic segmentation will use techniques of
multi-scale grouping and categorization that are capable in detecting the courses and ridges and merging the basins in
19
19
AutoMat
page 20 of 37
page 20 of 37
AutoMat
the 3D maps and associate to the nested structures other physical classifiers to which the areas are being “diggable”,
“traversable”, “forming an obstacle” in terms of physical observables acquired by the robots. These models will be
continously refined based on the robot’s experience.
Algorithms for representation of the 3D map to obtain a meaningful model for the mission planning will be developed.
This enables to employ the map not only for the motion planning and obstacle avoidance, but also for a planning of the
given task (the material handling) and for visualization of the plans.
B.1.2
Relation to the work program
Let us explain how your proposal addresses the specific challenge and scope of the topic Robotics, as set out in the
work programme of ICT 2014 - Information and Communications Technologies.
Specific Challenge: Research implementing the Strategic Research Agenda established by the euRobotics AISBL
(the private partner in the future Public-Private partnership in Robotics) will be essential to attain a world-leading
position in the robotics market. Driven by the applications needs identified in this Strategic Research Agenda (SRA),
challenging R&D problems will have to be addressed, to make substantial progress in robots capabilities and improve
the Technology Readiness Levels (TRL) of robotics R&D. In addition, a dedicated effort is necessary to close the
innovation gap, allow large scale deployment of robots and foster market take-up. Robotics is very broad, both in terms
of technologies and disciplines it involves, but also in terms of markets and stakeholders. It is therefore essential to
address the inherent fragmentation.
In accordance with the specific challenge of this call, the project AutoMat focuses on reaching new levels of autonomy, adaptability and reliability f o r s m a l l m a ch i nes to be able to deploy the autonomous system in industrial
applications, d e l i v e r i n g t r u l y u s a b l e p r a c t i c a l s o l u t i o n t o a u t o n o m y o f a s m a l l
construction machine and to deliver this in a form of a product on the
E u r o p e a n m a r k e t . This, together with prototyping of particular autonomous skills required by end- users of
construction machines increases TRL in the broad application of autonomous handling and moving of bulky
materials. Nowadays, the complex autonomous systems for control of construction machinery are successfully
verified in laboratory conditions (achieving TRL 4). Another use area of these existing technologies is with special and
dedicated large construction machines, that require solid functional infrastructure (as i.e. GPS navigation), and runtime supervision on
expert level (detailed activity plan given, etc.), achieving higher TRL but being extremely rare due to its; complexity of use, price and
running costs. TODO: add references here The hereby proposed project aims at d e l i v e r y of a complete system
qualified through test and demonstration in an operational environment of a n industrial test polygon (i.e. a waste dump
provided by t h e V O P member of t h e c o n s o r t i u m , t h e m a n u f a c t u r e r o f s m a l l - c o n s t r u c t i o n m a c h i n e s , a n d
t h e p r i m a r y d i s s e m i n a t o r a n d u s e r o f t h e h e r e i n d e v e l o p e d a p p l i c a t i o n o f t h e t e c h n o l o g y. M o r e o v e r ,
Destro company, Zetor tractors, and other parties involved in the AutoMat advisory board will also turally have
principal access to the investigated technologies and therefore will foster their wider dissemination. The final
demonstration and the corresponding use scenario will be fulfilled to achieve TRL8 and prototyping the target
autonomous functionalities. Achievement of extraordinarily high l e v e l s o f TRL is allowed by utilization of DAPPER
s m a l l c o n s t r u c t i o n m a c h i n e - b e i n g a f r e s h a n d m a r k e t e d product of VOP - as the robotic platform for
final demonstration. Originally, DAPPER has been developed for manned operation, with outlook to future remote and
autonomous control modes, investigated and delivered in the framework of this project. Within the project, practical
implementation, use scenarios and related autonomous functionalities will be tailored directly for this machine
based on demands of the target users, market investigations as well as of the m a n u f a c t u r e r c o m p a n y
needs.
The aim of the project endeavour is to develop a cognitive autonomous system enabling robotic c o n s t r u c t i o n
m a c h i n e to operate in dynamic, worksite and real-world environments, reaching measurable improvements of
autonomy in material handling, interacting in a safe way with the work environment and humans.
The target solution of the AutoMat project will be prepared for commercialization as an extending supplement of
services and products offered together with the DAPPER construction machine b y VOP company. Relying on the
advances of the methods of intelligent mobile robotics developed in this project, the proposed system will resolve
situations not anticipated at t h e design time, that enables operation in nondeterministic, dynamic, real-life workspaces.
Moreover, a new level of autonomy will be reached due to the fact that the autonomous system may suggest
20
20
AutoMat
page 21 of 37
page 21 of 37
AutoMat
improvements to the given material handling plan with the aim to increase the efficiency, reliability and overall performance
of its act i n g in the workspace towards the given task accomplishment. T h e Cognition, in the sense of continuous
sensing of the workspace and consequent adaptive system behavior and response, is crucial to be able to act in
realistic,
nondeterministic
workspaces
as
the
afore
given
use
scenarios
definitely
are.
21
21
AutoMat
page 22 of 37
page 22 of 37
AutoMat
Furthermore, we propose to build a system, which is not purely specialized for the listed particular workspaces, but
w e o f f e r a b a s e k i t o f n o v e l m e t h o d s which may be autonomously adapted for other similar kinds of
workspaces. The robots’ workspace is dynamic in the sense of its significant modification, performed by the
autonomous system itself. Due to the large changes in the workspace, the c o m m o n state-of-the-art approaches of
the key robotic components for mapping, localization and navigation cannot be employed since the sensory
information from the workspace are not persistent. In line with the Call requirements and conditions, the project
focuses advanced robotics systems, given their potential to underpin the competitiveness of key manufacturing
sectors in Europe.
AutoMat covers multi-disciplinary R&D and innovation activities via the strong cooperation between two robotic
partners, two AI partners and two companies that act as the goal-givers and the driving force of technology
transfer via use-cases, the three market- oriented verification scenarios. The Pre-Commercial Procurement (PCP)
is built in through involvement of the VOP partner in the project consortium. This partner enables and contributes massively to
prototype development and therewith stimulates deployment of the final product. Therefore AutoMat addresses the key
abilities necessary for commercialization of autonomy in the tackled application scenarios that are driven by market
needs (the end-users specified requirements).
From the list of bottlenecks preclusive of the commercialization and crucial cognitive capabilities which are missing
today, AutoMat deals with advanced sensing, perception and sensory data understanding (WP1), learning
towards adaptation and functional improvements (WP3), reasoning and action execution at increased level of
autonomy (WP4), all followed by bringing the project outcomes to experimentally verified application scenarios and high TRL8
prototyping towards a product (WP5) The spatio-temporal cognition in real-world is exactly the scope of the WP1.
Besides, AutoMat is built on an emerging inter-disciplinary approach c r o s s b r e e d i n g i n d u s t r i a l a n d
o u t d o o r robotics with artificial intelligence, which enables to significantly extend the area of applicability of intelligent
robotic systems with massive impacts into real-life applications.
B.1.3
Concept and approach
As mentioned the AutoMat objectives and also the three application scenarios are driven by wishes of customers
of VOP partner. These scenarios, the waste repository, the bulk materials storage and the landscape reclamation
activities, are consider by the VOP company as their target applications, for which the Dapper product is designed.
Taking into account the current level of development of autonomous systems of mobile robots, a set of technological
challenges and three main scientific challenges were selected as key components necessary for reaching a level of
development necessary for deployment of autonomy into the construction machines. As mentioned nowadays, the
complex autonomous systems designed for construction machines are tested mostly in laboratory conditions (TRL 4
technology validated in lab) and some components and subsystems have been validated or demonstrated in a relevant
environment (TRL 5-6). The objective of AutoMat is to go from the laboratory condition to market and to design a
prototype of complete system qualified through demonstration in relevant environment (the industrial waste dump and
the bulk materials storage area) with middle size construction machines, the Dapper product of VOP partner (this
matches to TRL 7 system prototype demonstration in operational environment). In addition, a set of autonomous
functionalities required by VOP’s customers will be completed, qualified in the real conditions of the final experiments
and offered to end-users as an added value of the Dapper product (this activity will be realized in accordance with the
TRL 8 system complete and qualified).
TODO: All: Describe any national or international research and innovation activities which will be linked with the
project, especially where the outputs from these will feed into the project; Where relevant, describe how sex and/or
gender analysis is taken into account in the projects content.
B.1.4
Ambition
TODO: ALL partners: Where relevant, refer to products and services already available on the market. Please refer to
the results of any patent search carried out.
The AutoMat project aims to develop methodologies and techniques for an autonomous mobile robotic system capable
of material handling. The study of related literature, projects web-sites and product portfolio of key manufacturers
22
22
AutoMat
page 23 of 37
page 23 of 37
AutoMat
worldwide has shown the most important technological aspect and the key scientific objectives that need to be tackled
to achieve a state, which enables to offer such a system on market. Although the state-of-the-art techniques for
autonomous navigation, localization and mapping enable autonomous deployment of robots in large scale outdoor
workspace, they tend to fail/degrade in performance in the case of substantial changes of the robots’ workspace, which
is the case in the identified market oriented scenarios. Some of the existing techniques are well-suited for continuous
23
23
AutoMat
page 24 of 37
page 24 of 37
AutoMat
and slight changes caused by different weather and seasonal conditions, for example, but the extensive modification
of the workspace by the robotic systems itself requires completely new approaches for perception and distributed
mapping. The active modification of the workspace may not only be a handicap for the methods in intelligent robotics.
On the contrary, properly realized modifications of the workspace may increase the efficiency of employed systems, as
is demonstrated in the proposed “Bulldozer SLAM” approach, going beyond the state-of-the-art in existing active SLAM
techniques.
The second important aspect, which needs to be considered, is the future employment of large, heavy and expensive
machines for the tasks of material handling and workspace modification. Although state-of-the-art machine learning
techniques are frequently used for evolving the skills of autonomous vehicles, it is difficult to guarantee safety, mainly
in the beginning of the learning process. It is necessary to go beyond the state-of-the-art of machine learning methods
to facilitate learning of large robots that may be dangerous to their surroundings as well as to themselves.
The AutoMat consortium proposes to enforce the ideas of material handling by enhancing the state of the art with cognitive capabilities, relying on safeness in the machine learning, perception and distributed 3D mapping and Bulldozer
SLAM. These issues together with related past and ongoing projects are considered in more detail in the following
sections. All these aspects were chosen by the construction machine manufacturers (project partner VOP, member of
the AutoMat’s Scientific Advisory Board Liebherr Group) as the key elements necessary for deployment of autonomy
together with their products in the above mentioned industrial applications.
B.1.4.1
Perception and Distributed 3D mapping
Several approaches for 3D spatial modeling have been proposed and used successfully in robotic mapping systems.
Elevation maps are a 2.5D parametrization of space, obtained by associating a height value to cells organized in a 2D
grid [12]. However, due to the dimension reduction elevation maps can model only a single surface per cell, making
overhanging structures like e.g. different layers in the waste repository scenario impossible to represent correctly.
Several authors have proposed to extend the elevation grid approach by modeling space using Gaussian Processes
(GP). Assuming that each point in 2D space x = (px , py ) can be associated to a height valuef (x) = pz , the central
idea of GPs is to represent f (x) as a Gaussian Distribution in function space. The available sensor data is used in
order to learn the hyperparameters of a GP, which can then be used to perform regression for any point in 2D space
and obtain an interpolated height value, resulting in a continuous spatial model. Lang et al. [61], as well as Plagemann
et al. [82], employ a non-stationary kernel that is adapted explicitly in order to handle different surface types. In another
series of influential works, Vasudevan et al. [118] propose to use GPs with a non-stationary neural-network kernel for
modeling of large-scale outdoor terrains in the context of open pit mining. As far as we are aware, however, no GP
frameworks have been applied in the context of online mapping or in dynamic workspace.
Different approaches to 3D representation attempt to relax the functional constraint of a unique height value per location, placed by the models discussed so far. Triebel et al. [114] propose the Multi-Level Surface (MLS) map as an
extension to elevation grids which allows for multiple height values to be stored per cell. The proposed models have
been used to perform localization and navigation in an outdoor environment. More recently, Rivadeneyra and Campbell
[86] propose an extension of MLS maps that incorporates Bayesian probability fusion for terrain estimation – Probabilistic Multi-Level (PML) maps. Both MLS and PML maps, however, do not explicitly model free space or track the
consistency of the surfaces in the map and are thus only targeting static workspace.
Another approach that removes the assumption of a single height value per cell relies on a re-formulated GP framework [81]. The proposed technique, Gaussian Beam Processes, performs a regression on the range in a typical 2D
(and by simple extension also 3D) laser scan, thus assuming only a single value per sensor ray, in a manner consistent
with the physical properties of range scanners. Recently, Smith et al. [102] present a similar formulation in three dimensions that is centered around a push broom laser configuration. These GP frameworks can be used to fully represent
3D workspace and are tightly coupled with the principle of operation of 3D range sensors, but are not suitable for online
mapping and assume static workspace.
Triangle meshes are another method for spatial representation popular in the computer graphics community. Each
triangle represents a facet in a mesh, a graph of interconnected vertices. The abundance of research on triangle mesh
visualization, reconstruction, and collision checking offers a powerful incentive to utilize them in robotics. In order to
24
24
AutoMat
page 25 of 37
page 25 of 37
AutoMat
obtain the best reconstruction results from noisy point clouds, special care has to be taken in filtering and handling
of uncertainty. For example, Wiemann et al. [121] present an approach for extracting triangle meshes from noisy
25
25
AutoMat
page 26 of 37
page 26 of 37
AutoMat
registered point cloud models. The model extraction is however performed as a post-processing step and the obtained
triangle mesh is not maintained online.
Finally, recent advances [78] in dense mapping methods utilize a volumetric representation of space, using a 3D
sample grid. Each stored sample represents the value of a signed distance function (SDF), which stores the Euclidean
distance to the nearest occupied cell. SDF methods have been demonstrated to produce extremely detailed models in
small-scale office-sized workspace using fine resolution grids, but are difficult to extend to large-scale workspace.
The Normal Distribution Transform (NDT) is a compact spatial representation, originally introduced by Biber and
Strasser [17] in the context of 2D scan matching. NDT represents space as a set of Gaussian probability density
functions. NDT has later been extended to three dimensions in the context of 3D scan registration [64, 65, 106] and for
planning and traversability analysis in 3D [107].
In our recent contributions [96, 95], we presented an extension of the NDT framework to explicitly model free space,
introducing a novel 3D representation – the Normal Distribution Transform Occupancy Map (NDT-OM). The proposed
framework natively supports multi-resolution maps, as well as consistent probabilistic updates of re-observed portions
of the environment. The resulting algorithm was shown to outperform a 3D occupancy map [123] in performance and it
was shown to be the only current solution capable of online, real-time modeling of large scale, dynamic 3D workspace.
However, NDT-OM is not a model of a dynamic environment as understood in the dynamic mapping literature in 2D
[16, 93, 7, 74, 122, 73].
Progress beyond state-of-the-art: In AutoMat we propose to study a representation of the dynamic workspace.
We will build upon the previous contributions of the consortium [16, 95, 93] and extend the state-of-the-art in several
aspects: 1) we will develop a novel 3D representation, which models several time-scales; 2) the timescales will be
learned online during operation; 3) we will model the state transition parameters by tracking the changes in a local
neighbourhood; 4) we will formulate a distributed approach for map updates and 5) we will provide real-time performance. Combining our contributions to existing contributions on NDT maps will push the frontier of 3D perception in
mobile robotics significantly, enabling a whole range of applications.
B.1.4.2
Control of learning and operating risks
In reinforcement learning, quite specific approaches have been designed for robotics, mainly aimed at scalability
through providing expert’s demonstrations (inverse reinforcement learning [1] and learning by imitation [21]), aimed at
the robot safety during learning through reversibility-related criteria [75], or aimed a unified Bayesian framework to deal
with several tasks [113] including SLAM [39]. Transfer learning for reinforcement learning [110] focuses on facilitating
new skill acquisition through extending existing skills, related similar tasks or environments.
Besides the safety and mechanical hazard issues, learning in situ raises an additional difficulty, that of defining appropriate rewards guiding the robots toward the acquisition of the desired skills. Taking inspiration from active learning, an
intrinsic motivation-based approach has been proposed by [10, 11]. The idea is to provide the robot with an internal
(autonomous) reward, measured as the error decrease of the forward model. The robot, driven by its internal reward,
thus focuses on visiting the fraction of the sensori-motor space most amenable to increase the coverage and improve
the performance of the forward model.
Another simpler internal reward has been proposed by [26], based on measuring the entropy of the robot log. With
the goal of maximizing this entropy, the robot thus demonstrates “curiosity” in the sense that it thrives to visit as many
different sensori-motor patterns as possible, thereby maximizing the coverage of the forward model. A limitation of
internal rewards is that they can lead the robot to experiment unsafe behaviors. Another approach, referred to as
preference learning-based has been proposed to drive the robot toward desirable behaviors, through exploiting the
sparse preferences of the human teacher [4, 3].
A key scientific challenge tackled in AutoMat is to enable safe in-situ skill acquisition. This challenge will be tackled
in two complementary ways. The first one, remotely inspired from developmental learning, starts with a very limited
actuator range of the robot and achieves motor babbling [25]; the robot being in an “infant” state can do little but at
the same time has little chance to endanger itself or others. Gradually, the robot extends the actuator range (“muskuloskeletal growth”) as it acquires the desired skills. The critical issue is to exploit the scarce human teacher’s feedback
to acquire the criteria of danger (what should not be done) and appropriateness (what should be done).
26
26
AutoMat
page 27 of 37
page 27 of 37
AutoMat
The second approach pursues the same line of developmental learning through expertise transfer among different
robotic systems. Note that the state of the art in transfer learning for reinforcement learning [110] to our best knowledge
only considers the transfer among different tasks (formally, reward functions) or environments (state spaces). While
27
27
AutoMat
page 28 of 37
page 28 of 37
AutoMat
the transfer of a controller among two robotic platforms with same sensor systems can be formulated as an inverse
reinforcement learning problem [1], the transfer among two different robotic platforms raises new and critical issues.
Progress beyond state-of-the-art: Taking into account the robot safety besides the controller accuracy has emerged
as a strategic research topic in the last 3 years (see [76, 75] and references therein). However the characterization
of “safe controllers” proposed so far in the literature to our best knowledge is built-in and hardly general (e.g. relying
on the reversibility of the policy, which indeed makes sense for aerial vehicles). The proposed approach in opposition
is based on a most intuitive metaphor: an infant can experience the world while incurring less hazards, due to his/her
own actions, than a child.
The progress beyond the state of the art thus involves two milestones:
• Starting from a very limited but safe initial state (the “robot infant state”), the robot gradually gains power and
knowledge through limited interaction with the human teacher: the active preference-based reinforcement learning setting [4, 3] is adapted to build an internal model of accuracy and danger.
• The skill transfer among different robotic platforms with same sensor systems will be achieved using inverse
reinforcement learning; a key difficulty here is to propose an adequate representation of the search space.
B.1.4.3 “Bulldozer SLAM”- System for Simultaneous Localization and Active Mapping with Environment Modification
In order to achieve our objectives related to performance in the material handling tasks outlined in the validation
scenarios (flattening the surface to create a “road”, making a small hillock to store the material, and transportation the
material), we must develop mechanisms for the goal-directed control of robots that exploit the novel representations
and functionalities developed by the rest of the project. We use the term “cognitive control” to refer to processes that
generate behaviour in a robot based on a higher-level interpretation of the semantic content of its sensors (as opposed
to low-level feedback control).
Most frameworks for goal-directed behaviour generation in robots rely on either reactive or deliberative multi-level
planning frameworks, from the pioneering early work on mobile systems [20, 112] to state-of-the-art cognitive mobile
robots [43, 14, 109]. These systems typically reason about descriptions of space encoded in relational form (e.g.
region connectivity, object placement, human location), but all assume to be descriptions of a static world. In AutoMat
we will develop a novel cognitive control framework for leveraging the long-term experience of the robot in pursuit of its
task-level goals in a dynamic workspace, including workspace modification by the robot itself.
Autonomous exploration to complete static 2D [5] or 3D maps [27] has been well-studied. For those applications of
static mapping well-established measures of information gain have been proposed and successfully employed [52,
103]. Another strand of research in robotic mapping is concerned with the addition of semantic information, where recent approaches build hierarchical representations featuring objects, categorised locations, and rooms [55, 117]. Such
semantic information was exploited for the problem of efficient visual search for objects by combining semantic mapping
and efficient planning [9]. Related approaches have also been applied to the problem of terrain classification for outdoor mobile robots [63]. In AutoMat, we will develop novel approaches for map exploration in dynamic workspace based
on information-gain driven view-point recommendation for dense scene reconstruction during workspace modification.
Additionally we will exploit the semantics of the workspace, by semantic segmentation of the acquired 3D models into
categories such as “landmarks”, “diggable regions”, “paved paths”, “traversable terrain”, etc, which are meaningful
and useful to the robot in achieving its high-level goals. These categories can be considered as “affordances” which
suggest a particular action, and will be learned and adapted based on the robot’s own experience.
Some previous works in robotics have exploited the principle of stigmergy [49], a mechanism that allows the coordination of actions within the same agent or across different agents by means of traces left in the workspace. Beckers
et al. [13] developed a system where a group of mobile robots gathered a set of randomly distributed objects and
clustered them into a pile, and also compared the results to similar clusters observed in ant colonies. Johansson and
Saffiotti [50] introduced a stigmergetic approach to robot navigation in which the robot creates and exploits a navigation
map which is stored in an array of RFID tags embedded in the floor. Our approach is related to stigmergy-based approaches because the robots in AutoMat will effectively create their own “landmarks” by favouring actions that produce
salient 3D features which enhance the robustness and accuracy of 3D mapping and localisation.
28
28
AutoMat
page 29 of 37
page 29 of 37
AutoMat
Progress beyond state-of-the-art: In AutoMat we propose to develop novel approaches for cognitive control of mobile
robots enabling “Bulldozer SLAM” for workspace modification. We will thus create a system that merges continual
29
29
AutoMat
page 30 of 37
page 30 of 37
AutoMat
planning-based control (to achieve the material handling plans obtained through interaction with human supervisors),
path planning using the representations obtained through the above-mentioned perception and distributed 3D mapping
capabilities, and robust execution mechanisms leveraging the skills acquired as action primitives for achieving the
anticipated workspace modifications. The innovations of AutoMat beyond the state of the art will include the following
aspects:
• We will extend the previous work on map exploration by enabling a robot to understand and predict dynamic
changes in 3D maps during workspace modification. Through conducting its own experimental trials the robot
will learn to predict the effects of its actions in modifying the workspace, and further learn which actions are likely
to achieve the best performance.
• We will extend the previous work on autonomous mapping by developing novel approaches for map exploration
in dynamic workspace based on information-gain driven view-point recommendation during workspace modification, and also by exploiting the semantics of the workspace, through segmentation of the acquired 3D models
into categories such as landmarks, diggable regions, traversable terrain, etc.
• We will further extend the previous work on robot map building by incorporating self-modification of the workspace
to actively create useful landmarks for increasing map quality and reducing localisation errors and uncertainty
during online operation.
B.1.4.4 Autonomous material handling
Finaly, let us mention state of the art of systems of autonomous material handling and their level of development
regarding market applicability.
TODO AALTO: please add references regarding material handling SOTA (e.g. related to your AVANT machine)
TODO UoL: please add references regarding material handling in agriculture
Progress beyond state-of-the-art: TODO All
B.1.4.5
Relation to other EU projects
There are many ongoing or finished EU projects related to the research areas investigated in AutoMat. The number
of related projects is even higher due to the necessary inter-disciplinarity and involved technologies from different
scientific fields. However, the proposed project is mostly related to ICT and cognitive robotics. Selected ongoing (and
a few finished) EU projects are listed in Table B.1.1, where their main research areas (also related to AutoMat) are
described.
The table shows that projects dealing with autonomous robotic systems tend to focus rarely on learning activities and
only few of them are focused on adaptation areas (ASCENS, CoCoRo, PANDORA, SYMBRION and REPLICATOR).
We will describe these projects in detail later on. Besides, we have selected several EU projects (SYMBRION, REPLICATOR, PeLoTe, DARWIN, NIFTY, LIVCODE, LOCUST, EYE2, STRANDS) with strong links to the AutoMat consortium to summarize possible sources of knowledge and scientific achievements that will be exploited within the AutoMat
project. We should mention that mainly due to this strong foundation of scientific achievements on which the ambitious
research activities of the AutoMat project may be built, the AutoMat budget remains within such a reasonable size.
The mentioned projects also significantly contributed to the endeavour of relatively high Technology Readiness Level
of autonomous mobile robotic systems designed for outdoor applications. It allows us to bring the level of development
in this area close to commercialization within such a short period.
Let us start the detailed summary of the most related project with SYMBRION and REPLICATOR, which appeared in
both lists due to their high relevance as well as strong inter-connections with the AutoMat consortium.
Symbrion & SYMBRION Enlarged EU
The flagship project of the EU in the area of evolutionary robotics aimed to develop a modular system of multiple robots
with an ability to form complex organisms for solving challenging tasks. The key idea of the project was to simulate
the evolution of complex robots from simple cells on-board of robots in real time and to try out in hardware realization
30
30
AutoMat
page 31 of 37
page 31 of 37
AutoMat
only effective results from the simulations. The AutoMat consortium have direct access to the results achieved in
Symbrion since two partners participate in both consortiums. Particularly, CTU will build on its scientific results in
31
31
AutoMat
page 32 of 37
page 32 of 37
AutoMat
Table B.1.1: Related ongoing and past EU projects with indications of the research areas considered.
Acronym
ALLOW
ASCENS
Adaptation
Learning
×
×
×
×
IM-CLEVER
MASH
NIFTi
×
×
PELOTE
PROMETHEUS
RACE
REFLECT
REPLICATOR
×
×
×
ROSETTA
SCOVIS
×
SEARISE
SF
×
×
×
×
×
×
×
SFLY
SHOAL
SPARKII
SYMBRION
Sensors
HMI
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
HUMOUR
PANDORA
Autonomous
Arch.
×
×
EYESHOTS
FRONTS
Perception
×
×
CO3 AUVS
CoCoRo
Cognitive
(Machine)
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
×
areas of environment modelling and simulations of complex robots behaviour. INRIA will utilize its achievements in
the area of machine learning and active learning that are used for robots motion evolution in Symbrion, similarly as
in AutoMat, where the deployment of robotic system in demanding outdoor environments is tackled. In comparison
with AutoMat, Symbrion was project of basic research with low TRL and its final verification scenario was realized in
structured laboratory conditions.
Replicator
Robotic evolutionary self-programming and self-assembling organisms were developed in Replicator with a direct correlation with Symbrion. In Replicator, intelligent, reconfigurable and adaptable “carriers” of sensors (sensor networks)
were built, and algorithms and novel approaches for cooperation within multi-robot system were developed.
Three AutoMat partners were involved in the Replicator project. Almende may further exploit their achievements in
mission planning of the modular systems into the approach for planning of the landscape plan realization. INRIA takes
benefit of the knowledge of learning mechanisms for self-assembling of organisms. CTU may contribute with its experience of autonomous navigation in dynamic environments. Similarly as Symbrion, also the Replicator achieved low TRL
in comparison with AutoMat, which is strongly focused on commercialization of project achievements. Although the
AutoMat project brings new challenges of real-world out-door deployment and the gained knowledge in the Replicator
and Symbrion projects may be used only partly, the whole consortium may take benefits of these expertises and also
the planned effort in particular WPs may be significantly lower than if starting from scratch.
CoCoRo
The Collective Cognitive Robots (CoCoRo) project aims to create an autonomous swarm of interacting, cognitive
robots (a swarm of autonomous underwater vehicles - AUVs). Although the used robots are different from the robots
32
32
AutoMat
page 33 of 37
page 33 of 37
AutoMat
considered in AutoMat, both projects share similar ideas of environment mapping. CoCoRo researches the potential
of cognition-generating software, which is supported by a suitable hardware concept. CoCoRo aims to develop an
33
33
AutoMat
page 34 of 37
page 34 of 37
AutoMat
embodied and distributed system of AUVs using collective cognitive capabilities derived from animals (e.g., social
insect societies). Thus, the project differs from AutoMat in the idea of the considering swarm of identical simple robots
in CoCoRo, in AutoMat rather more complex entities and highly heterogeneous units are considered. And of course the
possibility of active environment modifications is not considered in CoCoRo. Nevertheless, research streams in both
projects will be coordinated to maximise the overall outcome.
PANDORA
The PANDORA project (Persistent Autonomy through Learning, Adaptation, Observation and Re-planning) aims to
develop new computational models for reducing frequency of assistance requests, i.e., to make the robots persistently
autonomous. Three directions are investigated in the project to achieve this goal. The first is description of the
world using new probabilistic semantic representations. The second is directing and adapting intentions towards plan
adaptation under resource constraints. The third direction is robust acting learning methods and robot control. Again,
Pandora project differs mainly in lower TRL and therefore challenges associated with commercialization of robotic
system are not solved there. However, project;s achievements will be monitored and considered in AutoMat, especially
towards reliable navigation as it is also desired to have the required data acquisition persistently autonomous.
PeLoTe The PeLoTe project (funded under FP5) was targeted on concepts of creating the presence feature via remote
control and navigation, as well as knowledge sharing in combined communities of living and nonliving entities. Besides
other particular goals, the project aimed to study navigation algorithms in challenging environments, processing data
from standard as well as unique sensors (photonic mixed device camera) and research methods of cooperation, collaboration, and communication within a multi-robot team during the exploration of an unknown environment. Moreover,
graphical user interfaces were developed allowing to share information and knowledge from diverse origins and of different nature. As CTU coordinated the project and AALTO participated in it, the achieved experience will be utilized in
the above mentioned areas. Finally, several experiments were performed in a training facility of fire-fighters that gave
the project team members the experience with performing test in non-standard environments, which is also the case for
AutoMat and the new consortium will build on these results. The obvious difference between both projects represents
absolutely different application and again much lower TRL in PeLoTe.
DARWIN and NIFTY
Human-robot interaction is the subject of projects DARWIN and NIFTY, in which CTU participates. In the DARWIN
projects, two key tasks are solved: efficient interaction between assembling robots and human operators; and the
development of a cognitive architecture for learning from the interaction. The dexterous robots employed in the project
have to perform an assembly task. At the beginning, the robots operate using a detailed plan prepared by humans
and use its cognitive architecture to learn results of the actions. Later, a high level plan is given to the robots, which
translate it to a detailed one using the learned skills. Here, the robots are learned using the interaction with the
humans and objects in the environment. While assembly robots are employed in the DARWIN project, the NIFTY
project focuses on tasks in Urban Search & Rescue, where human-robot teams work in dynamic environments. Teams
of mobile autonomous cognitive robots coordinate their action during missions with humans. The robots learn from
actions performed by humans and adapt to their behaviours. This helps them to operate autonomously and even
estimate behaviors of the humans, which improve efficiency of the cooperation. NIFTi investigates how a cognitive
robot can complement models of its own capabilities and situation awareness, with cognitive user models of task load
and workflow. Although the approach of the projects to the active existence of robots in environments and the targeting
scenario are different from AutoMat, the gained expertise will be used to design efficient ways for communication
between robots and human supervisors, and for creating a common model (language) for exchanging modifications of
plans of material handling. Thanks to these preliminary achievements, it was not necessary to plan an additional effort
in AutoMat for these particular tasks.
LIVCODE, LOCUST and EYE2E
The multi-sensor fusion required for the all weather sensing will be built on achievements and knowledge obtained
in the LOCUST project, in which UoL partner participates. In this project, a novel architecture for bio-inspired visual
perception system has been developed. These results from LOCUST will be partially utilized in AutoMat as a foundation
of the development of the novel approach to the SLAM problem.
For the safe navigation in the outdoor environment, a robust collision detection system is required. It can be implemented using various sensor systems, which are studied in the LIVCODE project (UoL participates). The method
34
34
AutoMat
page 35 of 37
page 35 of 37
AutoMat
developed in the LIVCODE project is inspired by animal vision systems and provides a robust solution for intelligent
mobile systems. To allow fast image-based collision avoidance, a dedicated hardware was developed in the EYE2E
35
35
AutoMat
page 36 of 37
page 36 of 37
AutoMat
project, where UoL participated. The collision detection is realized using a neural network system, which is implemented on a VLSI chip. This allows fast computation and adaptation of the collision detection systems. Moreover, such
a system is more energy efficient. In AutoMat, some preliminary knowledge from EYE2E will be exploited for inclusion
of vision into the complex sensory system for the outdoor experiments.
STRANDS
The IP project STRANDS ”Spatio-Temporal Representations and Activities for Cognitive Control in Long-Term Scenarios”, in which UoL participates, aims to enable robots to achieve robust and intelligent behaviour through adaptation
to, and the exploitation of, long-term experience. The project will develop novel approaches to extract quantitative and
qualitative spatio-temporal structure from sensor data gathered during autonomous operation. The project will also
develop control mechanisms which exploit these structures to yield adaptive behaviour in highly demanding, real-world
security and care scenarios. Although the STRANDS project is concerned with indoor environments, it is expected that
the parallel development of cognitive control and outdoor mapping techniques for the highly dynamic environments in
36
36