Introduction to OpenRTM
National Institute of Advanced Industrial Science and Technology
Intelligence Systems Research Institute
Noriaki Ando
Outline
•
•
•
•
Introduction
History
Features
Applications
2
What is RT?
• RT = Robot Technology
cf. IT
– Not only standalone robots, but also robotic elements
(sensors, actuators, etc….)
+ OpenRTM-aist
+
+
+
+
Open (source) RT-Middlware implemented by AIST
RT-Middleware
• RT-Middleware (RTM)
– Middleware and platform for RT-element integration
• RT-Component (RTC)
– Basic software unit in RT-Middleware
The aim of RT-Middleware
Problem Solving by Modularization
•
•
•
仕様の明確化
Cost
最新技術を容易に利用可能
誰でもロボットが作れる
Mobile base
Manipulator
Technical Issue
Needs
The state of
the art
Various users
Sensors・・・
Spec.
！
！
RT-Component
Reusable modules
Realize
low-cost robots
！
！
System developers
Utilize
the state of the art
Easy to customize
Satisfy
various needs
Realizing innovation in service robot market
4
RT-Middleware & RT-Components
Logic
・device control
・algorithms
・applications
etc…
RT
Component
Framework
RT
Component
Logic/algorithm with common interfaces = RT-Component （RTC）
RTC
RTC
RTC
RTC
RTC
RTC
RTC
RTC
RT-Middleware
Execution environment for RTC＝RT-Middleware（RTM）
※RTCs can be distributed across a network
5
History of OpenRTM-aist
6
Takumi (匠), Ume (梅)
from 1996～ in ETL (Electrotechnical Laboratory, former AIST)
匠 (Takumi = meister (de))
•Transputer based multi-agent
system for manipulation
– Language: C, Occam and Euslisp
– Network: Transputer (real-time)
梅 (Ume = plum (en))
•Ethernet based Takumi
– Language: C, C++, Euslisp
– Interface definition: original definition
language similar to Occam
– Network: TCP/IP
– OS: Solaris, Windows, VxWorks
7
Another Takumi?
Takumi
TUM
Here
Ramen (noodle) shop Takumi
Heßstr.71 (ecke Schleisheimerstr.)
80798 München
8
History of RT-Middleware
RTM0.1
release
from 2002～
OpenRTM-aist
0.2.0 release
OMG RTC
spec. draft
OpenRTM-aist
0.4.0 release
OpenRTM-aist
1.0 release
OMG RTC
spec. release
OMG RTC
spec. approved
FY
2002
2003
2004
2005
2006
2007
2008
2009
2010
RT-Middleware Project
Kaken PJ
Distributed component based simulator
Robot award 2007
Key technology for next gen. robot PJ
AIST standardization PJ
Universal design for environment and manipulation PJ
OpenRTM-aist is used in many
national projects for robotics
research as a standard platform
AIST UCROA PJ
Robot element development PJ
Intelligent RT Software Project
2011
RT-Middleware features
• Component-based software development
– Execution contexts, ports, configuration, etc…
– Inter-OS/language communication
• Standards
– Open specification
– Specification continuity
• Many vendors with RTM implementations
– Various PSM implementations
– Implementation continuity
10
Basic functions of an RTC
Data ports
Activity/Execution contexts
Common state machine
Inactive
Composite execution
Sensor RTC
Active
•
•
•
Data-centric port
Continuous data I/O
Dynamic connection
ex. servo
Position
Ref
Voltage
Contoller RTC
-
Error
Actuator RTC
Manage RTC’s lifecycle and execution
Service ports
Service-oriented port
Provides detailed function
–
–
–
Service Port
•
•
•
3D depth
data
Image
Data
Controller
Component
Actuator
Component
Configuration
ex. Stereo vision
Stereo Vision
Interface
Position
Data centric communication
Getting/setting parameters
Changing modes
etc…
・Mode setting
・Coordinate setting
・Calibration
・etc…
Kp
T Ds
Encorder
component
•
•
1
+
TI s
Stereo vision
Component
Parameter management
Configuration sets
Dynamic configuration
An RTC can have several
configuration sets.
Runtime reconfiguration
and dynamic switching
are supported
key
Set name
Set name
value
key
value
Data Port
Service oriented communication
11
Standardization in OMG
• OMG （Object Management Group）
– Software standardization organization
– Ex. UML, CORBA, DDS specifications
• MDA (Model Driven Architecture)
– Improves system life-time and executable variable (quality x cost x lifetime).
– The model is described in a PIM, and systems are implemented based
on PSMs that are mapped automatically (or semi-automatically) from
the PIM.
• PIM (Platform Independent Model)
– A model independent of any target platform (CORBA, EJB, SOAP, etc.).
The model is described in a modelling language such as UML.
• PSM (Platform Specific Model)
– A model converted from a PSM for each target platform
– CORBA PSM, SOAP PSM, etc.
12
OMG RTC Standard
formal/08-04-04
•
OMG RTC specification defines
– Basic component interface
– Execution semantics
– Introspection
•
•
•
OMG RTC specification inherits SDO (Super distributed Object)
Specification available online: http://www.omg.org/spec/RTC/
Version 1.1 due soon
pd RTC Packages
«profile»
RTC
SDOPackage
Execution Semantics
Introspection
(from External Models)
Lightw eight RTC
(from Robotic Technology Components)
13
Features of RTC
• Provides rich component lifecycle to enforce
state coherency among components
• Defines data structures for describing
components and other elements
• Supports fundamental design patterns
– Collaboration of fine-grained components tightly
coupled in time (e.g. Simulink)
– Stimulus response with finite state machines
– Dynamic composition of components collaborating
synchronously or asynchronously
14
OMG RTC families
Name
Vendor
Feature
OpenRTM-aist
AIST
C++, Python, Java
OpenRTM.NET
SEC
.NET(C#,VB,C++/CLI, F#, etc..)
miniRTC, microRTC
SEC
RTC for embedded systems using CAN
Dependable RTM
SEC/AIST
Certified RTM for safety-critical systems
RTC CANOpen
SIT, CIA
Based on CANOpen standard and RTC
standard
PALRO
Fujisoft
C++ PSM impl. for small humanoid robot
OPRoS
ETRI
Korean national project
GostaiRTC
GOSTAI,
THALES
Based on C++ PSM running on URBI
Implementation diversity of the same specification
• Guarantees continuity of products
• Makes reuse between implementations easier
15
Applications
• Applications
– Robots
– DAQ system
• Project
– Intelligent RT Software Project
16
Success stories
DAQ-Middleware: KEK/J-PARC
KEK: High Energy Accelerator Research Organization
J-PARC: Japan Proton Accelerator Research Complex
HRP-4: Kawada/AIST
TAIZOU: General Robotics Inc.
HIRO: Kawada/GRX
HRP-4C: Kawada/AIST
17
Japan Proton Accelerator Research Complex
（J-PARC,大強度陽子加速器施設）
物質・生命科学実験施設
Materials and Life Science
Experimental Facility
核変換施設
Nuclear
Transmutation
ハドロン実験施設
Hadron Beam Facility
500 m
ニュートリノ
実験施設
Neutrino to
Kamiokande
Linac
(350m)
3 GeV Synchrotron
(25 Hz, 1MW)
50 GeV Synchrotron
(0.75 MW)
J-PARC = Japan Proton Accelerator Research Complex
© KEK
Joint Project between KEK and JAEA (former JAERI)
DAQ System
Working
Desktop
Common
module
IROHA
Begin
End
Configure
Status
User
module
Begin
End
デバイス
管理
XML/HTTP
Web
Server
DAQ
Operator
DAQ
Status
Gatherer
SiTCP
SiTCP
SiTCP
SiTCP
SiTCP
SiTCP
SiTCP
© KEK
Logger
Dispatcher
DAQ Middleware
Monitor
19
Communication Perfomance
Model : Dell PowerEdge SC1430
CPU :Intel Xeon 5120 @ 1.86GHz 2 Cores ×2
Memory: 2GB
NIC: Intel Pro 1000 PCI/e (1GbE)
OS: Scientific Linux 5.4 (i386)
Source
Source
Sink
Ethernet
88MB/s = 704 Mbps
© KEK
Repeater
Sink
Ethernet
MLF実験装置群
Muon D1
(KEK, JAEA)
Muon U
(KEK)
Muon D2
(KEK)
アマテラス
中性子実験装置：19台
ミュオン実験装置：3台
HRC
(KEK)
大観
(JAEA)
アマテラス
(JAEA)
PLANET
(U. of Tokyo,
JAEA)
SOFIA
(KEK)
千手
(JAEA)
iBIX
写楽
DNA
iMATERIA
ANNRI
匠
(JAEA)
iMATERIA
(Ibaraki
Pref.)
SOFIA
四季
NOBORU
(JAEA)
SPICA
(NEDO, KEK)
NOP
NOBORU
SPICA
写楽
匠 (JAEA)
SuperHRPD
大観
NOP (KEK)
NOVA
HRC
VIN_ROSE
(Kyoto U.,
KEK)
千手
PLANET
NOVA
DNA
(NEDO, KEK）
四季
(JAEA)
（JAEA, KEK, Tohoku U.）
SuperHRPD
(KEK)
ANNRI
(Tokyo Tech. U.,
JAEA,
Hokkaido U.)
iBIX (Ibaraki Pref.)
© KEK
Intelligent RT Software Project
(FY 2007-2011)
Ⅰ．Platform development
①-1 Software/hardware platform development
①-2 RT-Component reusability studies
Ⅱ．RTCs for Manipulation
② Manipulation intelligence for manufacturing
③ Manipulation intelligence for daily life
Ⅲ．RTCs for Mobile Robot
④ Mobile robot for service robot
⑤ High-speed mobile robot
⑥ Mobile robot for field service
Ⅳ．RTCs for Human Robot Interaction
AIST, SEC, Maekawa Industry,
General Robotics Inc., NEC,
Univ. of Tokyo, Fujisoft
AIST, Toyohashi Univ. of Tech.,
Univ of Tokyo, Tohoku Univ.,
NAIST, Osaka Univ., Tsukuba Univ.
Tokyo Univ. of Science, IDEC,
Mitsubishi Elec., Kyoto Univ.,
Kobe Univ., Yasukawa, Toshiba,
Kyushu Univ., Kyutech Univ.,
Metroporitan Univ. of Tokyo
SIT, NES software, fuRo,
Fujitsu, NAIST, Fujisoft,
Toyohashi Univ. of Tech.,
SEC, Univ. of Tokyo, Osaka Univ.,
Tsukuba Univ., Segway Japan,
Kyoto Univ., Tohoku Univ.,
International Rescue system
⑦ Communication intelligence for daily life robots
ATR, eager Inc., Omron, NEC
22
OpenRTP development process
and tool chain
RTCBuilder
RTSystemEditor
RTC development
System Development
Application Development
Test
Impl
Scenario
Editor
Complete
Design
Design
Design
Test
Real-Time design
Impl
Test
Impl
OpenRT Platform
OpenRTM + Eclipse + tool chain
RTC
Debugger
Hardware
design
Dynamics
Simulator
Mobile robot
Navigation
Motion Pattern
Design
23
Developed Components
• Each consortium made their
technologies into RTCs
• Products are registered as
projects on the openrtm.org
website
• 229 registered projects
• Developed RTCs are used
by other consortiums
• Based on this experience,
standard interfaces are
considered
Category
Tools
Manipulation RTCs
Number of Projects
7
106
Mobility RTCｓ
93
HRI RTCs
16
Commercial License RTCs
Total
5
229
24
ROBOSSA
(Robot Software Suite, AIST)
• Maintain middleware, tools and RTCs
developed in the RT Project
• ROBOSSA
– OpenRTM-aist (Middleware)
– OpenRTP-aist (Tools)
– OpenRTC-aist (RTC packages for vision,
manipulation, mobile robot and HRI)
• AIST continues maintenance of them and
will produce start-up companies
25
Summary
• RT-Middleware: OpenRTM-aist
– History (from 2002-)
– Features
• Component-based development
• Standard
• Multiple vendor RTM implementation
– Applications & projects
• Humanoid robots, DAQ
• RT software platform project: 229 projects
registered on the website
26
27
Reference Architecture
(for Manipulation)
Camera & image recognition
Camera
Planning
Image
Recoginition
Manipulator
Manipulator
frontend
Command
Servo
control
Manipulation
planning
Servo
Motor &
Motor &
Motor &
driver
Motor &
driver
driver
driver
28
Reference Architecture
(for Mobile Robot Navigation)
map sensor
map sensor
destination
UI
Sub goal points
Map
management
/navigation
status
Mobile robot
hardware
Sub goal
tracking
#include "BasicDataType.idl"
#include “ExtendedDataTypes.idl” // defined in RTM1.0
module IIS
{
struct TimedVelocity2D
{
RTC::Time tm;
sequence<long> id;
RTC::Velocity2D data; // defined in RTM1.0
sequence<double> error;
};
position
Motor &
Motor &
Motor &
driver
Motor &
driver
driver
driver
struct TimedPose2D
{
RTC::Time tm;
sequence<long> id;
RTC::Pose2D data; // defined in RTM 1.0
sequence<double> error;
};
struct TimedPose2DSeq
{
RTC::Time tm;
sequence<long> id;
sequence<RTC::Pose2D> data;
sequence<double> error;
}; };
29
Reference Architecture
(for Human Robot Interaction)
HRI
GUI
Voice
input
Voice
filter
Image
input
Speech
interval
detection
Person
detection
Object
recognition
Conversation
contents
management
Voice
Recoginition
Conversation
history
management
Conversation
control
Motion
creation
Speech
synthesize
Sound
output
Motion
control
30