Robots Unlimited
PHD Summerschool
July 2006
Alexander Brändle
Marco Combetto
1
Agenda
•
•
•
•
2
Who we are?
Why robotics?
Activity overview
Future ideas
Intelligent Environments Team
Alexander
Braendle
3
Marco
Combetto
Pierre-Louis
Xech
Andreas
Heil
(PHD Student)
Why robotics?
•
How will Future Environments look like?
– Challenges/Needs
– Personal Devices, Embedded Devices, Sensors, Robots, Appliances?
– Enabling technologies,Programming paradigms,Interaction paradigms?
•
How will future applications look like?
– Wide range of user types?
– Mixing real and virtual world?
Robots could form a part of it!
4
The Robotics Wave
• Service and consumer markets
just emerging
–
–
–
–
–
–
Remote assistance/presence
Assistive
Facilities maintenance
Security
Education
Entertainment
• Academic Research
–
–
–
–
–
Complex topic
Moving from 8/16 to 32 bit
Lots of hand-coded solutions
Education and Hobbyist channel
Still mostly 8 bit, starting to shift
• Strong governmental interest
worldwide
5
EURON roadmap:
2015 -> It‘s all about software
Robots!
•
Wide range of applications
– Commercial use
– Academic use
– Personal use
•
How to program them?
•
Challenges
–
–
–
–
–
–
–
–
–
6
Complexity
Reusability
Reliability
Resources
Tools
Technologies
Choice
Sharing
Transference of skills/experience
Links
• European Robotics Network
(EURON)
• European Robotics Platform
(EUROP)
• IEEE RAS Technical Commitee on
‚Programming Environments in
Robotics and Automation‘
7
Robots, too?
• Software
–
–
–
–
–
Internet Explorer
Media Player
MS Agent
Mobile Phones
…
Definition: A robot is a device, hard- or software with
the capability of sensing and (re-)acting.
8
Robotics at Microsoft [Research (Cambridge)]
Robots in human environments
• End-user programming
– Developer Tools (FischertechnikDemo, Scatterweb-Demo)
– Visual Programming (.FUN-Demo)
– End-user „debugging“
– Microsoft Robotics Studio
– Coding4Fun
• Enabling new applications
–
–
–
–
9
Emotional oriented computing
Personal robotics
Self-reconfigurable structures
Learn from interaction with animals
The Fischertechnik ROBO Interface
– The in- and outputs
•
•
•
•
10
8 digital inputs
2 digital and analog distance sensors
4 analog sensors for resistance and voltage
4 motors with 8 different speeds
• The board
•
•
•
•
•
TS%SystemRoot%\System32\mstsc.exe
Serial port, COM, RS232
USB
Infrared
R/F module available
Ethernet for the next
hardware revision planned
Adding Control
– Now we want to control the robot
– Let‘s take another off the shelf product
– Ordinary Joystick
+
11
Coding4Fun
http://msdn.microsoft.com/coding4fun/
12
Integrating Sensors
– Sensors are they eyes and ears of robots
– Increasing Demand for easy robust sensor platform from biology, ecology, civil
engineering et al.
How to get easy access to Sensor Nodes systems
A challenge on it‘s own:
13
•
Where are sensor node platforms
heading?
•
Dealing with vast amounts of real-time
data becoming available
Sensor Networks in the real world
•
Robustness …
–
–
–
–
–
–
–
•
Bluetooth, …
GW
SN
SN
SN
SN
SN
SN
SN
Realistic simulation tools
Heterogeneous testbeds
Useful traces
Tool integration (in well know platforms)
Development for heterogeneous systems
SN
GW
SN
SN
GW
GW
SN
…Better development support
–
–
–
–
–
14
WSNs have to work 24/7
WSNs have to offer uniform APIs
Long-lived, autonomous networks
Self-organizing and energy efficient
Routing
Data aggregation
Deployment & Setup
ScatterWeb/.NET
SN
Sensors (ScatterWeb)
An open and flexible platform for rapid protoyping & implementation of
wireless sensor networks
•
Nodes
–
with/without sensors
•
Sensors
–
•
•
–
Luminosity, noise detection,
vibration, PIR movement detection,
Microphone/speaker
IR sender/receiver
stand-alone/modular
•
–
•
Gateways
–
•
15
WLAN, Ethernet, Bluetooth, GPS, GSM/GPRS, USB, RS485 , serial…
Software
–
–
–
•
Acceleration, humidity, temperature, luminosity, noise detection,
vibration, PIR movement detection on demand
Stand-by: 7.6µA, 5 years life-time with AA battery and 1% duty-cycle
Management, flashing, routing, ns-2 simulation models
TCP/IP, web server (http://193.10.67.150/), Contiki, TinyOS, …
Basic functions for energy management, routing
www.scatterweb.net
Jochen Schiller,Free University Berlin
.NET for ScatterWeb
• Extend the .NET tools and architecture to small devices
•
•
•
•
•
•
•
16
Sensor world available to every
developer
Easy access to sensor values, events,
and functions
Understandable namespaces and
interfaces
Support for IntelliSense and dynamic
help
Well known programming model
(events, methods, properties)
Extensibility of the nodes’ logic (Tiny
C#) and instant IntelliSense-Update
Development, deployment and
debugging within Visual Studio
Jochen Schiller,Free University Berlin
A first step
• Attractive for developers
• End-users?
17
.FUN
18
•
A compelling & engaging
programmable environment
to play & learn for children
(introduce children to
Computing in new ways)
•
Make technologies of
tomorrow accessible to non
technical market (children,
nurse, elderly, machine
operator)
•
Linking real and virtual world
•
Wider market potential of
Robotics (Industrial,
Assistive technology, new
consumer products, health)
Technical University Berlin
Next steps
•
•
•
•
19
Keep graphical representation
Using context data
Domain specific language
Changing the paradigm …
Integrating a Context Server 1/2
Raw Sensor Data
Sensors
Context Events
Context Server
Application(s)
DB
Context Requests
20
Torben Weis, University of Stuttgart
Integrating a Context Server 2/2
Raw Sensor Data
Context Simulator
Context Events
Context Server
Application(s)
DB
3D Data
21
Context Requests
Torben Weis, University of Stuttgart
22
Torben Weis, University of Stuttgart
Microsoft Robotics Studio
A development platform for robotics community, supporting a wide variety of
users, hardware, and application scenarios.
Microsoft Robotics Studio
Runtime
• Concurrency
• Services
infrastructure
Authoring Tools
• Simulation Tool
• Visual Programming
Language
Services and Samples
• Samples and tutorials
• Robot services
• Robot models
•Technology services
•
Make it easy to manage asynchronous components
Avoid need to understand manual threading, semaphores, etc.
•Provide a scalable programming model
•Make state observable, easily accessible
•Provide for reusability and failure
•Support component discovery and composition
•Support remote/distributed execution
25
Microsoft Robotics Studio
Key Runtime Features
• Support standalone and distributed processing scenarios
Connected
operation
(remote execution on PC)
26
Disconnected
autonomous operation
(with optional networked monitoring)
Distributed execution
(execution across compute units)
Microsoft Robotics Studio
• Extensible to a wide variety of hardware
27
Microsoft Robotics Studio
Services and Samples
• Over 15 tutorials
– VB.Net, C#, JScript
• Support for
–
–
–
–
28
LEGO® Mindstorms® RCX
LEGO® Mindstorms® NXT
fischertechnik®
MobileRobots Pioneer P3™
Microsoft Robotics Studio
Other University Support
•
•
•
•
•
•
•
•
•
•
29
Bryn Mawr College
Carnegie Mellon University
Cornell University
Georgia Tech
Massachusetts Institute of Technology
Stanford University
University of Pennsylvania
University of Pisa
University of Southern California
University of Washington
Robotics at Microsoft [Research (Cambridge)]
Robots in human environments
• End-user programming
– Developer Tools (FischertechnikDemo, Scatterweb-Demo)
– Visual Programming (.FUN-Demo)
– End-user „debugging“
– Microsoft Robotics Studio
– Coding4Fun
• Enabling new applications
–
–
–
–
31
Emotional oriented computing
Personal robotics
Self-reconfigurable structures
Learn from interaction with animals
Emotion-Oriented Computing
• General Goal: Make interaction between human
and machine more natural for the humans
• Machine should be able to:
– To register human emotions
– To convey and comunicate emotion
– To understand the emotional relevance of the event
32
3/18/2016
Microsoft Internal Only
32
Human Centred, Affects and Emotions
•
Enhance communication through compelling, fun and
emotional interactions with computing. Seeking to make the
process more intuitive, interactive and appealing to a wider
group of people. Experimenting, evolving, evaluate
emotional models
•
Support privacy, intimacy and different level of information
sharing
Extending the software as a new medium
•
–
–
–
–
34
3/18/2016
Affective Media and Mobile media (affective loop)
Participatory creation, exchange, authoring and fruition
Storing the intimacy, the private value of the things
Fusion of different kind of data sensors, embodiments,
Microsoft Internal Only
34
Some examples
Sensing
Interaction
with Environments
35
Affective Diary
Designing for bodily expressiveness and self-reflection
•
•
•
•
Collecting memories – including body memorabilia mingled with
mobile materials (SMS, MMS, photographs, music listened to,
video,..
Offering a diary medium in which those memories can be mirrored
and organised
Empowering the user to create meaning and alter those
representations
Prototype
– build on TabletPC and Smartphone
– Sensordata (movement, arousal)
36
3/18/2016
Microsoft Internal Only
36
BSP: Interactive storytelling
in vast location based
•
•
•
•
Pervasive computing: Location based games provide navigational
challenges e.g. Chasing
Mobile Media: A flexible media and framework for interaction
Interactive storytelling: Balance linearity with user control, believable
characters
The concept of believable environments, and our implementation, put a
research focus on possibilities to:
– Enrich pervasive games with location
dependent narratives
– Design “the stage set” to improve
interactive storytelling
37
3/18/2016
Microsoft Internal Only
37
An example
Sensing
Interaction
with Robots
38
The mind and the body
• Neurophysiologists suggest that the body plays a crucial role
in our cognitive process
• The brain experiences the world through the body:
– The environment is well known to the brain
– Outside environment is a set of states of inside
• Agents with body (embodied agents) try to mimic this schema
to produce intelligent behavior
• Dautenhan and Jakobi show that body’s presence influences
the behavior of software (anything already heard?)
40
40
Component structure
Vision Roblet
Knowledge Base
Body Map
Arm Roblet
WiFi Roblet
MotionRoblet
Sensors
Internal perception
Network perception
41
41
Robotics4.net
• Realization of a platform for define the body of a robot as a
set of agents (Roblets) that acts as intermediaries between
the devices and the reasoning software
• The robot provides a physical body and a set of core services
to support the development of highly autonomous agent
situated in both cyberspace (e.g Internet, virtual entities) and
real world
• Definition of an Object Model, an API, a set of abstraction that
allows:
– Portability (on different type of platforms)
– Scalability from small (e.g. Lego) to complex systems
– Integration with the MS Development Platform and MS OS and
Application suites
42
42
Functions implemented
• The platform provide the following base services to
support the software implementing the roblets:
–
–
–
–
–
–
–
–
General purpose functions (I/O)
Basic motion abilities
Collision avoidance software
Vision software for face recognition
Speech and gesturing recognition
Positioning and navigation aided by video input
Network perception
IM interface
– The Architecture is available as a simple Software Development
Kit freely downloadable from http://www.robotics4.net
43
43
Perception
44
3/18/2016
Microsoft Internal Only
44
Evolving the framework
• Reduce the programming model of Robotics4.NET to the
one adopted by the Robotics Studio CTP
• Extending the Mind/Body model with self-developing
tools, model and methods to verify the behavior of
systems imposing very high level constrains and how
that can be both formally and programmatically verified
• Testing interaction in real scenarios
45
45
Questions
46