Auto Chasing Turtle
Hirotaka Niisato
Noritsuna Imamura
©SIProp Project, 2006-2008
1
About ourselves
Ourselves
Hirotaka Niisato(@hirotakaster)
Noritsuna Imamura(@noritsuna)
About SIProp Project
Open Source Software Project
The world of media (communication medium) is
extended.
©SIProp Project, 2006-2008
2
3/11
Earthquake
Tsunami
Nuclear Plant
©SIProp Project, 2006-2008
3
Agenda
Summary
Part “Software”
Detect the face
Calculate the course
Calculate the distance
Part “Hardware”
How to control the robot
How to process the analog data
Need more power supply
©SIProp Project, 2006-2008
4
Summary
This product is "Auto Chasing Turtle".
By autonomous control, this robot recognizes
people's face and approaches to the detected human.
©SIProp Project, 2006-2008
5
Robot behavior
1. Rotate and look for the human who becomes a target.
2. Detect the human with recognizing people's face by using
RGB camera of Kinect.
3. If it can detect the human, it calculates the course which it
should follow. And It direction is changed.
4. Calculate the distance to target by using Z(depth) camera of
Kinect.
5. Walk toward to target.
6. 2-5 are repeated until it becomes a suitable distance. And if
losts target, returns to 1.
7. The scene that it is working in real time can be seen by iPad.
©SIProp Project, 2006-2008
6
Movie
©SIProp Project, 2006-2008
7
Movie
YouTube
http://www.youtube.com/watch?v=8EgfAk5RBVo
Source Code & detail explanation
http://www.siprop.org/ja/2.0/index.php?product%2FA
utoChasingTurtle
©SIProp Project, 2006-2008
8
complete set parts
Robot
Sensors
KONDO Animal 01
Controllers
For servo
RCB3
For application
Beagleboard-xM
Linaro-Kernel
Android(Embedded
Master)
Kinect
RGB camera
Z(Depth) camera
Connect to outside
Radio wave
Wi-Fi router
Viewer
iPad
Power supply
12V1A output
10V1A output
5V3A output
©SIProp Project, 2006-2008
9
Why this project?
DENSUKE project
Mobile Internet Device with Cyber-Pet User Interface
©SIProp Project, 2006-2008
10
DENSUKE project
OESF(Open Embedded Software Foundation)
Future Systems WG’s mission
http://fswg.oesf.biz/
Partners
Project leader
Mr. Ohtsuki
The Father of AIBO
©SIProp Project, 2006-2008
11
Detail of “KINECT Turtle world”
©SIProp Project, 2006-2008
12
Why KINECT and Robot ?
Application have changed
Game, Motion Controller, Nunchuk
Cell Phone, Tablet
©SIProp Project, 2006-2008
13
Why KINECT and Robot ?
KINECT show the next future
Camera, Depth censor.
Recognition of human’s data.
Mobility Problem ...
©SIProp Project, 2006-2008
14
Why KINECT and Robot ?
KINECT on Robot is result of mobility problem.
INPUT
Camera data
Depth censor
OUTPUT
Analyze
Robot
movement
©SIProp Project, 2006-2008
15
Part “Software”
©SIProp Project, 2006-2008
16
ofxDroidKinect
openFrameworks
x
Kinect
x
Android
©SIProp Project, 2006-2008
17
About ofxDroidKinect
©SIProp Project, 2006-2008
18
About ofxDroidKinect
This is the Android Application Framework which
runs on openFrameworks & uses Kinect.
Using softwares
openFrameworks for Android
Linaro Android
©SIProp Project, 2006-2008
19
Control Robot and KINECT
ofxDroidKinect
USB
Serial
Robot
openFrameworks
Kinect
Linaro Android
Wifi
iPad
©SIProp Project, 2006-2008
20
Analyze KINECT data
Detect the Face
Calculate the course
1. detect
3. distance
Calculate the distance
2. course
©SIProp Project, 2006-2008
21
Detect the face
What’s data ?
640x480 RGB image, using KINECT’s camera,
Characteristics of human face.
KINECT Image
©SIProp Project, 2006-2008
22
Detect the face
Face recognition
Characteristics of human face
Statistical face detection and machine learning.
©SIProp Project, 2006-2008
23
Detect the face
Not all frame, but realtime analyzing.
Auto search around people.
Chase specified human.
©SIProp Project, 2006-2008
24
Calculate the course
Using width.
1. Specify center position of face.
2. Specify position of the face in 4-sections relavive
to KINECT’s image.
©SIProp Project, 2006-2008
25
Calculate the course
Using height.
Change Kinect’s angle.
Specify position of the face in image relavive to
KINECT’s image.
©SIProp Project, 2006-2008
26
Calculate the distance
1. Mesure the depth of specified point using
KINECT.
2. Robots walk forward or back, depending on the
mesured distance.
©SIProp Project, 2006-2008
27
Performance problem
Data analyze and image processing
Face recognition
More resource !!
GPU, Multi core.
©SIProp Project, 2006-2008
28
Part “Hardware”
©SIProp Project, 2006-2008
29
©SIProp Project, 2006-2008
30
How to control the robot?
Only control servos’ angle.
Accuracy
Reaction velocity
Torque
Compact(Size, Weight)
©SIProp Project, 2006-2008
31
How to control a servo?
RCB series
24 controls pins
Control commands
Control all servo at same time !
©SIProp Project, 2006-2008
32
More control
Do you know this IC?
©SIProp Project, 2006-2008
33
How to process more sensors?
Analog data is stream data!
Need powerful processor
Need to save power for Mobile
Now Solution
ARM base Processor
dual Cortex A9
MALI 400 GPU
Next trend
GPGPU
FPGA
©SIProp Project, 2006-2008
34
How to optimize for each sensors?
http://linaro.org/
Mission
Optimize for each SoC & Platform
We are committer!
©SIProp Project, 2006-2008
35
Trouble: Need more power supply
Don’t run beagleboard-xM by 5V/1A power supply.
It needs 5V/2A.
Standard USB battery has 5V/1A.
Create power supply by myself
My battery has 12V/2A.
My plan is to create convertor of 12V/2A ⇒ 5V/2A.
Can’t get power IC…
Using car’s cigarette adapter which convert to USB.
12V・24V/2A ⇒ 5V/2A
©SIProp Project, 2006-2008
36
MAKE:style
Hardware
beagleboard-xM
KONDO Animal
Kinect
Software
ofxDroidKinect
Linaro Kernel
Android(Embedded Master)
Do It Yourself ⇒Do It With Others!
©SIProp Project, 2006-2008
37