Quick Overview of
Robotics and Computer Vision
Computer Vision
Environment
Agent
camera
?
Light
Computer Vision Applications
Light is probably the most valuable information
that humans sense about their environment,
and CV has a ton of applications:
1. Object classification
2. 3D Reconstruction
3. Motion Analysis
Bag-of-visual-words processing:
A common CV pipeline for object
classification
Converts an image into a vector, much like the bag-of-words model for text.
After that, you can apply standard ML techniques to build a classifier.
Another common pipeline:
Feature detection + correspondences
1. Find “features” in an image.
These are typically things like corners, edges, or just
very distinctive image patches.
Another common pipeline:
Feature detection + correspondences
1. Find “features” in an image.
2. Find correspondences between features in
similar images.
Blue line shows a correspondence between features in stereo images.
Another common pipeline:
Feature detection + correspondences
1. Find “features” in an image.
2. Find correspondences between features in
similar images.
3. Recover depth
Improved Stereo Vision
Using two (or more) cameras to recover the “depth”
of pixels in a scene is called Stereo Vision or Stereo
Reconstruction.
Some techniques to improve it:
- Structured light (like the Kinect)
- Other sensors (eg, laser range finders)
https://www.youtube.com/watch?v=eEhBor_r5eY&
feature=player_embedded
Robotics: Putting it all together
Agent




cameras
Light,
Laser,
GPS
Representation of beliefs and
goals
Estimation of state based on
previous state and sensor
information
Planning of actions to reach
goals
Learning from previous
examples to improve over
time.
motors
Motion
Environment
Robotics Quiz
For each of the following problems, which
technique from our class would you use to solve
it?
1. Determine the position of the robot, based
on the sensor reading at the current time
step, and the belief about the position at the
previous time step.
Robotics Quiz
For each of the following problems, which
technique from our class would you use to solve
it?
2. Given a road map, a belief about the starting
position, and actions for driving forward and
turning, determine a sequence of actions to
get you to a desired point on the map.
Robotics Quiz
For each of the following problems, which
technique from our class would you use to solve
it?
3. Given a camera feed aimed at an object on a
table, come up with a sequence of robotic
arm/finger motions that will pick up the
object.
Robotics Quiz
For each of the following problems, which
technique from our class would you use to solve
it?
4. Given a set of examples of robotic car
maneuvers in a fixed area (several city blocks,
say), some that successfully got to their goal,
and some that didn’t (crashed or failed to
reach the goal), determine the maneuver for
each location in the area.
Additional Robotics Questions
Representation:
How complicated is it (how many variables/dimensions do you
need) to represent the position and orientation of a robot car
on a map?
How many variables/dimensions do you need to represent the
position, orientation, change in position, and change in
orientation of a robot car on a map?
The position and orientation are called the kinematic state of
a robot.
The position, orientation, change in position, and change in
orientation are called the dynamic state of a robot.
Answer: Additional Robotics
Questions
Representation:
How complicated is it (how many variables/dimensions do you need)
to represent the position and orientation of a robot car on a map?
How many variables/dimensions do you need to represent the
position, orientation, change in position, and change in orientation of a
robot car on a map?
For a 2D world, the kinematic state requires 3 variables (dimensions):
1 for position on the x axis, 1 for position on the y axis, and 1 for the
compass direction (or angle away from North).
For a 2D world, the dynamic state requires 5 variables (dimensions):
3 for the kinematic state, 1 for forward velocity, and 1 for “turning
velocity”, or yaw.
Additional Robotics Questions
3D Representation:
How many variables/dimensions are needed for
the kinematic and dynamic states of a robot
helicoptor?
Many-D Representation: How many
variables/dimensions do you need for a robotic
arm that has six rotation joints?
Answer: Additional Robotics
Questions
3D Representation:
How many variables/dimensions are needed for the kinematic and
dynamic states of a robot helicoptor?
6 for kinematic, 12 for dynamic.
Many-D Representation: How many variables/dimensions do you
need for a robotic arm that has six rotation joints?
The answer really depends on the orientation of the joints with
respect to one another. But if each joint provides independent motion
from the previous joints, you would need 6 variables for kinematic
state (angle of each joint), plus 6 additional variables for dynamic state
(change in the angle of each joint).
Some robotic videos
$3000 DARPA robot hand (very cheap for a robot
hand):
http://www.youtube.com/watch?feature=player_e
mbedded&v=NvhCk6BvLBE
Weird robot:
http://www.youtube.com/watch?v=UIWWLg4wLEY