Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University

Columbia University

Robotics Group

Robotic Grasp Planning

Peter Allen

Department of Computer Science

Columbia University


Robotics Group

Robotic grasping is a complex field

• hand design: high level (number of fingers, kinematic structure, etc.) and low-level (mechanism design, motors, materials, etc.);

• hand control algorithms: high level (find an appropriate posture for a given task) and low-level (execute the desired posture);

• information from sensors (tactile, vision, range sensing, etc.);

• any pre-existing knowledge of objects shape, semantics and tasks (e.g. a cup is likely to be found on a table, should not be held upside-down, etc.);

• all of these add up to a Grasp Planning System...and more!


Robotics Group

Human Grasping vs. Robotic Grasping

• Human performance provides both a benchmark to compare against, and a working example that we can attempt to learn from. However, it has proven very elusive to replicate:

• the human hand is a very complex piece of equipment, with amazing capabilities;

• humans benefit from an unmatched combination of visual and tactile sensing;

• human continuously practice grasping and manipulation, the amount of data they are exposed to dwarfs anything tried so far in robotics;

• are we setting the bar too high\

• Speed stacking!

Robotic Manipulation

• Process allowing a robot to make physical changes to the world around it.

• Includes moving objects, joining objects, reshaping objects, etc.

• Moving objects can be done by grasping, pushing, carrying, dropping, throwing, etc.

• Task accomplished by a manipulator with some sort of end-effecter.

Grippers vs. Hands

• Structured environments

• Reliable

• Simple

• Low cost

Suction Magnet

• Unstructured environments

• Adaptable

• Complex

• Expensive

Parallel Jaw

Barrett Utah/MIT Robonaut

Simulation for Grasp Planning

• Integrated grasp analysis

– Grasp quality, weak point, force optimization

• Perform many grasps quickly

– Faster than using a real arm and hand

• Build a library of saved grasps

– Recall grasp when object is encountered again

GraspIt!: A Tool for Grasping Research

• Library of hands and objects

• Intuitive user interface

• Visualize grasp wrench space

• Quality measures evaluate grasp

• Dynamic simulation

• Grasp Planning

GraspIt: A Tool for Grasping Research

• Library of hands and objects

• Intuitive user interface

• Visualize grasp wrench space

• Quality measures evaluate grasp

• Dynamic simulation

• Grasp Planning

GraspIt Components

World

Construction

• read object models

• read link models

• read kinematics

• assemble robots

User

Interface

• view 3D scene

• change hand pose

• auto-grip

• manually move joints

• Matlab interface

Grasp

Analysis

• compute grasp wrench space

• use metrics on space

• grasp force optimization

Wrench Space

Visualization

• create projections of

GWS

• worst case disturbance

Contact

Determination

• detect collisions

• adjust contact to object surface

• find contact area

• add friction cones

Rigid Body

Dynamics

• compute object motions

Grasp Planner

• generate and test grasps

Control Algorithms

• PD controllers

Hand Construction

• The hand kinematics (in D-H notation) specify the transforms between links, and can handle coupled joints.

• The 3D link geometries are accurately described in CAD model files.

• The format is flexible and can easily model many of the available complex articulated hands.

Hand Kinematics

Hand/Arm Library

Barrett Rutgers Robonaut

Parallel Jaw DLR Puma Paloma

Grasp Analysis

• Occurs when a contact is formed or broken

• Computes space of forces and torques that can be applied by the grasp

• Quality measures numerically evaluate grasp

• Provides a means to evaluate grasps

– Compare grasps of one hand, one object

– Compare grasps of many hands, one object

– Compare grasps of many hands, across a task specified object set.

Wrench Spaces

• In 3-space, a wrench is a 6D vector composed of a force and a torque: w





 f

τ

• The space of wrenches that may need to be



 applied during a task is the task wrench space.

• The space of wrenches that can be applied by a grasp is the grasp wrench space.

• A possible quality measure: TWS

GWS

Special Types of Grasps

• A force-closure grasp completely restrains the object.

– Origin is contained within grasp wrench space.

• A manipulable grasp can impart arbitrary velocities on the object without breaking contact.

Friction Cones

• Friction at a contact point allows forces in directions other than the contact normal

• COF, m, is determined by the contacting materials

• Estimate friction cone as convex sum of a force vectors on the boundary assuming a unit normal force, .

m f

  j f j j





1

2D Example

Object: square lamina

Grasp: 3 point contacts with a

COF of 0.3

In 2D, friction cones do not need to be approximated.

y x

Contact Wrenches

•

• Each force acts at a position on object.

• Compute the corresponding wrench with respect to object’s center of gravity: w i , j





 l

 f i , d i j

 f i , j





 i : contact number j : (1-8) around cone boundary l

- torque scale factor,

1 d max f

1,1 y d

1 d max f x t x f y

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Contact Wrenches

•







 l

 f i , d i j

 f i , j







1 d max y x

Grasp Wrench Space

• Objective: find total space of wrenches that can be applied by a grasp of unit magnitude.

• Grasp vector: g



 f

1

 f

2



 f n

 T



, g



1

L

1

– Sum magnitude of contact normal forces is 1.

• Compute grasp wrench space using qhull :







1





Two Measures of Quality

• Assume task wrench space is unknown.

• Estimate with wrench space ball good grasps will resist all wrenches equally well.

• Previously proposed measures of quality:

– Radius, e

, of the largest wrench space ball that can fit within the unit grasp wrench space.

– Volume, v , of unit grasp wrench space.

Ball radius e

: 0.201

Hull volume v : 0.902





– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201






– Radius, e



Ball radius e

: 0.201


Grasp Wrench Space Projections

• To visualize the 6D grasp wrench space project it to 3-space by fixing three coordinates.

• A useful choice:

– See the forces that can be applied without applying a net torque (or vice versa).

Visualizing the Results

Numeric quality &

Worst case indicator

Wrench Space

Projection

Comparing Grasps

Comparing Grasps

Comparing Grasps

Simulating Dynamics

Simulating Dynamics


Robotics Group

Hand Posture Subspaces

• A grasp can be a considered a point in a high-dimensional hand configuration space

• To find a grasp, need to search this space: expensive, intractable

20 DOF 12 DOF 4 DOF 14 DOF

• Low-dimensional subspaces can approximate most of the variance needed for common grasping tasks

• [ Santello] PCA on large dataset of human joint angles during grasping

• 2 PC’s contain approx.

85% of the variance!

• Continuous grasp subspace approximates common grasp posture e.g.

p = a

1 e

1

+ a

2 e

2


Robotics Group

Eigengrasps

• Can be seen as generalization of grasp taxonomy [ Napier ‘56 ,

Cutkosky ‘89 , Iberall ‘97 , etc.] continuous subspace discrete points, Cutkosky `89

E igengrasp method : video


Robotics Group

Grasp Planning using Eigengrasps

• Low-dimensionality basis for grasping

• Derived from human user studies and mapped to robotic hands

• Practical implications

 dimensionality reduction



Completeness: can it cover the space of good grasps?

• Search eigengrasp subspace for good grasps

 use simulated annealing as search algorithm o stochastic nature allows escape from local minima o no analytical gradient is necessary

• Planner works independent of hand kinematics

M. Ciocarlie, C. Goldfeder, P. Allen, Dimensionality reduction for hand-independent dexterous robotic grasping, IROS 2007


Robotics Group

Grasp planning using Eigengrasps

• Apply Eigengrasp concept to a range of hand models:

• From simple gripper (4 DOF) to human hand (20 DOF)

• For human hand, use available data

• For robotic hands, attempt to define grasp subspaces similar to the one obtained in the case of the human hand

• direct transfer possible for anthropomorphic models


Robotics Group


Robotics Group


• Energy function formulation attempts to bring pre-specified contact locations on the palm in contact with the object

• Simulated annealing search is performed over 8 variables o

6 for wrist position / orientation o

2 eigengrasp amplitudes

• Our implementation: 100K iterations, ~150 seconds


Robotics Group


Robotics Group

Grasp planning using Eigengrasps k = 1000 k = 5000 k = 8000 k = 15000 k = 20000 k = 25000 k = 29000 k = 30000 k = 35000 k = 40000 k = 55000 k = 60000 k = 75000 k = 100000

Each image shows best solution found until iteration k

Video


Robotics Group

Grasp Planning Examples


Robotics Group


300

200

100

-1

-2

0

1

0

2

0

0 20000 40000 60000 80000 100000

20000 40000 60000

Iteration number

80000 100000


Robotics Group


• We need the final posture for stable (form-closure) grasps

• Completeness: perfect grasp postures can not be found in EG space

• Can come very close with simple heuristic: close all fingers!

• Perform form-closure tests in parallel in a multi-threaded environment

 takes advantage of multi-core architecture

• Form-closure in 23 out of 30 cases


Robotics Group

Interactive Online Grasping

• Dexterous hand prosthetics

 human control combined with artificial hardware and algorithms



 benefits from results in both human and robotic grasping but also open new interface problems!

• Task: achieve a stable grasp using a dexterous prosthetic



3D tooltip positioning under neural control [ Taylor et al.

‘02 ]



finger posture has proven more difficult to specify o

EMG [ Zecca et al.

‘02 ], neural control [ Taylor et al.

‘03 ] o low-dimensional, noisy

• Approach:



reduce the dimensionality of the searched posture space



use on-line operator input to simplify hand position

Matei Ciocarlie and Peter K. Allen, Hand Posture Subspaces for Dexterous Robotic Grasping, The International Journal of

Robotics Research, Jul 2009; vol. 28: pp. 851 - 867.


Robotics Group

Neural Control of Prosthetic Hands


Robotics Group

Interactive Dexterous Grasping

• Operator

 specifies approach direction

 has no control over finger posture

• Automated grasp planner





 knows object shape and position records position input searches for a stable grasp according to user approach direction



sets finger posture accordingly

• Operator

 changes approach direction, or



completes grasp (binary “close all fingers” command)


Robotics Group

Interactive Grasp Planning

• Hand posture: 2 variables (eigengrasp amplitudes)

• Hand position:

 user not expected to fully specify final position

 o affects interaction, can not handle noise

3 variables to re-parameterize hand approach: o d, θ and φ define a conical search space

• Total: 5 variables

 loops of 2000 Simulated Annealing iterations

 continuously update base hand position



 search does not get stuck if one loop fails best pre-grasps tested for form-closure


Robotics Group

Interactive Grasp Planning Examples


Robotics Group



Robotics Group


Simulated Annealing search:

~1000 postures / s candidate pre-grasp tests:

~20 grasps / s overall: 5-15 seconds for task completion


Robotics Group

Is Grasping Indexable?

• Many previous attempts to taxonomize grasps

• Is there a finite set of grasps we can pre-compute?

• If so, can we build an indexable database of grasps?

• Given a new object to grasp, can we find a similar grasp

• Some Problems:

•

•

•

Lots of objects to grasp…

Lots of DOF in a hand (~20 + 6 in human hand)…

Lots of different robotic hands…

• Intractable? But maybe not….


Robotics Group

The Columbia Grasp Database

• We have built a large scale grasp database.

• High quality form closure grasps for:



Multiple hands





Thousands of objects

Hundreds of thousands of grasps

• We can use the grasp database for:





Data-driven grasp planning

Rigorous benchmarking of grasp planners


Robotics Group

Finding Stable Grasps

• Problem: Collecting grasp data from humans is expensive



Human studies do not scale to large datasets



How can we collect data on robotic hands?

• Solution: Automate the data collection



Use a powerful offline grasp planner as the source



Works for human or robotic hands, inexpensive

• Caveat: Database quality depends on the quality of this planner

• Advantages of Eigengrasps





Stochastic, so can give many grasps for one object

Gives form-closure grasps


Robotics Group

Objects to Grasp

• We reused the 3D models from the Princeton Shape

Benchmark *



Well known academic dataset of 1,814 models





Encourage collaboration with shape researchers

All models resized to “graspable” sizes

• We provide grasps at 4 scales



…because grasping is scale dependent



.75, 1.0, 1.25 and 1.5 times the size of each model



7,256 3D models in all

* Shilane et al ., SMI 2004


Robotics Group

Robotic Hands

• We provide grasps for 4 hands



Human hand model (20 DOF)









Barrett Hand (4 DOF + disengaging clutch)

Barrett Hand with rubber coating

Willow Garage gripper

More hands to come!


Robotics Group

The Completed Database

• Building the database



Took 1 month on 6 multicore computers.



Resulted in ~250,000 distinct form closure grasps

• For each grasp we saved:





Grasp and pre-grasp joint angles and position

Ferrari-Canny volume and epsilon metrics

• Available for download now!

 grasping.cs.columbia.edu


Robotics Group

Browsing the Grasp Database


Robotics Group

Data Driven Grasp Planning

• Given a new 3D model to grasp



Find nearest geometric neighbors in database







Initial implementation with Zernike descriptors

Collect pre-grasps from neighbor models

Evaluate candidates in GraspIt! simulator


Robotics Group


Robotics Group

CGDB: Benchmarking

• That seemed to work, but can we quantify it ?



Historically – not easily!





Evaluating a grasp planner is very difficult

No consistent benchmarks for comparison

• Our proposal: Benchmark on this database



Simulate planned grasps on each model







Report aggregate statistics

Easily repeatable , no expensive hardware required

Quality metrics are only possible in simulation


Robotics Group

Benchmarking

• We benchmarked two planners



Eigengrasp Planner and Database Planner

• We benchmarked two hands



Human and Barrett

• We report the average quality of the first 25 grasps from each planner over 1814 models (single scale)


Robotics Group

Planner Benchmarks

Barrett Hand Human Hand n th best grasp for data-driven & Eigengrasp planners


Robotics Group

Results

• Consistent results over both hands

• For first grasp



Database planner’s quality matches Eigengrasps





But is much faster (not shown in the chart)

1 minute vs. 10-15 minutes

• For subsequent grasps



Database planner produces many more good grasps


Robotics Group

Grasping a Sensed Object

• Data from a NextEngine laser scanner

• noisy, missing and occluded regions

• Do geometric match into database, test these grasps

• Works for human or robotic hands


Robotics Group

Data Driven Grasping Examples

Next: Using partial range scans…

Goldfeder, Ciocarlie, Allen, et. al., Data-Driven Grasping with Partial

Sensor Data, IROS 2009


Robotics Group

GraspIt!

• Simulation Engine focused on robotic grasping

• Can quickly test thousands of potential grasps, using arbitrary hand geometries and object models.

• Includes dynamics, visualization, GWS calculation

• Easy to add new hand designs

• Eigengrasps: Reduced subspace for grasping

• On-line grasp planning: Add user input to eigengrasps to reduce search space further

• Applied to a number of robotic and human hand models, using both real and simulated environments

• Columbia Grasp Database: indexed set of grasps across large model and hand space


Robotics Group

Acknowledgements

• Andy Miller: Original GraspIt! design and implementation

• Matei Ciocarlie: Eigengrasps, online planner, soft finger contacts

• Corey Goldfeder: Columbia Grasp Data Base

• NSF and NIH

Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University

GraspIt!: A Tool for Grasping Research

Is Grasping Indexable?

Related documents

Products

Support

Robotic Grasp Planning Peter Allen Department of Computer Science Columbia University

GraspIt!: A Tool for Grasping Research

Is Grasping Indexable?

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib