HANDS.DVI
Centro di Ricerca “E. Piaggio” – University of Pisa
Project Hands.DVI - KickOff Meeting
Italian Institute of Technology, IIT, Genova, 24.01.2011
Introduction to project HANDS.DVI
 The project HANDS.DVI will contribute to the development
of a unified structure for programming and controlling robotic hands
based on a number of fundamental primitives, and
abstracting, to the extent possible, from the specifics of their
kinematics, mechanical construction, sensor equipment
 The project will propose a new DeVice-Independent programming
and control framework for robotic HANDS
 This project builds upon well-founded results on sensorimotor
synergies developed in neuroscience and preliminary used in some
special robotic applications. The project HANDS.DVI will exploit initial
results of the research activity developed in the European Project The
Hand Embodied (THE)
Motivation and objective of project THE
 The project THE is focused on the study of how the embodied
characteristics of the human hand and its sensors, the sensorimotor
transformations, affect and determine the learning and control
strategies we use for such fundamental cognitive functions as
exploring, grasping and manipulating
 The project hinges about the conceptual structure and the
geometry of such enabling constraints, or synergies: correlations in
redundant hand mobility (motor synergies), correlations in redundant
cutaneous and kinaesthetic receptors readings (multi-cue
integration), and overall sensorimotor system synergies
 These key ideas aim at advancing the state of the art in artificial
systems for robotic manipulation, haptic, and neuroprosthetic
interfaces
Motivation and objective of project HANDS.DVI
 The device independent
mapping will be mediated by a
model of an anthropomorphic
robotic hand (paradigmatic hand)
able to capture the idea of
synergies in human hands. This
model will be used to design the
new control strategies for both
anthropomorphic and non–
anthropomorphic robotic hands
 The mapping between the paradigmatic hand and the robotic hands can
be obtained as the result of an optimal control strategy. A possible solution
is to organize this function as two hierarchical levels:
 in the high level stage only a few elements, the synergies, are controlled
 in the low level the synergies are mapped to the actual kinematic
variables of the robotic hand we want to control
Contents of the presentation
 Force control of robotic grasping: the role of hand synergies in the
optimal choice of grasping forces
 Extending the paradigmatic hand model: modelling and control of
biomorphic fingers using tendon actuation and rolling contacts
 Mapping synergies between the paradigmatic hand and robotic
hands with different kinematics: “DLR/HIT Hand II” (DLR) and “THE
First Hand” (UNIPI) - direct kinematic mapping of the contact points or
definition of an optimal function based on grasping forces
 Synergies definition: the Motion Capture system and statistical
methods for data analysis
On the Role of Hand Synergies
in the Optimal Choice of
Grasping Forces
Objective of the present study
Investigate analytically and numerically the role that limiting the DoFs of an
hand via synergistic underactuation have on:

making possible precision/whole-hand grasps;

grasping force distribution;

grasp optimality evaluated with respect to grasp quality indices;

robustness with respect to physical characteristics (e.g., compliance).
Fundamental Equations
Equilibrium Equations
(object)
(hand)
Congruence Equations
(object)
(hand)
where:
(Grasp matrix: maps contact forces
to object wrench
(Hand Jacobian: maps joint velocities
(maps contact forces
(maps the EE twist
to joint torques
to fingertip twists
)
)
)
to twists of the object contact points
)
Contact compliance model
j-th limb
object
Penetration of the fingertip point into the object surface
Contact force (penalty formulation)
By properly juxtaposing the expression for all the contact forces
(contact stiffness matrix)
(contact compliance matrix)
Joint compliance model
joint actual
positions
joint reference
positions
Joint torque costitutive equation
By properly juxtaposing the expression for all the joints
(joint stiffness matrix)
(joint compliance matrix)
Soft Synergy Model
 We introduce a synergy vector
whose components can be interpreted as
“high level knobs” controlling joint coordination patterns;
 In the classical synergistic model:
the hand configuration is rigidly constrained to a dimensional sub-manifold of
the joint configuration manifold. Therefore the virtual variations allowed are
(synergy matrix)
Synergy matrix found through Principal Component Analysis (PCA) of recorded
grasping experiments [Santello et al. 1998];
 In the soft synergy model (proposed) synergistic displacements
joint reference positions
command
The actual hand configuration depends on hand-object interaction (hand
adaptability to different grasping conditions).
Soft-Synergy Concept
actual joint position
coupling defines
the synergy vector
on references
synergistic displacements
(actual DoF)
Solution of the Grasping Problem with Synergies
For a general grasping system applying a wrench
to the object, i.e.,
the contact force distribution is given by
where:

= particular solution (minimizes the strain energy)
(

weighted right inverse of
)
= active internal forces that can be commanded via synergistic
displacements
If no underactuation takes place (independent joint control):

= passive (preload) internal contact forces
Internal Controllable Forces
Internal controllable forces directly dependent on synergistic variations
Defining
it is possible to write also
(coefficient of the linear combination)
The above form turns out to be very convenient in optimizing the
grasping forces.
Direct employment of
entails the explicit calculation of
is not efficient since it
.
A more efficient algorithm can be employed based on the intersection
of subspaces.
Qualitative analysis of Hand Embodiment
1) Assume each
has non-null projection onto
, then
2) Consider a grasp for which indep. joint control bears
3) For the underactuated case, increase the no. of synergies engaged one by one
(increase the number of columns in ):
 For
(increases)
 For
(a plateau is reached)
Therefore if is “small” (2, 3), embodiment with an equal number of synergies
does not endanger ability to exert the same grasping forces with great control
simplification.
Grasping Force Optimization Problem
Given a grasp characterized by

weighted right inverse of the Grasp Matrix
;

basis for the subspace of internal controllable forces
by employing a given set of synergies;

basis for the subspace on internal passive (preload) forces
by employing a given set of synergies;

External wrench to be resisted
Grasping Force Optimization Problem (GFOP)
In the tests performed:
Problem feasibility
,
force-closure
[N] (normal components)
Paradigmatic Hand Model
Paradigmatic Hand Model features:

model of the Cyberglove employed in the work
of Santello et al. (1998) for which high quality
experimental data were recorder and made
available to us;

the only “plant” for which postural synergies are
[T]
rigorously defined;

15 DoFs
 4 DoFs [T]: Rot., Abd., Metac., Interph.;
 3 DoFs [I, R, L]: Abd., Metac., Prox. Interph.;
 2 DoFs [M]: Metac., Prox. Interph.

limbs represented as cylinders;

anthropometric data relative to one subject
employed in the experimental tests.
[M]
[I]
[R]
[L]
Numerical tests – Precision grasp
As a representative of precision grasp layout we consider the grasp of a cherry.
Grasp analysis (independent joint control):

3 contact points: [T] Interph., [I] & [M] Prox. Interph.;

contact constraints: PCWF;

;

; (internal)

; (internal, controllable)

; (internal, passive)

force-closure: yes!
active limbs
Questions:
What is the effect of hand embodiment for this grasp?
Is the grasp still force-closure? For which synergy/synergies?
What is the effect of synergies on grasp optimality criteria?
cherry.avi
Precision grasp – Contact Stiffness Var.ns
 Variation of the cost function
(2-norm of contact force vector) as
more synergies are engaged into the grasp.
 The grasp is force-closure already for the 1-st synergy.
 Limited effect of Contact Stiffness variation, units [N/mm].
Precision grasp – Joint Stiffness Var.ns
 Variation of the cost function
(2-norm of contact force vector) as
more synergies are engaged into the grasp.
 The grasp is force-closure already for the 1-st synergy.
 Limited effect of Joint Stiffness variation, units [N/mm].
Numerical tests – Power grasp
As a representative of power grasp layout we consider the grasp of an ashtray.
Grasp analysis (independent joint control):

11 contact points: [T] 3 cps; [I], [M], [R], [L] 2cps each;

contact constraints: PCWF;

;

; (internal)

; (internal, controllable)

; (internal, passive)

force-closure: yes!
Questions:
What is the effect of hand embodiment for this grasp?
Is the grasp still force-closure? For which synergy/synergies?
What is the effect of synergies on grasp optimality criteria?
ashtray.avi
Power grasp – Contact Stiffness Var.ns
 Variation of the cost function
(2-norm of contact force vector) as
more synergies are engaged into the grasp.
 The grasp is not always force-closure for the 1-st synergy.
 Some effects of Contact Stiffness variation, units [N/mm].
Power grasp – Joint Stiffness Var.ns
 Variation of the cost function
(2-norm of contact force vector) as
more synergies are engaged into the grasp.
 The grasp is not always force-closure for the 1-st synergy.
 Some effects of Joint Stiffness variation, units [N/mm].
Conclusions
Theoretical results:

a way has been proposed to solve the force decomposition and optimization
problem for hands with synergies;

a way to map synergies from the kinematic domain (where have been
observed) to the force domain (soft-synergies).
Numerical results:

force-closure properties of grasps strongly depends on which synergies are
used to control the hand;

if the first few synergies (PCs) are not actively controlled, force-closure can be
obtained only in many more DoFs;

no improvement beyond the first three synergies for precision grasp,
continuous but small improvements in the whole-hand grasp case;

results are consistently robust with respect to different values of the stiffness
parameters (uncertainty in their knowledge/control).
Modelling and Control of a
Tendon Actuated
Biomorphic Hand
Biomorphic reference scheme
Contact points move over the surfaces
Object/finger contact
Finger
Object
Contact frames
Palm
[M. Gabiccini, A. Bicchi, GRSSP Workshop, 2008]
Convective coords
Total wrench on each limb
Resultant wrench on the -th body:
where :
is the wrench applied by the tendons expressed in body frame
is the wrench applied by the contact forces expressed in body frame
Solution of the problem
If we consider the complete system, we obtain:
the system must be in equilibrium under an external wrench
, therefore:
We search a vector
such that the system is in equilibrium and all the
friction/tendon constraints are satisfied (tendon tension must be positive):
where
is the particular solution;
is the homogeneous solution that not affect the overall system eq.
The vector
must be chosen by means of an optimization routine to fulfill all
the constraints. This can be formulated as a convex optimization problem.
Control scheme
TRAJECTORY
GENERATION
INITIAL
CONDITIONS
CONTACT
KINEMATICS
COMPUTED
TORQUE
OPTIMIZATION
ROUTINE
SYSTEM
DYNAMICS
SOLUTION
(TENDON TENSIONS)
System layout
The hand/object system consists of:
• Three-fingered hand grasping a ball;
• Each finger is a 4-d.o.f. serial manipulator with unilateral joints;
• The first joint (MCP) is a 2-d.o.f. hyperboloidic (saddle joint);
• The 2-nd and 3-rd joints are hinge joints with 1-d.o.f.
• There are 8 tendons which guaranteed the integrity of the system
Task
Task for the hand/object system:
• the object has to track a prescribed trajectory (e.g., G follows ellipse in vert. plane);
• integrity of the hand/object contacts and integrity of the finger limbs have to be
guaranteed by proper co-contraction forces in the tendons;
• integrity meaning:
• contact forces within friction cone boundaries;
• max/min contact force below/above prescribed limits;
• max/min tendon tensions below/above prescribed limits (e.g., tendons
cannot pull)
Videos of simulations
 Move the Center of Gravity of the ball along an ellipse in the vertical plane
 Rotate the ball around a vertical axis
 Move the Center of Gravity of the ball up and down
16 DoFs Robotic Hand
Prototype
Hand Features








16 DoFs
Five fingered hand
Tendon driven
One rotoidal spring for each joint
(only one tendon is needed)
Low cost
Human sized
Human-like movements
Servo controlled
Kinematic model of the “First Hand” is very
similar to the paradigmatic hand model
Hand Schematics
PHALANGES
Index, Middle, Ring and Little


Proximal phalanges are
independently controlled
w.r.t intermediate and
distal phalanges
Intermediate and distal
phalanges are
dependently controlled
THUMB

Phalanges are
independently controlled
ABDUCTIONS
 Middle finger doesn’t
have the abduction DOF
 Thumb and Little
abductions are
independently controlled
PALM
 Two rotoidal joints to
close palm on itself
Grasping Trials - 1
Grasping Trials - 2
Hand Synergies
First Synergy
Second Synergy
first.mp4
second.mp4
Hand Posture Tracking:
Motion Capture System
Experimental Setup
 Motion capture system:
active leds, impulse cameras, server
computer, led controllers, led base station
 Easy to reconfigure and calibrate
 Fast and reliable data
transmission (up to 480 Hz)
 Possibility to connect several clients and
integrate measurements of grasping forces
 Problems to overcome:
 Optimal marker positioning (phalanges or joints)
 Kinematic hand model that best fits experimental data
 Observability, model parameters estimation and posture reconstruction using
nonlinear filters (EKF)
 Statistical methods for data analysis
Statistical Methods
 Modelling grasping movements using statistical methods: Hidden
Markov Models and Gaussian Mixture Models
 Two Objectives:
 Task replication using regression techniques: Gaussian Mixture
Regression
 Tasks classification from continuous movements while grasping different
objects
 Problem of data dimension reduction (15 dofs for the paradigmatic
hand) using linear and nonlinear PCA
 Hand posture reconstruction with inaccurate data using a-priori
information and its application to a low cost glove (sensory synergies).
Thanks for your attention