Hand Features Detection
Oren Tabibi, i.d: 21519301
In this project I’ve tried to implement a hand Detection tool, designs to help
identify Hand Key Features witch can be used in real-time to identify hand
gestures.
Background:
If we look at a Hand’s palm
We can categorize the following
Hand Features:
Hand Features
* Finger Tip
* Finger
* Palm
* Palm Center
* Wrist
Hand Features Detection Algorithm
I Divide the Features Detection Algorithm to 4 stages
 hand and arm region Detection
 Identify Palm And Wrist from the Hand
 Identify Palm center and Radius
 Identify Palm Fingers and Finger Tips.
Hand and forearm region Extraction
I’ve tried several techniques, trying to detect hand and forearm as reliable and efficient as
possible.
At first I’ve filtered the hand from the background using skin tone of the hand, this method was
indeed efficient yet depends on light conditions, this method needs calibration often and not
very accurate with everyday background (my home closet had a color tone very similar to my
skin and messed up the results)
The second Method I tried was using Hough Transform, but the number of positions and shapes
a human hand can take is enormous therefore the implementation is too inefficient and hard to
implement.
Eventually I decided to use a relatively new way of filtering.
I assume a man stand in front of the Depth Sensor puts his hand in front of him, also
assuming an average human palm height/width is no more than 20-25 cm , so if a take the
closest point to the sensor as my anchor, the depth of all the points belong to the hand (in all
positions and angle) should be no more than 15 cm further, hence I could create efficient filter,
based on the distance from the sensor.
Here is an example of test I’ve made.
Before:
After:
Identify Palm and Wrist from the Hand
The next stage was to find the hand’s palm, in many cases, the user puts it’s hand parallel to the
sensor causing the Depth filtering include the user forearm within the extracted region
For example:
(This case is easy to identify, by examine the extracted region proportions and set a threshold)
In this case I’ve looked for the best way of identifying the palm from the object I’ve received.
First I defined the Palm (As can be seen in the background “chapter” – page 1)
I found the Wrist (which defined by two points) and filtered the forearm from the extracted
hand.
Finding the wrist was done by the following Algorithm:
1. I found The Minimum Area Rectangle of the hand (This was given for me “free” by
OPENCV library)
2. I found the forearm width along its length (marked in red in the following example)
(By subtracting the distances marked in yellow)
3. I’ve received a vector of lengths as a function of position along the rectangle longest
edge.
Wrist = local minimum in the lengthposition function
length
10
5
0
X
Positio
n
1
3
5
8
10 12 14 16 18 20
And looked for a local minimum, obviously I ignored the local minimum found in the end of the
object (generated by depth incontinency) .
Palm Center and Radius
I took the palm center to be the center of mass of the new object (after filtered the fore-arm)
And the radius to be a 1/3 of it.
Identify Finger Tips
The main idea of finding finger tips was relying to the fact the convexity of the hand contour at
the finger tips is the largest. This is assuming the hand contour is smooth (In practice it isn’t) ,
Therefore I had to made some manipulations to eliminate false positive.
The Finger tips I found using the following algorithm:
1. I found the hand contour.
2. In order to eliminate false positive finger tips (parts of the contour which have large
convexity mainly caused by depth discontinuity and distortion, I’ve made a Polygon fit to
the received contour (with Threshold as 1/2 from the hand radius)
3. I found the convex hulls of the hand, and eliminated the deepest points in the holes as
well as the wrist points I’ve found earlier.
4. Eventually I chose only the points that have angle small enough:
Example of such an angle
Example of a wide angle – which doesn’t define a fingertip
I calculated for each convex Point the vector to the next nearest points, and
threshold the once which the value Cos(V1,V2) <0.6, (where the V1 & V2 are
the vectors to the neighbors .
Result Example:
Using Skin tone filtering.
Bibliography:
1. Vision-Based Finger Action Recognition by Angle Detection and
Contour Analysis - Daeho Lee and SeungGwan Lee
2. Fingertip Detection for Hand Pose
Recognition - M.K. Bhuyan, Debanga Raj Neog and Mithun Kumar Kar
3. Real Time Fingers and Palm Locating using Dynamic Circle Templates Mokhtar M. Hasan & Pramod K. Mishra