Final Project for Computer Vision
Eye Feature Detection
Rui Liu
PhD student M.E.
CONTENT
‣ 1 Introduction
‣
‣
‣
1.1 Motivation
1.2 Goals
1.3 Experiment Devices and Environment
‣ 2 Methods & Algorithms
‣ 3 Experiment Results
‣ 4 Result Analysis
‣
‣
‣
4.1 Assessment
4.2 Small Problems and Its Analysis
4.3 Moving
Forward
1 Introduction
1.1 Motivation for Eye feature detection
(1) What could eye feature indicate?
reflecting psychological state
He sees the truth. It’s written all over our faces.
Indicating human intention
Maybe want to leave
Maybe want to drink
(2) Motivation: detecting eye features for intuitive human-robot
interaction
1.2 Goals
Pupil detection
pupil localization, pupil diameter/ mean area and its
standard deviation in a certain time.
Blink detection
blinking status detection, blinking times and
blinking rate in a certain time.
1.3 Experiment Devices and Environment
(1) Devices
HD Camera
Pic Captured
Head Mount
(2) Environment
Common lab (400 lux)
2 Methods & Algorithms
1.1 pupil detection
(1)Method:
a. Circle(d≈44)
b. Dark(Area≈1500)
c. Concentric
d. Region
Pupil
(2) Algorithm:
a. Circle detection: Two-stage Circular Hough Transform (‘imfindcircle’ )
b. Dark region detection: ‘regionprops’
Concentric
d= 42
‣ (3) Parameter Calculation
pupil center
(x, y)
circle detection
Pupil Diameter
Mean Area A =
𝑡2 𝜋𝑑2 𝑡
𝑡1
4
d
/(𝑛𝑡2 − 𝑛𝑡1 )
Standard Deviation of Pupil Diameter 𝜎 =
𝑡2
𝑡1(𝑑
𝑛𝑡 −𝑑)2
𝑛𝑡2 −𝑛𝑡1
1.2 blink detection
(1) Method:
pupil disappearing time >𝒕𝟎
Blinking
Blinking
Open
‣ (2) Parameter Calculation
Blinking Times N
Blinking Rate
Rate=N/(𝑡2 -𝑡1 )
3 Experiment Results
3.1 results in different situations
a. Beginning
b. Blinking
c. Open
d. Near Blinking
3.2 Result Video
3.3 Statistics Results
blink rate
Blinking Status
1.5
1 -- Blinking
0 -- Open
Eye Status
1
0.5
0
-0.5
0
5
10
Time(s)
a. Eye status
15
20
blink rate(Times/sec)
1.5
1
0.5
0
-0.5
0
5
10
Time(s)
15
b. Blinking Rate (n/sec)
20
Pupil Diameter
Mean Pupil Area
Mean Pupil Area(pixel 2)
40
20
1600
1500
1400
1300
0
0
0
5
10
Time(s)
c. Pupil Diameter
15
5
10
Time(s)
20
Standard Deviation of Pupil Size
Pupil Diameter(pixel)
60
3
2
1
0
5
10
Time(s)
15
20
d. Mean Pupil Area
Standard Deviation of Pupil Size
0
15
20
e. Standard Deviation of Pupil Size
4. Result Analysis
4.1 Assessments
Successful Rate for Pupil detection
99%
Successful Rate for Blink detection
100%
Successful Rate for Pupil Contour detection
90%
4.2 small problems and its analysis
Open
(a) A few pupil contours could not be detected accurately when eye is
near blinking/open
N
R
Because
Actual contour shape is ellipse which
is detected as 2 circles
—
A
(b) Pupil in some frames could not be detected near blinking/open.
Because
Dark region detection : Failed
Light intensity is too low near blinking/open.
4.3 Moving Forward
In the future, this project could be optimized in next aspects:
(1) The circle detection could be revised as circle/ellipse detection
(2) The sensitivities and thresholds of these algorithms could be
adjusted adaptively
Q&A
Thank You !
Rui Liu