Lecture 7
The Eye and Neuromorphic Vision
1
Outline
•
•
•
•
•
•
The eye and the retina
Artificial Vs biological vision systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
2
• Eyeball Cross Section and Retina
3
Retina Cells
4
Schematized Retinal Cells
5
Outline
•
•
•
•
•
•
The eye and the retina
Artificial Vs biological vision systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
6
Frame-based Sensors Vs Biological
Vision
Artificial Vision
Biological Systems
Frame-based
Continuous Detection
Global AGC
Local AGC
Redundancy
Low redundancy
High power consumption
Low power consumption
7
Notion of Frame
Tframe
Tintegration
Time
Tintegration<Tframe
8
Dynamic Range
The human eye has a dynamic range higher than 10 decades
Bad choice of the integration time
Rüedi et al, JSSC 2003 ”A 128x128 Pixel
120dB Dynamic-Range Vision-Sensor Chip
for Image Contrast and Orientation
Extraction”
9
Outline
•
•
•
•
•
•
The eye and the retina
Artificial Vs biological vision systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
10
Depletion Region and Phototransduction
11
Silicon
Electromagnetic Spectrum
12
Some Possible Photodetectors in CMOS Technology
A. Moini. ‘Vision Chips’. Kluwer Academic Publishers,
2000.
13
Photogates
14
Logarithmic Photoreceptors
(a)
(b)
15
Photoreceptors with Negative Feedback
(a)
(b)
16
Classic Active Pixel Sensor (APS)
17
Outline
•
•
•
•
•
•
The eye and the retina
Artificial vision Vs biological systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
18
Read-out Strategies:



Addressing/Scanning
Charge Coupled Devices (CCD)
Address Event Representation (AER)
19
Addressing/Scanning
Token System
20
Charge Coupled Devices (CCD)
21
Address Event Representation (AER) (I)
22
Address Event Representation (AER) (II)
23
Outline
•
•
•
•
•
•
The eye and the retina
Artificial Vs vision biological systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
24
Delbrück’s Adaptative Photo Cell (I)
25
Delbrück’s Adaptative Photo Cell (II)
Non-linear element
26
Mahowald’s Pixel
27
Retina Pixel Boahen
28
Dynamic Vision Sensor (DVS)
P. Lichtsteiner, C. Posch, and T. Delbrück, “A 128 128 120 dB 15 µs latency asynchronous temporal
contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008.
29
Common Source Amplifier
Can used as
comparator
30
Examples of Biomorphic Vision Sensors
 Spatio-Temporal Contrast Detector:
http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html
 Dynamic Vision Sensor:
http://folk.uio.no/juanle/Projects/DVS.html
31
Outline
•
•
•
•
•
•
The eye and the retina
Artificial vision Vs biological systems
Fundamentals of photo receptors
Read-out strategies
Examples of neuromorphic vision systems
Further processing
32
Further Image Processing
• Motion Detection
• Features extraction
33
Token Based Motion Processing: Reichardt Detector
34
Intensity Based
35
Features Extraction
36
’Difference of Gaussians’ Kernel (I)
37
’Difference of Gaussians’ Kernel (II)
38
Result of Edge Detection
39
45 Degrees Edge Kernel (I)
40
45 Degrees Edge Kernel (II)
41