Lecture 7 The Eye and Neuromorphic Vision 1 Outline • • • • • • The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 2 • Eyeball Cross Section and Retina 3 Retina Cells 4 Schematized Retinal Cells 5 Outline • • • • • • The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 6 Frame-based Sensors Vs Biological Vision Artificial Vision Biological Systems Frame-based Continuous Detection Global AGC Local AGC Redundancy Low redundancy High power consumption Low power consumption 7 Notion of Frame Tframe Tintegration Time Tintegration<Tframe 8 Dynamic Range The human eye has a dynamic range higher than 10 decades Bad choice of the integration time Rüedi et al, JSSC 2003 ”A 128x128 Pixel 120dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction” 9 Outline • • • • • • The eye and the retina Artificial Vs biological vision systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 10 Depletion Region and Phototransduction 11 Silicon Electromagnetic Spectrum 12 Some Possible Photodetectors in CMOS Technology A. Moini. ‘Vision Chips’. Kluwer Academic Publishers, 2000. 13 Photogates 14 Logarithmic Photoreceptors (a) (b) 15 Photoreceptors with Negative Feedback (a) (b) 16 Classic Active Pixel Sensor (APS) 17 Outline • • • • • • The eye and the retina Artificial vision Vs biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 18 Read-out Strategies: Addressing/Scanning Charge Coupled Devices (CCD) Address Event Representation (AER) 19 Addressing/Scanning Token System 20 Charge Coupled Devices (CCD) 21 Address Event Representation (AER) (I) 22 Address Event Representation (AER) (II) 23 Outline • • • • • • The eye and the retina Artificial Vs vision biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 24 Delbrück’s Adaptative Photo Cell (I) 25 Delbrück’s Adaptative Photo Cell (II) Non-linear element 26 Mahowald’s Pixel 27 Retina Pixel Boahen 28 Dynamic Vision Sensor (DVS) P. Lichtsteiner, C. Posch, and T. Delbrück, “A 128 128 120 dB 15 µs latency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008. 29 Common Source Amplifier Can used as comparator 30 Examples of Biomorphic Vision Sensors Spatio-Temporal Contrast Detector: http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html Dynamic Vision Sensor: http://folk.uio.no/juanle/Projects/DVS.html 31 Outline • • • • • • The eye and the retina Artificial vision Vs biological systems Fundamentals of photo receptors Read-out strategies Examples of neuromorphic vision systems Further processing 32 Further Image Processing • Motion Detection • Features extraction 33 Token Based Motion Processing: Reichardt Detector 34 Intensity Based 35 Features Extraction 36 ’Difference of Gaussians’ Kernel (I) 37 ’Difference of Gaussians’ Kernel (II) 38 Result of Edge Detection 39 45 Degrees Edge Kernel (I) 40 45 Degrees Edge Kernel (II) 41