Eyes and Displays © Dr. Zachary Wartell 3/23/2005 1 Light • Vision is perception of electromagnetic energy (EM radiation). • Humans can only perceive a very small portion of the EM spectrum: Gamma X UV Violet Blue 400 © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky Infra Radar Green 500 FM TV AM AC Yellow 600 Red 700 Wavelength (nm) 3/23/2005 2 Radiant-Energy Energy or Power Emission Spectrum Wavelength Energy [ joules]: E photon , f h f , Ef (h :Plank's Constant) hf all photons Esource f hf all photons Power[Watts=J/s] : © Dr. Zachary Wartell dE dt , radiant flux, radiant power 3/23/2005 3 Light in real world emission spectrum medium phototopic curve (eye sensitivity) reflection spectrum © Dr. Zachary Wartell 3/23/2005 4 Light in graphics RGB emission spectrum RGB RGB space “outside” display typically not computationally modeled Display medium reflection spectrum RGB pixels RGB © Dr. Zachary Wartell RGB 3/23/2005 5 Light interactions Light interacts with a surface in some combination of: • emission • reflection – on surface : mirror, specular or diffuse – suspended particles: random scattering • transmission – transparent, translucent, refraction • absorption © Dr. Zachary Wartell 3/23/2005 6 Eye Structure • The eye can be viewed as a dynamic, biological camera: it has a lens, a focal length, and an equivalent of film. • A simple diagram of the eye's structure: Cornea Retina Lens © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 7 Lens Basics: Light Refraction • Snell’s Law i sin r sin i r L ηi ηr N θi θi R reflected T θr refracted • η index of refraction – light speed in vacuum light speed in material – complications: varies with material temperature, light wavelength, anisotropic materials, double refraction © Dr. Zachary Wartell 3/23/2005 8 Thin Lens Equation o :object distance s :image distance 1 1 1 o i f f :focal distance o i f f © Dr. Zachary Wartell 3/23/2005 9 Thin Lens Equation o i f f • If the incident light comes from the object, we say it is a real object, and define the distance from the lens to it as positive. Otherwise, it is virtual and the distance is negative. • If the emergent light goes toward the image, we say it is a real image, and define the distance from the lens to it as positive. • f = positive for a converging lens • f often cited in measured in diopters (1/m) • A light ray through the center of the lens is undeflected. © Dr. Zachary Wartell, Dr. Larry Hodges 3/23/2005 10 Eye: The Lens • The lens must focus (accommodation) on directly on the retina for perfect vision: • But age, genetic factors, malnutrition and disease can unfocus the eye, leading to near- and farsightedness: Nearsighted © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 Farsighted 11 Eye: The Retina • The retina functions as the eye's "film". • It is covered with cells sensitive to light. These cells turn the light into electrochemical impulses that are sent to the brain. • There are two types of cells, rods and cones Retina © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 12 Numbers of rods or cones per mm2 The Retina: Cell Distribution 180,000 Blind spot 140,000 rod s 100,000 60,000 cones “Blind Spot Trick” © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky Optic di sk Te pe mpo rip ra he l ry Fovea 20,000 al ery s h Na rip pe (Right Eye) 3/23/2005 13 Dark Adaptation [Goldstein,pg 56] © Dr. Zachary Wartell 3/23/2005 14 The Retina: Rods • Sensitive to most visible frequencies (brightness). • About 120 million in eye. • Most located outside of fovea, or center of retina. • Used in low light (theaters, night) environments, result in achromatic (b&w) vision. • Absorption function: Cone Rod 400 © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky nm 3/23/2005 700 15 The Retina: Cones • R cones are sensitive to long wavelengths (nm), G to middle nm, and B to short nm. • R: 64%, 32% G, 2% B • About 8 million in eye. • Highly concentrated in fovea, with B cones more evenly distributed than the others (hence less in fovea). • Used for high detail color vision (CRTs!), so they will concern us most. © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 16 The Retina: Cones • The absorption functions of the cones are: 445 nm 535 nm 575 nm B G R 400 © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 700 3/23/2005 17 Color Constancy • If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? • This is the phenomenon of color constancy. • Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 18 Vision: Metamers • Because all colors are represented to the brain as ratios of three signals it is possible for different frequency combinations to appear as the same color. These combinations are called metamers. This is why RGB color works! • Example – [Goldstein,pg143] mix 620nm red light with 530nm green light matches color percept of 580 nm yellow 530 + 620 B 1.0 580 R B 5.0 8.0 1.0 G © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky,Wartell 3/23/2005 G R 5.0 8.0 19 Sensitivity vs Acuity • Sensitivity is a measure of the dimmest light the eye can detect. • Acuity is a measure of the smallest object the eye can see. • These two capabilities are in competition. – In the fovea, cones are closely packed. Acuity is at its highest, sensitivity is at its lowest (30 cycles per degree). – Outside the fovea, acuity decreases rapidly. Sensitivity increases correspondingly. © Kessler Dr. Zachary Wartell © , Watson, Hodges, Ribarsky 3/23/2005 20 Displays: Pixel • Pixel - The most basic addressable element in a image or on a display – CRT - Color triad (RGB phosphor dots) – LCD - Single color element • Resolution - measure of number of pixels on a image (m by n) – m - Horizontal image resolution – n - Vertical image resolution © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 21 Other meanings of resolution • Dot Pitch [Display] - Size of a display pixel, distance from center to center of individual pixels on display • Cycles per degree [Display] - Addressable elements (pixels) divided by twice the FOV measured in degrees. • Cycles per degree [Eye] - The human eye can resolve 30 cycles per degree (20/20 Snellen acuity). © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 22 Basic Image Synthesis Hardware (Raster Display) Peripheral Devices CPU System Memory raster images found here System Bus Framebuffer Display Processor Display Processor Memory © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, Video Controller 3/23/2005 23 Raster – Bit Depth • A raster image may be thought of as computer memory organized as a two-dimensional array with each (x,y) addressable location corresponding to one pixel. • Bit Planes or Bit Depth is the number of bits corresponding to each pixel. • A typical framebuffer resolution might be 1280 x 1024 x 8 1280 x 1024 x 24 1600 x 1200 x 24 © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 24 Displaying Color • There are no commercially available small pixel technologies that can individually change color. • spatial integration – place “mini”-pixels of a few fixed colors very close together. The eye & brain spatially integrate the “mini”-pixel cluster into a perception of a pixel of arbitrary color • temporal integration - field sequential color uses red, blue and green liquid crystal shutters to change color in front of a monochrome light source. The eye & brain temporally integrate the result into a perception of pixels of arbitrary color © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 25 CRT Display CRT Shadow Mask Electron Guns Red Input Green Input Blue Input Focusing System © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, Deflection Yoke Red, Blue, and Green Phosphor Dots 3/23/2005 26 Electron Gun •Contains a filament that, when heated, emits a stream of electrons. •Electrons are focused with an electromagnet into a sharp beam and directed to a specific point of the face of the picture tube. •The front surface of the picture tube is coated with small phosphor dots. •When the beam hits a phosphor dot it glows with a brightness proportional to the strength of the beam and how often it is excited by the beam. © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 27 Color CRT G •Red, Green and Blue electron guns. B G •Screen coated with phosphor triads. R B G R G R B B G •Each triad is composed of a red, blue and green phosphor dot. •Typically 2.3 to 2.5 triads per pixel. FLUORESCENCE - Light emitted while the phosphor is being struck by electrons. PHOSPHORESCENCE - Light given off once the electron beam is removed. PERSISTENCE - Is the time from the removal of excitation to the moment when phosphorescence has decayed to 10% of the initial light output. © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 28 Shadow Mask •Shadow mask has one small hole for each phosphor triad. •Holes are precisely aligned with respect to both the triads and the electron guns, so that each dot is exposed to electrons from only one gun. •The number of electrons in each beam controls the amount of red, blue and green light generated by the triad. SHADOW MASK Phosphor Dot Screen Red Green Blue © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, Convergence Point 3/23/2005 29 Scanning An Image Frame: The image to be scanned out on the CRT. •Some minimum number of frames must be displayed each second to eliminate flicker in the image. CRITICAL FUSION FREQUENCY •Typically 60-85 times per second for raster displays. •Varies with intensity, individuals, phosphor persistence, room lighting. © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 30 Interlaced Scanning 1/30 SEC 1/30 SEC 1/60 SEC 1/60 SEC 1/60 SEC 1/60 SEC FIELD 1 FIELD 2 FIELD 1 FIELD 2 FRAME FRAME Time •Display frame rate 30 times per second •To reduce flicker at lesser bandwidths (Bits/sec.), divide frame into two fields—one consisting of the even scan lines and the other of the odd scan lines. •Even and odd fields are scanned out alternately to produce an interlaced image. •non-interlaced also called “progressive” © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 31 Scanning (0,0) Device CS (alternate conventions) (0,0) VERTICAL SYNC PULSE — Signals the start of the next field. VERTICAL RETRACE — Time needed to get from the bottom of the current field to the top of the next field. HORIZONTAL SYNC PULSE — Signals the start of the new scan line. HORIZONTAL RETRACE — Time needed to get from the end of the current scan line to the start of the next scan line. © Dr. Zachary WartellZachary Wartell ©Larry F. Hodges, 3/23/2005 32 Example Video Formats NTSC – ? x 525, 30f/s, interlaced (60 fld/s) PAL – ? x 625, 25f/s, interlaced (50 fld/s) HDTV – 1920 x 1080i, 1280 x 720p XVGA – 1024x768, 60+ f/s, non-interlaced generic RGB – 3 independent video signals and synchronization signal, vary in resolution and refresh rate generic time-multiplexed color – R,G,B one after another on a single signal, vary in resolution and refresh rate © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell 3/23/2005 33 Calligraphic/Vector CRT Video Controller P0 P0 P1 P1 Line (P0,P1) older technology vector file instead of framebuffer wireframe engineering drawings flight simulators: combined raster-vector CRT © Dr. Zachary Wartell 3/23/2005 34 Flat-Panel Displays Flat-Panel Emissive Non-Emissive LCD DMD Plasma CRT (90°deflected) LED ActiveMatrix (TFT) PassiveMatrix Thin-Film electroluminescent © Dr. Zachary Wartell 3/23/2005 35 Flat-Panel Displays (Plasma) Flat-Panel Emissive Non-Emissive LCD DMD Plasma CRT (90°deflected) LED ActiveMatrix PassiveMatrix Thin-Film electroluminescent ToshibaTM, 42”, Plasma HTDV $4,500 (circa 2005) © Dr. Zachary Wartell 3/23/2005 36 Flat-Panel Displays (Plasma) [Hearn&Baker, pg 45] © Dr. Zachary Wartell 3/23/2005 37 Flat-Panel Displays (thin-film electroluminescent) [Hearn&Baker, pg 45] © Dr. Zachary Wartell 3/23/2005 38 Flat-Panel Displays (LED) Flat-Panel Emissive Non-Emissive LCD DMD Plasma CRT (90°deflected) LED ActiveMatrix PassiveMatrix Thin-Film electroluminescent © Dr. Zachary Wartell BarcoTM “Light Street” (LED) 3/23/2005 39 Flat-Panel Displays (DMD) Flat-Panel Emissive Non-Emissive LCD DMD Plasma CRT (90°deflected) LED Thin-Film electroluminescent ActiveMatrix PassiveMatrix 4 μm Digital Micro-mirror (DMD) © Dr. Zachary Wartell 3/23/2005 40 LCD • Liquid crystal displays use small flat chips which change their transparency properties when a voltage is applied. • LCD elements are arranged in an n x m array call the LCD matrix • Level of voltage controls gray levels. • LCDs elements do not emit light, use backlights behind the LCD matrix © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell 3/23/2005 41 LCD nematic liquid crystal details © Dr. Zachary Wartell 3/23/2005 [Hearn&Baker, pg 46] 42 LCD Components Small Diffuser LCD Linear fluorescent Module Polarizer Linear Color tubes Polarizer Filter © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell 3/23/2005 Wavefront distortion filter 43 LCD Resolution LCD resolution is occasionally quoted as number of pixel elements not number of RGB pixels. dot pitch Example: 3840 horizontal by 1024 vertical pixel elements = 4M elements Equivalent to 4M/3 = 1M RGB pixels "Pixel Resolution" is 1280x1024 © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell 3/23/2005 44 LCD • Passive LCD screens – Cycle through each element of the LCD matrix applying the voltage required for that element. – Once aligned with the electric field the molecules in the LCD will hold their alignment for a short time © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell • Active LCD (TFT) – Each element contains a small transistor that maintains the voltage until the next refresh cycle. – Higher contrast and much faster response than passive LCD – Circa 2005 this is the commodity technology 3/23/2005 45 LCD vs CRT flat & Lightweight low power consumption always some light pixel response-time (8-30ms) view angle limitations resolution interpolation required © Dr. Zachary Wartell ©Larry F. Hodges, Zachary Wartell heavy & bulky strong EM field & high voltage true black better contrast pixel response-time not noticeable inherent multi-resolution support 3/23/2005 46 Eye Versus 1280 x 1024 Display 66 cm 1280 pixel = 640 cycles 28° 33 cm Pictured: 22.8 c/d (cycles/degree with Vres=600/x → 26.25→ 20/26.25 vision (Snellen acuity) Widescreen at 60° → 20/56.25 vision © Dr. Zachary Wartell 3/23/2005 47 Measurements • Radiometry • Photometry • “Brightness” © Dr. Zachary Wartell 3/23/2005 48 Radiometry Measures Quantity SI unit Abbr. Symbol Notes radiant energy Joule J Q Energy radiant flux/power Watt W Φ Energy per unit time radiant intensity Watt per steradian W/sr I Power per unit solid angle radiance Watt per steradian per meter2 W/(sr L ∙m2) Power per unit solid angle per unit projected source area irradiance Watt per meter2 W/m2 E power incident on a surface radiant Watt per emittance/exitance meter2 W/m2 M power emitted from a surface © Dr. Zachary Wartell 3/23/2005 49 Intensity (Watts/steradian) d I d dω=dA/r2 dA r dΦ N θ dω y x © Dr. Zachary Wartell 3/23/2005 50 Irradiance (Watts/m2) ●sums radiance over hemisphere arriving on a surface dω θ N x © Dr. Zachary Wartell y 3/23/2005 51 Radiant Emittance (Watts/m2) ●sums radiance over hemisphere leaving a surface dω θ N x © Dr. Zachary Wartell y 3/23/2005 52 Radiance ●measure in specific direction d L d ds0 cos 2 dΦ N ds0 x © Dr. Zachary Wartell θ dω y 3/23/2005 53 Planckian Radiator • “black body” radiator – radiate energy perfectly – absorb light perfectly (no reflection) – as the radiator temperature rises how does the spectral power distribution change? SPD 3000K Φ (W) 2000K 1000K λ (nm) © Dr. Zachary Wartell 3/23/2005 54 Graph of Theoretic Planckian Radiator SPD 20000 K 10000 K 5600 K 3000 K 2000 K 1000 K © Dr. Zachary Wartell 3/23/2005 55 Planckian radiator SPD in Visible Region 1000K 2000K 3000K 4000K 5000K 6000K 8000K 10000K 100,000K • increasing temperature (1000K to 100,000K) yields red, orange-red, yellowish-white, bluish-white • tungsten bulb example © Dr. Zachary Wartell 3/23/2005 56 Examples SPD’s [McDonald] © Dr. Zachary Wartell 3/23/2005 57 Examples SPD’s [McDonald] © Dr. Zachary Wartell 3/23/2005 58 Color Temperature • if light source SPD is similar to black body radiator, associate the source’s color with the temperature at which a Planckian radiator will give similar color Source Temp. candle 1200K tungsten lamp 2800K bright midday 6000K sun heavy overcast sky © Dr. Zachary Wartell 10000K 3/23/2005 59 Photometry 45 W retinal stimulation ? Conductors 60 W Σ -15 W 1.0 photopic curve Vλ X W © Dr. Zachary Wartell ? = 900 lm nm 400 3/23/2005 700 60 Photometry & Lumens Φ (W) Σ 1.0 photopic curve Vλ X λ (nm) 400 700 760 Φv 683 V 380 1 W of 555nm yields 683 lm © Dr. Zachary Wartell 3/23/2005 61 Photometry Measures Quantity SI unit Abbr. Symbol Notes luminous energy lumen second lm ∙ s Qv Energy luminous flux/power lumen lm Φv Energy per unit time luminous intensity candela Iv cd (=lm/sr) luminance candela/meter2 cd/m2 Lv luminous flux per unit solid angle per unit projected area illuminance lux (lm/m2) lx Ev light incident on a surface luminous lux (lm/m2) emittance/exitance lx Mv light emitted from a surface © Dr. Zachary Wartell 3/23/2005 lum. flux per solid angle 62 Brightness Measures (!) • very hard to define analytically (no simple Vλ) • complications – contrast effects • Gelb’s (1929) experiment – breaks light constancy – adaptation effects • dark adapt one eye • above effects can cause same SPD to be perceived differently • active area of psychophysics research © Dr. Zachary Wartell 3/23/2005 63 Contrast Effects Example © Dr. Zachary Wartell 3/23/2005 64 Colorimetry: Measuring Color • Colorimeter: adjust primaries so that: “C[C] = R[R] + B[B] + G[G]” R[R] white screen G[G] B[B] eye black partition view hole with surround C[C] © Dr. Zachary Wartell 3/23/2005 65 Negative tristimulus values • very pure target color may be unmatchable C[C] ≠ R[R]+ G[G]+ B[B] for any (R,G,B) • all we can do is de-saturate the target color C[C] + R[R] = G[G]+ B[B] • this could be formulated as negative coordinates C[C] = -R[R] + G[G]+ B[B] • No set of real primaries will allow for positive coordinates to match all real colors! © Dr. Zachary Wartell 3/23/2005 66 An Early Experiment (1931) • Primaries: [R]=700nm, [G] = 546.1, [B] = 435.8 • Determine tristimulus values (R,G,B) for set of target stimulus {[Ci]} where [Ci] is a single spectral color (i.e. SPD contains 1 wavelength) • Use (R,G,B)λ to define distribution curves 435.8 546.1 700 r, g, b © Dr. Zachary Wartell 3/23/2005 67 Compute tristimulus value (R,G,B) for any SPD • Multiply SPD by r , g , b and compute area under curve: Let SPD be E E r , G energy 360 780 E g 360 , B Eλ λ R E b r λ Eλ r Yuck! (R,G,B) can be < 0! λ © Dr. Zachary Wartell 780 360 tristimulus R 780 3/23/2005 68 Transform tristimulus values between primaries If C[C] = Rc1[R1]+ Gc1[G1] + Bc1[B1] and 1[R1] = RR2[R2]+ GR2 [G2] + BR2 [B2] 1[G1] = RG2[R2]+ GG2[G2] + BG2[B2] 1[B1] = RB2[R2]+ GB2[G2] + BB2[B2] then C[C]=Rc1 (RR2[R2]+ GR2 [G2] + BR2 [B2])+ Gc1 (RG2[R2]+ GG2[G2] + BG2[B2])+ Bc1 (RB2[R2]+ GB2[G2] + BB2[B2]) =(Rc1RR2 +Gc1 RG2 +Bc1 RB2 ) [R2] + (Rc1GR2 +Gc1 GG2 +Bc1 GB2 ) [G2] + (Rc1BR2 +Gc1 BG2 +Bc1 BB2 ) [B2] + = Rc2[R1]+ Gc2[G1] + Bc2[B1] © Dr. Zachary Wartell 3/23/2005 69 Use “Imaginary” primaries to allow > 0 (R,G,B) • CIE primaries defined by: C700 0.73467[X] 0.26533[Y] 0.00000[Z] C546 0.27376[X] 0.71741[Y] 0.00883[Z] C435 0.16658[X] 0.00886[Y] 0.82456[Z] • Solve for (X,Y,Z) over spectrum yields: z y © Dr. Zachary Wartell x 3/23/2005 Yey! (X,Y,Z) always > 0! 70 CIE XYZ Space Y Features: - X,Y,Z > 0 -[Y] follows Vλ -equal energy SPD has coordinate (k,k,k) X Z © Dr. Zachary Wartell 3/23/2005 71 Chromaticity • chromaticity – independent on amount of luminous energy, dependent on dominant wavelength and saturation X Y Z x , y , z X Y Z X Y Z X Y Z • Note: – x+y+z=1 – x , y , z are on X+Y+Z=1 plane – if we know x , y, z is just 1- x – y, so we define chromaticity by just x , y. – use 2D plot of x , y by projecting X+Y+Z=1 plane onto X,Y plane – if we need full X,Y,Z we need to know (x , y ,Y) where Y encodes the luminous energy © Dr. Zachary Wartell 3/23/2005 72 Chromaticity Diagram C - white light B – dominant wavelength of A D & E – complementary AC/BC – excitation purity F – nonspectral (G) dom. wave. is complement of H B A H E D © Dr. Zachary Wartell C F G 3/23/2005 73 Color Gamut of Displays [G1] [G2] [B1] © Dr. Zachary Wartell [B2] [R1] [R2] 3/23/2005 74 Correlated Color Temperature Planckian locus (labeled in Kelvin) ∞ © Dr. Zachary Wartell (x,y) of colors with c.c.t of 10000K 3/23/2005 75 Color perception more complicated • Jameson (1985) – SPD of blue chip under tungsten light = yellow chip under sunlight, yet its still blue (color constancy) • color constancy only approximate • chromatic adaptation – red adapt one eye & compare perception between eyes – eye adapts to tungsten light long wv’s indoors (white paper appears white), but if adapted to nighttime looking inside light appears yellow © Dr. Zachary Wartell 3/23/2005 76 White Point • visual system scales response from photoreceptor according to illuminant color – white point of scene is reference point for perception • display white point – result of maximum output on all 3 channels to display – described by c.c.t. or chromaticity (x,y) – sometimes called c.c.t. or c.t. (yuck!) – adjust internally (bias & gain) or digitally in graphics card © Dr. Zachary Wartell 3/23/2005 77