Volume Graphics - Technology to Tools
advertisement
Volume Graphics Technology to Tools David R. Nadeau Principal Scientist San Diego Supercomputer Center University of California, San Diego, USA SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Technology & Tools • Technology = a useful idea – Polygons, images, volumes, transforms, shading, … • Tool = an idea put to use – Visualization, art, story telling, games, … • As a technology matures, it becomes a tool – We focus more on its use, than its mechanism – Use really explodes when artists begin to use it SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Technology & Tools • Written storytelling – dramatically advanced by… – Printing press, movable type, word processing, e-mail, Internet, … • Music composition – dramatically advanced by… – Standardized notation, equal-tempered tuning, piano, electric guitar, synthesizer, digital signal processing, … • Computer graphics – dramatically advanced by… – Hmmm…? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Topics of the Talk • History – What has happened so far? • Rendering – – – – What can we do now? What would we like to do? What might we be able to do in the future? How might this affect the way we render volumes? • Authoring – How might rendering issues affect how we create volumes? – What might future authoring tools look like? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring A Brief Look at Graphics History… • What are some major events? • What did they enable? • Surface vs. Volume graphics • Technology vs. Story telling vs. Games SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Evolution of surface graphics • Selected Technology events: 1963 “Sketchpad” 1965 Digital line drawing 1971 Gouraud shading Hidden surface removal 1974 Polygon clipping 1975 Phong shading 1972 Scene graphs 1976 Texture mapping Shadows 1977 Standard lighting model 1978 Bump mapping I.E Sutherland, "Sketchpad: A Man-Machine Graphical Communication System," Ph.D. Thesis, MIT, 1962. J.E. Bresenham, “Algorithm for computer control of a digital plotter,” IBM Systems Journal, 4(1), 1965. H. Gouraud, “Continuous shading of curved surfaces,” IEEE Transactions on Computers, C-20, 1971. M.E. Newell, R.G. Newell, T.L. Sancha, “A Solution to the Hidden Surface Problem,” Proceedings of ACM National Meeting, 1972. I.E. Sutherland, G.W. Hodgman, “Reentrant Polygon Clipping,” CACM, 17(1), 1974. B.T. Phong, “Illumination for Computer Generated Pictures,” CACM, 18(6), 1975. J.H. Clark, “Hierarchical Geometric Models for Visible Surface Algorithms,” CACM, 19(10), 1976. J.F. Blinn, M.E. Newell, “Texture and Reflection in Computer Generated Images,” CACM, 19(10), 1976. F.C. Crow, “Shadow Algorithms for Computer Graphics,” Computer Graphics, 11(2), 1977. J.F. Blinn, “Models of Light Reflection for Computer Synthesized Pictures,” Computer Graphics, 11(2), 1977. J.F. Blinn, “Simulation of Wrinkled Surfaces,” Computer Graphics, 12(3), 1978. SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Evolution of surface graphics • Selected Technology events: 1980 Ray tracing Geometry VLSI 1982 AutoCAD Particle systems 1983 Alias Radiosity Shaders 1984 SGI IRIS 1000 Wavefront 1986 Rendering eq. 1989 RenderMan 1992 OpenGL 1993 Direct3D 1994 GLINT 300SX 1995 VRML Voodoo 1997 Riva 128 T. Whitted, “An improved illumination model for shaded display,” CACM, 23(6), 1980. J.H. Clark, “The Geometry Engine: A VLSI geometry system for graphics,” SIGGRAPH 82. Autodesk Inc. W.T. Reeves, “Particle Systems – A Technique for Modeling a Class of Fuzzy Objects,” SIGGRAPH 83. Alias Research Inc. C.M. Goral, K.E. Torrance, D.P. Greenberg, B. Battaile, “Modeling the Interaction of Light Between Diffuse Surfaces,” SIGGRAPH 84. R.L. Cook, “Shade trees,” SIGGRAPH 84. Silicon Graphics Wavefront Inc. J.T. Kajiya, “The rendering equation,” SIGGRAPH 86. PIXAR OpenGL ARB Microsoft 3Dlabs VRML/Web3D Consortium 3dfx nVidia SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Too many papers to list History Rendering Authoring Evolution of surface graphics • Selected Story Telling events: 1982 1984 1985 1986 1987 1988 1989 1991 1992 Tron Wrath of Khan Andre & Wally B. The Last Starfighter Young Sherlock Holmes Luxo Jr. Red’s Dream Stanley & Stella Tin Toy Knickknack The Abyss Beauty and the Beast Terminator 2 Aladdin Lawnmower Man SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Disney Paramount PIXAR Universal Paramount PIXAR PIXAR Symbolics PIXAR PIXAR Fox Disney Tri-Star Disney New Line All images copyright by Disney, Fox, Paramount, New Line, PIXAR, and Tri-Star, as appropriate. History Rendering Authoring Evolution of surface graphics • Selected Story Telling events: 1993 Jurassic Park The Mask 1994 Reboot True Lies 1995 Toy Story Dragonheart 1996 Twister Geri’s Game Lost World 1997 Star Wars, Special Edition Titanic A Bug’s Life 1998 Antz Toy Story 2 1999 Star Wars, Episode 1 2000 Dinosaur SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Universal New Line MainFrame Fox PIXAR Universal Warner Bros. PIXAR Universal Lucasfilm Paramount PIXAR DreamWorks PIXAR Lucasfilm Disney All images copyright by Disney, DreamWorks, Paramount, Lucasfilm, MainFrame, PIXAR, and Universal, as appropriate. History Rendering Authoring Evolution of surface graphics Catacomb 3-D • Selected Game events: 1991 Catacomb 3-D Wolfenstein 3-D 1992 7th Guest DOOM 1993 Myst 1995 Dark Forces Nintendo 64 1996 Quake Tomb Raider Jedi Knight 1997 Quake II Riven Half-Life 1998 Thief Unreal 1999 Quake III: Arena 2000 RealMyst Id Software Id Software Tilobyte Studios Quake Id Software Cyan Productions Tomb Raider LucasArts Nintendo Id Software Eidos Interactive Jedi Knight Half-Life LucasArts Id Software Cyan Productions Sierra Eidos Interactive Epic MegaGames Unreal Quake III: Arena Id Software Cyan Productions SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego DOOM All images copyright by Id Software, Cyan Productions, Eidos Interactive, Epic MegaGames, LucasArts, and Sierra, as appropriate. History Rendering Authoring Evolution of surface graphics 1960’s 1970’s 1980’s 1990’s Games Story Telling Technology SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2000’s History Rendering Authoring Evolution of volume graphics • Selected Technology events: Volume ray casting 3D texture 1985 generation 1984 1987 Marching cubes 1989 Hypertexture 1990 Splatting 1991 Volume sculpting Shear-warp 1994 3D texture mapping VolumePro 1999 GeForce 256 GeForce2 2000 DirectX 8.0 H. Tuy, L. Tuy, “Direct 2D display of 3D objects,” Computer Graphics and Applications, 4(10), 1984. K. Perlin, “An image synthesizer,” SIGGRAPH 85. W.E. Lorensen, H.E. Cline, “Marching cubes: A high resolution 3D surface construction algorithm,” SIGGRAPH 87. K. Perlin, E.M. Hoffert, “Hypertexture,” SIGGRAPH 89. L. Westover, “Footprint evaluation for volume rendering,” SIGGRAPH 90. T.A. Galyean, J.F. Hughes, “Sculpting: An interactive volumetric modeling technique,” SIGGRAPH 91. P. Lacroute, M. Levoy, “Fast volume rendering using a shear-warp factorization of the viewing transformation,” SIGGRAPH 94. B. Cabral, N. Cam, J. Foran, “Accelerated volume rendering and tomographic reconstruction using texture mapping hardware,” Volume Visualization Symposium, 1994. Real-Time Visualization, Mitsubishi nVidia nVidia Microsoft SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Too many papers to list History Rendering Authoring Evolution of volume graphics • Selected Story Telling events: 1997 Contact 1998 Sphere Contact Warner Bros. Warner Bros. Sphere SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego All images copyright by Warner Bros., San Diego Supercomputer Center, and the American Museum of Natural History, as appropriate. History Rendering Authoring Evolution of volume graphics • Selected Game events: 2001 ? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego ? History Rendering Authoring Evolution of volume graphics 1960’s 1970’s 1980’s 1990’s Games Story Telling Technology SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2000’s History Rendering Authoring Evolution of volume graphics • Volume graphics today… …is where surface graphics was 15 years ago – We are at the start of a transition from technology to tool • What enabled story telling and games for surface graphics? • What might do the same for volume graphics? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Enabling events Surface graphics 1960’s 1970’s 1980’s 1990’s 2000’s Games Story Telling Technology • 1st business-level (expensive): 1982-4 – Attention-grabbing event: TRON – Interactive rendering hardware: SGI – Authoring tools: Alias, Wavefront • 1st consumer-level (cheap): – Attention-grabbing event: Catacomb3D, DOOM – Interactive rendering hardware: Voodoo SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 1991-3 1997 History Rendering Authoring Enabling events Volume graphics 1960’s 1970’s ? 1980’s Story Telling Technology – Attention-grabbing event: ? – Interactive rendering hardware: VolumePro, SGI? – Authoring tools: ? – Attention-grabbing event: ? – Interactive rendering hardware: nVidia? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2000’s Games • 1st business-level (expensive): • 1st consumer-level (cheap): 1990’s 1999 2000 History Rendering Authoring Interactive Volume Rendering… • What can we do now? • What would we like to do? • What might we be able to do in the future? • How might this affect the way we render volumes? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What can we do now? • Canonical volume visualizations… • What can we do interactively? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What can we do now? Memory Fill rate Texture rate Max stored volume Max volume RTviz VolumePro nVidia SGI IR2 GeForce2Ultra (1 pipe, 4 RMs) 256 MB N/A N/A 16 x 2563 (scalar) 2563 @ 30 fps 64 MB 1 Gpixels/s 2 Gtexels/s 2563 (RGBα) 2563 @ 15 fps 64 MB 896 Mpixels/s 768 Mtexels/s 2563 (RGBα) 2563 @ 12 fps • But 2563 is a small volume… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Product literature History Rendering Authoring What resolution can we see? • Eye’s lens focuses light onto retina – Fovea = focus area = center 20 • More receptors at the fovea – Mostly “Cones” (color) at fovea – Mostly “Rods” (intensity) in surrounding area Fovea Periphery SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Perception, Sekuler & Blake, 2nd ed, 1990, McGraw-Hill. History Rendering Authoring What resolution can we see? • Measure visual acuity by viewing “gratings” of parallel lines Gratings • How fine a grating can you see? • Normal vision: • 120 lines/degree in center 20 = 2,400 lines • Legally blind: • 1/10th normal vision • 12 lines/degree in center 20 = 240 lines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Visual Perception, Spillmann & Werner, 1990, Academic Press. History Rendering Authoring What resolution can we see? • A 2563 volume legally blind! – Very low resolution • Normal vision requires 2,4003! – If it only covers central 20 of visual field • 180 visual field requires 21,6003! – 120 lines/degree in 180 = 21,600 lines! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What resolution can we show? • We’re constrained by screen resolution – 1280x1024 on a large monitor – 30 pixels/degree at normal distance – ¼ normal vision = visually impaired • For now… – 10243 is a minimum for effective screen use • Smaller volumes are blurry (1 voxel covers multiple pixels) – But we want much higher volume resolutions… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What resolution do we want? • Medical imaging (cryosections) – State of the art: 2048 x 2048 images, 1/10mm apart – 2048 x 2048 x 18000 for full body (180cm person) – Visible Human Male: 2048 x 1216 x 1871 • Re-digitization of images: 4096 x 2700 x 1871 – Visible Human Female: 2048 x 1216 x 5189 – Recent brain only: 1789 x 1472 x 1152 SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Head data from the Visible Human Project, www.nlm.nih.gov/research/visible Brain data from Arthur Toga’s lab, LONI, UCLA, www.loni.ucla.edu History Rendering Authoring What resolution do we want? • Medical imaging (CT scan) – State of the art: 2048 x 2048 images, ½mm apart – 2048 x 2048 x 3600 for full body (180cm person) – Visible Human Male: SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 512 x 512 x 1871 Head data from the Visible Human Project, www.nlm.nih.gov/research/visible History Rendering Authoring What resolution do we want? • Microscopy imaging (fluorescence) – State of the art: 1024 x 1024, 0.15 microns apart – 1024 x 1024 x 75 for a 10 micron cell – Cell membranes: SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 768 x 768 x 72 Cell data from Hiro Tsukada, Eric Elenko, and Maria Pinhal, UCSD Cancer Center History Rendering Authoring What resolution do we want? • Simulations – Tend to use all available memory on a supercomputer – SDSC IBM SP “Blue Horizon” • 1152 processors (144 nodes, 8 cpus/node, +others), 1.7 teraflops • 576 Gbytes total memory (4 Gbytes/node) • Largest possible volume is 50003 – Ocean simulation: SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2160 x 960 x 30 Ocean data from Detlef Stammer, UCSD, Scripps Institute of Oceanography History Rendering Authoring What resolution do we want? • Combine to build volumetric scenes – Composite overlapping volumes – Mosaic volumes to fill space – Combine procedural elements SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego What resolution do we want? e e w C ry F em ) C os ec a l ry o t io e F B ra C r s e c n u y t i ll B n C o s e io n o d ry o c t y i C se on ry c o t io se n ct io n V H F M M u ll a le em B o CT b d ra y F C u ne ll T C M ic el l M ro s ic c o ro p O sc y S c D e o S an p y C S M i m ax u S la im t io u la n t io n 1 ,0 0 0 ,0 0 0 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 ,0 0 0 ,0 0 0 1 0 ,0 0 0 ,0 0 0 ,0 0 0 1 ,0 0 0 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 ,0 0 0 1 0 ,0 0 0 ,0 0 0 1 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 1 0 ,0 0 0 1 ,0 0 0 100 10 1 (n V H e V H M al V H M al N u m b er o f vo xels History Rendering Authoring • We’re a long way from what we want… why? It’s a logarithmic scale! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Full eye Fovea Full screen 2563 History Rendering Authoring Memory and bandwidth • Memory needed grows with cube of volume’s width 643 RGBα 1283 RGBα 2563 RGBα 5123 RGBα 10243 RGBα = 1 Mbyte = 8 Mbytes = 64 Mbytes = 512 Mbytes = 4096 Mbytes 4096 3072 Mbytes – – – – – 2048 1024 0 0 256 512 Resolution • Bandwidth needed also grows with frame rate – 10 fps minimum – 30 fps better – 70 fps best SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 768 1024 History Rendering Authoring What will volumes require? 2563 Memory 10 fps 30 fps 64 MB 640 MB/s 1.9 GB/s 70 fps 4.5 GB/s (legally blind) 10243 4 GB 40 GB/s (full screen) 24003 120 GB/s 180 GB/s (30 Gpixl/s) 55 GB 550 GB/s 1.6 TB/s 3.8 TB/s 40 TB 400 TB/s 280 TB/s (fovea) 216003 120 TB/s (full eye) • 120 GB/s is > 30 times current bandwidths! SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Growth of pixel fill rates 1200 F ill rate, Mp ixels/s 1000 800 600 SGI PC cards 400 200 • Fill rates recently growing by x2 every year SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 0 * Not counting custom hardware or special configurations 1996 30 Mpixels/s 3Dlabs Permedia 1997 100 Mpixels/s nVidia RIVA 128 1998 366 Mpixels/s 3dfx Voodoo 3 1999 540 Mpixels/s ATI Rage Fury MAXX 2000 1000 Mpixels/s nVidia GeForce2 Ultra Data source: Product literature History Rendering Authoring How long to get there? Bandwidth @ 30 fps Years 2563 1.9 GB/s (legally blind) (0.5 Gpixels/s) 10243 120 GB/s (full screen) (30 Gpixels/s) 24003 1600 GB/s (fovea) (400 Gpixels/s) 216003 120000 GB/s (full eye) (30,000 Gpixels/s) • But, this is not very realistic… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Last year 5 years 8 years 14 years History Rendering Authoring How long to get there? About trends… “A frequent criticism of predictions of the future is that they rely on mindless extrapolation of current trends without consideration of forces that may terminate or alter that trend.” – Ray Kurzweil, The Age of Spiritual Machines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring How long to get there? About the computer industry… “If the automobile industry had made as much progress in the past fifty years, a car today would cost a hundredth of a cent and go faster than the speed of light.” – Ray Kurzweil, The Age of Spiritual Machines SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring CPU performance growth 90 80 70 SPECfp95 60 50 40 30 20 10 0 1994 1995 1996 1997 1998 1999 2000 2001 • Floating point power grows by x2 every 2 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: SPEC benchmark database, www.spec.org History Rendering Authoring CPU performance growth • At this rate, by 2020 transistor insulators will be just a few atoms thick – A possible end to growth using this technology – What technology will arrive next? • Meanwhile, back to the present… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Memory speed growth 140 Memory bus speed, MHz 120 100 80 60 40 20 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 1990 1989 1988 1987 1986 1985 0 • PC processors only • Memory bandwidth only grows by x2 every 5 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Kingston Technology, www.kingston.com History Rendering Authoring Comparing growth rates 40 Processor performance growth 35 Increase factor 30 Memory bus speed growth Pixel fill rate growth 25 20 15 10 5 0 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 • Highly unlikely that fill rates can continue to grow faster than processor speed and memory bandwidth SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What should we do? • Don’t bet on rapid fill rate growth to satisfy our volume rendering needs – If fill rate growth drops to track memory bandwidth growth, full screen volume rendering may arrive in 25 years, not 5 years • And full eye in 70 years! • Faster hardware is no substitute for a better algorithm – How can we change the way we work? – Lots of ways… but I’ll focus on a few… SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Change how we render • Object-space rendering traversals are in common use – Scan through all voxels and project towards screen • Splatting, shear-warp, 3D texture mapping, point clouds – Memory streaming is possible • If the viewpoint is outside the volume • O(n3) time & space n = volume width SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Change how we render • Image-space traversals are more time/space friendly – Scan through all screen pixels and project towards voxels • Ray casting, … – Memory streaming not possible • Nearly random access • O(k r n) time & space n = volume width r = image resolution (width x height) k = additional computation factor (for comparisons) SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Change how we render • Is volume ray casting really O(k r n)? – Worst case: each ray takes longest path = n 3 – Normal case: early ray termination reduces it – Rays diverge, skipping voxels, introducing aliasing – For accuracy: supersample – For speed: use volume MIP-mapping • Upfront cost to build MIP-map volumes • Increases memory needed (but not bandwidth) – Memory is cheap, bandwidth is not SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Rendering time growth 8192 7168 6144 5120 4096 3072 2048 fovea full screen 1024 legally blind 0 Rendering Time Object space traversal Image space traversal (k=5) Image space traversal (k=10) Image space traversal (k=15) Image space traversal (k=20) Volume size • To interactively render larger volumes, we need to get on the better curves – such as ray casting SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring What can we do? • Changing our rendering algorithm can help M H H e V H M al V (n al e e w C ry F em ) C os ec a l ry o t io e F B ra C r s e c n u y t i ll B n C o s e io n o d ry o c t y i C se on ry c o t io se n ct io n V H F M M u ll a le em B o CT b d ra y F n C u ll e M T C e l ic r l M os ic c o ro p sc y S Oc D ea o S p y C nS M a x im u S la im t io u la n t io n 1 ,0 0 0 ,0 0 0 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 ,0 0 0 ,0 0 0 1 0 ,0 0 0 ,0 0 0 ,0 0 0 1 ,0 0 0 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 ,0 0 0 1 0 ,0 0 0 ,0 0 0 1 ,0 0 0 ,0 0 0 1 0 0 ,0 0 0 1 0 ,0 0 0 1 ,0 0 0 100 10 1 V N u m b er o f vo xels – For “large data,” image-space traversals are better than object-space traversals – We have “large data” now • What about changing our data? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Cross-over History Rendering Authoring Volume Authoring… • How might rendering issues affect how we create volumes? • What might future authoring tools look like? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring CPU vs. memory speeds 10.0 CPU MHz / Memory MHz 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 • Main memory access (cache miss) 1.0 0.0 1994 1995 1996 1997 1998 1999 2000 2001 • PC processors, excluding RDRAM & DDR SDRAM • Slower memory speed growth means memory references are getting more costly, in terms of cycles SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: SPEC benchmark database, www.spec.org History Rendering Authoring CPUs per Supercomputer 10000 Number of processors Average of top 100 1000 424 230 100 99 581 281 117 128 173 687 226 806 Average of top 500 281 10 1 1994 1995 1996 1997 1998 1999 2000 2001 • Adapt to slow memory by adding CPUs w/own memory • CPUs/Supercomputer grows by x2 every 3 years SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data source: Top 500 Supercomputers, www.top500.org History Rendering Authoring Cycles & bandwidth • Cycles are more available than memory bandwidth – “Easier” to add CPUs than increase bandwidth – Parallel computing is an obvious industry trend • But CPU coordination and data sharing is still bandwidth limited • Can we trade computation for memory accesses? – Data compression is clearly needed • Texture compression nearly standard in graphics hardware • Geometry compression becoming available – Additional techniques to reduce memory access costs • Interleaved frame buffers, Z-buffer cache • View frustum culling, occlusion culling SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring with cycles • “Shaders” compute parts of the scene as needed – Already common-place in software rendering • “Data amplification” – small number of parameters generates large amount of scene content – Programmable graphics pipelines arriving now – For surface graphics: splines, procedural textures, particles – For volume graphics: voxelization, procedural volumes SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring with cycles • Shaders aren’t just for shading any more – Procedural content creation • Noise and turbulence functions – Voxelization based upon geometry “parameters” • Voxelize on demand – don’t prevoxelize • Authoring uses a mix of data and shaders – Import data sets (CT, MRI, cryosection, etc.) – Sculpt & 3D paint volumes – Program “shaders” SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring with shaders • Procedural turbulence creates “clouds” – Defines a color & opacity at each point in space – Animate parameters to evolve the cloud – Voxelize during ray-casting • No volumes created • Very low memory use SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring with shaders • Manipulate shapes – Compute effects during ray-casting • No additional volumes created • For each point in space: – Compute shortest distance to surface – Perturb the distance with turbulence – Map distances to opacity Compute distances Add turbulence SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Map to opacity Do at high resolution History Rendering Authoring Authoring with shaders • Constructing shapes – Constructive Solid Geometry (CSG) – Cut using primitive shapes • (or anything) – Transform and “cut” during ray-casting • No pre-cut volumes created SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring with shaders • Repeat to build volumetric scenes – Multiple data sets, geometry, shaders, … – Evaluate some or all during ray-casting • Rendering is an active part of authoring SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Authoring scenes • Several existing scene structure metaphors – – – – – CSG trees Composite trees Shade trees Scene graphs Expression trees = combine primitives (most CAD apps.) = combine images (Houdini, Shake) = combine shaders (RenderMan) = lay out data sets (VRML, Java3D) = compute scene (any programming lang.) • “Compilation” prevoxelizes some, all, or none of the scene – Tune to minimize error, memory use, rendering time SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Visualizing the Orion nebula • Fluorescing clouds of hydrogen, helium, oxygen, ... – Complex structure with pillars, swirls, ripples, and cavities Eagle Nebula SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Lagoon Nebula Orion Nebula History Rendering Authoring Visualizing the Orion nebula • Build a surface for the ionization front – Derived from visual and infrared data • Make it fuzzy – Perturb distance field with turbulence • Project a color-corrected Hubble image through it – Jitter to reduce streaking SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Visualizing the Orion nebula • Add 85 proplyds (protostars), shock fronts, … – Each built in a similar way • Fly around in it all SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Visualizing the Orion nebula SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Visualizing the Orion nebula • 2112 shader nodes in the full scene – Surfaces, turbulence, image projection, transforms, … • Prevoxelized as 86 volumes – 2 Gbytes of multi-resolution data – 40 Gbytes if we didn’t voxelized separately • Ray-caster composited volumes and stars SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego History Rendering Authoring Concluding remarks • What are some directions to explore? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Data directions At WSCG 2001 • Volume registration “Multimodal Medical Volume Registration Based on Spherical Markers” – Align volumes for compositing • Volume compression – Make it take less space • Volume decimation – Express it well at a smaller size SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego “Geometric Processing of Volumetric Objects” “Towards Continuous Image Representations” “Exploiting Eigenvalues of the Hessian Matrix for Volume Decimation” Rendering directions At WSCG 2001 • Rendering from compressed data – Smaller data = less bandwidth needed • Large (out-of-core) data rendering – I/O during rendering to get data • Parallel rendering “A New Parallel Volume Rendering Algorithm” “Parallel Ray Tracing with 5D Adaptive Subdivision” – More processors to get more bandwidth • Hybrid volume + geometry rendering – Keep shapes in their smallest format SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Authoring directions At WSCG 2001 • Content creation “shaders” “Hypertexturing Complex Volume Objects” – Create using procedures • Volume sculpting / 3D painting – Create from scratch in an artist-natural way • Volume scene construction – Compose, composite, cut, … • Volume scene compilation – Prevoxelize for best memory/CPU use SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Tool directions • Work with users! – Put technology into user’s hands … make it a tool – Help create that attention-grabbing event ? SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego Acknowledgements • USA National Science Foundation • USA Department of Energy • San Diego Supercomputer Center – – – – – – – Mike Bailey Steve Cutchin Alex DeCastro Eric Enquist Jon Genetti Mike Houston John Moreland SAN DIEGO SUPERCOMPUTER CENTER University of California, San Diego
