Volume Visualization of Visible Korean Human (VKH)
Dataset on CAVE
Hyung-Seon Parkb, Joong-youn Leea, Minsu Joh a, Min-Suk Chungd,
Young-Hwa Chob, Insung Ihmc
a
Supercomputing Center, KISTI, Daejeon, Korea
Bioinformatics Center, KISTI, Daejeon, Korea
c
Dept. of Computer Science, Sogang Univ., Seoul, Korea
d
Dep. of Anatomy, School of Medicine, Ajou Univ., Suwon, Korea
b
ABSTRACT
Volume visualization is a research area on various techniques that helps generating
meaningful and visual information from two- or higher-dimensional volume dataset. It
has been increasingly important in the field of meteorology, medical science and
computational fluid dynamics, etc. On the other hand, virtual reality is a research field
that deals with various techniques that capable of experiencing the contents in the
virtual world with visual, auditory and tactile sense. Recently, there are many studies
going on about virtual reality worldwide.
We developed a visualization system for CAVE that produces the stereoscopic
images from the huge VKH volume data in real-time using an improved volume
visualization technique. CAVE is an innovative Immersive 3D virtual environment
system, which was developed in EVL (Electronic Visualization Laboratory) at the early
90's. Our system utilized an image-based rendering technique and 2D-texture mapping
hardware for real-time stereoscopic volume visualization since the current 3D-texture
mapping technique is too massive in operation. The system offers various functions of
user interface for visualizing the Visible Korea Human data. In this paper, the
visualization technique for real-time stereoscopic image generation and implementation
of that technique is described.
Keywords: Volume visualization, Virtual reality, CAVE, Visible Korea Human,
Stereoscopic image
1. INTRODUCTION AND MOTIVATION
Computer graphics is a research field on various technique that generate various
images using computer, and it is grown very rapidly for the last few decades. The photo
realistic visualization technologies based on computer graphics are utilized on many
fields. Volume visualization, the most representative technology on computer graphics,
is a research field on various technique that helps generating meaning and visual
information from two- or higher-dimensional volume data set. The volume data is the
sheer bulk of data, usually multivariate, produced by magnetic resonance imaging
(MRI), computed tomography (CT) or scientific computing model for medical science,
meteorology, computational fluid dynamics, etc.
Virtual reality is a research field on various techniques that aid gaining experience in
the virtual world with visual, auditory and tactile sense. The virtual world is generated
with computer graphics technology, and the world is not static, but responds to users
input. Recently there are many studies going on about virtual reality worldwide. CAVE
is a projection-based immersive virtual environment system, which was developed at
electronic visualization laboratory (EVL) in the university of Illinois at Chicago at the
early 90's[1]. This system is first announced at SIGGRAPH ‘92, and it is one of the
most immersive virtual environments in the world until now. CAVE is the cube shaped
VR system and users can view stereoscopic image with three dimensional shutter
glasses. Users also can manipulate the virtual world interactively via 3D wand in the
CAVE. This innovative virtual environment system is used in many application field
such as military, science, aerospace, automobile, medical science and so on. Nowadays,
there is 100+ CAVE systems in the many companies, universities, museums and
research institutes over the world. The Supercomputer Center at KISTI has equipped the
CAVE system, the first installation made in Korea year 2001, and it is used for
visualization of various scientific data sets such as meteorology, computational fluid
dynamics, molecular dynamics and medical science.
The KISTI and Ajou University created CT, MRI and RGB datasets of Korean old
man, and the data is named Visible Korea Human (VKH)[10]. The MR, CT, and
anatomical images were acquired. Length of the cadaver was 1,718 mm and interval of
the MR and CT images was 1 mm, so that 1,718 sets of MR and CT images were
acquired. Each cropped image had 505 X 276 resolution, 8 bit (b) gray color, and 769
kb file size. The length of the cadaver was 1,718 mm and the interval of the anatomical
images was 0.2 mm, so that 8,590 anatomical images were acquired. Each anatomical
image had 3,040 X 2,008 resolution, 24 b colors, and 17,890 kb file size. Several years
ago, the National Library of Medicine (NLM) is firstly reported the Visible Human
dataset that is generated from male and female human cadavers. However, the VH
dataset is quite different form body of Korean people, it is not suitable to use them for
educational purpose and research field of medical science in Korea.
In this paper, we developed the new real-time volume visualization technique for
huge volume dataset such as VKH, and implemented the stereoscopic visualization on
the CAVE and SGI Onyx3400 system.
2. PREVIOUS WORK
High-resolution volume dataset like VKH is too huge to visualize in real-time. So,
many research groups have proposed various volume visualization methods using 3D
graphics hardware but most do not achieve both of the reasonable image quality and
satisfied rendering speed.
Akeley proposed the possibility of accelerating volume visualization using 3D
texture hardware [2]. But the method considers only ambient light and produces only
unshaded images.
Cullip and Neumann proposed very fast volume rendering using 3D texture hardware
[3]. The method can generate 512x512 images only in 0.1 seconds on SGI Onyx Reality
Engine. But the image quality is still poor as before.
Van Gelder and Kim proposed a method that is not so fast but generates much
improved image [4]. The method is much slower than Cullip and Neumann’s one
because the approach reshades the volume and reloads the texture map for every frame
because colors in the texture memory are view dependant.
Dachille et al. proposed the little fast and high quality volume rendering method by
which perform shading and directional shading [5]. But it is still not enough fast for real
time application. The frame rate must be over 15 frames/sec at least for real time
applications but the method can render 128x128x113 CT data into 512x512 image
window in 4.68 frames/sec only.
Ihm et al. proposed the multi-pass rendering algorithm based on the Phong’s
illumination model which produce higher quality image than hardware-based methods
[6]. That emulates the Phong’s illumination model using combination of 2D- and 3Dtexture mapping hardware, since 3D-texture mapping is still expensive. Repeated 2Dtextures mapping was much cheaper than 3D-texture mapping. Still that method was not
supportive for the satisfied speed for the real-time application.
3.
VOLUME
VISUALIZATION
USING
TEXTURE
MAPPING
HARDWARE
In this section, a simple description of the volume rendering algorithm will be
given which was used for enhancing our visualization method. Ihm et al firstly proposed
the multi-pass rendering algorithm. The multi-pass algorithm is not suitable for realtime applications because of the bottleneck in the first step. We modified the first step
using by image based rendering technique.
3.1. Volume Rendering Technique based on Multi-pass Algorithm
The multi-pass algorithm is based on the Phong’s illumination model that produces
higher quality images. The algorithm utilized graphics hardware optimally for the fast
performing of Phong’s illumination model. This method consists of two steps. The first
step generates 2D normal vector images. In the second step, calculations for the ambient,
diffuse and specular components are performed via 2D-texture mapping using the 2D
image, which is generated by the first step as a texture map. Eqs (1) is the equation of
the Phong’s illumination model and Eqs (2) is the matrix equation for Eqs (1).
The multi-pass algorithm for this shading equation is described below.
1. First Step
(a) Generating 2D-texture (N) by composition of the normal vector textures using
3D texture mapping hardware
2. Second Step
(a) Generating ambient-diffuse reflection texture Mad using color matrix function
and normal vector texture N
(b) Generating specular reflection texture Ms using color matrix function and N
(c) Generating the specular-reflection coefficient  texture from specular coefficient
(d) Draw, n times, a rectangle mapped with Ms into the color buffer
(e) Draw a rectangle mapped with  texture into the color buffer
(f) Draw a rectangle mapped with Mad texture into the color buffer
The speed at second step is fast enough because of this is the combination of general
color matrix function and 2D-texture mapping, which is most graphics hardware
support the combination. But the first step takes most of rendering time because the
combination of 3D-texture mapping and composition is very expensive. So this
algorithm may not suitable for real-time application. We proposed the new volume
visualization method that applies image based rendering to prior multi-pass algorithm
3.2. Image Based Rendering
The volume rendering techniques using texture hardware are faster than traditional
way such as ray casting and it generates quite high quality images. However, it is still
not enough speed to apply real-time applications for the huge volume datasets, and the
rendering image is poor than ray casting’s one yet. The previous multi-pass method has
been used 3D-texture mapping and composition technique for the generation of 2D-
normal vector texture in the first step. But it took most of the rendering time because the
3D-texture mapping and its composition is very expensive. Here we proposed the new
method that produces normal vector texture using by image based rendering technique.
The second step can be performed to calculate Phong’s illumination model when the
normal vector information is available. So, we pre-calculated the high resolution normal
textures for the all possible viewing directions using ray casting and the proper image
for the current viewing was used for the normal texture. In the normal ray casting
method, the gradient may calculate at the point at which ray meets voxel, and the
shading was performed using the gradient as normal vector. However, the shading
algorithm was not performed and only the gradient is kept as normal vector to generate
normal textures in this algorithm.
The general image based rendering method has the critical defect such as
regeneration of the 2D-texture images required if the condition of rendering factor is
altered. There are several rendering factors, for example, locations and colors of light
sources, materials of objects in the scene. The new method we proposed is not the way
to use final rendering image for the image based rendering, but the normal vector image
used for the shading in second step in the multi-pass algorithm. By this new method
applied, the problem can be overcome. This method is not only able to visualize huge
volume datasets in real-time but also to produce very high quality images because ray
casting has used to generate normal texture.
4. IMPLEMENTATION AND RESULTS
Both images from left to right eyes are needed for the real-time volume visualization
method. It has to be rendered simultaneously for the interactive and stereoscopic
visualization system. We implemented the new multi-pass algorithm on SGI Onyx3400
InfiniteReality3 to visualize the huge VKH datasets on CAVE system. C/C++ and
OpenGL were used for core rendering routine, and the OpenGL Performer and tracked
API were used for stereoscopic visualization, user interface and tracking. We did not
apply the stereoscopic and Immersive ability, which can be supported by some APIs
such as CAVELib, Multipipe SDK, VR Juggler, etc. It was not able to use those APIs
because the new method was implemented on the image-based rendering.
4.1. Stereoscopic Visualization
The stereoscopic images are produced when images are rendered at each of left and
right eye position of viewpoint and images are displayed simultaneously. The images
are displayed stereoscopically using particular stereo glasses. Human beings perceive
depth as binocular disparity, which is the difference in the images projected onto the
back of the eye (and then onto the visual cortex) because the eyes are separated
horizontally by the interocular distance.
There are two frequently used methods for generation of stereoscopic images, Toe-in
and Off-axis[9]. Each camera located at left and right eye position is pointed at a single
focal point in the Toe-in method. This way can generate stereoscopic images but it
causes discomfort because of the vertical parallax. The Off-axis method is the way to
parallel the direction of camera. This method is more accurate than the Toe-in method.
The eyes may be more comfortable because vertical parallax is never generated. But it
is more expensive because viewing frustum must be modified for each of the eye point.
Figure 1 shows how the focal points and viewing frustums are generated for each
method.
Figure 1. The stereoscopic generation methods
We implemented the both stereoscopic methods and tried to choose optimal one.
Basically, we implemented that the viewpoint is moved following circumference of the
scene and pre-calculated the normal images using ray casting. Figure 2 shows how we
generate the normal images. We experientially found the angle of left and right eye at
which the image was displayed stereoscopically well. We just chose proper textures for
optimal eye angle and used them for normal images at rendering time. For example,
when the viewing position is at 3, if the angle  is optimal for current viewing position,
we can choose the 2 and 4 for the eye position. If the angle  is optimal, we can choose
1 and 5. In Toe-in method, this mechanism is very simply implemented. We created
normal images following circumference of the scene. And we don’t need to generate
each set of eyes. We can use same normal images for both eyes. But we must create
each set of eyes in Off-axis because the viewing frustum is different. So we must keep 2
normal image sets for Off-axis method. When two methods are implemented, the
stereoscopic image was not quite different. So we chose the Toe-in Method because of
efficiency of memory. If the eye position is changed, we adjust the angle, and the
normal images are reloaded.
Figure 2. Generation of the stereoscopic image
4.2. Rendering Engine and User Interface
The rendering engine was implemented with C/C++ and OpenGL. The size of whole
normal vector texture for coronal (x axis) view of VKH with 512X512 resolution was
360 Mbytes. Two kinds of normal vector texture, skin and bone, were generated. The
total size of texture maps is 720 Mbytes. Since 256Mbyte texture memory on Onyx3400
IR3 available, we loaded a part of the textures and continuously swapped the proper
texture when they are needed. But, it is not recognizable because swapping time was
very short on the machine. After texture is loaded, the shading is performed with proper
two normal images using on second step in the multi-pass algorithm. The rendering
engine can cover 5 screens in CAVE, and it is implemented by multipipe function in
OpenGL Performer. The shading factors such as ambient, diffuse, specular etc. can be
adjusted by user interface. We can manipulate those factors with wand that is the 6DOF
interface for CAVE. The transparency of skin is also changeable by user interface. We
could achieve it only with blending function on texture hardware. This program is also
able to map 512x512 texture images. Figure 3 shows the rotation of VKH and figure 4
shows several functions of this program. This new multi-pass rendering engine uses
several 2D-texture mapping and composition instead of 3D-texture slicing and
composition. So this engine performs much faster than prior algorithms. Table 1 shows
the improvement of rendering performance. Because the two images must be displayed
in stereo mode, the performance was twice slower than mono mode. 3D-texture method
in table 1. is the prior multi-pass algorithm which is the 3D-texturing used in the first
step, and 2D-texture method is the new multi-pass algorithm we applied in this study.
The method we applied was 24 times faster in speed than prior one and completely
suitable for real-time applications.
Speed/Frame (sec)
Frame Rate (fps)
3D Texture Method
0.529
1.89
2D Texture Method (Mono)
0.022
45.62
0.044
22.81
2D Texture Method
(Stereo)
Table 1. Comparison of the rendering performance (Resolution: 512x512)
Figure 3. Rotation of the VKH
Figure 4. Several operations to VKH
5. CONCLUSION AND FUTURE WORK
Volume visualization deals with various techniques for extracting meaningful and
visual information from various volume datasets. The case of the datasets is huge, it is
very difficult to render the datasets in real-time. Many research groups have hindered to
develop the fast volume visualization method with high quality images.
Virtual reality is the computer graphics research field that grows very rapidly. CAVE
is the most representative device in the field and it can generate fully immersive virtual
environment. We proposed high in speed and quality volume visualization method for
huge VKH datasets for the real-time applications and development of stereoscopic
visualization system. The system is able to manipulate the rendering functions with 3D
wand device interactively on CAVE. Figure 5. shows the visualization system of VKH
datasets on CAVE. This visualization method was not used for the 3D-texture hardware
but the 2D-texture mapping and image based rendering technique for the fast speed
rendering. We applied the normal vector textures for inputting images for the image
based rendering method to manipulate features of light sources and materials,
dynamically.
Even if the new multi-pass algorithm supports the speed and high quality volume
visualization, this algorithm requires very large amount of memory. Three full-sets of
normal vector textures were needed for arbitrary axis rotation, and the amount of
textures is about 1 gigabyte when the texture size is 512X512. So, the compression of
normal textures may be necessary. When the object zoomed in large scale, the size of
the pixel of image may be increased the reason that the algorithm was based on precomputed images.