CS3162 Introduction to Computer Graphics
Helena Wong, 2001
12 Window Systems
- A window system manages a computer screen.
- Divides the screen into overlapping regions.
- Each region displays output from a particular application.
X window system is widely used in most Unix machines.
We will use the X11 window system for the following discussion.
The Goals of X
- To provide low-level graphics workstation services
- windows
- 2-D color bitmap graphics
- multiprocess access to workstation screen (Multiple clients connected to a display
server)
- low-level input processing
1
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- To have local-area network transparency
-
applications on same or different network node of the workstation
portability of application software
multiple displays
open architecture
- To provide standard programming interfaces, eg. “Xlib” library interface
- To provide user interfaces
- “xterm” terminal emulator
- other tools such as editors and calculators
- programming toolkits
- To encourage further development
- Inexpensive, widely available source code
2
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Features of the X11
- Hardware and operating system independent.
- Developed jointly by MIT and DEC.
- The X11 provides programmer interfaces through Xlib and X Toolkits.
- Xlib provides basic functions for drawing and for manipulating the windows. Eg.,
XcreateWindow(display, parentwin, x,y,width,height,..);
XSetLineAttributes(display, gc, linewidth, linestyle, ..);
XdrawLine(display, win, gc, x1,y1,x2,y2);
- X Toolkits provide a higher level of interface to Xlib through widget. A widget is a
routine for creating and using user-interface components. A widget set may include
menus dialog boxes scrollbars and command buttons.
3
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
4
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Server and Client
The server is a program running on the workstation.
It acts as an intermediary between user programs (or clients) and the workstation
resources (keyboard, mouse, and screen.)
Client applications need not know the hardware of the server resources.
The server performs the following tasks:
- Allows access by multiple clients.
- Allows client applications to share resources (eg. screen space, graphics processor.)
- Interprets network messages (or requests) and acts on them. Requests include “move
window” and “draw polygon”.
- Sends user inputs to clients by sending network messages, eg. key presses, button
presses, and pointer motion.
- Maintains complex data structures including windows and fonts so that the server
can perform its tasks efficiently.
5
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
A Client is an application which requests services from the server.
A client may or may not be running on the same machine as the server (network
transparency).
All communication between a client and a server uses the X Protocol.
6
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Window Manager
In X11, a window manager is just another client, managing a particular screen at any
one time.
Tasks of the window manager:
- Controls the behavior of the screen such as the functionality of each mouse button.
- Controls the layout of the windows such as title bars and borders.
- Controls the size and displacement of windows.
- Manage icons.
Windows and Window Hierarchy
Dimensions of a window:
7
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Hierarchical organization of windows:
-
A window will obscure any overlapping windows that are behind it.
All sub-windows are bounded by their parent windows.
Any part of a sub-window going outside of its parent window will be clipped.
Any thing to be drawn but are outside of the specified window will be clipped too.
8
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- A window’s shape may become irregular when obscured by other windows.
- To simplify the clipping operation, an obscured window is broken into rectangular
windows:
Clipping an object against Window B is simplified to
clipping the object against each of the windows a,b,c.
9
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
13.Virtual Reality (VR)
A definition is from the book Silicon Mirage by Steve Aukstakalnis and David Blatner:
VR is a way for humans to visualize, manipulate and interact with computers and
extremely complex data. … Methods for human-computer interaction are called
computer interfaces, and virtual reality is just the newest in a long line of
interfaces.
General features of virtual reality:
- It emphases in the interaction with the computer through gestures and human senses.
- Instead of using screens and keyboards, people can put displays over their eyes,
gloves on their hands, and headphones on their ears.
- The computer controls what the user senses. The user can control the computer.
- The user may have a feeling of entering into a different world (virtual world).
- We may also want to obtain interactive data related to real world (Augmented VR).
10
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Levels of Virtual Reality
3 levels of VR:
- Passive: Not much control over the environment. Eg., watching a movie or TV
show, reading a book, and listening to the radio. During reading, though we can
choose a particular book, we cannot change its story.
- Exploratory: Instead of just seeing a3D space, we can move around in it (flying or
walking).
- Interactive: Can explore and interact with a virtual world. Eg., reaching out and
grabbing a virtual book, or moving furniture around in a virtual room.
The term “VR” normally refers to either 2nd level or 3rd level of VR.
11
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Human and Computer Communications
People communicate through variety of senses:
- Vision: Visual effects play a very important role in human communication.
- A lot of research has been done on improving the realism of the computer
generated images and the speed of generating them.
- Also, a lot of research has been done on improving the computer vision technology
to understand the user’s world through vision. (Far more difficult than the
previous one.)
- Sound: A direct way for human to exchange ideas, and let people sense the
surrounding, eg. the presence of an approaching car and its approximate distance.
Similar to vision, it is easier to generate sound than to recognize it.
12
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- Other Senses: Other senses include smell and touch.
- To produce different kind of smell as well as to identify them are both difficult to
do. These factors are rarely considered.
- There are existing equipment which can detect forces from or exert forces onto a
user. However, this kind of equipment is usually intrusive in that the user can feel
their presence even through they are not in operation at a particular moment. Eg.
one can feel the presence of the glove when he/she is wearing it.
13
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Types of VR Systems
- Different approaches to let users communicate with the computer.
- Common ones: Immersive VR, Non-immersive VR, Augmented VR, Telepresence.
Immersive VR
The ultimate VR system is to completely immerse the user to a computer simulated
environment.
- Because visual effect is important in human communication, and 3D graphics
hardware is getting cheaper, existing immersive VR systems emphasis on visual
immersion.
- A head Mounted Display (HMD) with position tracker provides virtual 360° vision.
- Some systems may use multiple large projection screens to visually surround users.
- Other systems may provide even the same physical environments. Eg., flight
simulators and car simulators in which the users are confined to a small room.
14
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Non-immersive VR
Non-immersive VR systems do not provide a 360° of
vision and the user is not required to wear an HMD or
be inside a simulation room. => “limited reality”.
- Usually uses traditional monitors for visual output.
- Some may use an additional equipment (such as the
CrystalEyes 3D stereo glasses) to provide stereo
vision. In the CrystalEyes glasses, the 2 lenses
allow light to pass through alternatively in
synchronization with the displayed images to
provide stereo vision.
The user is inside the virtual world only when he is
looking at the computer screen. When he looks away
from it, he is immediately back to the real world.
15
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Augmented VR
It uses transparent glasses onto which data, diagrams, animation or video can be
projected so that the user is able to see the real world with additional information
provided by the computer.
- There are 2 common types of displays available. One is a pair of glasses. A small
projector is attached to it so that video images can be projected onto the pair of
glasses.
- The other one is the flight helmet also used by the pilots of the fighter jets. A small
projector is attached to the helmet so that additional information from the computer
about the environment can be projected onto the transparent screen.
16
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Telepresence
Telepresence links the sensors and manipulators of a remote robot to a human
operator.
- Cameras, microphones and other sensors are attached to the remote robot. Signals
from these sensors are sent to the human operator in a control room. The operator
may be wearing an HMD and other devices which render the remote signals.
- The human operator is also wearing some sensors which sense the movement of the
operator. The signals are sent to the remote robot to control its movement.
Useful in performing some dangerous operations such as putting out the fire in a
nuclear plant, or in performing some otherwise too expensive operations such as Mars
exploration.
17
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Position Sensors
- 2D mouse: Difficult to use in 3D world. Only returns relative (not absolute) mouse
movement.
- Mechanical tracker: A mechanical device is attached to the object. The movement of
the object results in the movement of the mechanical device, which generate the
positional signal. This kind of devices are fast and accurate but has restrictions on
motion.
- Ultrasonic tracker: An ultrasonic transmitter is attached to the object to be tracked
and a few sensors are placed at different positions. By detecting the time it takes for
the sensors to receive the ultrasonic signal from the transmitter, it is possible to
locate the position of the object. However, ultrasonic systems are of low resolution
and are susceptible to noise.
18
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- Light pattern: By putting a pattern of light transmitters on the object and cameras on
the wall (or vice versa), it is possible to locate the position of the object by detecting
the patterns of light captured by the cameras. Limitations: it requires image
processing, which is not reliable enough, and a dedicated room.
- Magnetic tracker: Magnetic tracing is one of the widely used methods.
- A magnetic transmitter is placed at fixed location. A magnetic sensor is attached to
the object.
- Both the transmitter and sensor contain 3 coils at right angles to each other.
- Sensor’s position and orientation affect the amount of current induced to it by the
transmitter.
- As such, the sensor can be used to detect the 3D position (x,y,z) and orientation of
an object.
- Advantage: Magnetic tracking devices are cheap and accurate.
- Limitation: 1) The accuracy of these devices is affected by nearby metallic
materials. 2) It is generally believed that strong magnetic fields are harmful to
human body.
19
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
A Typical Immersive System
20
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Time Lag Problem
All VR systems suffer from the time lag problem.
- When one moves his head, the magnetic sensor senses the change and generates a
signal. There is a delay from head movement signal generation. =>A
- When the head tracking system receives the signal from the sensor, it processes the
signal to obtain the new position and orientation of the sensor. There is also a delay
in this process. =>B
21
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- When the tracking system sends out the information, the computer receives the
information, processes it and then updates the database. There is a delay in this
process. =>C
- After the database is updated, the image generator renders an updated image from
the updated database. There is also a delay in this process. =>D
- A+B+C+D : The time the head moves to the time the image is shown on screen =
total time lag.
- This time lag causes an uncomfortable feeling to the user.
- Some work is being done to reduce such problem by predicting what may the head
be few steps ahead. Problem: the prediction may not always be correct.
22
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Virtual Reality vs Computer Graphics
- In CG, we concern about the output image quality and also the generation speed.
- In VR, we concern about the image quality. However, with the limitation in current
hardware technology, we are more concerned about the generation speed.
- VR: at least 10 images per sec, time lag problem must be within an acceptable level.
- The Time critical rendering issue: given a time, how to generate the best picture.
- Eg., if time is not enough, we may render objects with flat shading instead of
Gouraud shading.
- We may also use simpler models for rendering less important objects.
- The image coherence issue: The time between 2 consecutive frames is small, the
change in the scene is small too. Hence, given the info about previous frame, we can
make use of it and minimize the amount of calculation in generating the next image.
- Eg., if an object is far away in the last frame, it is likely to be still far away in the
current frame.
23
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Current Areas of Research
- Multi-resolution modelling: This uses a simpler model for rendering when the object
is far away from the viewer. This is to reduce the number of polygons needed to be
rendered.
- Fast rendering
- Collision detection: This detects if 2 objects collide with each other.
- Visibility culling: This determines the visibility of a surface or an object.
24
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
14. Multimedia
Multimedia is the integration of media technologies including sound, video, document,
network and graphics with the computer technology. There are 2 major areas of
interest in multimedia:
- Multimedia technology: the technology itself.
- Multimedia applications: the application of the technology
25
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Multimedia Technology
The integration of media technologies may be considered as putting all the devices that
handle different media together with the computer being the central controlling device:
26
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
With a high-speed network,
it is possible to provide
video and audio services
through the network:
27
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
- These systems allow us to watch TV, to have video-conferencing with someone in
remote location, to read an electronic document, etc...
- Under such a situation, we can also have a large central database to provide variety
of information services.
One of the most demanding media technologies is the handling of video. Consider
color images of video quality – 512x512 in resolution:
- The memory size needed to store a single image, Msingle, is:
Msingle = 512 x 512 x 3 Bytes = 0.75 M Bytes
- Assuming 30 frames per second. The memory size needed for a second of image
sequence, Msec:
Msec = 0.75 M Bytes x 30 = 22.5 M Bytes
- A movie typically lasts for 90 minutes. The memory size needed, Mmovie, is:
Mmovie = 22.5 M Bytes x 60 x 90 = about 118.7 G Bytes
28
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
For color images of movie quality, eg. resolution of 1600 x 900 (16:9) with 24 frames
per second:
Msingle = 1600 x 900 x 3 Bytes = 4.12 M Bytes
Msec = 4.12 M Bytes x 24 = 98.88 M Bytes
Mmovie =98.88 M Bytes x 60 x 90 = about 521.4 G Bytes
With such a huge amount of memory, handling of videos digitally becomes a driving
force to improve other technologies such as
- database (to store the images)
- network (to transfer the images from the server to the computer)
- data compression (to compress / uncompress the images before / after transmission
or storage).
- Other related issues (eg. the demand for high resolution displays)
29
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Related Technologies
1. Display Technology:
With windowing systems being widely used in both PCs and workstations, and the
consumer community pushing for high definition television (HDTV), display
technology is advancing rapidly. Monitors of resolution 1600x1280 are commonly
available. At the same time hardware circuitries are also available which can read the
frame buffer fast enough to update the screen near to 100Hz.
2. Network Technology:
A local computer network connects many computers together so that a small
community can share resources such as disk space, printing facilities, and processing
power. With most of the local networks in the whole world being connected together
through the INTERNET, it is now possible to share some of the resources in a global
scale. A fast Ethernet network for video transmission is required. Recent
developments in this area include the 100 Mbits/s fast Ethernet network and the over 1
Gbits/s fiber optical network.
30
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
3. Database Technology
In multimedia environment, there are many different types of data that conventional
data management systems are not prepared to handle. The data objects can span an
enormous range of size and format.
Such a multimedia database contains a mixture of structured data (relations with fixed
fields) and of unstructured data (text and images). All these factors demand for new
models of data management systems.
Because not all existing multimedia systems are built using an object paradigm, the
new models must also provide object-oriented facilities for handling of data objects.
4. Image Compression
The needs for fast network to transmit digital videos and for large database to store
them demand for better and faster image compression methods.
31
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
5. Sound Technology
There are two major issues: music synthesis and speech processing.
Music synthesizing technology is becoming matured. In addition, the protocol called
MIDI is also widely accepted by manufacturers as a standard for networking musical
instruments.
Speech technology can be divided into 2 categories: Speech synthesis and Speech
recognition
6. Memory Technology
A multimedia system may need to have access to a large volume of information. This
demands for more RAMs and secondary storage. With the continuous improvement in
memory technology, both RAMs and secondary storage devices are getting cheaper
and cheaper.
32
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Multimedia Applications
Applications of the multimedia technology are continuously to be proposed and
developed. Some of them are:
1. Video conferencing: allows people at different physical locations to communication
with each other via voice and video through telephone or high speed networks.
2. Video services: are usually provided to users who have their machines connected to
a high speed network. In a cable network environment, subscribers receive different
video services from a central server(s) maintained by the cable company.
3. Video shopping: is a kind of services usually provided in a cable network
environment in which the user can purchase some products simply by pressing a
button on the remote controller while watching an advertisement on the cable TV.
4. Interactive environment for collaboration: is the use of multimedia technology to
provide an interactive environment for collaboration through work space sharing,
and video and audio communication.
5. CD-ROM / DVD-ROM publishing: is the distribution of CDROMs and DVDs which
store the interactive multimedia catalogues or interactive books.
33