CS3162 Introduction to Computer Graphics
Helena Wong, 2001
12 Window Systems
- A window system manages a computer screen.
- Divides the screen into overlapping regions.
- Each region displays output from a particular application.
X window system is widely used in most Unix machines.
We will use the X11 window system for the following discussion.
The Goals of X
-
To provide low-level graphics workstation services
-
-
windows
2-D color bitmap graphics
multiprocess access to workstation screen (Multiple clients connected to a display server)
low-level input processing
To have local-area network transparency
-
applications on same or different network node of the workstation
portability of application software
multiple displays
open architecture
-
To provide standard programming interfaces, eg. “Xlib” library interface
-
To provide user interfaces
- “xterm” terminal emulator
- other tools such as editors and calculators
- programming toolkits
-
To encourage further development
- Inexpensive, widely available source code
Features of the X11
-
Hardware and operating system independent.
-
Developed jointly by MIT and DEC.
-
The X11 provides programmer interfaces through Xlib and X Toolkits.
-
Xlib provides basic functions for drawing and for manipulating the windows. Eg.,
XcreateWindow(display, parentwin, x,y,width,height,..);
XSetLineAttributes(display, gc, linewidth, linestyle, ..);
XdrawLine(display, win, gc, x1,y1,x2,y2);
-
X Toolkits provide a higher level of interface to Xlib through widget. A widget is a routine for
creating and using user-interface components. A widget set may include menus dialog boxes
scrollbars and command buttons.
1
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Server and Client
The server is a program running on the workstation.
It acts as an intermediary between user programs (or clients) and the workstation resources (keyboard,
mouse, and screen.)
Client applications need not know the hardware of the server resources.
The server performs the following tasks:
-
Allows access by multiple clients.
-
Allows client applications to share resources such as screen space and graphics processor.
-
Interprets network messages (or requests) and acts on them. Requests include “move window”
and “draw polygon”.
-
Sends user inputs to clients by sending network messages, eg. key presses, button presses, and
pointer motion.
-
Maintains complex data structures including windows and fonts so that the server can perform its
tasks efficiently.
A Client is an application which requests services from the server.
A client may or may not be running on the same machine as the server (network transparency).
All communication between a client and a server uses the X Protocol.
2
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Window Manager
In X11, a window manager is just another client, managing a particular screen at any one time.
Tasks of the window manager:
-
Controls the behavior of the screen such as the functionality of each mouse button.
Controls the layout of the windows such as title bars and borders.
Controls the size and displacement of windows.
Manage icons.
Windows and Window Hierarchy
Dimensions of a window:
Hierarchical organization of windows:
-
A window will obscure any overlapping windows that are behind it.
-
All sub-windows are bounded by their parent windows.
-
Any part of a sub-window going outside of its parent window will be clipped.
-
Any thing to be drawn but are outside of the specified window will be clipped too.
-
A window’s shape may become irregular when obscured by other windows.
-
To simplify the clipping operation, an obscured window is broken into rectangular windows:
Clipping an object against Window B is simplified to
clipping the object against each of the windows a,b,c.
3
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
13.Virtual Reality (VR)
A definition is from the book Silicon Mirage by Steve Aukstakalnis and David Blatner:
Virtual Reality is a way for humans to visualize, manipulate and interact with computers and
extremely complex data. … Methods for human-computer interaction are called computer
interfaces, and virtual reality is just the newest in a long line of interfaces.
General features of virtual reality:
-
It emphases in the interaction with the computer through gestures and human senses.
-
Instead of using screens and keyboards, people can put displays over their eyes, gloves on their
hands, and headphones on their ears.
-
The computer controls what the user senses and the user, in turn, can control the computer.
-
The user may have a feeling of entering into a different world (virtual world).
-
We may also want to obtain interactive information related to the real world (Augmented VR).
Levels of Virtual Reality
3 levels of VR:
-
Passive: Not much control over the environment. Eg., watching a movie or TV show, reading a
book, and listening to the radio. During reading, though we can choose a particular book, we
cannot change its story.
-
Exploratory: Instead of just seeing a3D space, we can move around in it (flying or walking).
-
Interactive: Can explore and interact with a virtual world. Eg., reaching out and grabbing a virtual
book, or moving furniture around in a virtual room.
The term “VR” normally refers to either 2nd level or 3rd level of VR.
4
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Human and Computer Communications
People communicate through variety of senses:
-
Vision: Visual effects play a very important role in human communication.
-
A lot of research has been done on improving the realism of the computer generated images
and the speed of generating them.
-
Also, a lot of research has been done on improving the computer vision technology to
understand the user’s world through vision. (Far more difficult than the previous one.)
-
Sound: A direct way for human to exchange ideas, and let people sense the surrounding, eg. the
presence of an approaching car and its approximate distance. Similar to vision, it is easier to
generate sound than to recognize it.
-
Other Senses: Other senses include smell and touch.
-
To produce different kind of smell as well as to identify them are both difficult to do. These
factors are rarely considered.
-
There are existing equipment which can detect forces from or exert forces onto a user.
However, this kind of equipment is usually intrusive in that the user can feel their presence
even through they are not in operation at a particular moment. Eg. one can feel the presence
of the glove when he/she is wearing it.
Types of VR Systems
- Different approaches to let users communicate with the computer.
- Common ones: Immersive VR, Non-immersive VR, Augmented VR, and Telepresence.
Immersive VR
The ultimate VR system is to completely immerse the user to a computer simulated environment.
-
Because visual effect is important in human communication, and 3D graphics hardware is getting
cheaper, existing immersive VR systems emphasis on visual immersion.
-
A head Mounted Display (HMD) with position tracker provides a virtual 360° vision.
-
Some systems may use multiple large projection screens to visually surround the user.
-
Other systems may provide even the same physical environments. Eg., flight simulators and car
simulators in which the users are confined to a small room.
5
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Non-immersive VR
Non-immersive VR systems do not provide a 360° of vision and the user is not required to wear an
HMD or be inside a simulation room. => “limited reality”.
-
Usually uses traditional monitors for visual output.
-
Some may use an additional equipment (such as the CrystalEyes 3D stereo glasses) to provide
stereo vision. In the CrystalEyes glasses, the 2 lenses allow light to pass through alternatively in
synchronization with the displayed images to provide stereo vision.
The user is inside the virtual world only when he is looking at the computer screen. When he looks
away from it, he is immediately back to the real world.
Augmented VR
It uses transparent glasses onto which data, diagrams, animation or video can be projected so that the
user is able to see the real world with additional information provided by the computer.
-
There are 2 common types of displays available. One is a pair of glasses. A small projector is
attached to it so that video images can be projected onto the pair of glasses.
The other one is the flight helmet also used by the pilots of the fighter jets. A small projector is
attached to the helmet so that additional information from the computer about the environment
can be projected onto the transparent screen.
Telepresence
Telepresence links the sensors and manipulators of a remote robot to a human operator.
-
Cameras, microphones and other sensors are attached to the remote robot. Signals from these
sensors are sent to the human operator in a control room. The operator may be wearing an HMD
and other devices which render the remote signals.
-
The human operator is also wearing some sensors which sense the movement of the operator.
The signals are sent to the remote robot to control its movement.
Useful in performing some dangerous operations such as putting out the fire in a nuclear plant, or in
performing some otherwise too expensive operations such as Mars exploration.
6
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Position Sensors
-
2D mouse: Difficult to use in 3D world. Only returns relative (not absolute) mouse movement.
-
Mechanical tracker: A mechanical device is attached to the object. The movement of the object
results in the movement of the mechanical device, which generate the positional signal. This kind
of devices are fast and accurate but has restrictions on motion.
-
Ultrasonic tracker: An ultrasonic transmitter is attached to the object to be tracked and a few
sensors are placed at different positions. By detecting the time it takes for the sensors to receive
the ultrasonic signal from the transmitter, it is possible to locate the position of the object.
However, ultrasonic systems are of low resolution and are susceptible to noise.
-
Light pattern: By putting a pattern of light transmitters on the object and cameras on the wall (or
vice versa), it is possible to locate the position of the object by detecting the patterns of light
captured by the cameras. Limitations: it requires image processing, which is not reliable enough,
and a dedicated room.
-
Magnetic tracker: Magnetic tracing is one of the widely used methods.
-
A magnetic transmitter is placed at fixed location. A magnetic sensor is attached to the object.
-
Both the transmitter and sensor contain 3 coils at right angles to each other.
-
Sensor’s position and orientation affect the amount of current induced to it by the transmitter.
-
As such, the sensor can be used to detect the 3D position (x,y,z) and orientation of an object.
-
Advantage: Magnetic tracking devices are cheap and accurate.
-
Limitation: 1) The accuracy of these devices is affected by nearby metallic materials. 2) It is
generally believed that strong magnetic fields are harmful to human body.
A Typical Immersive System
7
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
Time Lag Problem
All VR systems suffer from the time lag problem.
-
When one moves his head, the magnetic sensor senses the change and generates a signal. There is
a delay from the head movement to the generation of the signal. =>A
-
When the head tracking system receives the signal from the sensor, it processes the signal to
obtain the new position and orientation of the sensor. There is also a delay in this process. =>B
-
When the tracking system sends out the information, the computer receives the information,
processes it and then updates the database. There is a delay in this process. =>C
-
After the database is updated, the image generator renders an updated image from the updated
database. There is also a delay in this process. =>D
-
A+B+C+D : The time the head moves to the time the image is shown on screen = total time lag.
-
This time lag causes an uncomfortable feeling to the user.
-
Some work is being done to reduce such problem by predicting what may the head be few steps
ahead. Problem: the prediction may not always be correct.
Virtual Reality vs Computer Graphics
-
In CG, we concern about the output image quality and also the speed in generating the image.
-
In VR, we concern about the quality of the images. However, with the limitation in current
hardware technology, we are more concerned about the speed in generating the images.
-
VR: at least 10 images per second and the time lag problem must be within an acceptable level.
-
The Time critical rendering issue: given the time we have, how can we generate the best picture.
-
Eg., if time is not enough, we may render objects with flat shading instead of Gouraud shading.
-
We may also use simpler models for rendering less important objects.
-
The image coherence issue: The time between 2 consecutive frames is small, the change in the
scene is small too. Hence, given the information about the previous frame, we can make use of it
and minimize the amount of calculation in generating the next image.
-
Eg., if an object is far away in the last frame, it is likely to be still far away in the current frame.
Current Areas of Research
-
Multi-resolution modelling: This uses a simpler model for rendering when the object is far away
from the viewer. This is to reduce the number of polygons needed to be rendered.
-
Fast rendering
-
Collision detection: This detects if 2 objects collide with each other.
-
Visibility culling: This determines the visibility of a surface or an object.
8
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
14. Multimedia
Multimedia is the integration of media technologies including sound, video, document, network and
graphics with the computer technology. There are 2 major areas of interest in multimedia:
-
Multimedia technology: the technology itself.
Multimedia applications: the application of the technology
Multimedia Technology
The integration of media technologies may be considered as putting all the devices that handle
different media together with the computer being the central controlling device:
With a high-speed network, it is possible to provide video and audio services through the network:
-
These systems allow us to watch TV, to have video-conferencing with someone in remote
location, to read an electronic document, etc...
-
Under such a situation, we can also have a large central database to provide variety of information
services.
9
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
One of the most demanding media technologies is the handling of video. Consider color images of
video quality – 512x512 in resolution:
-
The memory size needed to store a single image, Msingle, is:
Msingle = 512 x 512 x 3 Bytes = 0.75 M Bytes
-
Assuming 30 frames per second. The memory size needed for a second of image sequence, Msec:
Msec = 0.75 M Bytes x 30 = 22.5 M Bytes
-
A movie typically lasts for 90 minutes. The memory size needed, Mmovie, is:
Mmovie = 22.5 M Bytes x 60 x 90 = about 118.7 G Bytes
For color images of movie quality, eg. resolution of 1600 x 900 (16:9) with 24 frames per second:
Msingle = 1600 x 900 x 3 Bytes = 4.12 M Bytes
Msec = 4.12 M Bytes x 24 = 98.88 M Bytes
Mmovie =98.88 M Bytes x 60 x 90 = about 521.4 G Bytes
With such a huge amount of memory, handling of videos digitally becomes a driving force to improve
other technologies such as
- database (to store the images)
- network (to transfer the images from the server to the computer)
- data compression (to compress / uncompress the images before / after transmission or storage).
- Other related issues (eg. the demand for high resolution displays)
Related Technologies
1. Display Technology:
With windowing systems being widely used in both PCs and workstations, and the consumer
community pushing for high definition television (HDTV), display technology is advancing rapidly.
Monitors of resolution 1600x1280 are commonly available. At the same time hardware circuitries are
also available which can read the frame buffer fast enough to update the screen near to 100Hz.
2. Network Technology:
A local computer network connects many computers together so that a small community can share
resources such as disk space, printing facilities, and processing power. With most of the local
networks in the whole world being connected together through the INTERNET, it is now possible to
share some of the resources in a global scale. A fast Ethernet network for video transmission is
required. Recent developments in this area include the 100 Mbits/s fast Ethernet network and the over
1 Gbits/s fiber optical network.
3. Database Technology
In a multimedia environment, there may be many different types of data that conventional data
management systems are not prepared to handle. The data objects can span an enormous range of size
and format. Such a multimedia database contains a mixture of structured data (relations with fixed
fields) and of unstructured data (text and images). All these factors demand for new models of data
management systems. Because not all multimedia systems are built using an object paradigm, the
new models must also provide object-oriented facilities for handling of data objects.
4. Image Compression
The needs for fast network to transmit digital videos and for large database to store them demand for
better and faster image compression methods.
10
CS3162 Introduction to Computer Graphics
Helena Wong, 2001
5. Sound Technology
There are two major issues: music synthesis and speech processing.
Music synthesizing technology is becoming matured. In addition, the protocol called MIDI is also
widely accepted by manufacturers as a standard for networking musical instruments.
Speech technology can be divided into 2 categories: Speech synthesis and Speech recognition
6. Memory Technology
A multimedia system may need to have access to a large volume of information. This demands for
more RAMs and secondary storage. With the continuous improvement in memory technology, both
RAMs and secondary storage devices are getting cheaper and cheaper.
Multimedia Applications
Applications of the multimedia technology are continuously to be proposed and developed. Some of
them are:
1. Video conferencing: allows 2 or more people at different physical locations to communication
with each other via voice and video through telephone or high speed networks.
2. Video services: are usually provided to users who have their machines connected to a high speed
network. In a cable network environment, subscribers receive different video services from a
central server(s) maintained by the cable company.
3. Video shopping: is a kind of services usually provided in a cable network environment in which
the user can purchase some products simply by pressing a button on the remote controller while
watching an advertisement on the cable TV.
4. Interactive environment for collaboration: is the use of multimedia technology to provide an
interactive environment for collaboration through work space sharing, and video and audio
communication.
5. CD-ROM / DVD-ROM publishing: is the distribution of CDROMs and DVDs which store the
interactive multimedia catalogues or interactive books.
11