1.00/1.001 – Intro To Computation and Engineering Problem Solving
Fall 2002
Media in Java®
Topics
• The AWT, Java® 2D API and the newer Java®
Media Framework provide tools for working
with diverse media.
–
–
–
–
–
–
structure of digital media
the AWT Image class and its related methods
the MediaTracker class
the Java® Media Framework structure
use of the JMF for video and audio playing
media synchronization
1
Digital Media
• Includes text, video, audio, animation, images
and vector graphics
• video and audio have temporal dimension
• files are often large and compressed
• diverse file formats
• latency and network throughput are often
issues
The AWT and JMF
• The AWT supports text, limited drawn graphics and
text.
• Java® 1.2 provides much improved graphics and text
support, and additional image processing features
• the Java® Media Framework is an additional package
that provides support for temporal media-audio and
video
• JMF features include synchronization and
implementation using native platform features
2
Images in the AWT
• Image class supports the "push model". Images are
"pushed" to an ImageConsumer. Works well for
applets and other Web applications.
• A digital image (once loaded) is a rectangular
collection of pixels, where each pixel represents a
color
• Java® AWT has an Image class for loading and
displaying images
• AWT can load image from file or URL over the Web
• The Java® 2D API has BufferedImage class that has
much wider range of capabilities
Image in an Applet
• Simplest case is loading an image into an
application
• construct an Image object with code such as:
Image im = new Image(“myImageFile.gif”);
• display image in the paintComponent()
method of Component object using
drawImage() method:
boolean b drawImage(Image im, int x, int y,
ImageObserver iob)
returns true if Image is fully loaded, false otherwise
3
ImageObserver interface
• Images take a long time to load, and
may load incrementally over time
• ImageObserver is an interface that has
methods for handling notification of
state of image loading. It can use this
for redisplay as needed.
• JFrame and Applet both implement
ImageObserver interface
Drawing an Image
import java.awt.*;
import javax.swing.*;
public class ImagePanel extends JPanel {
private Image im;
public ImagePanel(Image i) {
im = i; }
public void paintComponent(Graphics g) {
super.paintComponent(g);
if(im == null)
g.drawString("Image object null", 30,30);
else
g.drawImage(im,10,10,this); }
} // end of ImagePanel class
4
Other Image loading methods
public Image getImage(URL u)
loads image from Web from URL u. URL is a
class in java.net package. Its simplest
constructor is
URL u = new URL(String s)
public Image getImage(URL u, String name)-u
is the base location on a Web server; name is
the file name
MediaTracker class
• Sometimes you want to (or have to wait) for
image to load fully before proceeding
• MediaTracker class allows you to register
images and wait until they are fully loaded
• supports waiting for groups of images
• you register images with MediaTracker object
• MediaTracker can establish priorities for
image loading
5
MediaTracker Constructor and Methods
MediaTracker(Component m) - m is the object where
the images are to be drawn
void addImage(Image im , int id)-im is the image to be
tracked, id is the group of objects with same id
void waitForID(int id) throws InterruptedException
void waitForAll() throws InterruptedException
void waitForID(int id, long ms) throws
InterruptedException
void waitForAll(long ms) throws InterruptedException
boolean checkID(int id)
boolean checkAll()
MediaTracker Use
MediaTracker iTrack = new MediaTracker(this);
Image tImage = new Image(“cecif2.jpg”);
iTrack.addImage(tImage, 1);
try {
iTrack.waitForID(1);
}
catch(InterruptedException e) {
System.err.println(“Error in Image loading”);
}
6
import java.awt.*;
import javax.swing.*;
public class ImageDisplay extends JFrame {
private Image testImage;
private ImagePanel iPanel;
public static void main(String[] args) {
new ImageDisplay(args[0]); }
public ImageDisplay(String imageFile) {
setTitle("Simple Image Displayer");
MediaTracker iTrack = new MediaTracker(this);
testImage = Toolkit.getDefaultToolkit().getImage(imageFile);
iTrack.addImage(testImage, 1);
try {
iTrack.waitForID(1); }
catch(InterruptedException e) {
System.out.println("Exception in loading image(s).");
System.exit(0);
}
iPanel = new ImagePanel(testImage);
getContentPane().add(iPanel);
setSize(testImage.getWidth(this)+50, testImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
}
} // end of ImageDisplay
7
Java® 2D API
• Java® 2D API provide "immediate mode"
model for images. the BufferedImage class is
central to this.
• BufferedImage class allows much greater
programmatic control than AWT's Image
class
• BufferedImage supports affine
transformations, image filtering, multiple color
models, compositing, multiple compression
methods
import java.awt.*;
import javax.swing.*;
import java.io.*;
import com.sun.image.codec.jpeg.*; // JPEG Decoder
public class BufferedImageDisplay extends JFrame {
private Image testImage;
private ImagePanel iPanel;
public static void main(String[] args)
{
new BufferedImageDisplay(args[0]);
}
8
public BufferedImageDisplay(String imageFile) {
setTitle("Simple Image Displayer");
try {
InputStream inStream = new FileInputStream(imageFile);
JPEGImageDecoder de =
JPEGCodec.createJPEGDecoder(inStream);
testImage = de.decodeAsBufferedImage();
}
catch(IOException e) {
System.out.println("Exception in loading image "+imageFile);
System.exit(0); }
iPanel = new ImagePanel(testImage);
getContentPane().add(iPanel);
setSize(testImage.getWidth(this)+50, testImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
}
} // end of ImageDisplay
Inside BufferedImage Class
• BufferedImage object has two major
parts:
– Raster-storage for pixels, each
representing small region of image
– ColorModel-interpretation of pixel value
(maps pixel contents to color components)
• BufferedImage class provides wide
range of options for packing pixels,
color models, indexed colors
9
Common Color Spaces
• RGB-Red, Green Blue, often with an
Alpha component for opacity
• CMYK – Cyan, Magenta, Yellow and
Black
• GrayScale
There are usually measured with respect
to standard references for color
Buffered Operations
• There are standard filters for operating on
BufferedImages that implement the
BufferedImageOp interface
• Basic coding pattern:
– create source BufferedImage object
– Use a "factory method" to create an appropriate
converter that implements BufferedImageOp
interface
– run converter object's filter method, as in
BufferedImage result = converter.filter(source, null); // null
can be the destination image
10
Image Sharpening Example
import
import
import
import
import
java.awt.*;
java.awt.image.*;
javax.swing.*;
java.io.*;
com.sun.image.codec.jpeg.*; // JPEG Decoder
public class SharpenOp extends JFrame {
private BufferedImage testImage;
private ImagePanel iPanel;
private BufferedImage convImage;
public static void main(String[] args)
{
new SharpenOp(args[0]);
}
public SharpenOp(String imageFile) {
setTitle("Sharpen Image Displayer");
try {
InputStream inStream = new
FileInputStream(imageFile);
JPEGImageDecoder de =
JPEGCodec.createJPEGDecoder(inStream);
testImage = de.decodeAsBufferedImage();
}
catch(IOException e) {
System.out.println("Exception in loading image
"+imageFile);
System.exit(0);
}
11
float kern[] = {-1f, -1f, -1f, -1f, 9f, -1f,
-1f, -1f, -1f};
Kernel sKernel = new Kernel(3,3,kern);
ConvolveOp sharp = new ConvolveOp(sKernel);
convImage = sharp.filter(testImage, null);
iPanel = new ImagePanel(convImage);
getContentPane().add(iPanel);
setSize(convImage.getWidth(this)+50,
convImage.getHeight(this)+50);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
setVisible(true);
}
} // end of SharpenOp
Java® Media Framework Goals
• make Java® applications using media portable
• relieve programmer of details of diverse media types
• provide means for using native decompression codes
in Java® applications
• tools for synchronizing two or more temporal media
• allow Java® programmer to treat different media types
uniformly independent of format and network
• provide useful defaults for media players and still
allow customization when desired
12
JMF Concepts
• DataSource - source of media content--file
server, etc.
• Two types are PushDataSource and
PullDataSource
• Example
URL mURL = new URL(“http://wwwceci.mit.edu/seg6.mpg”);
DataSource mySource = new DataSource(mURL)
Player Interface
• Player-an interface for a player of a media
type. Classes meeting this interface will
typically involve native code. Implementations
of Player can be “plug ins”.
• Concept is to allow many different groups
build Player implementations.
13
Manager class
• A “factory” that can create a player
appropriate to particular DataSource.
• Manager can locate Player classes if
available and provide them to applications
public static Player createPlayer (DataSource source)
throws IOException, NoPlayerException
public static Player createPlayer(URL sourceURL)
throws IOException, NoPlayerException
Sample Manager use
// assume mediaURL is a valid URL object
try {
player = Manager.createPlayer(mediaURL);
}
catch(NoPlayerException e) {
System.err.println("No player for URL");
return;
}
catch(IOException e) {
System.err.println("IO Exception encountered");
return;
}
14
Player States
• unrealized - Player has been created but doesn't
know anything about its media except the URL
• realizing - Player is in process of acquiring whatever
resouces it needs
• realized - Player finished realizing and knows about
its resources and media. It now knows how to display
itself.
• prefetching - Player preparing to present media
• prefetched - Player done preparing to present media
• started - clock of Player is running
State Diagram for Player
Source: JavaTM Media Player White Paper, Sun web site
15
Player Components
• public Component getControlPanelComponent() returns a panel with controls for media
(start,stop,etc.)
• public Component getVisualComponent()-returns a
Component in which media can be displayed
• public Component getGainComponent() - returns a
Component with audio volume controls
16
Player Events
• Player can generate a large number of
different Event types specific to JMF
• ControllerEvent is most useful
• Player can register ControllerListener objects
for notification. Uses AWT event model
• ControllerListener interface must have
controllerUpdate(ControllerEvent e) methods
JMF Event Types
17
Sample JMF Usage
• Create Frame for video player to display in
• create URL for video file
• Use Manager class to create appropriate
Player for media
• handle event where Player is fully realized,
and add visual and control Component of
Player to Frame
• start the Player by calling player.start()
method
import java.awt.*;
// import Java AWT package
import java.awt.event.*; // import Java event package
import java.net.*;
// import network package
import java.io.*;
// import Java IO package
import javax.media.*;
// import JMF package
public class VideoPlayer implements ControllerListener {
Player player = null;
Component visualComponent, controlComponent; Container m_container;
public static void main (String args[]) {
String mediaFile = args[args.length-1]; Frame fr = new DestroyableFrame("Simple Video Player");
fr.setSize(428, 398);
fr.setVisible(true);
VideoPlayer player = new VideoPlayer(mediaFile, fr);
} // end of main method for VideoPlayer class
18
// constructor method for VideoPlayer class
public VideoPlayer(String mediaFile, Container container) {
URL mediaURL = null;
m_container = container;
m_container.setLayout(new BorderLayout());
try {
mediaURL = new URL(mediaFile);
player = Manager.createPlayer(mediaURL);
player.addControllerListener(this);
player.realize();
}
catch (MalformedURLException e) {
System.err.println("Invalid media file URL!");
return; }
catch(NoPlayerException e) {
System.err.println("Unable to find a player for "+mediaURL);
return; }
catch(IOException e) {
System.err.println(”IO Exception encountered");
return; }
} // end of VideoPlayer constructor method
// When player is realized, get its visual and control Components and
// put them into the container and start() the complete player.
public void controllerUpdate(ControllerEvent event)
{
if(event instanceof RealizeCompleteEvent){
if((visualComponent = player.getVisualComponent()) != null)
m_container.add("Center", visualComponent);
if ((controlComponent = player.getControlPanelComponent())
!= null)
m_container.add("South",controlComponent);
m_container.validate(); // ensure layout is up to date
m_container.repaint(); // ensure that anything drawn painted
player.start();
// start the player
}
else if (event instanceof ControllerErrorEvent){
System.err.println("*** ControllerErrorEvent *** " +
((ControllerErrorEvent)event).getMessage());
}
} // end of controllerUpdate method
} // end of VideoPlayer class
Java® is a trademark or registered trademark of Sun Microsystems, Inc. in the United
States and other countries.
19