Java Industry News
Mar. 1, 1996 12:00 AM
As images come in different sizes and formats, image loading is often a complex and time consuming process. Image loading usually consists of fetching the image file, reading the header, decoding the compressed pixels' values, and then delivering the pixels. Java currently supports two types of images, JPEG and GIF. The loading mechanisms for these formats are already built-in. If other formats are involved, users will have to write their own loading programs.
Image loading may mean different things to different applications. Some applications may display images directly without explicitly using image properties or pixel values. In this case, image loading involves fetching the image as a single object. Image animation is an example of such an application. In some applications, pixels may have to be manipulated. Image loading for such applications will include fetching the image properties and pixel values explicitly, involving several stages. Image Analysis belongs to this category of applications. This article describes some loading issues, problems, and difficulties commonly encountered and a solution to some of these problems.
The Java environment, which is multithreaded, allows asynchronous loading of images. Whether it is from a network or local disk, it takes a finite amount of time to load an image; this duration is unpredictable. As an image is being loaded, the program which uses that image will require the loading status in order to proceed with its task. There can be two approaches to accomplishing this...polling and notification. In polling, the program that needs the image checks the loading status at regular intervals. In a multithreaded environment, this approach is inefficient because the program that waits for the images not only wastes CPU cycles but also hinders the progress of other tasks.
In the notification approach, the image loading program notifies the image user program as the image is being loaded. In this approach, CPU cycles are not wasted and the progress of other tasks is unhindered during loading. However, a proper synchronization mechanism is needed between the image loading and image using programs to trigger appropriate actions at different stages of loading.
The AWT imaging model facilitates image loading through notification. The image producer delivers the pixels to the image consumer asynchronously. The image observer monitors the delivery of pixels. The ImageObserver interface has just one method called imageUpdate(). Whether it is image loading or image drawing, whenever an asynchronous image operation is involved, the imageUpdate() method is invoked at regular intervals to notify the status of that image operation. Although the AWT component class has the built-in imageUpdate() method, it is often necessary to override the imageUpdate() method to provide a better control over the actions that are performed upon image loading.
A typical imageUpdate() method is shown in Listing 1. The arguments of the imageUpdate() method include the status flag, width, and height. The status flag is a bitmask with each bit in the flag representing an operation. If the imageUpdate () method returns false, the updates for that operation will not be notified further.
Image Loading Stages
As Java is used in a wide variety of applications, the loading requirements vary. For example, in image animation there is no need to explicitly extract pixels. In contrast, in image analysis it is a must.
Depending upon the applications, image loading can involve a number of steps. These steps can be grouped into the following three stages.
1. Fetching the Image Object
The getImage() methods in the Applet class or AWT Toolkit class will perform image fetching. Just the invocation of getImage() method does not force an image to load. Images are loaded only when they are needed. Thus, the getImage() method is just a request to load an image. Actual delivery of pixels takes place only when an image is required.
2. Fetching Properties
An image can be considered as a combination of raw pixel values and information about the image. This information may consist of geometrical parameters, viewing parameters, scanning parameters, ownership, image format, etc. The properties of an image can be obtained by explicit requests; i.e., by using the get methods in the Image class. There are three methods available for property fetching. The width and height of an image can be fetched by getWidth(ImageObserver observer) and getHeight(ImageObserver observer) methods respectively. The other image properties can be fetched by the getProperty(String name, ImageObserver observer). This method can be used to fetch any named property that is specific to an image format. All the properties may not be needed to view an image.
Getting the width, height, and other properties are asynchronous operations, which means that the properties requested may not be immediately redeemable if the image is not loaded. In such a case, these methods return a -1. As in any other asynchronous operation in AWT, an ImageObserver object monitors the status of the image property fetch. The ImageObserver receives updates about an Image operation through the imageUpdate() method. When the WIDTH bit is set, the width of the image is available; similarly, image height is available when the HEIGHT bit is set.
3. Extracting Pixel Values
When an image object is loaded, pixels are hidden within that object. In order to grab the pixel values, specific requests need to be made. The PixelGrabber class which implements the ImageConsumer interface is used for extracting pixels. A PixelGrabber object should first be created as:
PixelGrabber pg = new PixelGrabber(img, 0,0,width,height, pixMap, 0, width);
When the PixelGrabber object is created, resources are allocated to enable the pixel grabber operation. The actual fetching of pixels is done by the grabPixels() method, shown in Listing 2.
A request to extract pixels values will also trigger the delivery of pixels.
Image Loading Problems
In this section, some of the problems and difficulties encountered due to Java's image loading are described.
1. Image Painting Problem
As mentioned before, images are loaded only when they are needed. This could cause problems if the paint() method is executed before the image is fully loaded. If double buffering is not implemented, you can see the image slowly unfolding on the screen as it loads. Even with double buffering, if the actual paint() or update() is executed before it is drawn on the off screen graphics, the screen will be blank.
There are many solutions to the image paint problem. One of them is to force the image to load in advance. The AWT component and the Toolkit classes provide the prepareImage() methods to force image loading. When an image is loaded in advance, some resources are tied up for a longer time. When a large number of images is involved, loading in advance is not desirable. The other solution is to use the imageUpdate() method to paint the image only after the image is completely loaded.
2. Animation Problem
Late loading can also result in slow animation as the images are being loaded. It may also lead to undesirable flashing. A solution to this problem is to load all the images in advance and then start the animation. Although prepareImage() methods can be used to accomplish this, it may involve some additional programming. However, the MediaTracker class which uses prepareImage() and imageUpdate() has methods to accomplish advance image loading and tracking. One problem with advance loading is that the viewing area in the applet/application will remain blank until all of the images are loaded. This may not be desirable if the loading times are larger. A progressive painting solution can be resorted to in such cases.
3. Loading Images for Non-Display Applications
Although MediaTracker is easy to use, it can not be used universally. The MediaTracker constructor requires an AWT component to be an argument. Moreover, that component has to be visible on the screen. This is not a problem in multimedia types of applications, as images are always displayed in some subclasses of the AWT component class. However, in Image Analysis types of applications in which computations are often performed before display, a dummy component may have to be made visible in order to use the MediaTracker class. There can also be applications in which graphical user interfaces are separated from the image loading classes. In such cases, MediaTracker will force the use of a GUI component in the image loading classes.
4. Difficulty with imageUpdate() Method A number of actions may have to be performed in different objects when certain status information is available. Because of the asynchronous nature, these objects may be in different states. Managing these will be difficult because the state of the objects has to be known. This would require maintaining state variables in each object and triggering actions based on their state.
The imageUpdate() method has to trigger appropriate actions depending on the status. So, the imageUpdate() method becomes the center of control whenever image related asynchronous operations are involved. This may involve some programming difficulties in large and complex applications such as:
If several images are to be loaded together, it is often necessary to do different things with different images. For example, you may need to display a few images in an image set, but perform analysis on all the images in the set. This would require the imageUpdate() method to identify the image object and take appropriate actions depending on the object.
Typically, in an applet or a small application, a single object handles image loading and drawing. The ImageObserver interface is implemented in that object itself. In other words, the same object will have the imageUpdate() method. However, in a large application, loading may be performed by one object and drawing may be performed in another. The status notification may go to different objects and these objects will have to be aware of what actions are to be performed. A solution would be to create a separate ImageObserver object where the loading and drawing status related to an image is reported.
5. Scattered Image Loading APIs
As we have seen in the Image Loading Stages section, a number of APIs are required to load images. These are scattered over different classes in the java.awt and java.awt.image packages. This would mean a longer learning curve to understand image loading. What is desirable, therefore, is a single class that encapsulates all the image loading functionality. The next section describes such a class.
A Multi Purpose Image Loading Class
An image loading class called ScreenImage was developed to solve some of the problems discussed in the previous section. Unlike MediaTracker, it does not require a visible AWT component. The ScreenImage class implements the ImageObserver interface and its functionality centered around the imageUpdate() method. Clients can register with the ScreenImage class for callback and the imageUpdate() method will notify the registered objects at different stages of image loading. Because of this callback, the ScreenImage eases the programming difficulty involved in using the imageUpdate() method by allowing the clients themselves to perform appropriate actions.
The ScreenImage class has a number of APIs that help to load images, fetch properties, and extract pixels. It also provides the prepareImage() methods to enable the applications to load images in advance. In addition, it saves image properties such as width and height, and pixels' values if extracted. The client applications can monitor the progress in loading by checking the loadStatus variable. Thus, a number of related image loading functions is available under one class. The complete code listing for ScreenImage class is shown in Listing
3. A ScreenImage object needs to be constructed for every image that is going to be loaded. The client application is neither required to be of the type component nor is it required to implement the ImageObserver interface. However, it has to implement the Callback interface, which contains just one method called performCallback(int satus), shown in Listing 4.
The registered clients receive a callback notification whenever the loading status changes. Unlike the imageUpdate() method, this method is invoked when a certain loading status is attained. Currently, it is invoked on the following occasions: Height and width available
Image is ready for drawing
Pixel values are available
A Sample Application/applet.
A sample application/applet was developed to illustrate the use of ScreenImage class. It is used to load images for three different types of applications. Figure 1 shows the Frame that runs the application/applet.
It can display images in four tiled viewports, run animation, and extract pixels. Images can be selected using the ImageSelector Panel. Animation can be performed in any of the four windows. As the animation loop is running, it can be dynamically switched to other ports by selecting a desired viewport. The pixel computation can be performed without even displaying images. Some of the methods used in this application/applet are explained next.
The method shown in Listing 5 can be used in any application to load an image. This method can be a part of the image viewing class. The imageTable variable is a HashTable which saves the ScreenImage and its id. This is a convenient way of keeping track of whether a ScreenImage object has been created for a given image. The fileList variable contains the list of image file names. In case of applets, the setUrlInfo() method can be used.
Listing 6 is the program snippet to display an image. The actual image display takes place in a class called ImageCanvas.
All the methods in Listings 7, 8, and 9 belong to the ImageCanvas class, which performs image painting. In the ImageCanvas class, scnImage is an instance variable. The method in Listing 7 sets that variable. This method first removes itself from the callback list of the previous ScreenImage and then registers with the new ScreenImage object.
The imagePaint() methods shown in Listing 8 are used for painting images. The actual painting is done by the update() method. The drawOffScren() method, which is not shown here, implements the double buffering.
The method shown in Listing 9 is called by the ScreenImage method to notify the image loading status. The action is performed only when an image is ready (or when there is an error, but not shown in the code).
A method for extracting pixels is shown in Listing 10. Pixel values can be extracted without even displaying the image. The code is similar to the loadAnImage() method, except for the fetchPixels() method. In this method, a delay is added between load() and fetchPixels() method. Without this delay, loadPixels() will hang in some platforms.
The method in Listing 11 is called by the ScreenImage object. It checks for the PIXELS_AVAILABLE status and invokes a method to generate statistics.
Summary & Conclusions
The asynchronous nature of Java's image loading may pose problems, including image painting and animation problems, and difficulty in using the imageUpdate() method. The ScreenImage class described here not only solves some of the problems but can also be used in a variety of applications. It encapsulates a number of image loading-related methods that are scattered over different AWT classes. In addition, it keeps track of image loading and saves image pixels. Just like MediaTracker, it simplifies the programming involved in image loading synchronization. If an application has a large number of images involved for simple viewing or animation, using the ScreenImage class may not be as desirable as it may result in unnecessary code overhead.
1. Cohen et al, Professional Java Programming, Wrox Press, September 1996.
2. Graham, Jim, Image loading related articles posted to comp.lang.java. usenet group.
3. java.awt and java.awt.image API Documentation.